Method and device for identifying concave obstacles, processor and electronic device

Abstract

The application discloses a concave obstacle recognition method and device, a processor and an electronic device, relates to the technical field of target recognition and detection, and comprises the following steps: segmenting an original gray image to obtain a shadow area image and a highlight area image; clustering pixel points in the shadow area image and the highlight area image according to the distance between the pixel points to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas and a second center point coordinate of each class of highlight areas; matching each class of shadow areas and each class of highlight areas; determining a target sub-image containing a single concave obstacle from the original gray image, performing edge extraction on the target sub-image, obtaining target edge information, and determining information of the concave obstacle in the target sub-image. Through the application, the problem that the recognition accuracy of the concave obstacle is low due to the sparsity of three-dimensional imaging data points in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target recognition detection, in particular to a concave obstacle recognition method and device, a processor and an electronic device. BACKGROUND

[0002] The lunar surface topography environment is complex, and the widely distributed impact craters and other concave obstacles of various sizes are one of the key factors directly threatening the safe movement of the lunar rover. Once trapped, it will bring serious dangerous consequences such as tilting, landslide, and even rollover to the lunar rover. Therefore, effectively recognizing and detecting the lunar concave obstacles are beneficial to obstacle avoidance and provide necessary information for the safe movement of the lunar rover.

[0003] Target recognition detection, especially the rapid and accurate recognition and detection of obstacles, is a key point in the development of intelligent mobile robots, and has attracted high attention and in-depth research at home and abroad, and numerous detection methods have emerged. Common detection methods mainly include a method based on stereo vision and a method based on three-dimensional laser radar technology. Among them, the method based on binocular camera stereo vision can reconstruct the lunar terrain in three dimensions, and then can well recognize and detect various types of obstacles, but the calculation amount of three-dimensional construction is relatively large, the manual time is long, and the real-time performance is poor; the method based on three-dimensional laser radar technology can detect various obstacles by emitting laser beams to conduct high-precision three-dimensional mapping of the lunar environment, and is not affected by light conditions, but requires equipment and is expensive. Moreover, when recognizing the concave obstacles based on three-dimensional imaging data points, the recognition accuracy of the concave obstacles is relatively low due to the sparseness of the three-dimensional imaging data points.

[0004] At present, no effective solution has been proposed for the problem that the recognition accuracy of the concave obstacles is low due to the sparseness of the three-dimensional imaging data points when recognizing the concave obstacles based on three-dimensional imaging data points in the related art. SUMMARY

[0005] The main purpose of the present application is to provide a concave obstacle recognition method and device, a processor and an electronic device, to solve the problem that the recognition accuracy of the concave obstacles is low due to the sparseness of the three-dimensional imaging data points when recognizing the concave obstacles based on three-dimensional imaging data points in the related art.

[0006] In order to achieve the above object, according to one aspect of the present application, a method for identifying concave obstacles is provided. The method comprises: calculating a first target gray threshold and a second target gray threshold by using the maximum inter-class variance method on an original gray image to be identified, and segmenting the original gray image by using the first target gray threshold and the second target gray threshold to obtain a shadow area image and a highlight area image, wherein the original gray image includes a plurality of concave obstacles; clustering the pixel points in the shadow area image and the highlight area image according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas; calculating the solar azimuth angle and the solar elevation angle according to ephemeris prediction to obtain a light direction vector in the original gray image, and matching each class of shadow areas and each class of highlight areas based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image area corresponding to each matching group, wherein the matching group is one-to-one corresponding to the concave obstacle; determining a target sub-image containing a single concave obstacle from the original gray image according to the external rectangle of the image area corresponding to each matching group, performing edge extraction on the target sub-image to obtain target edge information, and performing elliptical fitting according to the target edge information to determine the information of the concave obstacle in the target sub-image.

[0007] Further, the first target gray threshold and the second target gray threshold calculated by using the maximum inter-class variance method comprise: calculating the total number of pixel points contained in the original gray image and the number of pixel points contained by each gray value, and calculating the probability value of each gray value in the original gray image according to the total number and the number of pixel points contained by each gray value; setting a first initial gray threshold and a second initial gray threshold, and dividing the pixel points in the original gray image into first, second, and third category pixel points according to the first initial gray threshold and the second initial gray threshold; calculating the gray value probability sum, the average gray value, and the gray variance value of each category of pixel points according to the probability value of each gray value; calculating the global average gray value of the gray values in the original gray image, and calculating the global gray variance value according to the global average gray value, and constructing an inter-class variance equation according to the gray value probability sum, the gray variance value, and the global gray variance value; obtaining the gray threshold corresponding to the maximum value of the inter-class variance equation by solving the inter-class variance equation, and taking the gray threshold as the first target gray threshold and the second target gray threshold.

[0008] Further, the dividing the pixel points in the original gray image into the first category pixel points, the second category pixel points and the third category pixel points according to the first initial gray threshold and the second initial gray threshold comprises: if the gray value corresponding to the pixel point in the original gray image is less than or equal to the first initial gray threshold, determining the pixel point as the first category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the first initial gray threshold and less than or equal to the second initial gray threshold, determining the pixel point as the second category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the second initial gray threshold, determining the pixel point as the third category pixel point.

[0009] Further, the segmenting the original gray image to be recognized by the first target gray threshold and the second target gray threshold to obtain the shadow region image and the highlight region image comprises: dividing the pixel points in the original gray image according to the first target gray threshold and the second target gray threshold to obtain fourth category pixel points, fifth category pixel points and sixth category pixel points, wherein the region corresponding to the fourth category pixel points is the shadow region, the region corresponding to the fifth category pixel points is the background region, and the region corresponding to the sixth category pixel points is the highlight region; segmenting the original gray image based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points to obtain the shadow region image and the highlight region image.

[0010] Further, before the clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different categories of shadow regions, the first center point coordinates of each category of shadow regions, a plurality of different categories of highlight regions and the second center point coordinates of each category of highlight regions, the method further comprises: removing the noise points in the shadow region image and the highlight region image by the image erosion method, and filling the gaps in the shadow region image and the highlight region image by the image dilation method.

[0011] Further, the sun azimuth and the sun elevation are calculated according to the ephemeris prediction to obtain a light direction vector in the original gray image, and each type of shadow region and each type of highlight region are matched based on the light direction vector, the first center point coordinate and the second center point coordinate to obtain a plurality of matching groups and a circumscribed rectangle of an image region corresponding to each matching group, including: calculating a position direction vector of a line between the first center point coordinate and the second center point coordinate; constructing an angle factor based on the light direction vector and the position direction vector; calculating a length value of each type of shadow region and each type of highlight region in the light direction, and constructing a distance factor according to the length value; and matching each type of shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of matching groups and a circumscribed rectangle of an image region corresponding to each matching group.

[0012] Further, the matching of each type of shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of matching groups and a circumscribed rectangle of an image region corresponding to each matching group includes: calculating a matching score of a shadow region to be matched and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of first matching score values; calculating a matching score of a highlight region to be matched and each type of shadow region according to the angle factor and the distance factor to obtain a plurality of second matching score values; matching each type of shadow region and each type of highlight region according to the first matching score values and the second matching score values to obtain a plurality of matching groups; calculating a vertex coordinate of a target direction of each matching group, and determining a circumscribed rectangle of an image region corresponding to each matching group according to the vertex coordinate.

[0013] Further, the edge extraction of the target sub-image to obtain target edge information includes: performing edge extraction on the target sub-image through an edge detection algorithm to obtain initial edge information; removing edge information with an edge length less than a first preset threshold in the initial edge information to obtain processed initial edge information; and removing pseudo edges and non-arc edges in the processed initial edge information to obtain the target edge information.

[0014] Further, before the pseudo edges and non-arc edges in the processed initial edge information are removed to obtain the target edge information, the method further includes: calculating a gradient vector of an edge point in the processed initial edge information, and obtaining a light direction vector in the original gray image; calculating a target included angle value between the gradient vector and the light direction vector; and if the target included angle value is greater than a second preset threshold, determining that edge information corresponding to the target included angle value is the pseudo edge.

[0015] Further, before the false edges and the non-arc edges in the processed initial edge information are removed to obtain the target edge information, the method further comprises: calculating a target value according to coordinates of an end point of a target edge in the processed initial edge information and a geometric center coordinate of the target edge; and if the target value is less than a third preset threshold, determining that the target edge is the non-arc edge.

[0016] Further, the elliptical fitting according to the target edge information to determine the information of the concave obstacle in the target sub-image comprises: mapping the coordinate information in the target edge information into an elliptical parameter space to obtain a plurality of points in a five-dimensional space; performing statistical voting on the plurality of points in the five-dimensional space to obtain a voting peak value of each point in the five-dimensional space, wherein the plurality of points in the five-dimensional space correspond to different elliptical parameters; and taking a point in the five-dimensional space corresponding to a voting peak value exceeding a fourth preset threshold as a target elliptical parameter; and determining the information of the concave obstacle in the target sub-image based on the target elliptical parameter.

[0017] To achieve the above object, according to another aspect of the present application, a device for identifying a concave obstacle is provided. The device comprises: a first calculation unit configured to calculate a first target gray threshold and a second target gray threshold by using the maximum inter-class variance method on an original gray image to be identified, and to segment the original gray image by using the first target gray threshold and the second target gray threshold to obtain a shadow area image and a highlight area image, wherein the original gray image comprises a plurality of concave obstacles; a clustering unit configured to cluster pixel points in the shadow area image and the highlight area image according to distances between the pixel points based on a K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas; a matching unit configured to calculate a solar azimuth angle and a solar altitude angle according to ephemeris prediction to obtain a light direction vector in the original gray image, and to match each class of shadow areas and each class of highlight areas based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image area corresponding to each matching group, wherein a matching group is one-to-one corresponding to the concave obstacle; and a first determination unit configured to determine a target sub-image containing a single concave obstacle from the original gray image according to the external rectangle of the image area corresponding to each matching group, to perform edge extraction on the target sub-image to obtain target edge information, and to perform elliptical fitting according to the target edge information to determine information of the concave obstacle in the target sub-image.

[0018] Further, the first calculation unit comprises: a first calculation module, configured to calculate a total number of pixel points contained in the original gray-scale image and a number of pixel points contained in each gray-scale value, and calculate a probability value of each gray-scale value in the original gray-scale image according to the total number and the number of pixel points contained in each gray-scale value; a setting module, configured to set a first initial gray-scale threshold and a second initial gray-scale threshold, and divide the pixel points in the original gray-scale image into first category pixel points, second category pixel points and third category pixel points according to the first initial gray-scale threshold and the second initial gray-scale threshold; a second calculation module, configured to calculate a gray-scale value probability sum, an average gray-scale value and a gray-scale variance value of each category pixel point according to the probability value of each gray-scale value; a third calculation module, configured to calculate a global average gray-scale value of the original gray-scale image, calculate a global gray-scale variance value according to the global average gray-scale value, and construct an inter-class variance equation according to the gray-scale value probability sum, the gray-scale variance value and the global gray-scale variance value; a solving module, configured to obtain a gray-scale threshold corresponding to a maximum value of the inter-class variance equation by solving the inter-class variance equation, and take the gray-scale threshold as the first target gray-scale threshold and the second target gray-scale threshold.

[0019] Further, the setting module comprises: a first determination submodule, configured to determine the pixel point in the original gray-scale image as the first category pixel point if a gray-scale value corresponding to the pixel point is less than or equal to the first initial gray-scale threshold; a second determination submodule, configured to determine the pixel point in the original gray-scale image as the second category pixel point if the gray-scale value corresponding to the pixel point is greater than the first initial gray-scale threshold and less than or equal to the second initial gray-scale threshold; and a third determination submodule, configured to determine the pixel point in the original gray-scale image as the third category pixel point if the gray-scale value corresponding to the pixel point is greater than the second initial gray-scale threshold.

[0020] Further, the first calculation unit comprises: a division module, configured to divide the pixel points in the original gray-scale image according to the first target gray-scale threshold and the second target gray-scale threshold to obtain fourth category pixel points, fifth category pixel points and sixth category pixel points, wherein a region corresponding to the fourth category pixel points is the shadow region, a region corresponding to the fifth category pixel points is a background region, and a region corresponding to the sixth category pixel points is the highlight region; and a segmentation module, configured to segment the original gray-scale image based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points to obtain the shadow region image and the highlight region image.

[0021] Further, the device further comprises: a processing unit, configured to, before clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow regions, a first center point coordinate of each class of shadow regions, a plurality of different classes of highlight regions, and a second center point coordinate of each class of highlight regions, remove noise points from the shadow region image and the highlight region image by using an image erosion device, and fill gaps in the shadow region image and the highlight region image by using an image expansion device.

[0022] Further, the matching unit comprises: an acquisition module, configured to calculate a position direction vector of a line connecting the first center point coordinate and the second center point coordinate; a construction module, configured to construct an angle factor based on the light direction vector and the position direction vector; a fourth calculation module, configured to calculate a length value of each class of shadow regions and each class of highlight regions in the light direction, and construct a distance factor according to the length value; and a matching module, configured to match each class of shadow regions and each class of highlight regions according to the angle factor and the distance factor, to obtain a plurality of matching groups and an external rectangle of an image region corresponding to each matching group.

[0023] Further, the matching module comprises: a first calculation submodule, configured to calculate a matching score of a shadow region to be matched and each class of highlight regions according to the angle factor and the distance factor, to obtain a plurality of first matching score values; a second calculation submodule, configured to calculate a matching score of a highlight region to be matched and each class of shadow regions according to the angle factor and the distance factor, to obtain a plurality of second matching score values; a matching submodule, configured to match each class of shadow regions and each class of highlight regions according to the first matching score values and the second matching score values, to obtain a plurality of matching groups; and a third calculation submodule, configured to calculate a vertex coordinate of a target direction of each matching group, and determine an external rectangle of an image region corresponding to each matching group according to the vertex coordinate.

[0024] Further, the first determination unit comprises: an extraction module, configured to perform edge extraction on the target sub-image by using an edge detection algorithm to obtain initial edge information; a first elimination module, configured to eliminate edges in the initial edge information with a length less than a first preset threshold value to obtain processed initial edge information; and a second elimination module, configured to eliminate pseudo edges and non-arc edges in the processed initial edge information to obtain the target edge information.

[0025] Further, the apparatus further comprises: a second calculation unit, configured to calculate a gradient vector of an edge point in the processed initial edge information and obtain a light direction vector in the original gray image before the target edge information is obtained by eliminating the pseudo edge and the non-arc edge in the processed initial edge information; a third calculation unit, configured to calculate a target included angle value between the gradient vector and the light direction vector; and a second determination unit, configured to determine that the edge information corresponding to the target included angle value is the pseudo edge if the target included angle value is greater than a second preset threshold.

[0026] Further, the apparatus further comprises: a fourth calculation unit, configured to calculate a target value according to a coordinate of an end point of a target edge in the processed initial edge information and a geometric center coordinate of the target edge before the target edge information is obtained by eliminating the pseudo edge and the non-arc edge in the processed initial edge information; and a third determination unit, configured to determine that the target edge is the non-arc edge if the target value is less than a third preset threshold.

[0027] Further, the first determination unit comprises: a solving module, configured to map coordinate information in the target edge information into an elliptic parameter space to obtain a plurality of points in a five-dimensional space; a statistical module, configured to statistically vote the points in the plurality of five-dimensional spaces to obtain a voting peak value of each point in the five-dimensional space, wherein the points in the plurality of five-dimensional spaces correspond to different elliptic parameters; and the point in the five-dimensional space corresponding to a voting peak value exceeding a fourth preset threshold is taken as a target elliptic parameter; and a determination module, configured to determine information of the concave obstacle in the target sub-image based on the target elliptic parameter.

[0028] In order to achieve the above-mentioned purpose, according to an aspect of the present application, a processor is provided, and the processor is used to run a program, wherein the program performs the concave obstacle identification method of any one of the above-mentioned aspects when running.

[0029] In order to achieve the above-mentioned purpose, according to an aspect of the present application, an electronic device is provided, and the electronic device comprises one or more processors and a memory, and the memory is used to store the one or more processors to implement the concave obstacle identification method of any one of the above-mentioned aspects.

[0030] By the present application, the following steps are adopted: the original gray image to be recognized is calculated by the maximum inter-class variance method to obtain a first target gray threshold and a second target gray threshold, and the original gray image is segmented by the first target gray threshold and the second target gray threshold to obtain a shadow area image and a highlight area image, wherein the original gray image includes a plurality of concave obstacles; pixel points in the shadow area image and the highlight area image are respectively clustered according to distances between the pixel points based on a K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas; a sun azimuth and a sun elevation are calculated according to ephemeris prediction to obtain a light direction vector in the original gray image, and each class of shadow areas and each class of highlight areas are matched based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image area corresponding to each matching group, wherein the matching groups are one-to-one corresponding to the concave obstacles; a target sub-image containing a single concave obstacle is determined from the original gray image according to the external rectangle of the image area corresponding to each matching group, edge extraction is performed on the target sub-image to obtain target edge information, and ellipse fitting is performed according to the target edge information to determine information of the concave obstacle in the target sub-image, thereby solving the problem in the related art that when a concave obstacle is recognized based on three-dimensional imaging data points, the recognition accuracy of the concave obstacle is low due to the sparseness of the three-dimensional imaging data points. The original gray image is segmented by the first target gray threshold and the second target gray threshold to obtain the shadow area image and the highlight area image, the pixel points in the shadow area image and the highlight area image are respectively clustered according to distances between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas, then a single concave obstacle is recognized through shadow and highlight area matching, and edge detection is performed on a sub-image of the original gray image containing the single concave obstacle, thereby solving the defect that multiple obstacles interfere with each other during detection and affect the detection effect, and finally realizing accurate detection of a position range of a concave obstacle area, and further achieving the effect of improving the recognition accuracy of the concave obstacle. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings, and their description, are presented to add generic scope to this application. In the drawings:

[0032] Figure 1 is a flow chart of a concave obstacle recognition method provided according to an embodiment of the present application;

[0033] Figure 2 is a feature diagram of a concave obstacle provided according to an embodiment of the present application;

[0034] Figure 3 is a schematic diagram of gray scale change of a concave obstacle edge according to an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of an ellipse according to an embodiment of the present application;

[0036] Figure 5 is a flowchart of an optional identification method of a concave obstacle according to an embodiment of the present application;

[0037] Figure 6 is a schematic diagram of an identification device of a concave obstacle according to an embodiment of the present application;

[0038] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0041] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0042] The present application will be described below in combination with preferred implementation steps, Figure 1 is a flowchart of an identification method of a concave obstacle according to an embodiment of the present application, as Figure 1 shown, the method comprises the following steps:

[0043] In step S101, the first target gray threshold and the second target gray threshold are obtained by calculating the original gray image to be identified by the maximum inter-class variance method, and the original gray image is segmented by the first target gray threshold and the second target gray threshold to obtain a shadow area image and a highlight area image.

[0044] Specifically, the lunar surface topography environment is complex, and the widely distributed impact craters and other concave obstacles of different sizes are one of the key factors directly threatening the safe movement of the lunar rover. Once trapped, it will bring serious dangerous consequences such as tilting, landslide, and even rollover to the lunar rover. Therefore, effectively identifying and detecting the lunar concave obstacles are beneficial to obstacle avoidance and provide necessary information for the safe movement of the lunar rover.

[0045] Therefore, based on the above technical background, a concave obstacle identification method is proposed. The original gray image of the concave obstacle on the lunar surface is obtained. According to the image features of the lunar concave obstacle, such as Figure 2 As shown in the figure, the edge of the lunar concave obstacle is basically elliptical, and the middle is concave with the edge forming a height difference. In the vacuum environment of the moon, under illumination, the edge and the internal backlight side form a obvious shadow area, and the edge and the internal light side form a obvious highlight area, and the shadow area and the highlight area have a one-to-one correspondence.

[0046] According to the above image features of the lunar concave obstacle, the gray value of the original gray image to be identified can be calculated based on the maximum inter-class variance method to obtain the first target gray threshold and the second target gray threshold. The shadow area and the highlight area of the concave obstacle are extracted from the background area by using the first target gray threshold and the second target gray threshold.

[0047] In step S102, the pixel points in the shadow area image and the highlight area image are respectively clustered according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas.

[0048] Specifically, after the shadow area and the highlight area of the concave obstacle are extracted from the background area, the K-means clustering algorithm is used according to the pixel distribution distance to aggregate a single shadow area and a highlight area, that is, a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas. It should be noted that the class here corresponds to different concave obstacles. The original gray image includes a plurality of concave obstacles, and each pixel point in the original gray image needs to be identified as belonging to which concave obstacle.

[0049] Step S103, according to the ephemeris prediction, the sun azimuth and the sun elevation are calculated to obtain the light direction vector in the original gray image, and based on the light direction vector, the first center point coordinate and the second center point coordinate, each type of shadow area and each type of highlight area are matched to obtain a plurality of matching groups and the circumscribed rectangle of the image area corresponding to each matching group, wherein the matching group is one-to-one corresponding to the concave obstacle;

[0050] Specifically, each highlight area and shadow area of the original gray image has been segmented from the graphic background by threshold segmentation and K-means clustering, and then rough matching needs to be performed on the two types of areas to make the shadow area and the highlight area one-to-one. By the first center point coordinate of the shadow area and the second center point coordinate of the highlight area, the shadow area and the highlight area are one-to-one matched to obtain a plurality of matching groups, the top point coordinates of the four directions of each pair of matching groups, i.e. the upper and lower points and the left and right points, are determined, and the circumscribed rectangle of each pair of matching light and dark areas is determined by the vertical coordinates of the upper and lower points and the horizontal coordinates of the left and right points, i.e. the circumscribed rectangle of each matching group (single concave obstacle) is obtained.

[0051] Step S104, according to the circumscribed rectangle of the image area corresponding to each matching group, a target sub-image containing a single concave obstacle is determined from the original gray image, edge extraction is performed on the target sub-image to obtain target edge information, and ellipse fitting is performed according to the target edge information to determine the information of the concave obstacle in the target sub-image.

[0052] Specifically, according to the obtained circumscribed rectangle of the image area corresponding to each matching group, a target sub-image containing a single concave obstacle is determined from the original gray image, edge extraction is performed on the target sub-image by an edge detection algorithm to obtain target edge information, and finally the precise contour information of the concave obstacle in the target sub-image is fitted according to the target edge information.

[0053] In summary, the adaptive double threshold based on the maximum inter-class variance method can automatically segment the shadow area and the highlight area of the concave obstacle, and after clustering analysis of the shadow area and the highlight area, the shadow area and the highlight area are one-to-one matched to identify a single concave obstacle. Then, edge extraction and fitting are performed on the sub-image of the original gray image containing a single concave obstacle to reduce the interference of processing multiple concave obstacles at the same time, all sub-images are traversed, and finally the identification and detection of all concave obstacles are completed. The above method is simple and intuitive, has high real-time performance and is easy to implement, which can make up for the error problem caused by sparse data points in the three-dimensional detection method, improve the efficient reliability of the detection of the lunar concave obstacle, and provide effective obstacle information for the lunar rover.

[0054] How to calculate the first target gray threshold and the second target gray threshold by the maximum inter-class variance method is crucial. In the concave obstacle identification method provided in the embodiments of the present application, the following steps are adopted for processing: the total number of pixel points contained in the original gray image and the number of pixel points contained in each gray value are calculated, and the probability value of each gray value in the original gray image is calculated according to the total number and the number of pixel points contained in each gray value; the first initial gray threshold and the second initial gray threshold are set, and the pixel points in the original gray image are divided into the first category pixel points, the second category pixel points and the third category pixel points according to the first initial gray threshold and the second initial gray threshold; the gray value probability sum, the average gray value and the gray variance value of each category pixel point are calculated according to the probability value of each gray value; the global average gray value is calculated by calculating the gray values in the original gray image, and the global gray variance value is calculated according to the global average gray value, and the inter-class variance equation is constructed according to the gray value probability sum, the gray variance value and the global gray variance value; the gray threshold corresponding to the maximum value of the inter-class variance equation is obtained by solving the inter-class variance equation, and the gray threshold is taken as the first target gray threshold and the second target gray threshold.

[0055] If the gray value corresponding to the pixel point in the original gray image is less than or equal to the first initial gray threshold, the pixel point is determined to be the first category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the first initial gray threshold and less than or equal to the second initial gray threshold, the pixel point is determined to be the second category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the second initial gray threshold, the pixel point is determined to be the third category pixel point.

[0056] Specifically, the adaptive double-threshold segmentation based on the maximum inter-class variance method extracts the shadow and highlight regions of the concave obstacle from the background, which includes:

[0057] The probability of the pixel point with the gray value i in the original gray image is calculated as pi (i.e., the above-mentioned probability value of each gray value in the original gray image):

[0058]

[0059] wherein the gray value range is [0, 1, …, k-1], the number of pixel points with the gray value i is ni, and the total number of pixel points is N.

[0060] Two initial gray thresholds T1 and T2 (0 < T1 < T2 < k-1) (corresponding to the first initial gray threshold and the second initial gray threshold) are set, and the pixel points in the original gray image are divided into three categories according to T1 and T2.

[0061] The pixels with the gray value of [0, 1,..., T1] are classified into a class, denoted as D0 (i.e. the first category of pixel points described above); the pixels with the gray value of [T1+1,..., T2] are classified into a class, denoted as D1 (i.e. the second category of pixel points described above); and the pixels with the gray value of [T2+1,..., k-1] are classified into a class, denoted as D2 (i.e. the third category of pixel points described above).

[0062] The sum of the gray value probabilities of the three classes D0, D1 and D2 is:

[0063]

[0064]

[0065]

[0066] The average gray value of the three classes D0, D1 and D2 is:

[0067]

[0068]

[0069]

[0070] The gray variance value of the three classes D0, D1 and D2 is:

[0071]

[0072]

[0073]

[0074] The global average gray value in the original gray image is:

[0075]

[0076] The equation of the initial inter-class variance obtained through the above calculation process is:

[0077]

[0078] In order to improve the accuracy of calculating the gray threshold value, the global gray variance value is used to replace the global average gray value described above, and the calculation formula of the global gray variance value is as follows:

[0079]

[0080] Based on the global gray variance value described above, the inter-class variance equation is constructed:

[0081]

[0082] The maximum inter-class variance method is to obtain the optimal double threshold value when the above formula reaches the maximum value:

[0083]

[0084] In summary, the target gray threshold value can be accurately calculated by the maximum inter-class variance method, and the original gray image can be accurately segmented, that is, the shadow area and the highlight area are accurately segmented from the background gray.

[0085] In the concave obstacle identification method provided in the embodiments of the present application, the original gray image to be identified is segmented by the first target gray threshold value and the second target gray threshold value to obtain a shadow area image and a highlight area image, including: dividing the pixel points of the original gray image according to the first target gray threshold value and the second target gray threshold value to obtain fourth category pixel points, fifth category pixel points and sixth category pixel points, wherein the area corresponding to the fourth category pixel points is the shadow area, the area corresponding to the fifth category pixel points is the background area, and the area corresponding to the sixth category pixel points is the highlight area; based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points, the original gray image is segmented to obtain the shadow area image and the highlight area image.

[0086] Specifically, the pixel points of the original gray image are divided by the first target gray threshold value and the second target gray threshold value to obtain fourth category pixel points, fifth category pixel points and sixth category pixel points, and the fourth category pixel points, the fifth category pixel points and the sixth category pixel points correspond to the shadow area, the background area and the highlight area respectively, and the shadow area and the highlight area are segmented from the background area. In an optional embodiment, the gray values of the pixel points in the fourth category pixel points, the fifth category pixel points and the sixth category pixel points are gray three-valued, and the gray values of the three types of pixel points are set to 0, 127 and 255 respectively, corresponding to the shadow area, the background area and the highlight area.

[0087] After the shadow area and the highlight area are segmented from the background area, the pixel points in the shadow area image and the highlight area image are clustered according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different categories of shadow areas, a first center point coordinate of each category of shadow area, a plurality of different categories of highlight areas and a second center point coordinate of each category of highlight area. Before that, the shadow area image and the highlight area image need to be processed, mainly including removing the noise points of the shadow area image and the highlight area image by the image erosion method, and filling the gaps of the shadow area image and the highlight area image by the image expansion method.

[0088] For the shadow area image and the highlight area image segmented by the target gray threshold, some impurities and small shadow areas and highlight areas are contained, which can be removed by a morphological image erosion method. Meanwhile, some gaps exist in the shadow and highlight areas, which can be filled by an image dilation method.

[0089] After the shadow area image and the highlight area image are processed, the K-means clustering algorithm is used according to the pixel distribution distance to aggregate a single shadow area and a single highlight area. The basic idea of the algorithm is to use the pixel coordinates of the shadow area or the highlight area as the similarity measurement for clustering analysis, the clustering criterion is the Euclidean distance between the pixel points, and the pixel points belonging to the same class are aggregated by finding the minimum Euclidean distance.

[0090] Supposing I(x) is the pixel coordinates of the shadow or highlight area, the target function of the clustering is:

[0091]

[0092] An iterative algorithm is used in the clustering process to make the target function reach a minimum value, and the pixel coordinate set of the target area is divided into K classes, that is, K shadow areas or highlight areas are contained, wherein c(j) is the mean value of the gray scale of the pixel coordinates in the jth class. Finally, a plurality of different classes of shadow areas, a plurality of different classes of highlight areas, a center point of each class of shadow area, and a center point of each different class of highlight area are obtained.

[0093] In order to better match each class of shadow area and each class of highlight area, in the method for identifying a concave obstacle provided in the embodiment of the application, the sun azimuth and the sun elevation are calculated according to the ephemeris prediction to obtain a light direction vector in the original gray scale image, and the light direction vector, the first center point coordinate and the second center point coordinate are used to match each class of shadow area and each class of highlight area to obtain a plurality of matching groups and an outer rectangle of an image area corresponding to each matching group, including: calculating a position direction vector of a line between the first center point coordinate and the second center point coordinate; constructing an angle factor based on the light direction vector and the position direction vector; calculating a length value of each class of shadow area and each class of highlight area in the light direction, and constructing a distance factor according to the length value; matching each class of shadow area and each class of highlight area according to the angle factor and the distance factor to obtain a plurality of matching groups and an outer rectangle of an image area corresponding to each matching group.

[0094] Specifically, each highlight area and each shadow area of the original gray scale image has been segmented from the graphic background by threshold segmentation and clustering, and then rough matching needs to be performed on the two types of areas so that the shadow areas and the highlight areas correspond to each other.

[0095] Based on the ephemeris forecast, the predicted solar azimuth and altitude angles are calculated to obtain the solar azimuth angle during the imaging of the lunar digital orthophoto, thus yielding the illumination direction vector in the original grayscale image:

[0096] S=(S x S y )

[0097] Among them, S x S y Let S be the component of S in the coordinate system of the original grayscale image, where |S| = 1.

[0098] like Figure 2 As shown, let the position direction vector of the line connecting the center point coordinates of the shadow region and the center point coordinates of the highlight region obtained from the previous clustering step (corresponding to the position direction vector of the line connecting the first and second center point coordinates calculated above) be:

[0099] C=(x l -x d y l -y d )

[0100] Among them, C d (x d y d C l (x l y l ) are the center point coordinates of the shadow area and the highlight area, respectively.

[0101] Calculate the angle between the direction vector of the line connecting the center coordinates of the shadow and highlight regions to be matched and the direction vector of the illumination:

[0102]

[0103] Where, θ s The angle value ranges from [0, 180]. The direction of the line connecting the centers of the correctly matched shadow and highlight areas should be consistent with the direction of the illumination, i.e., θ s It should be close to 0, therefore, construct the angle factor α:

[0104]

[0105] That is, the larger the angle factor α is, the closer it is to 1, and the higher the degree of matching with the direction of illumination.

[0106] To constrain the distance between the shadow and highlight regions participating in the matching to be too far and the size difference between them to be too large, the length values ​​of each type of shadow region and each type of highlight region in the illumination direction are calculated, and a distance factor δ is constructed based on the length values:

[0107]

[0108] Among them, L l L d L represents the length of the highlight area and the shadow area in the direction of illumination, respectively. max L is the longer of the two. min The shorter length is d = |C|, where d = |C| is the center distance between the two. Figure 2 As shown, the larger the distance factor δ is, the closer it is to 1, and the closer the distance and size of the matching regions are.

[0109] Finally, based on the aforementioned angle and distance factors, each type of shadow region and each type of highlight region are matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group.

[0110] In the method for recognizing concave obstacles provided in this application embodiment, matching each type of shadow region and each type of highlight region based on angle factors and distance factors to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group includes: calculating the matching score between the shadow region to be matched and each type of highlight region based on angle factors and distance factors to obtain multiple first matching score values; calculating the matching score between the highlight region to be matched and each type of shadow region based on angle factors and distance factors to obtain multiple second matching score values; matching each type of shadow region and each type of highlight region based on the first matching score values ​​and the second matching score values ​​to obtain multiple matching groups; calculating the vertex coordinates of the target direction of each matching group, and determining the bounding rectangle of the image region corresponding to each matching group based on the vertex coordinates.

[0111] In the specific pairing process, let's assume that the previous clustering step yielded n shaded regions D1, D2...D n m highlighted regions L1, L2...L m Where m and n are natural numbers. The cluster center of the k-th shaded region among n shaded regions. and length in the direction of illumination And the cluster center of the j-th highlighted region among the m highlighted regions. and length by Centered on the center of each highlighted area Connect the points to obtain m center distances d. j，k Calculate the scores of m matches using the following formula: SC = α·δ, and select the pair of matches with the largest SC value as the shaded area D. kThe final matching is obtained by taking the intersection of the two sets of matching results. It should be noted that the union of the two sets of matching results can also be selected as the final matching result according to actual needs.

[0112] For each pair of matching groups, the highlight region and the shadow region are merged. The top, bottom, left and right vertex coordinates of each pair of matching light and dark regions are determined, and the circumscribed rectangle of each pair of matching light and dark regions is determined by the vertical coordinates of the top and bottom points and the horizontal coordinates of the left and right points, that is, the circumscribed rectangle of a single concave obstacle is obtained. At this point, the concave obstacles in the image have been preliminarily identified and detected, but more accurate contour information and position information are still needed for actual application, and further edge detection is required.

[0113] In the concave obstacle identification method provided in the embodiments of the present application, the target edge information is obtained by performing edge extraction on the target sub-image, including: performing edge extraction on the target sub-image by an edge detection algorithm to obtain initial edge information; removing the edges with a length less than a first preset threshold in the initial edge information to obtain processed initial edge information; removing the pseudo edges and non-arc edges in the processed initial edge information to obtain target edge information.

[0114] The gradient vector of the edge point in the processed initial edge information is calculated, and the light direction vector in the original gray image is obtained; the target included angle value of the gradient vector and the light direction vector is calculated; if the target included angle value is greater than a second preset threshold, the edge information corresponding to the target included angle value is determined as a pseudo edge.

[0115] According to the coordinates of the end points of the target edge in the processed initial edge information and the geometric center coordinates of the target edge, a target value is obtained; if the target value is less than a third preset threshold, the target edge is determined as a non-arc edge.

[0116] Specifically, the target sub-image of the original gray image in the circumscribed rectangle range obtained in the last step is obtained, thereby reducing the interference of multiple impact craters, and an edge detection algorithm (Canny operator) is used to extract edge information to obtain initial edge information.

[0117] For the obtained initial edge information, a large number of non-real obstacle edges are contained, which cannot be directly used for contour fitting and need to be screened and removed.

[0118] Firstly, the very short edges are removed, and the edges with a length less than a first preset threshold λ are removed. λ is generally set to about 5 pixels.

[0119] Then, the false edges are removed, and for the false edges caused by the boundary between the shadow area and the highlight area, the gray level decreases in the direction opposite to the illumination direction, while the real edge gray level decreases in the same direction as the illumination direction, as shown in Figure 3 .

[0120] According to the illumination direction, the gradient constraint of the real edge is obtained, that is, the angle between the gradient direction of the real edge and the illumination direction is an acute angle, and the angle between the gradient direction of the false edge and the illumination direction is an obtuse angle, so the angle between the gradient direction vector of the edge and the illumination direction vector is less than the first preset threshold θ b of the pixel point, and the false edge that does not meet the condition is removed. Generally, θ b may be set to 40 degrees:

[0121]

[0122] Wherein, G x , G y is the gradient value of the gray level g(x, y) of the edge point (x, y) in the x and y directions:

[0123]

[0124] Finally, the non-arc-shaped edge is removed. Since the edges of the meteorite craters are all arc-shaped, according to this feature, the arc-shaped edge that meets the condition is retained, and the non-arc-shaped edge that does not meet the condition is removed according to the following formula:

[0125]

[0126] Wherein, P1 and P2 are the coordinates of the two end pixel points of the edge, P c is the coordinate of the geometric center of the edge, and ε p is the coordinate deviation threshold (i.e. the third preset threshold), and the third preset threshold can be set to about 3 pixels.

[0127] After the initial edge information is screened and removed, the target edge information is obtained.

[0128] For the target edge information extracted in the above steps, it is a binary discrete data point and cannot achieve complete extraction of the concave obstacle contour or accurately locate the position range of the concave obstacle, so it needs to be fitted. Considering the actual appearance shape of the lunar impact crater, the obstacle contour is regarded as an elliptical curve for fitting, and an elliptical fitting algorithm based on Hough transformation is used for fitting of the edge information. The central idea is to transform the image space to the parameter space, and to find the peak value by using the statistical voting mechanism to determine the elliptical parameters. The advantage is that it has good inclusiveness for the discontinuous and missing feature edge information, and is less affected by image noise.

[0129] Therefore, the process of determining the information of concave obstacles in the target sub-image by performing ellipse fitting based on the target edge information includes: mapping the coordinate information in the target edge information to the ellipse parameter space to obtain multiple points in five-dimensional space; performing statistical voting on the multiple points in five-dimensional space to obtain the voting peak value of each point in five-dimensional space, wherein the multiple points in five-dimensional space correspond to different ellipse parameters; and taking the point in five-dimensional space corresponding to the voting peak value exceeding a fourth preset threshold as the target ellipse parameter; and determining the information of concave obstacles in the target sub-image based on the target ellipse parameter.

[0130] Specifically, for any ellipse, let the coordinates of the ellipse's center be (O... x O y The semi-major and minor axes are a and b respectively, and the angle between the major axis and the x-axis is the ellipse rotation angle θ. e The equation of a standard ellipse in a plane is expressed as:

[0131]

[0132] Therefore, determining the standard equation of an ellipse requires determining five parameters {O}. x O y a, b, θ e},like Figure 4 As shown.

[0133] Ellipse fitting based on Hough transform first involves extracting edge information and saving the coordinates of edge pixels as feature point coordinates into array H.

[0134] Then, the feature points in array H are mapped to the five-dimensional parameter space of the ellipse based on the Hough transform, resulting in multiple points in the five-dimensional space. Finally, the points {O} in the multiple five-dimensional spaces are mapped in the five-dimensional parameter space. x O y a, b, θ e A statistical voting process is performed, and the parameters corresponding to points in five-dimensional space whose peak values ​​exceed a fourth preset threshold (generally, the fourth preset threshold can be set around 0.8) are used as the target ellipse parameters. Based on the target ellipse parameters, the edge detection results of a single concave obstacle are determined. After extracting the complete contour of a single concave obstacle using ellipse fitting, all target sub-images containing a single concave obstacle are traversed, and finally, the identification and detection of all concave obstacles are completed.

[0135] In an alternative embodiment, the following can be employed: Figure 5The flowchart shown realizes the identification of the concave obstacle. First, the adaptive double threshold based on the maximum inter-class variance method can automatically segment the shadow and highlight regions of the concave obstacle. Then, clustering analysis is performed on the shadow and highlight regions to obtain a plurality of different classes of shadow regions, a first center point coordinate of each class of shadow region, a plurality of different classes of highlight regions, and a second center point coordinate of each class of highlight region. The shadow regions and the highlight regions are matched one by one according to the first center point coordinate and the second center point coordinate, a single concave obstacle is identified, edge extraction and fitting are performed on a target sub-image containing the single concave obstacle, the interference influence of simultaneously processing a plurality of obstacles is reduced, all target sub-images are traversed, and finally the identification and detection of all concave obstacles are completed.

[0136] In summary, the concave obstacle identification method proposed in the present application can be used to improve the accuracy of lunar surface concave obstacle identification and detection, and is particularly suitable for concave obstacle regions with obvious illumination characteristics. The concave obstacle identification method proposed in the present application is simple and intuitive, has high real-time performance, is easy to implement, can compensate for the error problem caused by sparse data points in the three-dimensional detection method, and has very important significance for improving the efficient reliability of lunar surface concave obstacle detection and providing effective obstacle information for the lunar surface inspection probe.

[0137] The method for identifying concave obstacles provided in the embodiments of the present application performs calculation on the original gray-scale image to be identified by the maximum inter-class variance method to obtain a first target gray-scale threshold and a second target gray-scale threshold, and performs segmentation on the original gray-scale image by the first target gray-scale threshold and the second target gray-scale threshold to obtain a shadow area image and a highlight area image, wherein the original gray-scale image includes a plurality of concave obstacles; the pixel points in the shadow area image and the highlight area image are respectively clustered according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas; the sun azimuth and the sun elevation are calculated according to ephemeris prediction to obtain a light direction vector in the original gray-scale image, and each class of shadow areas and each class of highlight areas are matched based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of the image area corresponding to each matching group, wherein the matching group is one-to-one corresponding to the concave obstacle; the target sub-image containing a single concave obstacle is determined from the original gray-scale image according to the external rectangle of the image area corresponding to each matching group, and edge extraction is performed on the target sub-image to obtain target edge information, and ellipse fitting is performed according to the target edge information to determine the information of the concave obstacle in the target sub-image, thereby solving the problem that in the related art, when the concave obstacle is identified based on three-dimensional imaging data points, the three-dimensional imaging data points are sparse, which leads to low accuracy of identifying the concave obstacle. The original gray-scale image is segmented by the first target gray-scale threshold and the second target gray-scale threshold to obtain the shadow area image and the highlight area image, the pixel points in the shadow area image and the highlight area image are respectively clustered according to the distance between the pixel points by the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas, then a single concave obstacle is identified by shadow and highlight area matching, and edge detection is performed on the sub-image of the original gray-scale image containing a single concave obstacle, thereby solving the defect that a plurality of obstacles interfere with each other during detection, affecting the detection effect, and finally realizing accurate detection of the position range of the concave obstacle area, and further achieving the effect of improving the accuracy of identifying the concave obstacle.

[0138] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0139] The embodiment of the present application further provides a concave obstacle identification device. It should be noted that the concave obstacle identification device of the embodiment of the present application can be used to execute the concave obstacle identification method provided by the embodiment of the present application. The concave obstacle identification device provided by the embodiment of the present application is introduced as follows.

[0140] Figure 6 is a schematic diagram of the concave obstacle identification device according to the embodiment of the present application. As shown in the figure, the device comprises a first calculation unit 601, a clustering unit 602, a matching unit 603 and a first determination unit 604. Figure 6

[0141] The first calculation unit 601 is configured to calculate a first target gray threshold value and a second target gray threshold value by using the maximum inter-class variance method on the original gray image to be identified, and to segment the original gray image by using the first target gray threshold value and the second target gray threshold value to obtain a shadow region image and a highlight region image, wherein the original gray image comprises a plurality of concave obstacles.

[0142] The clustering unit 602 is configured to cluster the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow regions, a first center point coordinate of each class of shadow region, a plurality of different classes of highlight regions and a second center point coordinate of each class of highlight region.

[0143] The matching unit 603 is configured to calculate the solar azimuth angle and the solar elevation angle according to the ephemeris prediction to obtain a light direction vector in the original gray image, and to match each class of shadow region and each class of highlight region based on the light direction vector, the first center point coordinate and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image region corresponding to each matching group, wherein the matching group is one-to-one corresponding to the concave obstacle.

[0144] The first determination unit 604 is configured to determine a target sub-image containing a single concave obstacle from the original gray image according to the external rectangle of the image region corresponding to each matching group, to perform edge extraction on the target sub-image to obtain target edge information, and to perform elliptical fitting according to the target edge information to determine the information of the concave obstacle in the target sub-image.

[0145] ​The identification device for concave obstacles provided in the embodiments of the present application, through the first computing unit 601, the original gray scale image to be identified is calculated by the maximum inter-class variance method to obtain a first target gray scale threshold and a second target gray scale threshold, and the original gray scale image is segmented by the first target gray scale threshold and the second target gray scale threshold to obtain a shadow area image and a highlight area image, wherein the original gray scale image includes a plurality of concave obstacles; the clustering unit 602 respectively clusters the pixel points in the shadow area image and the highlight area image according to the distance between the pixel points based on the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas; the matching unit 603 calculates the solar azimuth and the solar elevation angle according to the ephemeris prediction to obtain a light direction vector in the original gray scale image, and matches each class of shadow areas and each class of highlight areas based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image area corresponding to each matching group, wherein the matching group is one-to-one corresponding to the concave obstacle; the first determination unit 604 determines a target sub-image containing a single concave obstacle from the original gray scale image according to the external rectangle of the image area corresponding to each matching group, performs edge extraction on the target sub-image to obtain target edge information, and performs elliptical fitting according to the target edge information to determine the information of the concave obstacle in the target sub-image, thereby solving the problem in the related art that when the concave obstacle is identified based on three-dimensional imaging data points, the three-dimensional imaging data points are sparse, which easily leads to low accuracy of identifying the concave obstacle. The original gray scale image is segmented by the first target gray scale threshold and the second target gray scale threshold to obtain the shadow area image and the highlight area image, the pixel points in the shadow area image and the highlight area image are respectively clustered according to the distance between the pixel points by the K-means clustering algorithm to obtain a plurality of different classes of shadow areas, a first center point coordinate of each class of shadow areas, a plurality of different classes of highlight areas, and a second center point coordinate of each class of highlight areas, then a single concave obstacle is identified by matching the shadow and highlight areas, and the edge of the sub-image of the original gray scale image containing the single concave obstacle is detected, thereby solving the defect that a plurality of obstacles interfere with each other during detection, affecting the detection effect, and finally realizing accurate detection of the position range of the concave obstacle area, and further achieving the effect of improving the accuracy of identifying the concave obstacle.

[0146] Optionally, in the recess obstacle identification device provided by the embodiment of the present application, the first calculation unit 601 comprises: a first calculation module, configured to calculate the total number of pixel points contained in the original gray image and the number of pixel points contained in each gray value, and calculate the probability value of each gray value in the original gray image according to the total number and the number of pixel points contained in each gray value; a setting module, configured to set a first initial gray threshold and a second initial gray threshold, and divide the pixel points in the original gray image into first category pixel points, second category pixel points and third category pixel points according to the first initial gray threshold and the second initial gray threshold; a second calculation module, configured to calculate the gray value probability sum, the average gray value and the gray variance value of each category pixel point according to the probability value of each gray value; a third calculation module, configured to calculate the gray values in the original gray image to obtain a global average gray value, calculate a global gray variance value according to the global average gray value, and construct an inter-class variance equation according to the gray value probability sum, the gray variance value and the global gray variance value; and a solving module, configured to obtain the gray threshold corresponding to the maximum value of the inter-class variance equation by solving the inter-class variance equation, and take the gray threshold as the first target gray threshold and the second target gray threshold.

[0147] Optionally, in the recess obstacle identification device provided by the embodiment of the present application, the setting module comprises: a first determination submodule, configured to determine the pixel point as the first category pixel point if the gray value corresponding to the pixel point in the original gray image is less than or equal to the first initial gray threshold; a second determination submodule, configured to determine the pixel point as the second category pixel point if the gray value corresponding to the pixel point in the original gray image is greater than the first initial gray threshold and less than or equal to the second initial gray threshold; and a third determination submodule, configured to determine the pixel point as the third category pixel point if the gray value corresponding to the pixel point in the original gray image is greater than the second initial gray threshold.

[0148] Optionally, in the recess obstacle identification device provided by the embodiment of the present application, the first calculation unit 601 comprises: a division module, configured to divide the pixel points of the original gray image according to the first target gray threshold and the second target gray threshold to obtain fourth category pixel points, fifth category pixel points and sixth category pixel points, wherein the region corresponding to the fourth category pixel points is a shadow region, the region corresponding to the fifth category pixel points is a background region, and the region corresponding to the sixth category pixel points is a highlight region; and a segmentation module, configured to segment the original gray image based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points to obtain a shadow region image and a highlight region image.

[0149] Optionally, in the concave obstacle recognition device provided by the embodiment of the present application, the device further comprises a processing unit, configured to: before clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain the multiple different classes of shadow regions, the first center point coordinates of each class of shadow regions, the multiple different classes of highlight regions, and the second center point coordinates of each class of highlight regions, remove the miscellaneous points from the shadow region image and the highlight region image by using the image erosion device, and fill the gaps in the shadow region image and the highlight region image by using the image expansion device.

[0150] Optionally, in the concave obstacle recognition device provided by the embodiment of the present application, the matching unit 603 comprises: an acquisition module, configured to calculate a position direction vector of a line connecting the first center point coordinates and the second center point coordinates; a construction module, configured to construct an angle factor based on the light direction vector and the position direction vector; a fourth calculation module, configured to calculate the length value of each class of shadow regions and each class of highlight regions in the light direction, and construct a distance factor according to the length value; and a matching module, configured to match each class of shadow regions and each class of highlight regions according to the angle factor and the distance factor, to obtain the multiple matching groups and the circumscribed rectangle of the image region corresponding to each matching group.

[0151] Optionally, in the concave obstacle recognition device provided by the embodiment of the present application, the matching module comprises: a first calculation sub-module, configured to calculate the matching score of the shadow region to be matched and each class of highlight regions according to the angle factor and the distance factor, to obtain multiple first matching score values; a second calculation sub-module, configured to calculate the matching score of the highlight region to be matched and each class of shadow regions according to the angle factor and the distance factor, to obtain multiple second matching score values; a matching sub-module, configured to match each class of shadow regions and each class of highlight regions according to the first matching score values and the second matching score values, to obtain the multiple matching groups; and a third calculation sub-module, configured to calculate the vertex coordinates of the target direction of each matching group, and determine the circumscribed rectangle of the image region corresponding to each matching group according to the vertex coordinates.

[0152] Optionally, in the concave obstacle recognition device provided by the embodiment of the present application, the first determination unit 604 comprises: an extraction module, configured to perform edge extraction on the target sub-image by using an edge detection algorithm to obtain initial edge information; a first elimination module, configured to eliminate the edges with a length less than a first preset threshold in the initial edge information to obtain processed initial edge information; and a second elimination module, configured to eliminate the pseudo edges and the non-arc edges in the processed initial edge information to obtain target edge information.

[0153] Optionally, in the concave obstacle identification device provided by the embodiment of the present application, the device further comprises: a second calculation unit, configured to calculate a gradient vector of an edge point in the processed initial edge information and obtain a light direction vector in the original gray image before the pseudo edge and the non-arc edge in the processed initial edge information are removed to obtain the target edge information; a third calculation unit, configured to calculate a target included angle value between the gradient vector and the light direction vector; and a second determination unit, configured to determine that the edge information corresponding to the target included angle value is a pseudo edge if the target included angle value is greater than a second preset threshold value.

[0154] Optionally, in the concave obstacle identification device provided by the embodiment of the present application, the device further comprises: a fourth calculation unit, configured to calculate a target value according to the coordinates of an end point of a target edge in the processed initial edge information and the coordinates of a geometric center of the target edge before the pseudo edge and the non-arc edge in the processed initial edge information are removed to obtain the target edge information; and a third determination unit, configured to determine that the target edge is a non-arc edge if the target value is less than a third preset threshold value.

[0155] Optionally, in the concave obstacle identification device provided by the embodiment of the present application, the first determination unit 604 comprises: a solving module, configured to map the coordinate information in the target edge information to an elliptical parameter space to obtain a plurality of points in a five-dimensional space; a statistical module, configured to statistically vote the points in the plurality of five-dimensional spaces to obtain a voting peak value of each point in the five-dimensional space, wherein the points in the plurality of five-dimensional spaces correspond to different elliptical parameters; and the points in the five-dimensional space corresponding to the voting peak value exceeding a fourth preset threshold value are taken as target elliptical parameters; and a determination module, configured to determine the information of the concave obstacle in the target sub-image based on the target elliptical parameters.

[0156] The concave obstacle identification device comprises a processor and a memory, and the first calculation unit 601, the clustering unit 602, the matching unit 603 and the first determination unit 604 are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0157] The processor comprises a core, and the core retrieves the corresponding program units from the memory. The core can be set to one or more, and the core parameters are adjusted to realize the identification and detection of the concave obstacle.

[0158] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0159] This invention provides a processor for running a program, wherein the program executes a method for recognizing concave obstacles during runtime.

[0160] like Figure 7 As shown, this embodiment of the invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: calculating a first target grayscale threshold and a second target grayscale threshold using the maximum inter-class variance method; segmenting the original grayscale image using the first and second target grayscale thresholds to obtain a shadow region image and a highlight region image, wherein the original grayscale image includes multiple concave obstacles; and clustering the pixels in the shadow region image and the highlight region image according to the distance between pixels using the K-means clustering algorithm to obtain multiple different classes of shadow regions, the coordinates of the first center point of each class of shadow regions, and so on. Multiple different types of highlighted regions and the coordinates of the second center point of each type of highlighted region; the solar azimuth and solar altitude angles are calculated based on ephemeris forecasts to obtain the illumination direction vector in the original grayscale image; based on the illumination direction vector, the coordinates of the first center point, and the coordinates of the second center point, each type of shadow region and each type of highlighted region are matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group, wherein the matching group corresponds one-to-one with the concave obstacle; based on the bounding rectangle of the image region corresponding to each matching group, the target sub-image containing a single concave obstacle is determined from the original grayscale image, and the edge of the target sub-image is extracted to obtain the target edge information; and ellipse fitting is performed based on the target edge information to determine the information of the concave obstacle in the target sub-image.

[0161] Optionally, calculating the first target grayscale threshold and the second target grayscale threshold using the maximum inter-class variance method includes: calculating the total number of pixels in the original grayscale image and the number of pixels contained in each grayscale value, and calculating the probability value of each grayscale value appearing in the original grayscale image based on the total number and the number of pixels contained in each grayscale value; setting the first initial grayscale threshold and the second initial grayscale threshold, and classifying the pixels in the original grayscale image into first category pixels, second category pixels, and third category pixels based on the first initial grayscale threshold and the second initial grayscale threshold. For each grayscale pixel, based on the probability of each grayscale value, calculate the sum of grayscale probabilities, the average grayscale value, and the grayscale variance for each category of pixels. Calculate the global average grayscale value from the grayscale values ​​in the original grayscale image, and calculate the global grayscale variance based on the global average grayscale value. Construct an inter-class variance equation based on the sum of grayscale probabilities, the grayscale variance, and the global grayscale variance. Solve the inter-class variance equation to obtain the grayscale threshold corresponding to the maximum value of the inter-class variance equation, and use this grayscale threshold as the first target grayscale threshold and the second target grayscale threshold.

[0162] Optionally, the dividing the pixel points in the original gray image into the first category pixel points, the second category pixel points and the third category pixel points according to the first initial gray threshold and the second initial gray threshold comprises: if the gray value corresponding to the pixel point in the original gray image is less than or equal to the first initial gray threshold, determining the pixel point as the first category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the first initial gray threshold and less than or equal to the second initial gray threshold, determining the pixel point as the second category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the second initial gray threshold, determining the pixel point as the third category pixel point.

[0163] Optionally, the segmenting the original gray image to be recognized by the first target gray threshold and the second target gray threshold to obtain the shadow region image and the highlight region image comprises: dividing the pixel points in the original gray image according to the first target gray threshold and the second target gray threshold to obtain the fourth category pixel points, the fifth category pixel points and the sixth category pixel points, wherein the region corresponding to the fourth category pixel points is the shadow region, the region corresponding to the fifth category pixel points is the background region, and the region corresponding to the sixth category pixel points is the highlight region; segmenting the original gray image based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points to obtain the shadow region image and the highlight region image.

[0164] Optionally, before the clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain the multiple different categories of shadow regions, the first center point coordinates of each category of shadow regions, the multiple different categories of highlight regions and the second center point coordinates of each category of highlight regions, the method further comprises: removing the miscellaneous points in the shadow region image and the highlight region image by the image erosion method, and filling the gaps in the shadow region image and the highlight region image by the image dilation method.

[0165] Optionally, the calculating the sun azimuth and the sun altitude according to the ephemeris prediction to obtain the light direction vector in the original gray image, and matching each category of shadow regions and each category of highlight regions based on the light direction vector, the first center point coordinates and the second center point coordinates to obtain the multiple matching groups and the circumscribed rectangle of the image region corresponding to each matching group comprises: calculating the position direction vector of the line connecting the first center point coordinates and the second center point coordinates; constructing an angle factor based on the light direction vector and the position direction vector; calculating the length value of each category of shadow regions and each category of highlight regions in the light direction, and constructing a distance factor according to the length value; matching each category of shadow regions and each category of highlight regions according to the angle factor and the distance factor to obtain the multiple matching groups and the circumscribed rectangle of the image region corresponding to each matching group.

[0166] Optionally, the matching of each type of shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of matching groups and a circumscribed rectangle of an image region corresponding to each matching group comprises: calculating a matching score of the to-be-matched shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of first matching score values; calculating a matching score of the to-be-matched highlight region and each type of shadow region according to the angle factor and the distance factor to obtain a plurality of second matching score values; matching each type of shadow region and each type of highlight region according to the first matching score values and the second matching score values to obtain a plurality of matching groups; calculating a vertex coordinate of a target direction of each matching group, and determining a circumscribed rectangle of an image region corresponding to each matching group according to the vertex coordinate.

[0167] Optionally, the edge extraction on the target sub-image to obtain target edge information comprises: performing edge extraction on the target sub-image through an edge detection algorithm to obtain initial edge information; eliminating edge information with an edge length less than a first preset threshold in the initial edge information to obtain processed initial edge information; and eliminating pseudo edges and non-arc edges in the processed initial edge information to obtain the target edge information.

[0168] Optionally, before the elimination of the pseudo edges and the non-arc edges in the processed initial edge information to obtain the target edge information, the method further comprises: calculating a gradient vector of an edge point in the processed initial edge information, and obtaining a light direction vector in the original gray image; calculating a target included angle value of the gradient vector and the light direction vector; and if the target included angle value is greater than a second preset threshold, determining that edge information corresponding to the target included angle value is a pseudo edge.

[0169] Optionally, before the elimination of the pseudo edges and the non-arc edges in the processed initial edge information to obtain the target edge information, the method further comprises: calculating a target value according to a coordinate of an end point of a target edge in the processed initial edge information and a geometric center coordinate of the target edge; and if the target value is less than a third preset threshold, determining that the target edge is a non-arc edge.

[0170] Optionally, the elliptical fitting according to the target edge information to determine information of a concave obstacle in the target sub-image comprises: mapping coordinate information in the target edge information to an elliptical parameter space to obtain a plurality of points in a five-dimensional space; performing statistical voting on the plurality of points in the five-dimensional space to obtain a voting peak value of each point in the five-dimensional space, wherein the plurality of points in the five-dimensional space correspond to different elliptical parameters; and taking a point in the five-dimensional space corresponding to a voting peak value exceeding a fourth preset threshold as a target elliptical parameter; and determining information of a concave obstacle in the target sub-image based on the target elliptical parameter.

[0171] The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0172] The application further provides a computer program product, which is suitable for executing the program of the following method steps when executed on a data processing device: calculating a to-be-identified original gray image by a maximum inter-class variance method to obtain a first target gray threshold and a second target gray threshold, and segmenting the original gray image by the first target gray threshold and the second target gray threshold to obtain a shadow region image and a highlight region image, wherein the original gray image includes a plurality of concave obstacles; clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on a K-means clustering algorithm to obtain a plurality of different classes of shadow regions, a first center point coordinate of each class of shadow regions, a plurality of different classes of highlight regions, and a second center point coordinate of each class of highlight regions; calculating a solar azimuth angle and a solar altitude angle according to ephemeris prediction to obtain a light direction vector in the original gray image, and matching each class of shadow regions and each class of highlight regions based on the light direction vector, the first center point coordinate, and the second center point coordinate to obtain a plurality of matching groups and an external rectangle of an image region corresponding to each matching group, wherein the matching groups are one-to-one corresponding to the concave obstacles; determining a target sub-image containing a single concave obstacle from the original gray image according to the external rectangle of the image region corresponding to each matching group, performing edge extraction on the target sub-image to obtain target edge information, and performing elliptical fitting according to the target edge information to determine the information of the concave obstacle in the target sub-image.

[0173] Optionally, the calculation of the first target gray threshold and the second target gray threshold by the maximum inter-class variance method comprises: calculating the total number of pixel points contained in the original gray image and the number of pixel points contained in each gray value, and calculating a probability value of each gray value in the original gray image according to the total number and the number of pixel points contained in each gray value; setting a first initial gray threshold and a second initial gray threshold, and dividing the pixel points in the original gray image into first-class pixel points, second-class pixel points, and third-class pixel points according to the first initial gray threshold and the second initial gray threshold; calculating a gray value probability sum, an average gray value, and a gray variance value for each class of pixel points according to the probability value of each gray value; calculating the gray values in the original gray image to obtain a global average gray value, and calculating a global gray variance value according to the global average gray value, and constructing an inter-class variance equation according to the gray value probability sum, the gray variance value, and the global gray variance value; obtaining a gray threshold corresponding to the maximum value of the inter-class variance equation by solving the inter-class variance equation, and taking the gray threshold as the first target gray threshold and the second target gray threshold.

[0174] Optionally, the dividing the pixel points in the original gray image into the first category pixel points, the second category pixel points and the third category pixel points according to the first initial gray threshold and the second initial gray threshold comprises: if the gray value corresponding to the pixel point in the original gray image is less than or equal to the first initial gray threshold, determining the pixel point as the first category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the first initial gray threshold and less than or equal to the second initial gray threshold, determining the pixel point as the second category pixel point; if the gray value corresponding to the pixel point in the original gray image is greater than the second initial gray threshold, determining the pixel point as the third category pixel point.

[0175] Optionally, the segmenting the original gray image to be recognized by the first target gray threshold and the second target gray threshold to obtain the shadow region image and the highlight region image comprises: dividing the pixel points in the original gray image according to the first target gray threshold and the second target gray threshold to obtain the fourth category pixel points, the fifth category pixel points and the sixth category pixel points, wherein the region corresponding to the fourth category pixel points is the shadow region, the region corresponding to the fifth category pixel points is the background region, and the region corresponding to the sixth category pixel points is the highlight region; segmenting the original gray image based on the fourth category pixel points, the fifth category pixel points and the sixth category pixel points to obtain the shadow region image and the highlight region image.

[0176] Optionally, before the clustering the pixel points in the shadow region image and the highlight region image according to the distance between the pixel points based on the K-means clustering algorithm to obtain the multiple different categories of shadow regions, the first center point coordinates of each category of shadow regions, the multiple different categories of highlight regions and the second center point coordinates of each category of highlight regions, the method further comprises: removing the miscellaneous points in the shadow region image and the highlight region image by the image erosion method, and filling the gaps in the shadow region image and the highlight region image by the image dilation method.

[0177] Optionally, the calculating the sun azimuth and the sun elevation according to the ephemeris prediction to obtain the light direction vector in the original gray image, and matching each category of shadow regions and each category of highlight regions based on the light direction vector, the first center point coordinates and the second center point coordinates to obtain the multiple matching groups and the circumscribed rectangle of the image region corresponding to each matching group comprises: calculating the position direction vector of the line connecting the first center point coordinates and the second center point coordinates; constructing an angle factor based on the light direction vector and the position direction vector; calculating the length value of each category of shadow regions and each category of highlight regions in the light direction, and constructing a distance factor according to the length value; matching each category of shadow regions and each category of highlight regions according to the angle factor and the distance factor to obtain the multiple matching groups and the circumscribed rectangle of the image region corresponding to each matching group.

[0178] Optionally, the matching each type of shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of matching groups and a bounding rectangle of an image region corresponding to each matching group comprises: calculating a matching score of the to-be-matched shadow region and each type of highlight region according to the angle factor and the distance factor to obtain a plurality of first matching score values; calculating a matching score of the to-be-matched highlight region and each type of shadow region according to the angle factor and the distance factor to obtain a plurality of second matching score values; matching each type of shadow region and each type of highlight region according to the first matching score values and the second matching score values to obtain a plurality of matching groups; calculating a vertex coordinate of a target direction of each matching group, and determining a bounding rectangle of an image region corresponding to each matching group according to the vertex coordinate.

[0179] Optionally, the edge extraction on the target sub-image to obtain target edge information comprises: performing edge extraction on the target sub-image by an edge detection algorithm to obtain initial edge information; eliminating edge information with an edge length less than a first preset threshold in the initial edge information to obtain processed initial edge information; eliminating pseudo edges and non-arc edges in the processed initial edge information to obtain target edge information.

[0180] Optionally, before the eliminating pseudo edges and non-arc edges in the processed initial edge information to obtain target edge information, the method further comprises: calculating a gradient vector of an edge point in the processed initial edge information, and obtaining a light direction vector in the original gray image; calculating a target included angle value of the gradient vector and the light direction vector; if the target included angle value is greater than a second preset threshold, determining that edge information corresponding to the target included angle value is a pseudo edge.

[0181] Optionally, before the eliminating pseudo edges and non-arc edges in the processed initial edge information to obtain target edge information, the method further comprises: calculating a target value according to a coordinate of an end point of a target edge in the processed initial edge information and a geometric center coordinate of the target edge; if the target value is less than a third preset threshold, determining that the target edge is a non-arc edge.

[0182] Optionally, the elliptical fitting according to the target edge information to determine information of a concave obstacle in the target sub-image comprises: mapping coordinate information in the target edge information to an elliptical parameter space to obtain a plurality of points in a five-dimensional space; performing statistical voting on the plurality of points in the five-dimensional space to obtain a voting peak value of each point in the five-dimensional space, wherein the plurality of points in the five-dimensional space correspond to different elliptical parameters; and taking a point in the five-dimensional space corresponding to a voting peak value exceeding a fourth preset threshold as a target elliptical parameter; and determining information of a concave obstacle in the target sub-image based on the target elliptical parameter.

[0183] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0184] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0185] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0186] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0187] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0188] The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory, etc. in the form of a computer-readable medium, such as read only memory (ROM) or flash memory. The memory is an example of computer-readable media.

[0189] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0190] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0191] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0192] The above merely provides embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method of identifying a concave obstacle, characterized by, include: The original grayscale image to be identified is calculated using the maximum inter-class variance method to obtain a first target grayscale threshold and a second target grayscale threshold. The original grayscale image is then segmented using the first target grayscale threshold and the second target grayscale threshold to obtain a shadow region image and a highlight region image. The original grayscale image includes multiple concave obstacles. Based on the K-means clustering algorithm, the pixels in the shadow region image and the highlight region image are clustered according to the distance between the pixels, resulting in multiple different classes of shadow regions, the coordinates of the first center point of each class of shadow regions, multiple different classes of highlight regions, and the coordinates of the second center point of each class of highlight regions. The solar azimuth and solar altitude angles are calculated based on the ephemeris forecast to obtain the illumination direction vector in the original grayscale image. Based on the illumination direction vector, the coordinates of the first center point, and the coordinates of the second center point, each type of shadow region and each type of highlight region are matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group. The matching group corresponds one-to-one with the concave obstacle. Based on the bounding rectangle of the image region corresponding to each matching group, a target sub-image containing a single concave obstacle is determined from the original grayscale image, and edge extraction is performed on the target sub-image to obtain target edge information. Ellipse fitting is then performed based on the target edge information to determine the information of the concave obstacle in the target sub-image. Specifically, ellipse fitting based on the target edge information to determine the information of concave obstacles in the target sub-image includes: The coordinate information in the target edge information is mapped to the ellipse parameter space to obtain multiple points in five-dimensional space, wherein the multiple points in five-dimensional space correspond to different ellipse parameters; Statistical voting is performed on the points in the multiple five-dimensional spaces to obtain the peak voting value of each point in the five-dimensional space, and the point in the five-dimensional space corresponding to the peak voting value that exceeds the fourth preset threshold is used as the target ellipse parameter. Information about concave obstacles in the target sub-image is determined based on the target ellipse parameters.

2. The method of claim 1, wherein, The first and second target grayscale thresholds are calculated using the maximum inter-class variance method, including: Calculate the total number of pixels in the original grayscale image and the number of pixels in each grayscale value, and calculate the probability value of each grayscale value in the original grayscale image based on the total number and the number of pixels in each grayscale value. Set a first initial grayscale threshold and a second initial grayscale threshold, and divide the pixels in the original grayscale image into a first category of pixels, a second category of pixels, and a third category of pixels based on the first initial grayscale threshold and the second initial grayscale threshold; Based on the probability value of each gray value, calculate the gray value probability, average gray value, and gray value variance for each category of pixel. The gray values ​​in the original grayscale image are calculated to obtain the global average gray value, and the global gray variance value is calculated based on the global average gray value. Based on the sum of gray value probabilities, the gray variance value, and the global gray variance value, an inter-class variance equation is constructed. By solving the inter-class variance equation, the gray threshold corresponding to the maximum value of the inter-class variance equation is obtained, and the gray threshold is used as the first target gray threshold and the second target gray threshold.

3. The method of claim 2, wherein, The pixels in the original grayscale image are divided into a first category of pixels, a second category of pixels, and a third category of pixels based on the first initial grayscale threshold and the second initial grayscale threshold, including: If the gray value corresponding to a pixel in the original grayscale image is less than or equal to the first initial grayscale threshold, then the pixel is determined to be a pixel of the first category. If the gray value corresponding to a pixel in the original grayscale image is greater than the first initial grayscale threshold and less than or equal to the second initial grayscale threshold, then the pixel is determined to be a second category pixel. If the gray value corresponding to a pixel in the original grayscale image is greater than the second initial grayscale threshold, then the pixel is determined to be a third category pixel.

4. The method of claim 1, wherein, The original grayscale image to be identified is segmented using the first target grayscale threshold and the second target grayscale threshold to obtain a shadow region image and a highlight region image, including: The pixels of the original grayscale image are divided according to the first target grayscale threshold and the second target grayscale threshold to obtain a fourth category of pixels, a fifth category of pixels, and a sixth category of pixels. The area corresponding to the fourth category of pixels is the shadow area, the area corresponding to the fifth category of pixels is the background area, and the area corresponding to the sixth category of pixels is the highlight area. Based on the fourth category of pixels, the fifth category of pixels, and the sixth category of pixels, the original grayscale image is segmented to obtain the shadow region image and the highlight region image.

5. The method of claim 1, wherein, Before clustering the pixels in the shadow region image and the highlight region image according to the distance between pixels using the K-means clustering algorithm to obtain multiple different classes of shadow regions, the coordinates of the first center point of each class of shadow regions, multiple different classes of highlight regions, and the coordinates of the second center point of each class of highlight regions, the method further includes: The image erosion method is used to remove noise from the shadow area image and the image highlight area image, and the image dilation method is used to fill the gaps in the shadow area image and the image highlight area image.

6. The method of claim 1, wherein, The solar azimuth and solar altitude angles are calculated based on ephemeris forecasts to obtain the illumination direction vector in the original grayscale image. Based on the illumination direction vector, the coordinates of the first center point, and the coordinates of the second center point, each type of shadow region and each type of highlight region is matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group, including: Calculate the position and direction vector of the line connecting the coordinates of the first center point and the coordinates of the second center point; An angle factor is constructed based on the illumination direction vector and the position direction vector. Calculate the length of each type of shadow region and each type of highlight region in the direction of illumination, and construct a distance factor based on the length values; Based on the angle factor and the distance factor, each type of shadow region and each type of highlight region are matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group.

7. The method of claim 6, wherein, Based on the angle factor and the distance factor, each type of shadow region and each type of highlight region are matched to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group, including: Based on the angle factor and the distance factor, the matching score between the shadow region to be matched and each type of highlight region is calculated to obtain multiple first matching score values; Based on the angle factor and the distance factor, the matching score between the highlight area to be matched and each type of shadow area is calculated to obtain multiple second matching score values; Based on the first matching score and the second matching score, each type of shadow region and each type of highlight region are matched to obtain multiple matching groups; Calculate the vertex coordinates of the target direction for each matching group, and determine the bounding rectangle of the image region corresponding to each matching group based on the vertex coordinates.

8. The method of claim 1, wherein, Edge extraction is performed on the target sub-image to obtain target edge information, including: The target sub-image is edge-extracted using an edge detection algorithm to obtain initial edge information; Edges with a length less than a first preset threshold in the initial edge information are removed to obtain the processed initial edge information. The false edges and non-arc edges in the processed initial edge information are removed to obtain the target edge information.

9. The method of claim 8, wherein, Before removing false edges and non-arc edges from the processed initial edge information to obtain the target edge information, the method further includes: Calculate the gradient vector of the edge points in the processed initial edge information, and obtain the illumination direction vector in the original grayscale image; Calculate the target angle between the gradient vector and the illumination direction vector; If the target included angle value is greater than the second preset threshold, then the edge information corresponding to the target included angle value is determined to be the pseudo edge.

10. The method of claim 8, wherein, Before removing false edges and non-arc edges from the processed initial edge information to obtain the target edge information, the method further includes: The target value is obtained by calculating the coordinates of the endpoints of the target edge and the coordinates of the geometric center of the target edge in the processed initial edge information. If the target value is less than the third preset threshold, then the target edge is determined to be the non-arc edge.

11. A device for identifying a pothole, characterized in that include: The first calculation unit is used to calculate the original grayscale image to be identified using the maximum inter-class variance method to obtain a first target grayscale threshold and a second target grayscale threshold, and to segment the original grayscale image using the first target grayscale threshold and the second target grayscale threshold to obtain a shadow region image and a highlight region image, wherein the original grayscale image includes multiple concave obstacles. The clustering unit is used to cluster the pixels in the shadow region image and the highlight region image according to the distance between the pixels based on the K-means clustering algorithm, to obtain multiple different classes of shadow regions, the coordinates of the first center point of each class of shadow regions, multiple different classes of highlight regions, and the coordinates of the second center point of each class of highlight regions. The matching unit is used to calculate the solar azimuth angle and solar altitude angle based on the ephemeris forecast to obtain the illumination direction vector in the original grayscale image, and to match each type of shadow region and each type of highlight region based on the illumination direction vector, the coordinates of the first center point and the coordinates of the second center point to obtain multiple matching groups and the bounding rectangle of the image region corresponding to each matching group, wherein the matching group corresponds one-to-one with the concave obstacle; The first determining unit is configured to determine a target sub-image containing a single concave obstacle from the original grayscale image based on the bounding rectangle of the image region corresponding to each matching group, extract the edge of the target sub-image to obtain target edge information, and perform ellipse fitting based on the target edge information to determine the information of the concave obstacle in the target sub-image. The first determining unit includes: a solving module, used to map the coordinate information in the target edge information to an elliptical parameter space to obtain multiple points in a five-dimensional space; a statistics module, used to perform statistical voting on the multiple points in the five-dimensional space to obtain the voting peak value of each point in the five-dimensional space, wherein the multiple points in the five-dimensional space correspond to different elliptical parameters; and to take the point in the five-dimensional space corresponding to the voting peak value exceeding a fourth preset threshold as the target elliptical parameter; and a determining module, used to determine the information of the concave obstacle in the target sub-image based on the target elliptical parameter.

12. A processor, comprising: The processor is used to run a program, wherein the program executes the method for identifying concave obstacles according to any one of claims 1 to 10.

13. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for identifying concave obstacles as described in any one of claims 1 to 10.

Images