Road detection method based on multifunctional road detection vehicle

Abstract

The present invention relates to the field of image processing technology, and in particular to a road detection method based on a multifunctional road detection vehicle. By collecting multiple road surface images along a road position, obtaining a sub-stain area in each road surface image and performing image stitching, an overall stain area can be obtained. The overall stain area is compared with a shadow area (stain, tree shade) in each road surface image, and the stain part is removed to obtain a tree shade area in each road surface. In this way, the technical effect of automatically identifying road stains and roadside tree shades can be achieved. Then, the tree shade area is eliminated to obtain a complete road surface stain area. By calculating a road damage degree value for the complete road surface stain area, a road detection result can be generated through the road damage degree value. Since the adverse effects of the tree shade area on road detection are avoided, the detection result can be made more accurate, thereby improving the detection efficiency and the practicality of the multifunctional road detection vehicle.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a road detection method based on a multifunctional road detection vehicle. Background Art

[0002] With the acceleration of urbanization, urban road networks are expanding rapidly and traffic volume is increasing. In order to ensure the safety, comfort and life of roads, regular road inspection and maintenance must be carried out. Traditional manual inspection methods are often inefficient, inaccurate, and difficult to cover a wide range of road networks. Therefore, the development and application of multifunctional road inspection vehicles has become a demand for modern road management and maintenance. Through systematic and automated inspection, multifunctional road inspection vehicles can automatically complete the collection, analysis, classification and storage of road pavement images, road shape, road facility stereo images, flatness and road geometric parameters under normal vehicle driving conditions, providing effective data for the digital management of road ancillary facilities.

[0003] However, during the road inspection process, the road inspection vehicle cannot automatically identify road stains and roadside tree shadows, which is not conducive to the inspection of road conditions, resulting in reduced inspection efficiency and reduced equipment practicality. Summary of the invention

[0004] In order to solve the technical problem that the road detection vehicle cannot automatically identify road stains and roadside tree shades during road detection, which is not conducive to detecting road conditions, resulting in reduced detection efficiency and reduced equipment practicality, the purpose of the present invention is to provide a road detection method based on a multifunctional road detection vehicle, and the technical solution adopted is as follows:

[0005] The present invention provides a road detection method based on a multifunctional road detection vehicle, which is applied to the multifunctional road detection vehicle. The method comprises:

[0006] Collect multiple road surface images along the road location;

[0007] Obtain the sub-stain area in each road surface image;

[0008] Splicing multiple sub-stain areas to obtain the overall stain area;

[0009] Obtain the tree shade area in each road surface image according to the overall stain area, and eliminate the tree shade area;

[0010] The road surface images with multiple tree-shaded areas removed are stitched together to obtain a complete road stain area;

[0011] A road damage degree value is calculated according to the complete road surface stain area, and a road detection result is generated according to the road damage degree value.

[0012] Preferably, the step of obtaining the sub-stain area in each road surface image comprises:

[0013] Perform Harris corner point detection on each road surface image to obtain all corner points in each road surface image;

[0014] Perform threshold segmentation on each road surface image, and obtain abnormal road surface area and normal road surface area according to the threshold segmentation result;

[0015] Obtaining the shadow area in the road surface abnormality area according to preset conditions;

[0016] A road image is randomly selected as a target image, and any corner point in the shadow area of ​​the target image is obtained as a target corner point, and a left neighborhood and a right neighborhood of the target corner point are respectively constructed, wherein the left neighborhood and the right neighborhood respectively include the same number of first corner points and second corner points;

[0017] Obtain a first distance change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood;

[0018] Obtain a first angle change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood;

[0019] Obtain a second distance change value between each second corner point and an adjacent second corner point in the right neighborhood of the target corner point and a second angle change value between each second corner point and an adjacent second corner point;

[0020] Calculate a first change value of the target corner point according to the first distance change value and the first angle change value;

[0021] Calculate a second change value of the target corner point according to the second distance change value and the second angle change value;

[0022] Calculate the probability value of the target corner point belonging to the boundary corner point according to the first distance change degree value, the first angle change degree value, the second distance change degree value, and the second angle change degree value;

[0023] Return to the step of obtaining any corner point in the shadow area of ​​the target image as the target corner point, so as to traverse all corner points and obtain probability values ​​corresponding to all target corner points;

[0024] Sort all probability values ​​and take the two target corner points with the largest probability values ​​as the first fixed points;

[0025] Determine whether the first change value and the second change value corresponding to all target corner points are both less than a first preset value;

[0026] If there is at least one target corner point whose first change value and second change value are both smaller than the first preset value, the at least one target corner point whose value is smaller than the preset value is used as the second fixed point;

[0027] Draw a connecting line between the first fixed point and the second fixed point to obtain a sub-smudge area of ​​the target image;

[0028] Traverse all road images and obtain the sub-stain area of ​​each road image.

[0029] Preferably, the step of obtaining a first distance change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood includes:

[0030] Get the total number of corner points of all first corner points in the left neighborhood;

[0031] Get the distance between each first corner point and the adjacent first corner point in the left neighborhood;

[0032] The first distance change degree value is calculated according to the total number of corner points of the first corner point and the two distance values, wherein the calculation formula is:

[0033] ;

[0034] in, Indicates The first distance change value of the target corner point, Indicates The target corner point is in the left neighborhood The distance between a first corner point and an adjacent first corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the target corner point, Indicates The maximum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point. No. The minimum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point.

[0035] Preferably, the step of obtaining a first angle change value between each first corner point and an adjacent first corner point in the left neighborhood of the target corner point comprises:

[0036] Get the total number of corner points of all first corner points in the left neighborhood;

[0037] Obtain the slope of each first corner point and the adjacent first corner point in the left neighborhood, and calculate the average of the slopes of the first corner point and the adjacent first corner points based on the multiple slopes;

[0038] The first angle change value is calculated according to the total number of corner points and the average value of the first corner point, wherein the calculation formula is:

[0039] ;

[0040] in, Indicates The first angle change value of the target corner point, Indicates The corner point is in the left neighborhood The slope between a corner point and the adjacent first corner point; Indicates The average of the slopes of the first corner point and the adjacent first corner point in the left neighborhood of the target corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the corner points; represents the absolute value function.

[0041] Preferably, after the step of traversing all road surface images to obtain the sub-stain area of ​​each road surface image, the method further comprises:

[0042] Align all road images based on the ORB feature extraction algorithm;

[0043] Obtain the predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image;

[0044] The sub-stain areas of all road surface images are subjected to dilation and erosion processing, and the sub-stain areas and predicted areas are identified and merged based on the connected domain analysis algorithm to obtain the overall stain area.

[0045] Preferably, the step of obtaining the predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image comprises:

[0046] Obtaining the acquisition time of each road surface image, and obtaining the interval time between each road surface image and an adjacent road surface image according to the acquisition time;

[0047] Obtain the vehicle's driving speed, and calculate the vehicle's driving distance from the acquisition of each road surface image to the adjacent road surface image based on the vehicle's driving speed and the interval time;

[0048] The predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image is obtained according to the vehicle travel distance.

[0049] Preferably, before the step of dilating and corroding the sub-stain areas of all road surface images, the method further includes:

[0050] Obtaining the area and the first brightness mean of the sub-stain region of each road surface image;

[0051] Obtaining a normal road surface area in each road surface image, and obtaining a second brightness mean value of the normal road surface area;

[0052] The damage degree value of each sub-stain area is calculated according to the area of ​​the sub-stain area, the first brightness average value and the second brightness average value, wherein the calculation formula is:

[0053] ;

[0054] in, Indicates the number of the sub-smudge area, Indicates The damage degree value of each sub-stain area; Indicates The area of ​​the sub-stain area; Indicates The first brightness mean of the sub-stain areas; Indicates the second brightness mean value of the normal road area; Represents a linear normalization function that normalizes the data value to within the interval; represents the absolute value function.

[0055] Preferably, after the step of obtaining the overall stain area, the method further comprises:

[0056] Get the shadow area in each road surface image;

[0057] The overall stain area is compared with the shadow area to obtain the overlapping part of each road surface image;

[0058] The overlapping parts are removed to obtain the tree shade area of ​​each road image.

[0059] Preferably, after the step of removing the overlapping parts to obtain the tree shade area of ​​each road surface image, the method further comprises:

[0060] The shadow area is divided into sub-stain areas blocked by the tree shade area as the blocked area, sub-stain areas not blocked by the tree shade area as the unblocked area, and the shadow area other than the blocked stain area and the unblocked stain area as the sub-tree shade area;

[0061] Based on the image cloning technology, the pixels of the unobstructed stain area are copied to the obstructed stain area;

[0062] Pixels in the normal road area are acquired, and the pixels in the normal road area are copied to the sub-tree shadow area to eliminate the sub-tree shadow area.

[0063] Preferably, the step of calculating the road damage degree value according to the complete road stain area and generating the road detection result according to the road damage degree value comprises:

[0064] Obtain the damage degree value corresponding to each sub-stain area;

[0065] The total number of multiple sub-stain areas is obtained, and the road damage degree value of the complete road surface stain area is calculated according to the damage degree value and the total number, wherein the calculation formula is:

[0066] ;

[0067] in, The road damage value representing the intact road surface stain area; Indicates The damage degree value of each sub-stain area; represents the number of sub-smudge areas, Indicates the number of the sub-smudge area, Indicates The sub-stain area is in the sub-stain quantity interval The maximum value of the damage degree of all sub-stain areas within the Indicates The sub-stain area is in the sub-stain quantity interval The minimum value of the damage degree of all sub-stain areas within.

[0068] Determining whether the road damage degree value is greater than or equal to a second preset value;

[0069] If the road damage degree value is greater than or equal to a second preset value, the road detection result of the complete road surface stain area corresponding to the road damage degree value is determined as causing an impact;

[0070] If the road damage degree value is less than a second preset value, the road detection result of the complete road surface stain area corresponding to the road damage degree value is determined to have no impact.

[0071] The present invention has the following beneficial effects:

[0072] When the multifunctional road inspection vehicle is traveling on the road, it collects multiple road surface images along the road position, and then performs image processing on the road surface images to obtain sub-stain areas in each road surface image. By splicing the road surface images corresponding to the multiple sub-stain areas, the overall stain area can be obtained. Since the overall stain area and position are fixed on the road surface, the complete stain area can be compared with the shadow area (stains, tree shade) in each road surface image. The stain part is removed to obtain the tree shade area in each road surface. In this way, the technical effect of automatically identifying road stains and roadside tree shade can be achieved. The tree shade area is then eliminated, and the road surface images with multiple tree shade areas eliminated are spliced ​​to obtain the complete road stain area. By calculating the road damage degree value of the complete road stain area, the road inspection result can be generated according to the road damage degree value. Since the adverse effects of the tree shade area on road inspection are avoided, the inspection result can be made more accurate, thereby improving the inspection efficiency and the practicality of the multifunctional road inspection vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 A flowchart of a road detection method based on a multifunctional road detection vehicle provided by an embodiment of the present invention;

[0075] Figure 2 A flow chart of step S25 of a road detection method based on a multifunctional road detection vehicle provided by an embodiment of the present invention;

[0076] Figure 3 A flow chart of step S26 of a road detection method based on a multifunctional road detection vehicle provided by an embodiment of the present invention;

[0077] Figure 4 A flow chart of step S297 of a road detection method based on a multifunctional road detection vehicle provided by an embodiment of the present invention;

[0078] Figure 5 This is a structural diagram of the division of shaded areas in steps S297361-S297363 of a road detection method based on a multifunctional road detection vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0079] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a road detection method based on a multifunctional road detection vehicle proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0080] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0081] A specific solution of a road detection method based on a multifunctional road detection vehicle provided by the present invention is described in detail below in conjunction with the accompanying drawings.

[0082] See also Figure 1-5 The road detection method based on the multifunctional road detection vehicle provided by the present invention is applied to the multifunctional road detection vehicle, and the method comprises:

[0083] S1, collecting multiple road surface images along the road position and determining all corner points and shadow areas in each road surface image;

[0084] S2, taking any corner point located in the shadow area as the target corner point, obtaining the degree of change of the target corner point and calculating the probability value of the target corner point being the intersection corner point, traversing all corner points, selecting the two target corner points with the largest probability values ​​as the first fixed points, and determining the sub-stain area of ​​each road surface image according to the first fixed points and the degree of change;

[0085] S3, splicing the multiple road surface images according to the multiple sub-stain areas to obtain an overall stain area;

[0086] S4, obtaining the tree shade area in each road surface image according to the overall stain area, and eliminating the tree shade area;

[0087] S5, splicing the road surface images with multiple tree-shaded areas removed to obtain a complete road stain area;

[0088] S6. Calculate the road damage degree value according to the complete road stain area, and generate a road detection result according to the road damage degree value.

[0089] In the above step S1: Harris corner point detection is performed on each collected road surface image to obtain all corner points in each road surface image, wherein Harris corner point detection is a feature extraction method that can be implemented based on mathematical principles and simple image processing; threshold segmentation is performed on each road surface image, and the road surface abnormal area and the road surface normal area are obtained according to the threshold segmentation result; the shadow area in the road surface abnormal area is obtained according to the preset conditions. Specifically, the road surface image obtained by the multifunctional road inspection vehicle may contain abnormal areas other than tree shades and stains. These abnormal areas may include potholes, cracks, damaged road surfaces, etc., which are significantly different from the normal road surface area. The multifunctional road inspection vehicle can automatically detect and mark abnormal areas such as potholes, cracks, damaged road surfaces, etc., while road stains and roadside tree shades cannot be automatically identified by the multifunctional road inspection vehicle. Therefore, threshold segmentation is performed on several collected road surface images to obtain abnormal road surface areas and normal road surface areas in each road surface image. Specifically, the area with a threshold greater than or equal to 0.5 is recorded as the abnormal road surface area in the road surface image. The multifunctional road inspection vehicle can identify potholes, cracks, damaged road surfaces, etc. in the abnormal road surface area. Preset conditions are set to mark the areas in the abnormal area except potholes, cracks, damaged road surfaces, etc. as shadow areas. The shadow areas are composed of tree shades and stains.

[0090] In the above steps S2-S6: during the road detection process, although the shade of roadside trees will not have a direct impact on the road detection, road stains will have a significant impact on the detection results. The shape of road stains is relatively regular and not easy to change, while the shade of roadside trees sways with the breeze, and the shape and position of the shade will also change, overlapping with road stains. This causes the multifunctional road detection vehicle to be unable to automatically identify road stains and the shade of roadside trees, which is not conducive to the detection of road conditions, resulting in reduced detection efficiency and reduced practicality of the multifunctional road detection vehicle. Based on this, the multifunctional road inspection vehicle of the present invention travels on the road, and the high-resolution camera installed thereon collects a road surface image every second along the road position, and collects the images for half an hour to obtain multiple road surface images, and then performs image processing on the road surface images. Specifically, any corner point located in the shadow area is taken as the target corner point, and the degree of change of the target corner point is obtained, wherein the degree of change includes the degree of change of the distance and the degree of change of the angle between the target corner point and the adjacent corner point, and the probability value of the target corner point being the intersection corner point is calculated according to the degree of change of the distance and the degree of change of the angle, and all corner points are traversed, and the two target corner points with the largest probability values ​​are selected as the first fixed points, and the sub-stain area of ​​each road surface image is determined according to the first fixed point and the degree of change. In order to obtain the overall stain area of ​​the road surface passed by the multifunctional road inspection vehicle within half an hour, multiple The road surface images corresponding to the sub-stained areas are spliced ​​to obtain the overall stained area. Since the overall stained area and position are fixed on the road surface, the complete stained area can be compared with the shadow area (stains, tree shade) in each road surface image. The stained part is removed to obtain the tree shade area in each road surface. In this way, road stains and roadside tree shades can be automatically identified, and then the tree shade area is eliminated. The road surface images with multiple tree shade areas eliminated are spliced ​​to obtain the complete road stain area. By calculating the road damage degree value of the complete road stain area, the road detection result can be generated according to the road damage degree value. Since the adverse effects of the tree shade area on road detection are avoided, the detection result can be made more accurate, thereby improving the detection efficiency and the practicality of the multi-functional road detection vehicle.

[0091] In one embodiment, the step S2 of taking any corner point located in the shadow area as the target corner point, obtaining the degree of change of the target corner point and calculating the probability value of the target corner point being the intersection corner point, traversing all corner points, selecting two target corner points with the largest probability values ​​as the first fixed points, and determining the sub-stain area of ​​each road surface image according to the first fixed points and the degree of change includes:

[0092] S24, taking any corner point located in the shadow area as a target corner point, and constructing a left neighborhood and a right neighborhood of the target corner point, respectively, wherein the left neighborhood and the right neighborhood respectively include the same number of first corner points and second corner points;

[0093] S25, obtaining a first distance change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood;

[0094] S26, obtaining a first angle change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood;

[0095] S27, obtaining a second distance change value between each second corner point and an adjacent second corner point in the right neighborhood of the target corner point and a second angle change value between each second corner point and an adjacent second corner point;

[0096] S28, calculating a first change value of the target corner point according to the first distance change value and the first angle change value;

[0097] S29, calculating a second change value of the target corner point according to the second distance change value and the second angle change value;

[0098] S291, calculating a probability value of the target corner point being a boundary corner point according to the first distance change degree value, the first angle change degree value, the second distance change degree value, and the second angle change degree value;

[0099] S292, returning to the step of obtaining any corner point in the shadow area of ​​the target image as the target corner point, so as to traverse all corner points and obtain probability values ​​corresponding to all target corner points;

[0100] S293, sorting all probability values, and taking the two target corner points with the largest probability values ​​as the first fixed points;

[0101] S294, determining whether the first change value and the second change value corresponding to all target corner points are both smaller than a first preset value;

[0102] S295, if there is at least one target corner point corresponding to a first change value and a second change value that are both smaller than the first preset value, use the at least one target corner point corresponding to the value smaller than the preset value as the second fixed point;

[0103] S296, drawing a connecting line between the first fixed point and the second fixed point to obtain a sub-stain area of ​​the target image;

[0104] S297. According to the order of collecting road images, select the next road image adjacent to the target image as the target image, and return to the step of obtaining any corner point in the shadow area of ​​the target image as the target corner point, until all road images are traversed to obtain the sub-stain area of ​​each road image.

[0105] In the above steps S24-S296: for the shadow area in the road image, since the shade of the roadside trees swings with the breeze, the shape and swing direction of the shade are random, so there may be overlaps between the stained area and the shade area. Due to the natural swing and layer changes of leaves and tree crowns under wind or light changes, the edge of the shade area usually presents an irregular, complex and natural outline, and the corner points of the shade edge are usually irregular and unevenly distributed; due to the complexity of the shade edge, the distance between adjacent corner points may vary greatly, some areas may have dense corner points, while other areas are relatively sparse, and the angles between corner points vary greatly. The edges formed in the stained area are usually more linear or regular, less affected by external factors, the corner points at the edge of the stained area are usually more regular and stable, the corner points at the edge of the stained area are usually more regular and stable, the curvature of the stained area changes less, the corner points often appear at the edge corners, but the curvature changes tend to be stable, the spacing between the corner points at the edge of the stained area is relatively uniform, and the angles between the corner points change less. Based on this, a road surface image can be randomly selected or the first road surface image can be selected in the acquisition order as the target image, and then a corner point can be randomly selected in the target image as the target corner point, and the left neighborhood and the right neighborhood of the target corner point are respectively constructed. The left neighborhood and the right neighborhood contain the same number of corner points. According to the difference between the change degree of the distance of the target corner point in the left neighborhood and the change degree of the angle between the corner points, and the change degree of the distance of the corner point in the right neighborhood and the change degree of the angle between the corner points, it can be judged whether the target corner point belongs to the intersection corner point of the shade and stain area. Specifically, the probability value of the target corner point belonging to the intersection corner point can be calculated by the first distance change degree value, the first angle change degree value, the second distance change degree value, and the second angle change degree value. The calculation formula is:

[0106] ;

[0107] in, Indicates The probability that the target corner point belongs to the boundary corner point of the shade and stain area; Indicates The first distance change value in the left neighborhood of the target corner point, Indicates The first angle change value in the left neighborhood of the target corner point; Indicates The second distance change value in the right neighborhood of the target corner point; Indicates The second angle change value in the right neighborhood of the target corner point; Represents a linear normalization function that normalizes data values ​​to the interval [0,1]; represents the absolute value function.

[0108] In the above formula, The larger the value of is, the greater the change in distance is, and the greater the change in the angle between the corner points is. The more the corner points in the left neighborhood of the target corner point are, the more likely they are to belong to the shaded area. The smaller the value, the smaller the change in distance and the change in angle between corner points. The more likely the corner points on the right side of the target corner point belong to the stain area. The larger the value, the The greater the probability that the target corner point belongs to the intersection corner point of the shade and stain area.

[0109] By traversing all corner points in the target image in the above manner, the possibility of all corner points in the first target image belonging to the corner points at the intersection of the shade and stain areas can be calculated, all possibilities are sorted, and the two target corner points with the greatest possibility are screened out as the first fixed points, and the first distance change degree value is multiplied by the first angle change degree value to obtain the first change value of the target corner point, and the second distance change degree value is multiplied by the second angle change degree value to obtain the second change value of the target corner point. The target corner points whose first change value and second change value are both less than 0.5 are screened out as the second fixed points, and the first fixed point and the second fixed point are connected by a straight line to obtain the sub-stain area in the first target image.

[0110] In the above step S297: when the sub-stain area of ​​the first target image is confirmed, the next road image of the target image can be selected as the target image according to the acquisition order of the road images, and steps S24-S296 can be repeated to traverse all road images, thereby obtaining the sub-stain area of ​​each road image.

[0111] In one embodiment, the step S25 of obtaining the first distance change value between each first corner point of the target corner point and the adjacent first corner point in the left neighborhood includes:

[0112] S251, obtaining the total number of corner points of all first corner points in the left neighborhood;

[0113] S252, obtaining the distance value between each first corner point and the adjacent first corner point in the left neighborhood;

[0114] S253. Calculate a first distance change degree value according to the total number of corner points of the first corner point and the two distance values.

[0115] The calculation formula is:

[0116] ;

[0117] in, Indicates The first distance change value of the target corner point, Indicates The target corner point is in the left neighborhood The distance between a first corner point and an adjacent first corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the target corner point, Indicates The maximum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point. No. The minimum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point.

[0118] As described in the above steps S251-S253, the calculation method of the first distance change value of the target corner point in the left neighborhood is the same as the calculation method of the second distance change value of the right neighborhood, and the total number of the first corner points on the left is the same as the total number of the second intersection points on the right. Therefore, the distance change degree of the target corner point on the left and right sides can be calculated respectively by the above formula. Due to the complexity of the edge of the shade, the distance between adjacent corner points varies greatly. Some areas may have dense corner points, while other areas are relatively sparse. The edges formed in the smudge area are usually more linear or regular, and are less affected by external factors. The corner points at the edge of the smudge area are usually more regular and stable. Therefore, the spacing between the corner points at the edge of the smudge area is more uniform, and the distance change degree is smaller.

[0119] In one embodiment, the step S26 of obtaining the first angle change value between each first corner point and the adjacent first corner point in the left neighborhood of the target corner point includes:

[0120] S261, obtaining the total number of corner points of all first corner points in the left neighborhood;

[0121] S262, obtaining the slope of each first corner point and the adjacent first corner point in the left neighborhood, and calculating the average of the slopes of the first corner point and the adjacent first corner points according to the multiple slopes;

[0122] S263, calculating the first angle change value according to the total number of corner points and the average value of the first corner point. The calculation formula is:

[0123] ;

[0124] in, Indicates The first angle change value of the target corner point, Indicates The corner point is in the left neighborhood The slope between a corner point and the adjacent first corner point; Indicates The average of the slopes of the first corner point and the adjacent first corner point in the left neighborhood of the target corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the corner points; represents the absolute value function.

[0125] As described in the above steps S251-S253, the calculation method of the first angle change value of the target corner point in the left neighborhood is the same as the calculation method of the second angle change value of the right neighborhood, and the total number of the first corner points on the left is the same as the total number of the second intersection points on the right. Therefore, the angle change degrees of the target corner points on the left and right sides can be calculated respectively by the above formula. Due to the complexity of the edge of the shade, some areas may have dense corners, while other areas are relatively sparse, and the angle change degree between corner points is large. The edges formed in the stained area are usually more linear or regular, and are less affected by external factors. The corner points on the edge of the stained area are usually more regular and stable, and the curvature of the stained area changes less. Corner points often appear at the corners of the edge, but the curvature changes tend to be stable, and the angle change degree between corner points is small.

[0126] In one embodiment, after the step S297 of traversing all road surface images to obtain the sub-stain area of ​​each road surface image, the following steps are included:

[0127] S2971. Align all road surface images based on ORB feature extraction algorithm;

[0128] S2972, obtaining a predicted area of ​​a sub-stain area of ​​each road surface image in an adjacent road surface image;

[0129] S2973. Perform dilation and erosion processing on the sub-stain areas of all road surface images, and identify and merge the sub-stain areas and the predicted areas based on the connected domain analysis algorithm to obtain the overall stain area.

[0130] As described in steps S2971-S2973 above, in the actual road detection task, the shade of the roadside trees sways due to the breeze, which causes the shape and occlusion direction of the shade area to change. This randomness may cause the stain area to be partially covered, resulting in the inability to accurately detect some stain areas in the image. Therefore, in order to obtain the complete stain area, it is necessary to process the sub-stain area of ​​each road surface more accurately and obtain the overall stain area through splicing technology. Based on this, this embodiment first aligns the images using the ORB feature extraction algorithm. Since a random road image will have part of the sub-stain area in the previous road image during vehicle driving, it is necessary to obtain the predicted area of ​​the sub-stain area of ​​each road image in the adjacent road image, and use the dilation and erosion operations in morphological operations to process the spliced ​​sub-stain areas. The dilation operation can expand the boundary of the stain area and fill small holes; while the erosion operation can remove small noise points and tiny artifacts. Through these operations, the noise can be cleaned and the holes in the sub-stain area can be filled. The connected domain analysis is used to identify and merge the stain area and the predicted area. Through marking and analysis, it can be ensured that the sub-stain areas in different road images can be accurately spliced, and the sub-stain areas can be merged into a complete overall stain area.

[0131] In one embodiment, the step S2972 of obtaining the predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image includes:

[0132] S29721. Obtaining the acquisition time of each road surface image, and obtaining the interval time between each road surface image and an adjacent road surface image according to the acquisition time;

[0133] S29722, obtaining the vehicle's driving speed, and calculating the vehicle's driving distance from the acquisition of each road surface image to the acquisition of the adjacent road surface image according to the vehicle's driving speed and the interval time;

[0134] S29723. Obtain a predicted area of ​​a sub-stain area of ​​each road surface image in an adjacent road surface image according to the vehicle travel distance.

[0135] As described in the above steps S29721-S29723, since the position and shape of the stain area on the road surface are fixed, and the multifunctional road detection vehicle travels at a constant speed during the road detection process, the position of the stain in each image, i.e., the predicted area, can be obtained based on the distance traveled by the multifunctional road detection vehicle. Specifically, the vehicle travel distance from the acquisition of each road surface image to the adjacent road surface image can be calculated, where the calculation formula is:

[0136] ;

[0137] in, Indicates that the multifunctional road detection vehicle collects The distance traveled from one road image to the adjacent road image; Indicates the travel speed of the multifunctional road inspection vehicle; Indicates collection of The time from one road image to the adjacent road image.

[0138] By analogy, based on the distance traveled by the multifunctional road inspection vehicle from collecting the first road surface image to collecting the second road surface image, the predicted position of the sub-stain area of ​​the first road surface image in the second road surface image can be determined, and the sub-stain area in each road surface image can be obtained according to the above steps.

[0139] In one embodiment, before step S2973 of dilating and corroding the sub-stain areas of all road surface images, the method further includes:

[0140] S29731, obtaining the area and the first brightness average of the sub-stain region of each road surface image;

[0141] S29732, obtaining a normal road surface area in each road surface image, and obtaining a second brightness mean value of the normal road surface area;

[0142] S29733. Calculate the damage degree value of each sub-stain area according to the area of ​​the sub-stain area, the first brightness average value, and the second brightness average value, wherein the calculation formula is:

[0143] ;

[0144] in, Indicates the number of the sub-smudge area, Indicates The damage degree value of each sub-stain area; Indicates The area of ​​the sub-stain area; Indicates The first brightness mean of the sub-stain areas; Indicates the second brightness mean value of the normal road area; Represents a linear normalization function that normalizes the data value to within the interval; represents the absolute value function.

[0145] As described in steps S29731-S29733 above, before the sub-stain areas are spliced, the degree of damage of each sub-stain area can be calculated. The greater the degree of damage, the greater the impact of the spliced ​​stain area on road detection. Specifically, the area of ​​the sub-stain area directly affects its impact on the overall road surface. A larger sub-stain area usually indicates a more serious degree of damage. It is an important indicator to measure the size of the stain area. A larger area may mean a greater degree of damage. Indicates The brightness of the sub-stain area, and the second brightness mean of the normal road area is the average brightness of the normal road area, and the difference between the two It reflects the visual significance of the stained area. The greater the brightness difference, the stronger the significance of the stain, indicating that the degree of damage may be more serious. Stains usually cause changes in the brightness of the road surface, and this change is the key to judging the degree of damage. This product combines the area of ​​the sub-smudge region and the brightness difference. Difference in brightness The product of provides a comprehensive measure reflecting the severity of the sub-stain areas, with larger areas and more pronounced brightness differences generally indicating a higher degree of damage.

[0146] In one embodiment, after step S2973 of obtaining the entire stain area, the following steps are included:

[0147] S29734, obtaining a shadow area in each road surface image;

[0148] S29735, comparing the overall stain area with the shadow area to obtain an overlapping portion of each road surface image;

[0149] S29736. Remove the overlapping parts to obtain the tree shade area of ​​each road surface image.

[0150] As described in the above steps S29734-S29736, the obtained overall stain area is compared with the shadow area in each road surface image, and the overlapping parts are removed to obtain the tree shade area in each road surface image.

[0151] In one embodiment, after the step S29736 of removing the overlapping parts to obtain the tree shade area of ​​each road surface image, the following steps are included:

[0152] S297361, dividing the shadow area, taking the sub-stain area in the shadow area that is blocked by the tree shade area as the blocked area, the sub-stain area that is not blocked by the tree shade area as the unblocked area, and the shadow area other than the blocked stain area and the unblocked stain area as the sub-tree shade area;

[0153] S297362, copying pixels of the unblocked stain area to the blocked stain area based on the image cloning technology;

[0154] S297363. Obtain pixels of a normal road surface area, and copy the pixels of the normal road surface area to the sub-tree shadow area to eliminate the sub-tree shadow area.

[0155] As described in steps S297361-S297363 above, please refer to Figure 5 The shadow area includes the shade area and the sub-stain area. The sub-stain area in the shadow area that is blocked by the shade area is taken as the blocked area, the sub-stain area that is not blocked by the shade area is taken as the unblocked area, and the shadow area other than the blocked area and the unblocked area is taken as the sub-shade area. The image cloning technology is used to copy the pixels of the unblocked stain area in the shadow area to the shaded stain area to obtain a sub-shade area and sub-stain area with obvious contrast. The image cloning technology is used to copy the pixels of the normal area of ​​the road surface except the abnormal area in the road surface image to the sub-shade area to obtain a road surface image without the shade area. All road surface images are traversed according to the above steps to eliminate the shade areas in all road surface images.

[0156] In one embodiment, the step S6 of calculating the road damage degree value according to the complete road stain area and generating the road detection result according to the road damage degree value includes:

[0157] S61, obtaining a damage degree value corresponding to each stain sub-region;

[0158] S62, obtaining the total number of the plurality of sub-stain areas, and calculating the road damage degree value of the complete road surface stain area according to the damage degree value and the total number, wherein the calculation formula is:

[0159] ;

[0160] in, The road damage value representing the intact road surface stain area; Indicates The damage degree value of each sub-stain area; represents the number of sub-smudge areas, Indicates the number of the sub-smudge area, Indicates The sub-stain area is in the sub-stain quantity interval The maximum value of the damage degree of all sub-stain areas within the Indicates The sub-stain area is in the sub-stain quantity interval The minimum value of the damage degree of all sub-stain areas within.

[0161] S63, determining whether the road damage degree value is greater than or equal to a second preset value;

[0162] S64, if the road damage degree value is greater than or equal to a second preset value, determining the road detection result of the complete road surface stain area corresponding to the road damage degree value as causing an impact;

[0163] S65: If the road damage degree value is less than a second preset value, the road detection result of the complete road surface stain area corresponding to the road damage degree value is determined to have no impact.

[0164] As described in the above steps S61-S65, by comprehensively considering and analyzing the damage levels of multiple sub-stain areas, the road damage level of the complete road stain area can be evaluated as a whole, and whether the stain area affects road detection can be evaluated based on the road damage level value.

[0165] Specifically, The simple average of the damage degree of all sub-stain areas is calculated. By averaging the damage degree of all sub-stain areas, a representative value of the overall damage degree can be obtained. This step reflects the average damage of the entire stain area.

[0166] The ratio of the average damage degree to the range of the damage degree of the sub-stain area (i.e., the difference between the maximum and minimum values) is calculated. The range is used to measure the distribution range of the damage degree of the sub-stain area. The purpose of this step is to normalize the damage degree to a unified scale to avoid affecting the overall evaluation due to excessive differences in the damage degree of different areas.

[0167] By calculating the average value of the damage degree of the sub-stain area, the formula can reflect the overall damage of the entire stain area. Even if the damage degree of each sub-stain area is different, the average value provides a comprehensive evaluation. The purpose of normalization is to eliminate the influence of the absolute size of the damage degree value on the overall evaluation. By dividing the average damage degree by the range of the damage degree, the formula takes into account the relative distribution of the damage degree, so that the evaluation of the damage degree is not affected by extreme values. This treatment can more fairly reflect the relative damage of the entire stain area.

[0168] When the road damage degree value is greater than or equal to 0.5, it means that the complete road surface stain area affects the road detection, and the complete road surface stain area is marked as a quasi-stained area; when the road damage degree value is less than 0.5, it means that the complete road surface stain area does not affect the road detection, and the complete road surface stain area is marked as a normal area.

[0169] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0170] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A road detection method based on a multifunctional road detection vehicle, applied to a multifunctional road detection vehicle, characterized in that: The method comprises: Collect multiple road surface images along the road location and determine all corner points and shadow areas in each road surface image; Take any corner point in the shadow area as the target corner point, obtain the change degree of the target corner point and calculate the probability value of the target corner point being the intersection corner point, traverse all corner points, select the two target corner points with the largest probability value as the first fixed point, and determine the sub-stain area of ​​each road surface image according to the first fixed point and the change degree; Based on the multiple sub-stain areas, multiple road surface images are stitched together to obtain the overall stain area; Obtain the tree shade area in each road surface image according to the overall stain area, and eliminate the tree shade area; The road surface images with multiple tree-shaded areas removed are stitched together to obtain a complete road stain area; The road detection result is generated according to the complete road stain area; the steps of taking any corner point located in the shadow area as the target corner point, obtaining the degree of change of the target corner point and calculating the probability value of the target corner point being the intersection corner point, traversing all corner points, selecting two target corner points with the largest probability values ​​as the first fixed points, and determining the sub-stain area of ​​each road surface image according to the first fixed point and the degree of change include: Take any corner point in the shadow area as the target corner point, and construct the left neighborhood and the right neighborhood of the target corner point, respectively, wherein the left neighborhood and the right neighborhood include the same number of first corner points and second corner points respectively; Obtain a first distance change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood; Obtain a first angle change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood; Obtain a second distance change value between each second corner point and an adjacent second corner point in the right neighborhood of the target corner point and a second angle change value between each second corner point and an adjacent second corner point; Calculate a first change value of the target corner point according to the first distance change value and the first angle change value; Calculate a second change value of the target corner point according to the second distance change value and the second angle change value; Calculate the probability value of the target corner point belonging to the boundary corner point according to the first distance change degree value, the first angle change degree value, the second distance change degree value, and the second angle change degree value; Return to the step of obtaining any corner point in the shadow area of ​​the target image as the target corner point, so as to traverse all corner points and obtain probability values ​​corresponding to all target corner points; Sort all probability values ​​and take the two target corner points with the largest probability values ​​as the first fixed points; Determine whether the first change value and the second change value corresponding to all target corner points are both less than a first preset value; If there is at least one target corner point whose first change value and second change value are both smaller than the first preset value, the at least one target corner point whose value is smaller than the preset value is used as the second fixed point; A connecting line is drawn between the first fixed point and the second fixed point to obtain a sub-stain area of ​​the road surface image.

2. The road detection method based on the multifunctional road detection vehicle according to claim 1 is characterized in that: The step of obtaining a first distance change value between each first corner point of the target corner point and an adjacent first corner point in the left neighborhood includes: Get the total number of corner points of all first corner points in the left neighborhood; Get the distance between each first corner point and the adjacent first corner point in the left neighborhood; The first distance change degree value is calculated according to the total number of corner points of the first corner point and the two distance values, wherein the calculation formula is: ； in, Indicates The first distance change value of the target corner point, Indicates The target corner point is in the left neighborhood The distance between a first corner point and an adjacent first corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the target corner point, Indicates The maximum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point. No. The minimum value of the distance between all first corner points and adjacent first corner points in the left neighborhood of the target corner point.

3. The road detection method based on the multifunctional road detection vehicle according to claim 1 is characterized in that: The step of obtaining a first angle change value between each first corner point and an adjacent first corner point in a left neighborhood of the target corner point comprises: Get the total number of corner points of all first corner points in the left neighborhood; Obtain the slope of each first corner point and the adjacent first corner point in the left neighborhood, and calculate the average of the slopes of the first corner point and the adjacent first corner points based on the multiple slopes; The first angle change value is calculated according to the total number of corner points and the average value of the first corner point, wherein the calculation formula is: ； in, Indicates The first angle change value of the target corner point, Indicates The corner point is in the left neighborhood The slope between a corner point and the adjacent first corner point; Indicates The average of the slopes of the first corner point and the adjacent first corner point in the left neighborhood of the target corner point; Indicates The total number of corner points of all first corner points in the left neighborhood of the corner points; represents the absolute value function.

4. The road detection method based on the multifunctional road detection vehicle according to claim 1 is characterized in that: After the step of traversing all road surface images to obtain the sub-stain area of ​​each road surface image, the method further includes: Align all road images based on the ORB feature extraction algorithm; Obtain the predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image; The sub-stain areas of all road surface images are subjected to dilation and erosion processing, and the sub-stain areas and predicted areas are identified and merged based on the connected domain analysis algorithm to obtain the overall stain area.

5. The road detection method based on the multifunctional road detection vehicle according to claim 4 is characterized in that: The step of obtaining the predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image includes: Obtaining the acquisition time of each road surface image, and obtaining the interval time between each road surface image and an adjacent road surface image according to the acquisition time; Obtain the vehicle's driving speed, and calculate the vehicle's driving distance from the acquisition of each road surface image to the adjacent road surface image based on the vehicle's driving speed and the interval time; The predicted area of ​​the sub-stain area of ​​each road surface image in the adjacent road surface image is obtained according to the vehicle travel distance.

6. The road detection method based on the multifunctional road detection vehicle according to claim 4 is characterized in that: Before the step of dilating and corroding the sub-stained areas of all road surface images, the following steps are also included: Obtaining the area and the first brightness mean of the sub-stain region of each road surface image; Obtaining a normal road surface area in each road surface image, and obtaining a second brightness mean value of the normal road surface area; The damage degree value of each sub-stain area is calculated according to the area of ​​the sub-stain area, the first brightness average value and the second brightness average value, wherein the calculation formula is: ； in, Indicates the number of the sub-smudge area, Indicates The damage degree value of each sub-stain area; Indicates The area of ​​the sub-stain area; Indicates The first brightness mean of the sub-stain areas; Indicates the second brightness mean value of the normal road area; Represents a linear normalization function that normalizes the data value to within the interval; represents the absolute value function.

7. The road detection method based on a multifunctional road detection vehicle according to claim 4, characterized in that: After the steps of obtaining the overall stain area, include: Get the shadow area in each road surface image; The overall stain area is compared with the shadow area to obtain the overlapping part of each road surface image; The overlapping parts are removed to obtain the tree shade area of ​​each road image.

8. The road detection method based on the multifunctional road detection vehicle according to claim 7 is characterized in that: After the step of removing the overlapping parts to obtain the tree shade area of ​​each road surface image, the method includes: The shadow area is divided into sub-stain areas blocked by the tree shade area as the blocked area, sub-stain areas not blocked by the tree shade area as the unblocked area, and the shadow area other than the blocked stain area and the unblocked stain area as the sub-tree shade area; Based on the image cloning technology, the pixels of the unobstructed stain area are copied to the obstructed stain area; Pixels in the normal road area are acquired, and the pixels in the normal road area are copied to the sub-tree shadow area to eliminate the sub-tree shadow area.

9. The road detection method based on a multifunctional road detection vehicle according to claim 1, characterized in that: The step of calculating the road damage degree value according to the complete road stain area and generating the road detection result according to the road damage degree value includes: Obtain the damage degree value corresponding to each sub-stain area; The total number of multiple sub-stain areas is obtained, and the road damage degree value of the complete road surface stain area is calculated according to the damage degree value and the total number, wherein the calculation formula is: ； in, The road damage value representing the intact road surface stain area; Indicates The damage degree value of each sub-stain area; represents the number of sub-smudge areas, Indicates the number of the sub-smudge area, Indicates The sub-stain area is in the sub-stain quantity interval The maximum value of the damage degree of all sub-stain areas within the Indicates The sub-stain area is in the sub-stain quantity interval The minimum value of the damage degree of all sub-stain areas within the area; Determining whether the road damage degree value is greater than or equal to a second preset value; If the road damage degree value is greater than or equal to a second preset value, the road detection result of the complete road surface stain area corresponding to the road damage degree value is determined as causing an impact; If the road damage degree value is less than a second preset value, the road detection result of the complete road surface stain area corresponding to the road damage degree value is determined to have no impact.

Images