A method and system for detecting road pavement defects

Abstract

This invention provides a method and system for detecting road surface defects, relating to the field of image processing technology. The method includes: acquiring real-time road surface photographed images and real-time road surface infrared images; obtaining a first weight and a second weight through a weight allocation model; acquiring a first initial feature set corresponding to the real-time road surface photographed images and a second initial feature set corresponding to the real-time road surface infrared images; performing feature complementation on the first initial feature set and the second initial feature set to convert them into a first final feature set and a second final feature set, respectively; and obtaining a final image to be used through the first final feature set, the second final feature set, the first weight, and the second weight. Based on illumination features, the distribution of illumination and shadow is identified to form a combined weight between the real-time road surface photographed images and the real-time road surface infrared images, adaptively eliminating interference from external environmental factors, improving the accuracy of defect identification, and reducing the error rate.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for detecting road surface defects. Background Technology

[0002] Highways are public transportation routes used by cars and other vehicles. As an important carrier of people's lives and one of the important infrastructures for urban construction, they drive the economic development of the entire city, and their importance in urban construction and development has been increasingly recognized.

[0003] As highways age and are subjected to external forces, their quality gradually declines, leading to defects such as cracks. These defects severely impact highway performance and driving safety. Therefore, accurately and promptly identifying and repairing road surface defects is crucial for preventing highway accidents.

[0004] Currently, the detection of road surface defects generally involves acquiring images of the road surface, processing these images to obtain processed images, and then using these processed images to detect defects. However, external environmental factors such as low light, partial obstruction, and poor weather conditions can easily lead to interference in the subsequently acquired processed images, resulting in inaccurate identification of road surface defects and significant detection errors. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method and system for detecting road surface defects. The aim is to solve the technical problem that in the prior art, there are many external environmental factors that interfere with the road surface images during the shooting process, which easily leads to many interference factors in the subsequently acquired and processed road surface images, resulting in inaccurate identification of road surface defects and large detection errors.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide a method for detecting pavement defects, comprising the following steps:

[0007] Acquire real-time road surface images and real-time road surface infrared images, and obtain a first weight corresponding to the real-time road surface images and a second weight corresponding to the real-time road surface infrared images through a weight allocation model;

[0008] Multi-scale feature extraction is performed on the real-time road surface captured image and the real-time road surface infrared image based on several feature extraction layers to obtain a first initial feature set corresponding to the real-time road surface captured image and a second initial feature set corresponding to the real-time road surface infrared image. Feature complementation is performed on the first initial feature set and the second initial feature set to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps.

[0009] The final image to be used is obtained by using the first final feature set, the second final feature set, the first weight and the second weight. The final image to be used is then subjected to grayscale processing, binarization processing and edge detection in sequence to obtain several candidate disease contours in the final image to be used.

[0010] The candidate disease contours are false positives to separate them into several contours to be removed and the final disease contours. The contours to be removed are then removed to complete the disease detection.

[0011] Furthermore, real-time road surface images and real-time road surface infrared images are acquired, and a first weight corresponding to the real-time road surface images and a second weight corresponding to the real-time road surface infrared images are obtained through a weight allocation model.

[0012] Multi-scale feature extraction is performed on the real-time road surface captured image and the real-time road surface infrared image based on several feature extraction layers to obtain a first initial feature set corresponding to the real-time road surface captured image and a second initial feature set corresponding to the real-time road surface infrared image. Feature complementation is performed on the first initial feature set and the second initial feature set to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps.

[0013] The final image to be used is obtained by using the first final feature set, the second final feature set, the first weight and the second weight. The final image to be used is then subjected to grayscale processing, binarization processing and edge detection in sequence to obtain several candidate disease contours in the final image to be used.

[0014] The candidate disease contours are false positives to separate them into several contours to be removed and the final disease contours. The contours to be removed are then removed to complete the disease detection.

[0015] Furthermore, the formula for obtaining the first weight is:

[0016] ,

[0017] in, Indicates the first weight. Indicates clear components, Indicates interference components, This represents an exponential function with the natural constant e as its base.

[0018] Furthermore, the first initial feature set includes several initial captured feature maps, and the second initial feature set includes several initial infrared feature maps. The number of initial captured feature maps, initial infrared feature maps, and feature extraction layers corresponds. The step of performing feature complementation on the first initial feature set and the second initial feature set to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively, wherein the first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps, includes:

[0019] Based on the initial captured feature map and the initial infrared feature map, capture complementary feature values ​​and infrared complementary feature values ​​are obtained;

[0020] The average vector value of each channel in the captured complementary feature value is obtained to form a first channel vector, and the average vector value of each channel in the infrared complementary feature value is obtained to form a second channel vector.

[0021] The shooting attention weight is obtained through the first channel vector, and the infrared attention weight is obtained through the second channel vector;

[0022] The final image feature map is obtained by using the initial image feature map, the image attention weight, and the image complementary feature value; and the final infrared feature map is obtained by using the initial infrared feature map, the infrared attention weight, and the infrared complementary feature value.

[0023] The several final captured feature maps are combined into a first final feature set, and the several final infrared feature maps are combined into a second final feature set.

[0024] Furthermore, the formula for calculating the complementary feature values ​​is as follows:

[0025] ,

[0026] in, This represents the complementary feature value captured by the image corresponding to the i-th feature extraction layer. This represents the initial infrared feature map corresponding to the i-th feature extraction layer. This represents the initial captured feature map corresponding to the i-th feature extraction layer;

[0027] The formula for calculating the infrared complementary feature value is as follows:

[0028] ,

[0029] in, This represents the infrared complementary feature value corresponding to the i-th feature extraction layer;

[0030] The formula for obtaining the final captured feature map is as follows:

[0031] ,

[0032] in, This represents the final captured feature map corresponding to the i-th feature extraction layer. This represents the shooting attention weight corresponding to the i-th feature extraction layer. This indicates multiplication by channel.

[0033] Furthermore, the step of obtaining the final image to be used through the first final feature set, the second final feature set, the first weight, and the second weight includes:

[0034] Based on the final captured feature map, the first weight, the final infrared feature map, and the second weight, a feature map to be used is obtained;

[0035] Several of the proposed feature maps are upsampled and stitched together to obtain a multi-scale feature map, which is then converted into the final proposed image.

[0036] Furthermore, the formula for obtaining the feature map to be used is:

[0037] ,

[0038] in, This represents the feature map to be used corresponding to the i-th feature extraction layer. Indicates the first weight. This represents the final captured feature map corresponding to the i-th feature extraction layer. Indicates the second weight. This represents the final infrared feature map corresponding to the i-th feature extraction layer.

[0039] Furthermore, the step of identifying false positives in the candidate disease contours to separate the candidate disease contours into several deselected contours and the final disease contours includes:

[0040] Obtain a set of connected components corresponding to the candidate disease contour. The set of connected components includes several connected components. Dilate the connected components to obtain a set of dilated components. Remove the connected components from the set of dilated components to obtain a set of centered neighbor thresholds.

[0041] Based on the set of de-centered neighbor thresholds, a region to be determined is selected in the final image to be used, and the gray-level histogram of the region to be determined is obtained. The two gray-level values ​​with the largest peaks in the gray-level histogram are selected to obtain the first determination gray level and the second determination gray level.

[0042] Obtain the grayscale difference between the first and second determination grayscale values, compare the grayscale difference with a difference threshold, if the grayscale difference is greater than the difference threshold, then the candidate disease contour is determined as a contour to be removed, if the grayscale difference is less than the difference threshold, then the candidate disease contour is determined as the final disease contour.

[0043] Furthermore, the difference threshold is 25~45.

[0044] Secondly, embodiments of this application provide a highway pavement distress detection system, applied to the highway pavement distress detection method described in the first aspect above, the system comprising:

[0045] The analysis module is used to acquire real-time road surface images and real-time road surface infrared images, and to obtain a first weight corresponding to the real-time road surface images and a second weight corresponding to the real-time road surface infrared images through a weight allocation model.

[0046] The processing module is used to perform multi-scale feature extraction on the real-time road surface captured image and the real-time road surface infrared image based on several feature extraction layers, so as to obtain a first initial feature set corresponding to the real-time road surface captured image and a second initial feature set corresponding to the real-time road surface infrared image, and to perform feature complementation on the first initial feature set and the second initial feature set, so as to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps.

[0047] The recognition module is used to obtain a final image to be used through the first final feature set, the second final feature set, the first weight and the second weight, and to perform grayscale processing, binarization processing and edge detection on the final image to obtain a number of candidate lesion contours in the final image.

[0048] The verification module is used to identify false detections of the candidate disease contours, so as to separate the candidate disease contours into several contours to be removed and the final disease contours, and remove the contours to be removed to complete the disease detection.

[0049] Thirdly, embodiments of this application provide a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road surface distress detection method as described in the first aspect above.

[0050] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the road surface distress detection method as described in the first aspect above.

[0051] Compared with existing technologies, the beneficial effects of this invention are as follows: the real-time road surface image retains the texture details and morphology of road surface defects, while the real-time road surface infrared image, based on the characteristics of thermal radiation, can detect structural anomalies. The combination of the two forms a multi-dimensional expression of defect features, improving the completeness of information. Through the weight allocation model, based on the illumination characteristics of the real-time road surface image, the distribution of illumination and shadow is identified, thereby forming a combination weight between the real-time road surface image and the real-time road surface infrared image. This adaptively eliminates interference from external environmental factors, improves the accuracy of subsequent defect identification, and reduces identification errors. By performing feature complementation, some infrared structural anomaly features are incorporated into the texture details, while spatial details are incorporated into the structural anomaly features, enhancing the feature response of the defect area. Combined with the elimination of interference from external environmental factors, this further improves the accuracy of defect identification. After obtaining the final image to be used, by performing false detection identification, the interference of the candidate defect contours formed by water stains, road lines, etc., on the real defect contours is eliminated, reducing the probability of false defect detection. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method for detecting road surface defects in the first embodiment of the present invention;

[0053] Figure 2 This is a structural block diagram of the highway pavement defect detection system in the second embodiment of the present invention;

[0054] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0055] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0056] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0058] Please see Figure 1 The method for detecting road surface defects provided in the first embodiment of the present invention includes the following steps:

[0059] S10: Acquire real-time road surface captured images and real-time road surface infrared images, and obtain the first weight corresponding to the real-time road surface captured images and the second weight corresponding to the real-time road surface infrared images through a weight allocation model;

[0060] In this embodiment, the real-time road surface image and the real-time road surface infrared image are acquired through drone aerial photography. The weight allocation model includes a group of convolutional layers, pooling layers, and a group of fully connected layers connected in sequence. The convolutional layer group includes several convolutional kernels; in this embodiment, the convolutional layer group includes 64 convolutional kernels (3*3). The essence of the weight allocation model is to distinguish the illumination characteristics of the real-time road surface image and determine the clear and interfering areas in the real-time road surface image.

[0061] Step S10 includes:

[0062] S110: Input the real-time road surface image into the convolutional layer group to obtain several illumination feature maps;

[0063] After the real-time road surface image is entered into the convolutional layer group, all the convolutional kernels output illumination feature maps of different spatial dimensions. Both the real-time road surface image and the illumination feature maps of different spatial dimensions have width, height and number of channels.

[0064] S120: Input the illumination feature map into the pooling layer to extract the maximum dimension of each channel in the illumination feature map, and construct a global vector based on several of the maximum dimensions, wherein the dimension of the global vector corresponds to the number of channels in the illumination feature map;

[0065] If a certain illumination feature map has 256 channels, then the maximum value of the dimension of the 256 channels is extracted to form a 256-dimensional global vector.

[0066] S130: Input the global vector into the fully connected layer group to reduce the dimensionality of the global vector into clear components and interference components;

[0067] Understandably, the fully connected layer group includes several fully connected sub-layers. In this embodiment, the fully connected layer group includes two fully connected sub-layers. If the global vector is 256-dimensional, the fully connected sub-layer connected to the pooling layer is used to reduce the 256-dimensional global vector to 32-dimensional, and the other fully connected sub-layer is used to reduce the 32-dimensional global vector to 2-dimensional, thereby completing the differentiation of different illumination feature regions in the real-time road surface image.

[0068] S140: Determine a first weight corresponding to the real-time road surface image and a second weight corresponding to the real-time road surface infrared image based on the clear component and the interference component;

[0069] The formula for obtaining the first weight is:

[0070] ,

[0071] in, Indicates the first weight. Indicates clear components, Indicates interference components, This represents an exponential function with the natural constant e as its base. The formula for obtaining the second weight is the same as that for obtaining the first weight, except that the numerator is replaced; it will not be described again here.

[0072] It should be noted that, in constructing the weight allocation model, it can be equipped with the ability to distinguish weights by inputting past road images under various shooting conditions. The model training follows a fairly conventional method, which will not be elaborated upon here.

[0073] S20: Multi-scale feature extraction is performed on the real-time road surface image and the real-time road surface infrared image based on several feature extraction layers to obtain a first initial feature set corresponding to the real-time road surface image and a second initial feature set corresponding to the real-time road surface infrared image. Feature complementarity is performed on the first initial feature set and the second initial feature set to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps.

[0074] In this embodiment, multi-scale feature extraction is performed using a multi-scale feature extraction model. Understandably, the multi-scale feature extraction model includes several feature extraction layers. The training of the model is consistent with conventional training methods and will not be described in detail here.

[0075] The first initial feature set includes several initial captured feature maps, and the second initial feature set includes several initial infrared feature maps. The number of initial captured feature maps, initial infrared feature maps, and feature extraction layers corresponds. In this embodiment, the several feature extraction layers are used to acquire features of different spatial dimensions, realizing layer-by-layer downsampling of the real-time road surface captured image and the real-time road surface infrared image. The initial captured feature map and the initial infrared feature map acquired by the same feature extraction layer have the same spatial dimension.

[0076] Step S20 includes:

[0077] S210: Obtain complementary feature values ​​for shooting and complementary feature values ​​for infrared based on the initial shooting feature map and the initial infrared feature map;

[0078] The formula for calculating the complementary feature value of the image is:

[0079] ,

[0080] in, This represents the complementary feature value captured by the image corresponding to the i-th feature extraction layer. This represents the initial infrared feature map corresponding to the i-th feature extraction layer. This represents the initial captured feature map corresponding to the i-th feature extraction layer;

[0081] The formula for calculating the infrared complementary feature value is as follows:

[0082] ,

[0083] in, This represents the infrared complementary feature value corresponding to the i-th feature extraction layer. Since both the initial infrared feature map and the initial captured feature map have multiple channels, it can be understood that the captured complementary feature value and the infrared complementary feature value have the same number of channels as the former.

[0084] S220: Obtain the vector mean of each channel in the captured complementary feature values ​​to form a first channel vector, and obtain the vector mean of each channel in the infrared complementary feature values ​​to form a second channel vector;

[0085] Understandably, the number of dimensions of the first channel vector and the second channel vector is the same as the number of channels of the captured complementary feature value and the infrared complementary feature value.

[0086] S230: Obtain the shooting attention weight through the first channel vector, and obtain the infrared attention weight through the second channel vector;

[0087] In this embodiment, the first channel vector and the second channel vector are normalized by the Sigmoid function, and the shooting attention weight and the infrared attention weight are formed in the range of 0 to 1.

[0088] S240: Obtain the final shooting feature map through the initial shooting feature map, the shooting attention weight and the shooting complementary feature value, and obtain the final infrared feature map through the initial infrared feature map, the infrared attention weight and the infrared complementary feature value;

[0089] The formula for obtaining the final captured feature map is as follows:

[0090] ,

[0091] in, This represents the final captured feature map corresponding to the i-th feature extraction layer. This represents the shooting attention weight corresponding to the i-th feature extraction layer. This indicates channel-by-channel multiplication. The formula for obtaining the final infrared feature map is the same as the formula for obtaining the final captured feature map, and will not be repeated here.

[0092] S250: Combine a number of the final captured feature maps into a first final feature set, and combine a number of the final infrared feature maps into a second final feature set.

[0093] S30: Obtain the final image to be used through the first final feature set, the second final feature set, the first weight and the second weight, and perform grayscale processing, binarization processing and edge detection on the final image to obtain a number of candidate lesion contours in the final image to be used.

[0094] Step S30 includes:

[0095] S310: Obtain a feature map to be used based on the final captured feature map, the first weight, the final infrared feature map, and the second weight;

[0096] The formula for obtaining the feature map to be used is:

[0097] ,

[0098] in, This represents the feature map to be used corresponding to the i-th feature extraction layer. Indicates the first weight. This represents the final captured feature map corresponding to the i-th feature extraction layer. Indicates the second weight. This represents the final infrared feature map corresponding to the i-th feature extraction layer.

[0099] S320: Upsample several of the proposed feature maps and stitch them together to obtain a multi-scale feature map, and convert the multi-scale feature map into the final proposed image;

[0100] The essence of the transformation is to resample the multi-scale feature map to the same spatial dimension as the real-time road surface image and the real-time road surface infrared image, thereby restoring the image resolution. Contour recognition algorithms are now widely used, so grayscale processing, binarization processing, and edge detection will not be elaborated here.

[0101] S40: Perform false detection identification on the candidate disease contours to separate several candidate disease contours into several contours to be removed and the final disease contours, remove the contours to be removed, and complete the disease detection.

[0102] In this embodiment, the outline to be removed is an interference outline formed by water stains or road lines in the image. However, it differs from the final defect outline in that the two adjacent ends of the outline to be removed have a more obvious grayscale difference, while the two ends of the final defect outline are both road surfaces with a smaller grayscale difference.

[0103] Step S40 includes:

[0104] S410: Obtain a set of connected components corresponding to the candidate disease contour, the set of connected components includes several connected components, expand the connected components to obtain an expanded set of connected components, and remove the connected component set from the expanded set of connected components to obtain a set of centered neighbor thresholds.

[0105] The actual extent of the connected domain set is the enclosed region of the candidate disease contour.

[0106] S420: Based on the set of de-centered neighbor thresholds, select a region to be determined in the final image to be used, and obtain the gray-level histogram of the region to be determined. Select the two gray-level values ​​with the largest peaks in the gray-level histogram to obtain the first determination gray level and the second determination gray level.

[0107] S430: Obtain the grayscale difference between the first determination grayscale and the second determination grayscale, compare the grayscale difference with the difference threshold, if the grayscale difference is greater than the difference threshold, then determine the candidate disease contour as a contour to be removed, if the grayscale difference is less than the difference threshold, then determine the candidate disease contour as the final disease contour.

[0108] By eliminating interference from the disease itself, the grayscale differences at different locations in the area to be determined are analyzed, thereby eliminating interference items. Preferably, the difference threshold is 25~45; in this embodiment, the difference threshold is 30.

[0109] The real-time road surface image capture preserves the texture details and morphology of road surface defects, while the real-time road surface infrared image, based on the characteristics of thermal radiation, can detect structural anomalies. The combination of these two images forms a multi-dimensional representation of defects, enhancing the completeness of the information. Through the weight allocation model, based on the illumination characteristics of the real-time road surface image capture, the distribution of light and shadow is identified, thus forming a combined weight between the real-time road surface image capture and the real-time road surface infrared image. This adaptively eliminates interference from external environmental factors, improving the accuracy of subsequent defect identification and reducing identification errors. By performing feature complementation, some infrared structural anomaly features are incorporated into the texture details, while spatial details are incorporated into the structural anomaly features, enhancing the feature response of the defect area. Combined with the elimination of interference from external environmental factors, this further improves the accuracy of defect identification. After acquiring the final candidate image, the false detection identification process eliminates interference from candidate defect contours formed by water stains, lane lines, etc., reducing the probability of false defect detection.

[0110] Please see Figure 2 The second embodiment of the present invention provides a highway pavement distress detection system, which is applied to the highway pavement distress detection method described in the above embodiments, and will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] The system includes:

[0112] Analysis module 10 is used to acquire real-time road surface captured images and real-time road surface infrared images, and to acquire a first weight corresponding to the real-time road surface captured images and a second weight corresponding to the real-time road surface infrared images through a weight allocation model;

[0113] The analysis module 10 includes:

[0114] The first unit is used to input the real-time road surface image into the convolutional layer group to obtain several illumination feature maps.

[0115] The second unit is used to input the illumination feature map into the pooling layer to extract the maximum dimension of each channel in the illumination feature map, and to construct a global vector based on several of the maximum dimensions, wherein the dimension of the global vector corresponds to the number of channels in the illumination feature map.

[0116] The third unit is used to input the global vector into the fully connected layer group to reduce the dimensionality of the global vector into clear components and interference components;

[0117] The fourth unit is used to determine a first weight corresponding to the real-time road surface image and a second weight corresponding to the real-time road surface infrared image based on the clear component and the interference component.

[0118] Processing module 20 is configured to perform multi-scale feature extraction on the real-time road surface captured image and the real-time road surface infrared image based on several feature extraction layers, to obtain a first initial feature set corresponding to the real-time road surface captured image and a second initial feature set corresponding to the real-time road surface infrared image, and to perform feature complementation on the first initial feature set and the second initial feature set, so as to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps.

[0119] The processing module 20 includes:

[0120] The fifth unit is used to obtain complementary feature values ​​for shooting and complementary feature values ​​for infrared based on the initial shooting feature map and the initial infrared feature map;

[0121] The sixth unit is used to obtain the vector mean of each channel in the captured complementary feature values ​​to form a first channel vector, and to obtain the vector mean of each channel in the infrared complementary feature values ​​to form a second channel vector.

[0122] The seventh unit is used to obtain the shooting attention weight through the first channel vector and the infrared attention weight through the second channel vector;

[0123] The eighth unit is used to obtain a final shooting feature map through the initial shooting feature map, the shooting attention weight and the shooting complementary feature value, and to obtain a final infrared feature map through the initial infrared feature map, the infrared attention weight and the infrared complementary feature value;

[0124] The ninth unit is used to combine several of the final captured feature maps into a first final feature set, and to combine several of the final infrared feature maps into a second final feature set;

[0125] The recognition module 30 is used to obtain a final image to be used through the first final feature set, the second final feature set, the first weight and the second weight, and to perform grayscale processing, binarization processing and edge detection on the final image to obtain a number of candidate lesion contours in the final image.

[0126] The identification module 30 includes:

[0127] The tenth unit is used to obtain a feature map to be used based on the final captured feature map, the first weight, the final infrared feature map, and the second weight;

[0128] The eleventh unit is used to upsample several of the proposed feature maps and perform stitching to obtain a multi-scale feature map, and then convert the multi-scale feature map into the final proposed image.

[0129] The verification module 40 is used to identify false detections of the candidate disease contours, so as to separate several candidate disease contours into several contours to be removed and the final disease contours, and remove the contours to be removed to complete the disease detection.

[0130] The verification module 40 includes:

[0131] The twelfth unit is used to obtain a set of connected components corresponding to the candidate disease contour. The set of connected components includes several connected components. The connected components are expanded to obtain an expanded set of connected components. The connected components are removed from the expanded set of connected components to obtain a set of centered neighbor thresholds.

[0132] The thirteenth unit is used to select a region to be determined in the final image to be used based on the set of de-centered neighbor thresholds, and to obtain the gray-level histogram of the region to be determined. The two gray-level values ​​with the largest peaks in the gray-level histogram are selected to obtain the first determination gray level and the second determination gray level.

[0133] The fourteenth unit is used to obtain the grayscale difference between the first determination grayscale and the second determination grayscale, compare the grayscale difference with the difference threshold, and if the grayscale difference is greater than the difference threshold, the candidate disease contour is determined as a contour to be removed; if the grayscale difference is less than the difference threshold, the candidate disease contour is determined as the final disease contour.

[0134] The present invention also provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road surface distress detection method as described in the above technical solutions.

[0135] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the road surface defect detection method as described in the above technical solution.

[0136] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0137] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for detecting pavement defects on highways, characterized in that, Includes the following steps: Acquire real-time road surface images and real-time road surface infrared images, and obtain a first weight corresponding to the real-time road surface images and a second weight corresponding to the real-time road surface infrared images through a weight allocation model. The weight allocation model includes a group of convolutional layers, a pooling layer and a group of fully connected layers connected in sequence. Multi-scale feature extraction is performed on the real-time road surface image and the real-time road surface infrared image based on several feature extraction layers to obtain a first initial feature set corresponding to the real-time road surface image and a second initial feature set corresponding to the real-time road surface infrared image. The first initial feature set includes several initial shooting feature maps, and the second initial feature set includes several initial infrared feature maps. Feature complementation is performed on the first initial feature set and the second initial feature set to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The steps for feature complementarity include: Based on the initial captured feature map and the initial infrared feature map, capture complementary feature values ​​and infrared complementary feature values ​​are obtained; The average vector value of each channel in the captured complementary feature value is obtained to form a first channel vector, and the average vector value of each channel in the infrared complementary feature value is obtained to form a second channel vector. The shooting attention weight is obtained through the first channel vector, and the infrared attention weight is obtained through the second channel vector; The final image feature map is obtained by using the initial image feature map, the image attention weight, and the image complementary feature value; and the final infrared feature map is obtained by using the initial infrared feature map, the infrared attention weight, and the infrared complementary feature value. The several final captured feature maps are combined into a first final feature set, and the several final infrared feature maps are combined into a second final feature set; The final image to be used is obtained by using the first final feature set, the second final feature set, the first weight and the second weight. The final image to be used is then subjected to grayscale processing, binarization processing and edge detection in sequence to obtain several candidate disease contours in the final image to be used. The candidate disease contours are falsely identified to separate several candidate disease contours into several contours to be removed and the final disease contours. The contours to be removed are then removed to complete the disease detection. The steps for false positive identification include: Obtain the set of connected components corresponding to the candidate disease contours, expand the connected components to obtain an expanded set of components, and remove the connected component set from the expanded set of components to obtain a de-centered neighbor set. Based on the set of de-centered neighbor thresholds, a region to be determined is selected in the final image to be used, and the gray-level histogram of the region to be determined is obtained. The two gray-level values ​​with the largest peaks in the gray-level histogram are selected to obtain the first determination gray level and the second determination gray level. The grayscale difference between the first and second determination grayscale values ​​is obtained, and the grayscale difference is compared with the difference threshold to determine the candidate disease contour as a contour to be removed or the final disease contour.

2. The method for detecting road surface defects according to claim 1, characterized in that, The step of obtaining the first weight corresponding to the real-time road surface image and the second weight corresponding to the real-time road surface infrared image through the weight allocation model includes: The real-time road surface image is input into the convolutional layer group to obtain several illumination feature maps; The illumination feature map is input into the pooling layer to extract the maximum dimension of each channel in the illumination feature map. A global vector is constructed based on several of the maximum dimensions, and the dimension of the global vector corresponds to the number of channels in the illumination feature map. The global vector is input into the fully connected layer group to reduce the dimensionality of the global vector into clear components and interference components; Based on the clear component and the interference component, a first weight corresponding to the real-time road surface image and a second weight corresponding to the real-time road surface infrared image are determined.

3. The method for detecting road surface defects according to claim 2, characterized in that, The formula for obtaining the first weight is: ， in, Indicates the first weight. Indicates clear components, Indicates interference components, This represents an exponential function with the natural constant e as its base.

4. The method for detecting road surface defects according to claim 1, characterized in that, The formula for calculating the complementary feature value of the image is: ， in, This represents the complementary feature value captured by the image corresponding to the i-th feature extraction layer. This represents the initial infrared feature map corresponding to the i-th feature extraction layer. This represents the initial captured feature map corresponding to the i-th feature extraction layer; The formula for calculating the infrared complementary feature value is as follows: ， in, This represents the infrared complementary feature value corresponding to the i-th feature extraction layer; The formula for obtaining the final captured feature map is as follows: ， in, This represents the final captured feature map corresponding to the i-th feature extraction layer. This represents the shooting attention weight corresponding to the i-th feature extraction layer. This indicates multiplication by channel.

5. The method for detecting road surface defects according to claim 1, characterized in that, The step of obtaining the final image to be used through the first final feature set, the second final feature set, the first weight, and the second weight includes: Based on the final captured feature map, the first weight, the final infrared feature map, and the second weight, a feature map to be used is obtained; Several of the proposed feature maps are upsampled and stitched together to obtain a multi-scale feature map, which is then converted into the final proposed image.

6. The method for detecting pavement defects according to claim 5, characterized in that, The formula for obtaining the feature map to be used is: ， in, This represents the feature map to be used corresponding to the i-th feature extraction layer. Indicates the first weight. This represents the final captured feature map corresponding to the i-th feature extraction layer. Indicates the second weight. This represents the final infrared feature map corresponding to the i-th feature extraction layer.

7. The method for detecting road surface defects according to claim 1, characterized in that, The difference threshold is 25~45.

8. A highway pavement distress detection system, applied to the highway pavement distress detection method as described in any one of claims 1 to 7, characterized in that, The system includes: The analysis module is used to acquire real-time road surface images and real-time road surface infrared images, and to acquire a first weight corresponding to the real-time road surface images and a second weight corresponding to the real-time road surface infrared images through a weight allocation model. The weight allocation model includes a group of convolutional layers, a pooling layer and a group of fully connected layers connected in sequence. The analysis module includes: The first unit is used to input the real-time road surface image into the convolutional layer group to obtain several illumination feature maps. The second unit is used to input the illumination feature map into the pooling layer to extract the maximum dimension of each channel in the illumination feature map, and to construct a global vector based on several of the maximum dimensions, wherein the dimension of the global vector corresponds to the number of channels in the illumination feature map. The third unit is used to input the global vector into the fully connected layer group to reduce the dimensionality of the global vector into clear components and interference components; The fourth unit is used to determine a first weight corresponding to the real-time road surface image and a second weight corresponding to the real-time road surface infrared image based on the clear component and the interference component. The processing module is used to perform multi-scale feature extraction on the real-time road surface captured image and the real-time road surface infrared image based on several feature extraction layers, so as to obtain a first initial feature set corresponding to the real-time road surface captured image and a second initial feature set corresponding to the real-time road surface infrared image, and to perform feature complementation on the first initial feature set and the second initial feature set, so as to convert the first initial feature set and the second initial feature set into a first final feature set and a second final feature set, respectively. The first final feature set includes several final captured feature maps, and the second final feature set includes several final infrared feature maps. The recognition module is used to obtain a final image to be used through the first final feature set, the second final feature set, the first weight and the second weight, and to perform grayscale processing, binarization processing and edge detection on the final image to obtain a number of candidate lesion contours in the final image. The verification module is used to identify false detections of the candidate disease contours, so as to separate the candidate disease contours into several contours to be removed and the final disease contours, and remove the contours to be removed to complete the disease detection.

Images