Non-destructive testing method and system for road surface in highway engineering

Abstract

The present invention relates to the technical field of image processing, and particularly to a method and system for non-destructive detection of road surfaces for highway engineering. The method performs threshold segmentation on a thermal road surface map to segment out suspected defect regions; determines surface defect regions based on the texture feature differences of the surface defect regions in the optical road surface map; according to the optical road surface map, distinguishes the surface defect regions from the regions with high temperatures generated by vehicle friction in the suspected defect regions of the thermal road surface map; according to the heat changes in consecutive thermal images in the thermal road surface map, differentiates the regions with high temperatures generated by vehicle friction from internal defect regions to obtain the regions where road surface damage is located. The method mainly uses the thermal road surface map and supplements it with the optical road surface map, which can eliminate the influence of the friction high-temperature regions on the accuracy of judging road surface defects through the road surface temperature distribution, and also takes into account factors such as shooting angle, distance, and inter-frame differences, thereby improving the accuracy of the regions where road surface damage is located.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method and a system for non-destructive testing of road surfaces for highway engineering. Background Art

[0002] The non-destructive testing of road surfaces is an important application field of the subgrade and pavement non-destructive testing technology. The non-destructive testing of road surfaces mainly involves the performance evaluation of road paving materials and the monitoring of surface conditions. By means of the thermal images of the road surface area taken by a thermal imager, the temperature distribution of the road surface is detected, so as to judge the compactness of the paving materials and the possible voids and defects.

[0003] However, since the heat absorbed by the road surface comes not only from sunlight as a heat source, but also from the friction between the vehicle and the road surface during driving, there are friction high-temperature areas in the road surface without defects. Judging the defects in the road surface only by the temperature distribution of the road surface obtained from the thermal image may identify normal road surface areas as defective areas. Therefore, it is urgent to develop a method and a system for non-destructive testing of road surfaces for highway engineering to solve the above problems. Summary of the Invention

[0004] In order to solve the technical problem that the friction high-temperature area affects the accuracy of judging road surface defects through the road surface temperature distribution, the purpose of the present invention is to provide a method and a system for non-destructive testing of road surfaces for highway engineering, and the specific technical solutions adopted are as follows:

[0005] In the first aspect, the present invention provides a method for non-destructive testing of road surfaces for highway engineering, and the method includes:

[0006] Step 101, obtaining a thermal road surface map and an optical road surface map, where the thermal road surface map is multiple frames of thermal images of the road surface area taken by a thermal imager, the optical road surface map is multiple frames of optical images of the road surface area taken by an optical camera, the multiple frames of thermal images and the multiple frames of optical images correspond one by one, and the road surface area photographed by a certain frame of thermal image and its corresponding frame of optical image is the same road surface area;

[0007] Step 102, performing threshold segmentation on the thermal road surface map to obtain the suspected defect area in the thermal road surface map;

[0008] Step 103, determining the surface defect area in the optical road surface map based on the texture feature difference between the surface defect area and the normal road surface area in the optical road surface map, and taking the corresponding area of the suspected defect area in the thermal road surface map in the optical road surface map as the suspected surface defect area, where the suspected defect area in the thermal road surface map includes a friction high-temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area;

[0009] Step 104: Determine the surface defect area in the thermal road map according to the optical road map;

[0010] Step 105: Determine the internal defect area in the thermal road map according to the heat change of consecutive multi-frame thermal images in the thermal road map;

[0011] Step 106: Determine that the surface defect area and the internal defect area in the thermal road map are the areas where road damage is located.

[0012] In some embodiments, Step 104 includes:

[0013] Step 1041: Obtain the area in the optical road map corresponding to the suspected surface defect area in the thermal road map;

[0014] Step 1042: Based on the eigenvector of the gray-level co-occurrence matrix of the area in the optical road map corresponding to the suspected surface defect area in the thermal road map, determine the possibility that the area in the optical road map corresponding to the suspected surface defect area in the thermal road map is a surface defect area;

[0015] Step 1043: In response to the possibility that the area in the optical road map corresponding to the suspected surface defect area in the thermal road map being greater than the first threshold, determine the surface defect area in the thermal road map.

[0016] In some embodiments, Step 105 includes:

[0017] Step 1051: Determine the centroid of other defect areas in the suspected defect area in the thermal road map except for the surface defect area determined, where the other defect areas include the friction high-temperature area and the internal defect area;

[0018] Step 1052: Draw a horizontal ray through the centroid of the other defect area, and rotate the ray in the counterclockwise direction by an integer multiple of 30° until a 360° rotation is completed, obtaining twelve rays in sequence. The areas where the twelve rays overlap with the other defect area obtain twelve line segments. The gray-scale values of the pixel points corresponding to the twelve line segments on the other defect area form the heat dissipation sequences of twelve defect areas. The heat dissipation sequence satisfies that the first element is the gray-scale value of the pixel point corresponding to the centroid of the other defect area, and the last element is the gray-scale value of the pixel point corresponding to the intersection of the outermost edge of the other defect area and the end of the line segment;

[0019] Step 1053: Calculate the temporal change factor of the heat dissipation sequence;

[0020] Step 1054: Determine the timing change factor of the other defect regions based on the timing change factor of the heat dissipation sequence.

[0021] Step 1055: Determine the timing change factor of the surface defect regions in the thermal road map by the method of Steps 1051 to 1054.

[0022] Step 1056: For the same road surface area, project the line connecting the centroid of the other defect regions and the centroid of the surface defect regions in the thermal road map onto the road driving direction to obtain a deformation difference segment.

[0023] Step 1057: For the same road surface area, determine the comparison weight between the other defect regions and the surface defect regions in the thermal road map based on the deformation difference degree.

[0024] Step 1058: Determine the possibility that the other defect regions are internal defect regions based on the timing change factor of the other defect regions, the timing change factor of the surface defect regions in the thermal road map, and the comparison weight between the other defect regions and the surface defect regions in the thermal road map.

[0025] Step 1059: Use the other defect regions with the possibility of internal defect regions greater than the second threshold as the internal defect regions in the thermal road map to obtain the internal defect regions in the thermal road map.

[0026] In some embodiments, according to the following formula, determine the possibility that the region corresponding to the suspected surface defect region in the thermal road map in the optical road map is a surface defect region:

[0027] ;

[0028] In the formula, represents the possibility that the region corresponding to the suspected surface defect region in the thermal road map in the optical road map is a surface defect region, represents the eigenvector of the gray-level co-occurrence matrix of the region corresponding to the suspected surface defect region in the thermal road map in the optical road map, represents the eigenvector of the gray-level co-occurrence matrix of the normal region in the optical road map, represents the cosine function.

[0029] In some embodiments, for the timing change factor of the th heat dissipation sequence in the th other defect region in the th frame of the thermal road map, calculate the timing change factor of the heat dissipation sequence according to the following formula:

[0030] ;

[0031] In the formula, represents the th thermal change factor of the th heat dissipation sequence in the th other defect area in the thermal road surface map of the th other defect area, represents the total number of frames of the thermal road surface map including the th heat dissipation sequence in the th other defect area, represents the frame number sequence of the thermal road surface map including the th heat dissipation sequence in the th other defect area, and respectively represent the th and th heat dissipation sequences in the th and th frames of the thermal road surface map, in the th other defect area, and represents the DTW distance between the

[0032] In some embodiments, according to the following formula, the temporal change factor of the other defect area is determined:

[0033] ;

[0034] In the formula, represents the temporal change factor of the other defect area, represents the th th angle between the direction of the line segment corresponding to the th heat dissipation sequence in the th other defect area in the thermal road surface map of the th frame and the road driving direction,

[0035]

[0036] ;

[0037] In the formula, Indicates the comparison weight between the other defect area and the surface defect area in the thermal road map. Indicates the length of the deformation difference segment. Indicates the total length of the road surface area in the thermal road map. Indicates the distance from the midpoint of the deformation difference segment to the lower edge of the road surface area in the thermal road map. Is an exponential function.

[0038] In some embodiments, according to the following formula, determine the possibility that the other defect area is an internal defect area:

[0039] ;

[0040] In the formula, Indicates the possibility that the th other defect area is an internal defect area, Indicates the time series change factor of the th surface defect area, Indicates the comparison weight between the th surface defect area and the th other defect area, Indicates the time series change factor of the th other defect area, Indicates an S-shaped function for normalization operation, Indicates the serial number of the other defect area, Indicates the total number of surface defect areas.

[0041] In some embodiments, in step 102, perform threshold segmentation on the thermal road map by using the Otsu threshold method to obtain the suspected defect area in the thermal road map.

[0042] In a second aspect, the present invention provides a pavement non-destructive testing system for highway engineering, and the system includes:

[0043] An acquisition module for acquiring a thermal road map and an optical road map, where the thermal road map is multiple frames of thermal images of a road surface area captured by a thermal imager, the optical road map is multiple frames of optical images of the road surface area captured by an optical camera, the multiple frames of thermal images and the multiple frames of optical images correspond one by one, and the road surface area captured by a certain frame of thermal image and its corresponding frame of optical image is the same road surface area;

[0044] A segmentation module for performing threshold segmentation on the thermal road map to obtain the suspected defect area in the thermal road map;

[0045] The first determination module is configured to determine the surface defect area in the optical road surface map based on the texture feature difference between the surface defect area and the normal road surface area in the optical road surface map, and use the corresponding area of the suspected defect area in the thermal road surface map in the optical road surface map as the suspected surface defect area, where the suspected defect area in the thermal road surface map includes the friction high-temperature area and the real defect area, and the real defect area includes the surface defect area and the internal defect area;

[0046] The second determination module is configured to determine the surface defect area in the thermal road surface map according to the optical road surface map;

[0047] The third determination module is configured to determine the internal defect area in the thermal road surface map according to the heat change of consecutive frames of thermal images in the thermal road surface map;

[0048] The fourth determination module is configured to determine that the surface defect area and the internal defect area in the thermal road surface map are the areas where road surface damage is located.

[0049] The present invention has the following beneficial effects:

[0050] The method and system for non-destructive detection of road surface for highway engineering provided by the present invention first obtains the thermal road surface map and the optical road surface map, so as to facilitate subsequent comprehensive analysis of the texture features and temperature distribution of the same road surface area; then, performs threshold segmentation on the thermal road surface map to obtain the suspected defect area in the thermal road surface map, so as to separate the suspected defect area with a gray value difference from the normal road surface area; furthermore, based on the texture feature difference between the surface defect area and the normal road surface area in the optical road surface map, determines the surface defect area in the optical road surface map, and uses the corresponding area of the suspected defect area in the thermal road surface map in the optical road surface map as the suspected surface defect area, where the suspected defect area in the thermal road surface map includes the friction high-temperature area and the real defect area, the real defect area includes the surface defect area and the internal defect area, and the surface defect area in the optical road surface map is the area where there is actual damage on the road surface; then, according to the optical road surface map, determines the surface defect area in the thermal road surface map, so as to distinguish the surface defect area from the area where high temperature is generated by vehicle friction in the suspected defect area of the thermal road surface map; determines the internal defect area in the thermal road surface map according to the heat change of consecutive frames of thermal images in the thermal road surface map, and distinguishes the area where high temperature is generated by vehicle friction and the internal defect area in the thermal road surface map; finally, determines that the surface defect area and the internal defect area in the thermal road surface map are the areas where road surface damage is located. This method mainly uses the thermal road surface map and supplements it with the optical road surface map, which can eliminate the influence of the friction high-temperature area on the accuracy of judging road surface defects through the road surface temperature distribution, and also considers factors such as shooting angle, distance, and inter-frame difference, thereby improving the accuracy of the area where road surface damage is located. Brief Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0052] Figure 1 It is a method flow chart of a pavement non-destructive testing method for highway engineering provided by an embodiment of the present invention;

[0053] Figure 2 It is a structural schematic diagram of a pavement non-destructive testing system for highway engineering provided by an embodiment of the present invention. Detailed Embodiments

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0056] In the first aspect, the present invention provides a pavement non-destructive testing method for highway engineering. Refer to Figure 1 , the method includes:

[0057] Step 101, obtain a thermal pavement map and an optical pavement map. The thermal pavement map is multiple frames of thermal images of a pavement area captured by a thermal imager, and the optical pavement map is multiple frames of optical images of the pavement area captured by an optical camera. The multiple frames of thermal images correspond to the multiple frames of optical images one by one. The pavement area captured by a certain frame of thermal image and its corresponding frame of optical image is the same pavement area.

[0058] Obtain a thermal road surface map and an optical road surface map, so as to obtain an optical image and a thermal image of the road surface area to be defect-detected. The optical road surface map can reflect the texture characteristics of the road surface area. The texture characteristics of the normal road surface area and the defective road surface area are different, so as to judge whether there are defects in the road surface area. For the optical road surface map, it is inconvenient to reflect the internal situation of the road surface area. The thermal road surface map can reflect the temperature distribution of the road surface area. The temperature of the defective road surface area or the area where the vehicle friction generates high temperature is higher than that of the normal road surface area, and the area where the vehicle friction generates high temperature is the normal road surface area. Therefore, it is necessary to comprehensively consider the thermal road surface map and the optical road surface map to judge whether there are defects.

[0059] In some embodiments, an optical camera and a thermal imager are installed in front of the vehicle. The acquisition frequencies of the optical camera and the thermal imager are both set to 30 Hz, so as to ensure that each frame of the captured image is of the same road surface area. And the vehicle travels uniformly on the road surface, and the traveling speed can be 10 km / h. The RGB image and the infrared image of the road surface are collected. The RGB image is grayscale processed to obtain an optical road surface map. According to the temperature scale in the infrared image, the temperature values in the infrared image are mapped into the gray scale interval of [0, 255] to obtain a thermal road surface map.

[0060] When there are damages such as internal defects and surface defects on the road surface, sunlight will enter the surface defects and be reflected multiple times in the surface defects, resulting in the temperature of the surface defects being higher than that of the normal road surface; and the internal defects in the road will have poor heat conduction ability, and thus cannot dissipate heat in time, making the temperature of the internal defects higher than that of the normal road surface, that is, the heat absorption efficiency and heat dissipation efficiency at the defective positions in the road surface are different from those of the surrounding normal road surface, which is manifested as the difference in the gray scale values at the defective positions and other normal road surfaces in the thermal road surface map.

[0061] Step 102, perform threshold segmentation on the thermal road surface map to obtain the suspected defect area in the thermal road surface map.

[0062] In some embodiments, in step 102, the Otsu threshold method is used to perform threshold segmentation on the thermal road surface map to obtain the suspected defect area in the thermal road surface map.

[0063] Threshold segmentation can separate the suspected defect areas with different gray scale values from the normal road surface area, so as to obtain the suspected defect areas in the thermal road surface map.

[0064] Since the heat absorbed by the road surface has not only sunlight as the heat source, but also comes from the friction between the vehicle and the road surface during driving, there are friction high-temperature regions in the road surface without defects. Judging the defects in the road surface only by obtaining the temperature distribution of the road surface through the thermal image may identify the normal road surface area as a defective area, that is, the heat received under the road surface position is not the same. If simply based on the gray value of the thermal road map as the basis for identifying the defective position, the area with high temperature generated by vehicle friction will be identified as a defective area. Therefore, the suspected defective area may be the area with high temperature generated by vehicle friction, or may be the defective area with surface and internal damage.

[0065] In order to distinguish the area with high temperature generated by vehicle friction from the defective area in the suspected defective area, step 103 is introduced.

[0066] Step 103: Based on the texture feature difference between the surface defective area and the normal road surface area in the optical road map, determine the surface defective area in the optical road map, and use the corresponding area of the suspected defective area in the thermal road map in the optical road map as the suspected surface defective area, where the suspected defective area in the thermal road map includes the friction high-temperature area and the real defective area, and the real defective area includes the surface defective area and the internal defective area.

[0067] The surface defects on the road surface will damage the original texture structure of the road surface. At the same time, because the defects in the road surface are not regular, that is, the textures of the defects in different positions are quite different, while the textures on the normal road surface are similar. Therefore, based on the texture feature difference between the surface defective area and the normal road surface area in the optical road map, the surface defective area in the optical road map can be determined. The surface defective area in the optical road map is the real surface defective area with damage. Using the corresponding area of the suspected defective area in the thermal road map in the optical road map as the suspected surface defective area, the texture feature information of the suspected surface defective area in the optical road map is obtained, which is convenient for subsequent using the optical road map as an auxiliary to distinguish the area with high temperature generated by vehicle friction from the defective area in the suspected defective area of the thermal road map.

[0068] It should be noted that the surface defective area in the optical road map is the area where there is actual damage on the road surface. The non-surface defective area in the optical road map is the normal road surface area, which can be used as the standard for subsequent judgment of texture features, so as to judge whether the suspected defective area in the thermal road map is a surface defective area.

[0069] In some embodiments, when the difference between the texture features of a certain road surface area and the texture features of all other local road surface areas is greater than the texture feature difference threshold, then this road surface area is the surface defective area.

[0070] In some embodiments, by performing Fourier transform, wavelet transform, etc., the texture feature values of each local road surface area are extracted, and then the average value of the absolute values of the differences between the texture feature values of each local road surface area and those of all other local road surface areas is obtained. The average value of the absolute values of the differences is the difference between the texture features of each local area of the road surface.

[0071] Since the local high-temperature areas generated by tire friction heating in the thermal infrared image will interfere with the normal detection of the road surface, while the optical image is not affected by friction heating, therefore, the thermal road surface map is assisted in detection by combining with the optical road surface map.

[0072] Step 104, determine the surface defect area in the thermal road surface map according to the optical road surface map.

[0073] Determine the surface defect area in the thermal road surface map, so as to distinguish the surface defect area from the area with high temperature generated by vehicle friction in the suspected defect area of the thermal road surface map.

[0074] In some embodiments, step 104 includes:

[0075] Step 1041, obtain the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map as the basis for subsequent discrimination.

[0076] Step 1042, based on the eigenvector of the gray-level co-occurrence matrix of the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map, determine the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area.

[0077] The eigenvector of the gray-level co-occurrence matrix of the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map can reflect the texture feature of the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map. Thus, the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area can reflect the possible degree that the area is a surface defect area in the thermal road surface map.

[0078] In some embodiments, according to the following formula, determine the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area:

[0079] ;

[0080] In the formula, represents the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area, reflecting the possible degree that the area is a surface defect area in the thermal road surface map, It represents the eigenvector of the gray-level co-occurrence matrix of the suspected surface defect area in the optical road surface map and the thermal road surface map, reflecting the texture features of the suspected surface defect area in the optical road surface map and the thermal road surface map. It represents the eigenvector of the gray-level co-occurrence matrix of the normal area in the optical road surface map, reflecting the texture features of the normal area in the optical road surface map. It represents the cosine function. It can reflect the similarity degree between the texture features of the normal area in the optical road surface map and the texture features of the suspected surface defect area in the optical road surface map and the thermal road surface map. The larger it is, that is, the more similar the two are and the smaller the difference is, then The smaller it is, that is The larger it is, the more likely this area is the surface defect area in the thermal road surface map.

[0081] Step 1043: In response to the possibility that the area corresponding to the suspected surface defect area in the optical road surface map and the thermal road surface map being a surface defect area being greater than the first threshold, determine the surface defect area in the thermal road surface map.

[0082] The possibility that the area corresponding to the suspected surface defect area in the optical road surface map and the thermal road surface map being a surface defect area is greater than the first threshold indicates that the possibility of this area being a surface defect area in the thermal road surface map is very high, and it can be determined as the surface defect area in the thermal road surface map.

[0083] In some embodiments, the first threshold may be 0.8.

[0084] In some embodiments, since the thermal road surface map is multiple frames of thermal images of the road surface area captured by a thermal imager, and the optical road surface map is multiple frames of optical images of the road surface area captured by an optical camera, the possibility that the area corresponding to the suspected surface defect area in the optical road surface map and the thermal road surface map being a surface defect area may come from multiple frames. The average value of the possibilities that the areas corresponding to the suspected surface defect areas in the optical road surface maps from multiple frames and the thermal road surface map being surface defect areas can be compared with the first threshold.

[0085] Since there are no defects in the areas with high temperature generated by vehicle friction, there is a large difference between its heat dissipation efficiency and the thermal radiation at the positions where there are actual defects in the road surface. Therefore, step 105 is introduced to distinguish the areas with high temperature generated by vehicle friction and the internal defect areas in the thermal road surface map.

[0086] Step 105: Determine the internal defect area in the thermal road surface map according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map.

[0087] By analyzing the heat changes in consecutive frames of thermal images in the thermal road surface map, the areas with high temperatures generated by vehicle friction and the internal defect areas in the thermal road surface map are distinguished. Since the embodiments of the present invention collect the thermal road surface map and the optical road surface map during the driving of the vehicle by installing a thermal imager and an optical camera on the vehicle, there may be some changes between each frame of the image and the previous frame. In order to better analyze the suspected defect areas in the thermal road surface map to obtain the internal defect areas in the thermal road surface map, it is necessary to correspond the suspected defect areas in different frames of images. Therefore, the heat changes in consecutive frames of thermal images in the thermal road surface map are analyzed to master the information of the same suspected defect area in the thermal images of different frames, eliminate the influence of the changes of the same suspected defect area in different frames of thermal images, and thus ensure the accuracy of the judgment of the internal defect areas in the thermal road surface map.

[0088] In some embodiments, the method for obtaining the same suspected defect area in consecutive frames of thermal images and optical images may be: for the suspected defect area in any frame of the thermal image, obtain the corresponding area in the optical image of the corresponding frame of any frame. Furthermore, through the optical flow matching method, obtain the corresponding area in the optical image of the corresponding frame of any frame. in the previous frame of the optical image corresponding to the area. Obtain the area in the previous frame of the thermal image of any frame corresponding to the area .

[0089] It should be noted that the optical road surface map can detect the areas with surface defects on the road surface. The surface defect areas and the internal defect areas are both caused by actual damage in the road surface. Therefore, the heat dissipation efficiency of the surface defect areas and the internal defect areas in the thermal road surface map is similar, while the heat dissipation efficiency is different from that of the friction high-temperature areas. The friction high-temperature areas and the internal defect areas can be distinguished by the difference in the heat dissipation efficiency between the suspected defect areas in the thermal road surface map and other defect areas determined to be outside the surface defect areas.

[0090] In some embodiments, step 105 includes:

[0091] Step 1051, determine the centroid of other defect areas determined to be outside the surface defect areas in the suspected defect areas in the thermal road surface map. The other defect areas include the friction high-temperature areas and the internal defect areas.

[0092] Step 1052: Draw a horizontal ray through the centroid of other defect regions, and rotate the ray in a counterclockwise direction by integer multiples of 30° until a 360° rotation is completed, obtaining twelve rays in sequence. The regions where the twelve rays overlap with other defect regions yield twelve line segments. The gray values of the pixel points corresponding to the twelve line segments on other defect regions form the heat dissipation sequences of twelve defect regions. The heat dissipation sequence satisfies that the first element is the gray value of the pixel point corresponding to the centroid of other defect regions, and the last element is the gray value of the pixel point corresponding to the intersection of the outermost edge of other defect regions and the end of the line segment. The heat dissipation sequences of twelve defect regions can reflect the change in gray value, that is, the heat change, from the centroid of the region outward.

[0093] Step 1053: Calculate the temporal change factor of the heat dissipation sequence, thereby obtaining the temporal change situation, that is, the heat change situation, in the heat dissipation sequence in consecutive frames in a certain other defect region.

[0094] In some embodiments, for the th frame of the thermal road surface map, for the th other defect region, and for the th heat dissipation sequence, according to the following formula, calculate the temporal change factor of the heat dissipation sequence:

[0095] ;

[0096] In the formula, represents the temporal change factor of the th heat dissipation sequence in the th other defect region in the th frame of the thermal road surface map, reflecting the heat change situation of the th heat dissipation sequence in the th other defect region in the th frame of the thermal road surface map, represents the total number of frames of the thermal road surface map containing the th other defect region and the th heat dissipation sequence, represents the frame number sequence of the thermal road surface map containing the th other defect region and the th heat dissipation sequence, and respectively represent the th frame and the th frame of the thermal road surface map, and the th other defect region and the th heat dissipation sequence, represents the th frame and the th frame of the thermal road surface map, and the The DTW distance between the th heat dissipation sequences within another defect area can reflect the th frame and the th frame in the thermal road surface map, and the th heat dissipation sequences within another defect area, indicating the similarity of the heat dissipation sequences. Here, the larger is, the smaller the similarity between the th frame and the th frame in the thermal road surface map, and the th heat dissipation sequences within another defect area. Then, the larger is, the greater the temporal variation of the th heat dissipation sequences within another defect area in the th frame of the thermal road surface map, and the greater the heat variation.

[0097] It should be noted that the DTW distance is calculated by aligning the elements of two heat dissipation sequences to obtain the best matching distance between them. Different from the traditional Euclidean distance, DTW allows non-linear alignment of the time axis, thus better handling the time warping problem in the heat dissipation sequences, that is, the variation problem between different frames.

[0098] Step 1054: Based on the temporal variation factor of the heat dissipation sequence, determine the temporal variation factor of another defect area, so as to obtain the temporal variation situation of another defect area.

[0099] In some embodiments, according to the following formula, determine the temporal variation factor of another defect area:

[0100] ;

[0101] In the formula, represents the temporal variation factor of another defect area, reflecting the temporal variation situation of another defect area, represents the th frame in the thermal road surface map, and the th heat dissipation sequences within another defect area, and is the angle between the direction of the line segment corresponding to the th heat dissipation sequences and the road driving direction, represents the sine function, is not zero, and

[0102] Due to the influence of the shooting angle and distance on other defect areas, the other defect areas in the thermal road surface map will be compressed and overlapped inward along two sections in the road driving direction, and thus the change characteristics of some road surface areas will be lost; among them, the sequence parallel to the road driving direction is most affected, while the part perpendicular to the road driving direction is not affected by the shooting angle and distance; therefore, when calculating the temporal change factor of other defect areas, a credibility weight needs to be assigned to the angle between the direction of the line segment corresponding to the heat dissipation sequence and the road driving direction, that is , and thus the credibility weights of the temporal change factors of all heat dissipation sequences in other defect areas are adjusted to obtain the temporal change factor of other defect areas . When the angle between the direction of the line segment corresponding to the heat dissipation sequence and the road driving direction is smaller, that is is smaller, the degree of deformation generated by this heat dissipation sequence is greater, the more heat change characteristics it loses, and its credibility weight is lower, that is, the greater the impact on the heat change information of the collected road surface area. When is smaller, then is smaller

[0103] Step 1055: Determine the temporal change factor of the surface defect area in the thermal road surface map by the methods of steps 1051 to 1054

[0104] Step 1056: For the same road surface area, project the connection line between the centroid of the other defect area and the centroid of the surface defect area in the thermal road surface map onto the road driving direction to obtain a deformation difference segment, so as to facilitate subsequent analysis of the deformation of the other defect area

[0105] Step 1057: For the same road surface area, determine the comparison weight between the other defect area and the surface defect area in the thermal road surface map based on the deformation difference degree

[0106] Since the degree of stretching and shrinking of other defect areas is different in thermal road surface maps of different frames, and in order to better compare the other defect area and the surface defect area in the thermal road surface map, it is necessary to combine the deformation difference degree and assign a comparison weight to the other defect area and the surface defect area in the thermal road surface map

[0107] In some embodiments, according to the following formula, determine the comparison weight between the other defect area and the surface defect area in the thermal road surface map

[0108] ;

[0109] In the formula Indicates the comparison weight between other defect regions and the surface defect regions in the thermal road map, reflecting the magnitude of the comparison weight between other defect regions and the surface defect regions in the thermal road map. Indicates the length of the deformation difference segment. Indicates the total length of the road surface region in the thermal road map and the value is not zero. Indicates the distance from the midpoint of the deformation difference segment to the lower edge of the road surface region in the thermal road map. Is an exponential function. Can indirectly reflect the position of the deformation difference segment in the road surface region of the thermal road map.

[0110] When the deformation difference segment Is longer, the projection distance of the line segment connecting the centroid of other defect regions and the centroid of the surface defect regions in the thermal road map in the road driving direction is longer, and the possibility of its deformation degree is greater. Therefore, assign A negatively correlated comparison weight to obtain ; At the same time, due to the principle of objects appearing smaller when farther away, when the deformation difference segment is farther from the thermal imager, the actual distance corresponding to the deformation difference segment is larger. Therefore, it is also necessary to assign a calculation weight according to the position of the deformation difference segment in the road surface region of the thermal road map . The larger is, the is smaller.

[0111] Step 1058, based on the time series change factor of other defect regions, the time series change factor of the surface defect regions in the thermal road map, and the comparison weight between other defect regions and the surface defect regions in the thermal road map, determine the possibility that other defect regions are internal defect regions, so as to obtain the possible degree that other defect regions are internal defect regions.

[0112] In some embodiments, according to the following formula, determine the possibility that other defect regions are internal defect regions:

[0113] ;

[0114] In the formula, Indicates the possibility that the th other defect region is an internal defect region, reflecting the possible degree that the th other defect region is an internal defect region. Indicates the time series change factor of the th surface defect region. Indicates the comparison weight between the th surface defect region and the th other defect region. Indicates the The temporal change factor of other defect regions Represents an S-shaped function for normalization operations Represents the number sequence of other defect regions Represents the total number of surface defect regions

[0115] When the temporal change factor of the th surface defect region and the temporal change factor of the th other defect region have a greater difference, it indicates that the difference in heat dissipation efficiency between the th surface defect region and the th other defect region is greater. When the comparison weight between the th surface defect region and the th other defect region is larger, the possibility that the th other defect region is an internal defect region

[0116] Step 1059: Use other defect regions with the possibility of internal defect regions greater than the second threshold as the internal defect regions in the thermal road surface map to obtain the internal defect regions in the thermal road surface map

[0117] When the possibility that other defect regions are internal defect regions is greater than the second threshold, it indicates that the other defect regions in the thermal road surface map may be internal defect regions

[0118] In some embodiments, the second threshold may be 0.9

[0119] Step 106: Determine that the surface defect regions and internal defect regions in the thermal road surface map are the regions where road damage is located

[0120] The surface defect regions and internal defect regions in the thermal road surface map are the regions where road damage is located

[0121] In summary, the non-destructive testing method for road surfaces in highway engineering provided by the present invention mainly uses the thermal road surface map and supplements it with the optical road surface map, which can eliminate the influence of the friction high-temperature region on the accuracy of judging road surface defects through the road surface temperature distribution, and also considers factors such as shooting angle, distance, and inter-frame difference, thereby improving the accuracy of the region where road damage is located

[0122] Second, the present invention provides a non-destructive testing system for road surfaces in highway engineering. Refer to Figure 2 , this system includes:

[0123] An acquisition module 201, configured to acquire a thermal road surface map and an optical road surface map, where the thermal road surface map is multiple frames of thermal images of a road surface area captured by a thermal imager, and the optical road surface map is multiple frames of optical images of the road surface area captured by an optical camera. The multiple frames of thermal images correspond to the multiple frames of optical images one by one, and the road surface area captured by a certain frame of thermal image and its corresponding frame of optical image is the same road surface area.

[0124] A segmentation module 202, configured to perform threshold segmentation on the thermal road surface map to obtain suspected defect areas in the thermal road surface map.

[0125] A first determination module 203, configured to determine surface defect areas in the optical road surface map based on the texture feature differences between the surface defect areas and normal road surface areas in the optical road surface map, and use the corresponding areas of the suspected defect areas in the thermal road surface map in the optical road surface map as suspected surface defect areas. Among them, the suspected defect areas in the thermal road surface map include friction high-temperature areas and real defect areas, and the real defect areas include surface defect areas and internal defect areas.

[0126] A second determination module 204, configured to determine the surface defect areas in the thermal road surface map according to the optical road surface map.

[0127] A third determination module 205, configured to determine the internal defect areas in the thermal road surface map according to the heat changes of multiple consecutive frames of thermal images in the thermal road surface map.

[0128] A fourth determination module 206, configured to determine that the surface defect areas and internal defect areas in the thermal road surface map are the areas where road damage is located.

[0129] In summary, the pavement non-destructive testing method for highway engineering provided by the present invention uses the thermal road surface map as the main and the optical road surface map as the auxiliary. It can eliminate the influence of friction high-temperature areas on the accuracy of judging road surface defects through the road surface temperature distribution, and also takes into account factors such as shooting angle, distance, and inter-frame difference, thereby improving the accuracy of the areas where road damage is located.

[0130] It should be noted that: the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A pavement non-destructive testing method for highway engineering, characterized in that, The method includes: Step 101: Obtain a thermal road surface map and an optical road surface map. The thermal road surface map is multiple frames of thermal images of a road surface area captured by a thermal imager, and the optical road surface map is multiple frames of optical images of the road surface area captured by an optical camera. The multiple frames of thermal images correspond one by one to the multiple frames of optical images, and the road surface area captured by a certain frame of thermal image and its corresponding frame of optical image is the same road surface area. Step 102: Perform threshold segmentation on the thermal road surface map to obtain suspected defect areas in the thermal road surface map. Step 103: Based on the texture feature differences between the surface defect areas and the normal road surface areas in the optical road surface map, determine the surface defect areas in the optical road surface map, and use the corresponding areas of the suspected defect areas in the thermal road surface map in the optical road surface map as the suspected surface defect areas in the thermal road surface map. Among them, the suspected defect areas in the thermal road surface map include friction high-temperature areas and real defect areas, and the real defect areas include surface defect areas and internal defect areas. Step 104: Determine the surface defect areas in the thermal road surface map according to the areas in the optical road surface map corresponding to the suspected surface defect areas in the thermal road surface map. Step 105: Determine the internal defect areas in the thermal road surface map according to the heat changes of consecutive multiple frames of thermal images in the thermal road surface map. Step 106: Determine that the surface defect areas and the internal defect areas in the thermal road surface map are the areas where road damage is located. The step 105 includes: Determine the centroids of the other defect areas except the surface defect areas determined in the suspected defect areas in the thermal road surface map. The other defect areas include friction high-temperature areas and internal defect areas. Draw a horizontal ray through the centroid of the other defect area, and rotate the ray in a counterclockwise direction by an integer multiple of 30° until a 360° rotation is completed, and twelve rays are obtained in sequence. The areas where the twelve rays overlap with the other defect area obtain twelve line segments, and the gray values of the pixel points corresponding to the twelve line segments on the other defect area form the heat dissipation sequences of the twelve defect areas. The heat dissipation sequence satisfies that the first element is the gray value of the pixel point corresponding to the centroid of the other defect area, and the last element is the gray value of the pixel point corresponding to the intersection of the outermost edge of the other defect area and the end of the line segment. Calculate the temporal change factor of the heat dissipation sequence. Based on the temporal change factor of the heat dissipation sequence, determine the temporal change factor of the other defect area. Through the method of determining the temporal change factor described above, determine the temporal change factor of the surface defect area in the thermal road surface map. For the same road surface area, project the line connecting the centroid of the other defect area and the centroid of the surface defect area in the thermal road surface map onto the road driving direction to obtain a deformation difference segment. For the same road surface area, based on the deformation difference segment, determine the comparison weights of the other defect area and the surface defect area in the thermal road surface map. Determine the possibility that the other defect area is an internal defect area based on the temporal change factor of the other defect area, the temporal change factor of the surface defect area in the thermal road map, and the comparison weight between the other defect area and the surface defect area in the thermal road map; Use the other defect areas with the possibility of internal defect areas greater than the second threshold as the internal defect areas in the thermal road map to obtain the internal defect areas in the thermal road map.

2. The method for non-destructive flaw detection of road surface for highway engineering according to claim 1, characterized in that, The step 104 includes: Step 1041, obtain the area in the optical road map corresponding to the suspected surface defect area in the thermal road map; Step 1042, determine the possibility that the area in the optical road map corresponding to the suspected surface defect area in the thermal road map is a surface defect area based on the eigenvector of the gray-level co-occurrence matrix of the area in the optical road map corresponding to the suspected surface defect area in the thermal road map; Step 1043, in response to the possibility that the area in the optical road map corresponding to the suspected surface defect area in the thermal road map is greater than the first threshold, determine the surface defect area in the thermal road map.

3. The pavement non-destructive testing method for highway engineering according to claim 2, characterized in that, Determine the possibility that the area in the optical road map corresponding to the suspected surface defect area in the thermal road map is a surface defect area according to the following formula: ； In the formula, represents the possibility that the area corresponding to the suspected surface defect area in the thermal road surface map in the optical road surface map is a surface defect area, represents the eigenvector of the gray-level co-occurrence matrix of the area corresponding to the suspected surface defect area in the thermal road surface map in the optical road surface map, represents the eigenvector of the gray-level co-occurrence matrix of the normal area in the optical road surface map, represents the cosine function.

4. The non-destructive flaw detection method for road surface in road engineering according to claim 1, characterized in that, For the in the in the For the timing change factor of the heat dissipation sequence in the th other defect area in the thermal road surface map of the th frame, calculate the timing change factor of the heat dissipation sequence according to the following formula: ； Wherein, represents the th thermal time series variation factor of the th heat dissipation sequence in the th other defect area in the thermal road surface map of the th other defect area, represents the total number of frames of the thermal road surface map containing the th heat dissipation sequence in the th other defect area, represents the frame number of the thermal road surface map containing the th heat dissipation sequence in the th other defect area, and respectively represent the th heat dissipation sequence in the th and th frames of the thermal road surface map, and the th DTW distance between the heat dissipation sequences in the represents the serial number of the number of heat dissipation sequences.

5. The method for non-destructive testing of road surface for highway engineering according to claim 4, characterized in that, Determine the temporal change factor of the other defect area according to the following formula: ； In the formula, represents the temporal change factor of other defect areas, represents the angle between the direction of the line segment corresponding to the th heat dissipation sequence in the th other defect area in the th heat map of the hot pavement and the road driving direction, represents the total number of heat dissipation sequences.

6. The pavement non-destructive flaw detection method for highway engineering according to claim 4, characterized in that, Determine the comparison weight between the other defect area and the surface defect area in the thermal road map according to the following formula: ； In the formula, represents the comparison weight between the other defect area and the surface defect area in the thermal road map, represents the length of the deformation difference section, represents the total length of the road surface area in the thermal road map, represents the distance from the midpoint of the deformation difference section to the lower edge of the road surface area in the thermal road map, is an exponential function.

7. The method for non-destructive flaw detection of road surface for highway engineering according to claim 6, characterized in that, Determine the possibility that the other defect area is an internal defect area according to the following formula: ； Wherein, represents the probability that the th other defect area is an internal defect area, represents the temporal variation factor of the th surface defect area, represents the comparison weight between the th surface defect area and the th other defect area, represents the temporal variation factor of the th other defect area, represents the S-shaped function for normalization operation, represents the serial number of the number of other defect areas, represents the total number of surface defect areas.

8. The method for non-destructive flaw detection of road surface for highway engineering according to claim 1, characterized in that, In step 102, perform threshold segmentation on the thermal road map by the Otsu threshold method to obtain the suspected defect areas in the thermal road map.

9. A pavement non-destructive testing system for highway engineering, characterized in that, The system includes: An acquisition module, configured to acquire a thermal road map and an optical road map, where the thermal road map is multiple frames of thermal images of a road area captured by a thermal imager, and the optical road map is multiple frames of optical images of the road area captured by an optical camera. The multiple frames of thermal images correspond to the multiple frames of optical images one by one, and the road area captured by a certain frame of thermal image and its corresponding frame of optical image is the same road area; A segmentation module, configured to perform threshold segmentation on the thermal road map to obtain the suspected defect areas in the thermal road map; A first determination module, configured to determine the surface defect areas in the optical road map based on the texture feature difference between the surface defect areas and the normal road areas in the optical road map, and use the corresponding areas of the suspected defect areas in the thermal road map in the optical road map as the suspected surface defect areas in the thermal road map, where the suspected defect areas in the thermal road map include friction high-temperature areas and real defect areas, and the real defect areas include surface defect areas and internal defect areas; A second determination module, configured to determine the surface defect areas in the thermal road map according to the areas in the optical road map corresponding to the suspected surface defect areas in the thermal road map; A third determination module, configured to determine an internal defect area in the thermal road surface map according to the heat change of consecutive multi-frame thermal images in the thermal road surface map; A fourth determination module, configured to determine that the surface defect area and the internal defect area in the thermal road surface map are the areas where road damage is located; Determining the internal defect area in the thermal road surface map includes: Determining the centroid of other defect areas other than the surface defect area determined in the suspected defect area in the thermal road surface map, where the other defect areas include a friction high-temperature area and an internal defect area; Drawing a horizontal ray through the centroid of the other defect area, and rotating the ray in a counterclockwise direction by an integer multiple of 30° until a 360° rotation is completed, to obtain twelve rays in sequence. The areas where the twelve rays overlap with the other defect area obtain twelve line segments. The gray values of the pixel points corresponding to the twelve line segments on the other defect area form a heat dissipation sequence of twelve defect areas. The heat dissipation sequence satisfies that the first element is the gray value of the pixel point corresponding to the centroid of the other defect area, and the last element is the gray value of the pixel point corresponding to the intersection of the outermost edge of the other defect area and the end of the line segment; Calculating the temporal change factor of the heat dissipation sequence; Based on the temporal change factor of the heat dissipation sequence, determining the temporal change factor of the other defect area; By the method of determining the temporal change factor described above, determining the temporal change factor of the surface defect area in the thermal road surface map; For the same road surface area, projecting the connection line between the centroid of the other defect area and the centroid of the surface defect area in the thermal road surface map onto the road driving direction to obtain a deformation difference segment; For the same road surface area, based on the deformation difference segment, determining the comparison weight between the other defect area and the surface defect area in the thermal road surface map; Based on the temporal change factor of the other defect area, the temporal change factor of the surface defect area in the thermal road surface map, and the comparison weight between the other defect area and the surface defect area in the thermal road surface map, determining the possibility that the other defect area is an internal defect area; Taking the other defect areas with the possibility of the internal defect area being greater than a second threshold as the internal defect areas in the thermal road surface map, to obtain the internal defect areas in the thermal road surface map.

Images