Pavement nondestructive inspection method and system for highway engineering

Abstract

The invention relates to the technical field of image processing, in particular to a pavement nondestructive inspection method and system for highway engineering. The method comprises the following steps: carrying out threshold segmentation on a thermal pavement diagram, and segmenting a suspected defect area; determining the surface defect area based on the texture feature difference of the surface defect area in the optical pavement map; according to the optical pavement diagram, distinguishing a surface defect region in the suspected defect region of the thermal pavement diagram from a region where high temperature is generated by vehicle friction; and according to the heat change of the continuous multi-frame thermodynamic images in the thermodynamic pavement diagram, distinguishing a high-temperature region generated by vehicle friction from an internal defect region to obtain a region where the pavement damage is located. According to the method, the thermal pavement diagram is taken as a main part and the optical pavement diagram is taken as an auxiliary part, the influence of a friction high-temperature area on the accuracy of judging pavement defects through pavement temperature distribution can be eliminated, and factors such as a shooting angle, a shooting distance and inter-frame difference are also considered, so that the accuracy of a pavement damage area is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a road surface nondestructive flaw detection method and system for highway engineering. Background Art

[0002] Nondestructive testing of road surfaces is an important application area of ​​nondestructive testing technology for roadbed and pavement. Nondestructive testing of road surfaces mainly involves performance evaluation of pavement paving materials and monitoring of surface conditions. The thermal image of the pavement area taken by a thermal imager can detect the temperature distribution of the pavement to determine the density of the pavement material and possible gaps and defects.

[0003] However, the heat absorbed by the road surface is not only from sunlight, but also from the friction between the vehicle and the road surface when driving, which makes the friction high temperature area exist in the road surface without defects. Judging the defects in the road surface only by the temperature distribution of the road surface obtained by thermal image may misidentify the normal road surface area as the defective area. Therefore, it is urgent to develop a non-destructive road surface flaw detection method and system for highway engineering to solve the above problems. Summary of the invention

[0004] In order to solve the technical problem that the friction high temperature area affects the accuracy of judging road surface defects through road surface temperature distribution, the purpose of the present invention is to provide a road surface non-destructive flaw detection method and system for highway engineering. The technical scheme adopted is as follows: In a first aspect, the present invention provides a road surface nondestructive flaw detection method for highway engineering, the method comprising: Step 101, obtaining a thermal road surface map and an optical road surface map, wherein the thermal road surface map is a plurality of frames of thermal images of a road surface area taken by a thermal imager, and the optical road surface map is a plurality of frames of optical images of a road surface area taken by an optical camera, wherein the plurality of frames of thermal images correspond to the plurality of frames of optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image are the same road surface area; Step 102, performing threshold segmentation on the thermal road surface map to obtain a suspected defect area in the thermal road surface map; Step 103, based on the difference in texture features between the surface defect area in the optical road surface image and the normal road surface area, the surface defect area in the optical road surface image is determined, and the corresponding area of ​​the suspected defect area in the thermal road surface image in the optical road surface image is used as the suspected surface defect area, wherein the suspected defect area in the thermal road surface image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area; Step 104, determining the surface defect area in the thermal road surface map according to the optical road surface map; Step 105, determining an internal defect area in the thermal road surface map according to the heat change of a plurality of consecutive frames of thermal images in the thermal road surface map; Step 106: Determine the surface defect area and the internal defect area in the thermal road surface map as the road surface damage area.

[0005] In some embodiments, step 104 includes: Step 1041, obtaining an area in the optical road surface image corresponding to the suspected surface defect area in the thermal road surface image; Step 1042, determining the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area based on the eigenvector of the gray level co-occurrence matrix of the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map; Step 1043, in response to the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area is greater than a first threshold, determining the surface defect area in the thermal road surface map.

[0006] In some embodiments, step 105 includes: Step 1051, determining the centroid of other defective areas other than the surface defective areas in the suspected defective areas in the thermal pavement map, wherein the other defective areas include friction high temperature areas and internal defective areas; Step 1052: draw a horizontal ray through the centroid of the other defective area, rotate the ray in a counterclockwise direction by integer multiples of 30° until a rotation of 360° is completed, and obtain twelve rays in sequence. The areas where the twelve rays overlap with the other defective area obtain twelve line segments. The grayscale values ​​of the pixels corresponding to the twelve line segments on the other defective area constitute a heat dissipation sequence of the twelve defective areas. The heat dissipation sequence satisfies that the first element is the grayscale value of the pixel corresponding to the centroid of the other defective area, and the last element is the grayscale value of the pixel corresponding to the intersection of the outermost edge of the other defective area and the end of the line segment. Step 1053, calculating the time series variation factor of the heat dissipation sequence; Step 1054, determining the timing variation factors of the other defective areas based on the timing variation factors of the heat dissipation sequence; Step 1055, determining the time series variation factor of the surface defect area in the thermal pavement map through the method of steps 1051 to 1054; Step 1056, for the same road surface area, projecting the line connecting the centroid of the other defective area and the centroid of the surface defective area in the thermal road surface map to the road driving direction to obtain a deformation difference segment; Step 1057, for the same road surface area, based on the deformation difference, determining a comparison weight between the other defect areas and the surface defect areas in the thermal road surface map; Step 1058, determining the possibility that the other defect area is an internal defect area based on the time series variation factor of the other defect area, the time series variation factor of the surface defect area in the thermal pavement map, and the comparison weight between the other defect area and the surface defect area in the thermal pavement map; Step 1059, taking the other defect areas whose probability of being internal defect areas is greater than the second threshold as internal defect areas in the thermal road surface map, to obtain internal defect areas in the thermal road surface map.

[0007] In some embodiments, the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area is determined according to the following formula: ; In the formula, Indicates the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area, an eigenvector of a gray-level co-occurrence matrix representing a suspected surface defect region in the optical pavement image and the thermal pavement image, The eigenvector of the gray-level co-occurrence matrix representing the normal area in the optical road map, Represents the cosine function.

[0008] In some embodiments, for The first The other defect areas The timing change factor of the heat dissipation sequence is calculated according to the following formula: ; In the formula, Indicates The first The other defect areas The timing variation factor of a heat dissipation sequence, Indicates that it contains The other defect areas The total number of frames of the thermal road map for the cooling sequence, Indicates that it contains The other defect areas The frame number of the thermal road map of the heat dissipation sequence, and Respectively represent Frame and Frame No. 1 in the thermal road map The other defect areas A cooling sequence, Indicates Frame and Frame No. 1 in the thermal road map The other defect areas The DTW distance between the heat dissipation sequences, Indicates the number of cooling sequences.

[0009] In some embodiments, the timing variation factor of the other defective regions is determined according to the following formula: ; In the formula, represents the timing variation factor of other defect areas, Indicates that for The first The other defect areas The angle between the direction of the line segment corresponding to the heat dissipation sequence and the direction of the road travel, represents the sine function, Indicates the total number of cooling sequences.

[0010] In some embodiments, the comparison weights of the other defect areas and the surface defect areas in the thermal pavement map are determined according to the following formula: ; In the formula, represents the comparison weight between the other defective areas and the surface defective areas in the thermal pavement map, represents the length of the deformation difference segment, represents the total length of the road surface area in the thermal road surface map, represents the distance from the midpoint of the deformation difference segment to the lower edge of the road surface area in the thermal road surface map, is an exponential function.

[0011] In some embodiments, the possibility that the other defect regions are internal defect regions is determined according to the following formula: ; In the formula, Indicates The possibility that the other defect areas are internal defect areas, Indicates The temporal variation factor of the surface defect area, Indicates The surface defect area and the The contrast weight between other defect areas, Indicates Timing variation factors for other defect areas, Represents the S-type function, which is used for normalization operation. Indicates the number of other defect areas. Indicates the total number of surface defect areas.

[0012] In some embodiments, in step 102, the thermal road surface map is threshold segmented by using the Otsu threshold method to obtain a suspected defect area in the thermal road surface map.

[0013] In a second aspect, the present invention provides a road surface nondestructive flaw detection system for highway engineering, the system comprising: An acquisition module, used to acquire a thermal road surface map and an optical road surface map, wherein the thermal road surface map is a multi-frame thermal image of a road surface area taken by a thermal imager, and the optical road surface map is a multi-frame optical image of a road surface area taken by an optical camera, wherein the multi-frame thermal images correspond to the multi-frame optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image are the same road surface area; A segmentation module, used to perform threshold segmentation on the thermal pavement map to obtain a suspected defect area in the thermal pavement map; A first determination module is used to determine the surface defect area in the optical road surface image based on the difference in texture features between the surface defect area and the normal road surface area in the optical road surface image, and to use the corresponding area of ​​the suspected defect area in the thermal road surface image in the optical road surface image as the suspected surface defect area, wherein the suspected defect area in the thermal road surface image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area; A second determination module is used to determine the surface defect area in the thermal road surface map according to the optical road surface map; A third determination module is used to determine the internal defect area in the thermal road surface map according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map; The fourth determination module is used to determine that the surface defect area and the internal defect area in the thermal road surface map are the road surface damage areas.

[0014] The present invention has the following beneficial effects: The present invention provides a road surface nondestructive flaw detection method and system for highway engineering. The method first obtains a thermal road surface map and an optical road surface map, so as to facilitate the subsequent comprehensive analysis of the texture characteristics and temperature distribution of the same road surface area; then, threshold segmentation is performed on the thermal road surface map to obtain suspected defect areas in the thermal road surface map, so as to segment out the suspected defect areas that have grayscale value differences with the normal road surface area; further, based on the texture feature difference between the surface defect area in the optical road surface map and the normal road surface area, the surface defect area in the optical road surface map is determined, and the corresponding area of ​​the suspected defect area in the thermal road surface map in the optical road surface map is used as the suspected surface defect area, wherein the suspected defect area in the thermal road surface map includes The method includes friction high temperature area and real defect area, and the real defect area includes surface defect area and internal defect area. The surface defect area in the optical road surface map is the area where the road surface is actually damaged; then, according to the optical road surface map, the surface defect area in the thermal road surface map is determined, so that the surface defect area in the suspected defect area of ​​the thermal road surface map and the area where the vehicle friction generates high temperature are distinguished; according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map, the internal defect area in the thermal road surface map is determined, and the area where the vehicle friction generates high temperature and the internal defect area in the thermal road surface map are distinguished; finally, the surface defect area and the internal defect area in the thermal road surface map are determined as the area where the road surface damage is located. This method is mainly based on the thermal road surface map and supplemented by the optical road surface map. It can eliminate the influence of the friction high temperature area on the accuracy of judging road surface defects through road surface temperature distribution, and also considers factors such as shooting angle, distance and frame difference, thereby improving the accuracy of the road surface damage area. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A method flow chart of a non-destructive flaw detection method for a road surface used in a highway project provided by one embodiment of the present invention; Figure 2 A schematic diagram of the structure of a road surface nondestructive flaw detection system for highway engineering provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] In a first aspect, the present invention provides a road surface nondestructive testing method for highway engineering, see Figure 1 , the method comprising: Step 101, obtaining a thermal road surface map and an optical road surface map. The thermal road surface map is a multi-frame thermal image of a road surface area taken by a thermal imager, and the optical road surface map is a multi-frame optical image of a road surface area taken by an optical camera. The multi-frame thermal images correspond to the multi-frame optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image are the same road surface area.

[0020] Acquire a thermal pavement map and an optical pavement map, thereby obtaining an optical image and a thermal image of the pavement area to be inspected for defects. The optical pavement map can reflect the texture characteristics of the pavement area. The texture characteristics of the normal pavement area and the defective pavement area are different, so as to judge whether there are defects in the pavement area. The optical pavement map is inconvenient to reflect the internal conditions of the pavement area. The thermal pavement map can reflect the temperature distribution of the pavement area. The temperature of the defective pavement area or the area where high temperature is generated by vehicle friction is higher than that of the normal pavement area, while the area where high temperature is generated by vehicle friction is the normal pavement area. Therefore, it is necessary to comprehensively consider the thermal pavement map and the optical pavement map to judge whether there are defects.

[0021] In some embodiments, an optical camera and a thermal imager are installed in front of the vehicle, and the acquisition frequencies of the optical camera and the thermal imager are set to 30 Hz to ensure that each frame of the image taken is of the same road surface area, and the vehicle travels at a constant speed on the road surface, and the driving speed can be 10 km / h. RGB images and infrared images of the road surface are collected, and the RGB images are grayed to obtain an optical road surface map. The temperature values ​​in the infrared image are mapped to a grayscale range of [0, 255] according to the temperature scale in the infrared image to obtain a thermal road surface map.

[0022] When the road surface has damage such as internal defects and surface defects, sunlight will enter the surface defects and be reflected in the surface defects for multiple times, causing the temperature at the surface defects to be higher than that of the normal road surface. The internal defects in the road will not be able to dissipate heat in time due to their poor thermal conductivity, causing the temperature at the internal defects to be higher than that of the normal road surface. That is, the heat absorption efficiency and heat release efficiency at the defective position in the road surface are different from those of the surrounding normal road surface, which is manifested in the thermal road surface map as a difference in the grayscale value at the defective position and the grayscale value of other normal road surfaces.

[0023] Step 102, performing threshold segmentation on the thermal road surface map to obtain suspected defect areas in the thermal road surface map.

[0024] In some embodiments, in step 102, threshold segmentation is performed on the thermal pavement map using the Otsu threshold method to obtain suspected defect areas in the thermal pavement map.

[0025] Threshold segmentation can segment out suspected defect areas that have grayscale value differences from normal road areas, thereby obtaining suspected defect areas in the thermal road surface map.

[0026] Since the heat absorbed by the road surface is not only from sunlight as a heat source, but also from the friction between the vehicle and the road surface when driving, there are friction high temperature areas in the road surface without defects. Judging the defects in the road surface only by obtaining the temperature distribution of the road surface through thermal images may identify the defective area as a normal road surface area, that is, the heat received at the road surface position is different. If the gray value of the thermal road surface image is simply used as the basis for identifying the defect position, the area where the vehicle friction generates high temperature will be identified as the defective area. Therefore, the suspected defective area may be the area where the vehicle friction generates high temperature, or it may be a defective area with damage on the surface and inside.

[0027] In order to distinguish the area where high temperature is generated by vehicle friction from the defective area in the suspected defective area, step 103 is introduced.

[0028] Step 103, based on the difference in texture features between the surface defect area in the optical pavement image and the normal pavement area, the surface defect area in the optical pavement image is determined, and the corresponding area of ​​the suspected defect area in the thermal pavement image in the optical pavement image is used as the suspected surface defect area, wherein the suspected defect area in the thermal pavement image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area.

[0029] Surface defects on the road surface will destroy the original texture structure of the road surface. At the same time, since the defects in the road surface are not regular, that is, the textures of defects in different positions are quite different, while the textures on normal road surfaces are similar, the surface defect area in the optical road surface image can be determined based on the difference in texture features between the surface defect area in the optical road surface image and the normal road surface area. The surface defect area in the optical road surface image is the surface defect area where real damage exists. The corresponding area of ​​the suspected defect area in the thermal road surface image in the optical road surface image is used as the suspected surface defect area, and the texture feature information of the suspected surface defect area in the optical road surface image is obtained, so as to facilitate the subsequent use of the optical road surface image as a supplement to distinguish the area where high temperature is generated by vehicle friction and the defect area in the suspected defect area of ​​the thermal road surface image.

[0030] It should be noted that the surface defect area in the optical pavement map is the area where there is actual damage on the pavement surface, and the non-surface defect area in the optical pavement map is the normal pavement area, which can be used as a standard for subsequent judgment of texture features, thereby determining whether the suspected defect area in the thermal pavement map is a surface defect area.

[0031] In some embodiments, when the difference between the texture features in a certain road surface area and the texture features of all other local road surface areas is greater than a texture feature difference threshold, the road surface area is a surface defect area.

[0032] In some embodiments, the texture feature values ​​of each local road surface area are extracted by performing Fourier transform or wavelet transform, and then the average value of the absolute value of the difference in texture feature values ​​between each local road surface area and all other local road surface areas is obtained. The average value of the absolute value of the difference is the difference between the texture features of each local area of ​​the road surface.

[0033] Since the local high temperature areas caused by tire friction heat in the thermal infrared image will interfere with the normal detection of the road surface, while the optical image will not be affected by friction heat, the thermal road surface map is combined with the optical road surface map for auxiliary detection.

[0034] Step 104, determining the surface defect area in the thermal road surface map according to the optical road surface map.

[0035] The surface defect area in the thermal road map is determined, so that the surface defect area in the suspected defect area of ​​the thermal road map is distinguished from the area where high temperature is generated by vehicle friction.

[0036] In some embodiments, step 104 includes: Step 1041, obtaining the area in the optical road surface image corresponding to the suspected surface defect area in the thermal road surface image, as a basis for subsequent resolution.

[0037] Step 1042, based on the eigenvector of the gray level co-occurrence matrix of the area in the optical pavement image corresponding to the suspected surface defect area in the thermal pavement image, determine the possibility that the area in the optical pavement image corresponding to the suspected surface defect area in the thermal pavement image is a surface defect area.

[0038] The eigenvector of the grayscale co-occurrence matrix of the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map can reflect the texture characteristics of the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map. The possibility that the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map is a surface defect area can reflect the possibility that the area is a surface defect area in the thermal pavement map.

[0039] In some embodiments, the possibility that the area in the optical road surface image corresponding to the suspected surface defect area in the thermal road surface image is a surface defect area is determined according to the following formula: ; In the formula, It indicates the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is the surface defect area, reflecting the possibility that the area is the surface defect area in the thermal road surface map. The eigenvector of the gray-level co-occurrence matrix of the suspected surface defect area in the optical pavement image and the thermal pavement image reflects the texture characteristics of the suspected surface defect area in the optical pavement image and the thermal pavement image. The eigenvector of the gray-level co-occurrence matrix representing the normal area in the optical pavement image reflects the texture characteristics of the normal area in the optical pavement image. represents the cosine function, It can reflect the similarity between the texture features of the normal area in the optical pavement image and the texture features of the suspected surface defect area in the optical pavement image and the thermal pavement image. The larger the difference, the more similar the two are. The smaller, that is The larger it is, the more likely the area is a surface defect area in the thermal pavement map.

[0040] Step 1043, in response to the possibility that the area in the optical road surface image corresponding to the suspected surface defect area in the thermal road surface image is a surface defect area is greater than a first threshold, determining the surface defect area in the thermal road surface image.

[0041] The possibility that the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map is a surface defect area is greater than the first threshold, indicating that the area is very likely to be a surface defect area in the thermal pavement map and can be determined to be a surface defect area in the thermal pavement map.

[0042] In some embodiments, the first threshold may be 0.8.

[0043] In some embodiments, since the thermal pavement map is a multi-frame thermal image of a pavement area taken by a thermal imager, and the optical pavement map is a multi-frame optical image of a pavement area taken by an optical camera, the possibility that the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map is a surface defect area may come from multiple frames, and the average value of the possibility that the area in the optical pavement map corresponding to the suspected surface defect area in the thermal pavement map is a surface defect area from multiple frames can be compared with the first threshold.

[0044] Since there are no defects in the area where high temperature is generated by vehicle friction, there is a large difference between the heat dissipation efficiency and the thermal radiation at the actual defect location in the road surface. Therefore, step 105 is introduced to distinguish the area where high temperature is generated by vehicle friction and the internal defect area in the thermal road surface map.

[0045] Step 105, determining the internal defect area in the thermal road surface map according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map.

[0046] Through the heat changes of multiple consecutive frames of thermal images in the thermal pavement map, the areas where high temperature is generated by vehicle friction and the internal defect areas in the thermal pavement map are distinguished. Since the embodiment of the present invention collects thermal pavement maps and optical pavement maps while the car is driving by installing a thermal imager and an optical camera on the vehicle, each frame of the image may be slightly different from the previous frame. In order to better analyze the suspected defect areas in the thermal pavement map to obtain the internal defect areas in the thermal pavement map, it is necessary to correspond the suspected defect areas in different frames of images. Therefore, the heat changes of multiple consecutive frames of thermal images in the thermal pavement map are analyzed to grasp the information of the same suspected defect area in the thermal images of different frames, eliminate the influence of the changes in the same suspected defect area in the thermal images of different frames, and thus ensure the accuracy of the judgment of the internal defect area in the thermal pavement map.

[0047] In some embodiments, the method for acquiring the same suspected defect area of ​​multiple consecutive frames of thermal images and optical images may be: for the suspected defect area in any frame of thermal image, the corresponding area in the optical image of any frame corresponding to the suspected defect area is acquired. , and then obtain the corresponding area in the optical image of any frame by the streamer matching method The corresponding area in the previous optical image , get the area in the previous thermal image of any frame The corresponding area.

[0048] It should be noted that the optical pavement map can detect areas with surface defects on the road surface. Both the surface defect area and the internal defect area are caused by actual damage in the road surface. Therefore, the heat dissipation efficiency of the surface defect area and the internal defect area in the thermal pavement map is similar, but different from the heat dissipation efficiency of the friction high temperature area. The friction high temperature area and the internal defect area can be distinguished by the difference in heat dissipation efficiency between the suspected defect area in the thermal pavement map and other defect areas determined to be outside the surface defect area.

[0049] In some embodiments, step 105 includes: Step 1051, determining the centroid of other defect areas other than the surface defect areas in the suspected defect areas in the thermal pavement map, wherein the other defect areas include friction high temperature areas and internal defect areas.

[0050] Step 1052: draw a horizontal ray through the centroid of other defective areas, rotate the ray counterclockwise by integer multiples of 30° until 360° is completed, and obtain twelve rays in sequence. The areas where the twelve rays overlap with other defective areas obtain twelve line segments. The grayscale values ​​of the pixels corresponding to the twelve line segments on other defective areas constitute the heat dissipation sequence of the twelve defective areas. The heat dissipation sequence satisfies that the first element is the grayscale value of the pixel corresponding to the centroid of other defective areas, and the last element is the grayscale value of the pixel corresponding to the intersection of the outermost edge of other defective areas and the end of the line segment. The heat dissipation sequence of the twelve defective areas can reflect the grayscale value change of the area from its centroid to the outside, that is, the heat change.

[0051] Step 1053, calculating the time series variation factor of the heat dissipation sequence, thereby obtaining the time series variation of the heat dissipation sequence in a certain other defective area in continuous frames, that is, the heat variation.

[0052] In some embodiments, for The first The other defect areas The timing change factor of the heat dissipation sequence is calculated according to the following formula: ; In the formula, Indicates The first The other defect areas The time series variation factor of the heat dissipation sequence reflects the The first The other defect areas The heat changes of a cooling sequence, Indicates that it contains The other defect areas The total number of frames of the thermal road map for the cooling sequence, Indicates that it contains The other defect areas The frame number of the thermal road map of the heat dissipation sequence, and Respectively represent Frame and Frame No. 1 in the thermal road map The other defect areas A cooling sequence, Indicates Frame and Frame No. 1 in the thermal road map The other defect areas The DTW distance between the heat dissipation sequences can reflect the Frame and Frame No. 1 in the thermal road map The other defect areas The similarity of the heat dissipation sequence, Indicates the number of cooling sequences. The larger the Frame and Frame No. 1 in the thermal road map The other defect areas The smaller the similarity of the heat dissipation sequences, the The bigger, the The first The other defect areas The greater the timing variation of a heat dissipation sequence, the greater the thermal variation.

[0053] It should be noted that the DTW distance is calculated by aligning the elements of two heat dissipation sequences to obtain the best matching distance between them. Unlike the traditional Euclidean distance, DTW allows nonlinear alignment of the time axis, thus better handling the time distortion problem in the heat dissipation sequence, that is, the change problem between different frames.

[0054] Step 1054, based on the timing variation factor of the heat dissipation sequence, determine the timing variation factors of other defective areas, so as to obtain the timing variation conditions of other defective areas.

[0055] In some embodiments, the timing variation factor of other defective regions is determined according to the following formula: ; In the formula, Indicates the timing change factor of other defect areas, reflecting the timing change of other defect areas. Indicates that for The first The other defect areas The angle between the direction of the line segment corresponding to the heat dissipation sequence and the direction of the road travel, represents the sine function, is not zero, Indicates the total number of cooling sequences.

[0056] Since other defect areas are affected by the shooting angle and distance, other defect areas in the thermal pavement map will be compressed and overlapped inward along the two sections of the road driving direction, thereby losing the change characteristics of some road areas; the sequence parallel to the road driving direction is most affected, and the part perpendicular to the road driving direction will not be affected by the shooting angle and distance; therefore, when calculating the time series change factor of other defect areas, it is necessary to give a credibility weight to the angle between the direction of the line segment corresponding to the heat dissipation sequence and the road driving direction, that is, , and thus adjust the credibility weight of the timing change factors of all heat dissipation sequences in other defective areas to obtain the timing change factors of other defective areas When the angle between the direction of the line segment corresponding to the heat dissipation sequence and the direction of road travel is smaller, that is, The smaller it is, the greater the deformation degree of the heat dissipation sequence, the more characteristics of the heat loss change, and the credibility weight The lower the value, the greater the impact on the heat change information of the road surface area collected. The smaller the time, the The smaller.

[0057] Step 1055, determining the time series variation factor of the surface defect area in the thermal pavement map through the method of steps 1051 to 1054.

[0058] Step 1056, for the same road surface area, project the line connecting the centroid of other defective areas and the centroid of the surface defective area in the thermal road surface map to the road driving direction to obtain a deformation difference segment, so as to facilitate the subsequent analysis of the deformation of other defective areas.

[0059] Step 1057: for the same road surface area, based on the deformation difference, determine the comparison weights of other defect areas and the surface defect areas in the thermal road surface map.

[0060] Since the degree to which other defect areas are stretched and scaled is different in the thermal pavement map of different frames, in order to better compare other defect areas with the surface defect areas in the thermal pavement map, it is necessary to combine the deformation difference and give comparison weights to other defect areas and the surface defect areas in the thermal pavement map.

[0061] In some embodiments, the comparison weights of other defect areas and the surface defect areas in the thermal pavement map are determined according to the following formula: ; In the formula, Indicates the comparison weight of other defect areas and surface defect areas in the thermal pavement map, reflecting the size of the comparison weight of other defect areas and surface defect areas in the thermal pavement map. represents the length of the deformation difference segment, It represents the total length of the road surface area in the thermal road surface map and its value is not zero. Indicates the distance from the midpoint of the deformation difference segment to the lower edge of the pavement area in the thermal pavement map, is an exponential function, It can indirectly reflect the position of the deformation difference section in the pavement area in the thermal pavement map.

[0062] When the deformation difference The longer the projection distance of the line segment between the centroid of other defective areas and the centroid of the surface defective area in the thermal road surface map in the road driving direction is, the greater the possibility of deformation. Therefore, the A negative correlation contrast weight, resulting in At the same time, due to the principle that the closer the deformation is, the smaller the deformation is. The farther the deformation difference segment is from the thermal imager, the greater the actual distance corresponding to the deformation difference segment. Therefore, a calculation weight should be assigned according to the position of the deformation difference segment in the road surface area in the thermal road surface map. . The bigger, The larger the The smaller.

[0063] Step 1058, based on the time series variation factors of other defect areas, the time series variation factors of the surface defect areas in the thermal pavement map, and the comparison weights of other defect areas and the surface defect areas in the thermal pavement map, determine the possibility that other defect areas are internal defect areas, thereby obtaining the possibility degree that other defect areas are internal defect areas.

[0064] In some embodiments, the possibility that other defect regions are internal defect regions is determined according to the following formula: ; In the formula, Indicates The possibility that the other defect areas are internal defect areas reflects the The possibility that other defect areas are internal defect areas, Indicates The temporal variation factor of the surface defect area, Indicates The surface defect area and the The contrast weight between other defect areas, Indicates The timing variation factor of other defect areas, Represents the S-type function, which is used for normalization operation. Indicates the number of other defect areas. Indicates the total number of surface defect areas.

[0065] When The temporal variation factor of the surface defect area is Timing variation factors for other defect areas The greater the difference, the The surface defect area and the The greater the difference in heat dissipation efficiency of the other defective areas, the greater the difference in heat dissipation efficiency of the The surface defect area and the The greater the contrast weight between the other defect areas, the The possibility that the other defect areas are internal defect areas The smaller, that is The smaller.

[0066] Step 1059, taking other defect areas whose probability of being internal defect areas is greater than the second threshold as internal defect areas in the thermal road surface map, and obtaining internal defect areas in the thermal road surface map.

[0067] When the possibility that the other defect areas are internal defect areas is greater than the second threshold, it indicates that the other defect areas in the thermal road surface map may be internal defect areas.

[0068] In some embodiments, the second threshold may be 0.9.

[0069] Step 106 , determining the surface defect area and the internal defect area in the thermal road surface map as the road surface damage area.

[0070] The surface defect area and internal defect area in the thermal pavement map are the areas where the pavement damage is located.

[0071] In summary, the present invention provides a non-destructive pavement flaw detection method for highway engineering. The method is based on a thermal pavement map and supplemented by an optical pavement map. It can eliminate the influence of the friction high temperature area on the accuracy of judging pavement defects by pavement temperature distribution, and also takes into account factors such as shooting angle, distance and frame difference, thereby improving the accuracy of the area where the pavement damage is located.

[0072] In a second aspect, the present invention provides a road surface nondestructive flaw detection system for highway engineering, see Figure 2 The system comprises: The acquisition module 201 is used to acquire a thermal road surface map and an optical road surface map. The thermal road surface map is a multi-frame thermal image of a road surface area taken by a thermal imager, and the optical road surface map is a multi-frame optical image of a road surface area taken by an optical camera. The multi-frame thermal images correspond to the multi-frame optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image is the same road surface area.

[0073] The segmentation module 202 is used to perform threshold segmentation on the thermal road surface map to obtain suspected defect areas in the thermal road surface map.

[0074] The first determination module 203 is used to determine the surface defect area in the optical pavement image based on the difference in texture features between the surface defect area in the optical pavement image and the normal pavement area, and to use the corresponding area of ​​the suspected defect area in the thermal pavement image in the optical pavement image as the suspected surface defect area, wherein the suspected defect area in the thermal pavement image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area.

[0075] The second determination module 204 is used to determine the surface defect area in the thermal road surface map according to the optical road surface map.

[0076] The third determination module 205 is used to determine the internal defect area in the thermal road surface map according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map.

[0077] The fourth determination module 206 is used to determine that the surface defect area and the internal defect area in the thermal road surface map are the road surface damage areas.

[0078] In summary, the present invention provides a non-destructive road surface flaw detection method for highway engineering. The system is based on thermal road surface maps and supplemented by optical road surface maps. It can eliminate the influence of friction high temperature areas on the accuracy of judging road surface defects through road surface temperature distribution, and also considers factors such as shooting angle, distance and frame difference, thereby improving the accuracy of the road surface damage area.

[0079] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0080] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A non-destructive testing method for road surface used in highway engineering, characterized in that: The method comprises: Step 101, obtaining a thermal road surface map and an optical road surface map, wherein the thermal road surface map is a plurality of frames of thermal images of a road surface area taken by a thermal imager, and the optical road surface map is a plurality of frames of optical images of a road surface area taken by an optical camera, wherein the plurality of frames of thermal images correspond to the plurality of frames of optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image are the same road surface area; Step 102, performing threshold segmentation on the thermal road surface map to obtain a suspected defect area in the thermal road surface map; Step 103, based on the difference in texture features between the surface defect area in the optical road surface image and the normal road surface area, the surface defect area in the optical road surface image is determined, and the corresponding area of ​​the suspected defect area in the thermal road surface image in the optical road surface image is used as the suspected surface defect area, wherein the suspected defect area in the thermal road surface image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area; Step 104, determining the surface defect area in the thermal road surface map according to the optical road surface map; Step 105, determining an internal defect area in the thermal road surface map according to the heat change of a plurality of consecutive frames of thermal images in the thermal road surface map; Step 106: Determine the surface defect area and the internal defect area in the thermal road surface map as the road surface damage area.

2. The road surface nondestructive flaw detection method for highway engineering according to claim 1, characterized in that: The step 104 includes: Step 1041, obtaining an area in the optical road surface image corresponding to the suspected surface defect area in the thermal road surface image; Step 1042, determining the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area based on the eigenvector of the gray level co-occurrence matrix of the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map; Step 1043, in response to the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area is greater than a first threshold, determining the surface defect area in the thermal road surface map.

3. The road surface nondestructive flaw detection method for highway engineering according to claim 2, characterized in that: The step 105 includes: Step 1051, determining the centroid of other defective areas other than the surface defective areas in the suspected defective areas in the thermal pavement map, wherein the other defective areas include friction high temperature areas and internal defective areas; Step 1052: draw a horizontal ray through the centroid of the other defective area, rotate the ray in a counterclockwise direction by integer multiples of 30° until a rotation of 360° is completed, and obtain twelve rays in sequence. The areas where the twelve rays overlap with the other defective area obtain twelve line segments. The grayscale values ​​of the pixels corresponding to the twelve line segments on the other defective area constitute a heat dissipation sequence of the twelve defective areas. The heat dissipation sequence satisfies that the first element is the grayscale value of the pixel corresponding to the centroid of the other defective area, and the last element is the grayscale value of the pixel corresponding to the intersection of the outermost edge of the other defective area and the end of the line segment. Step 1053, calculating the time series variation factor of the heat dissipation sequence; Step 1054, determining the timing variation factors of the other defective areas based on the timing variation factors of the heat dissipation sequence; Step 1055, determining the time series variation factor of the surface defect area in the thermal pavement map through the method of steps 1051 to 1054; Step 1056, for the same road surface area, projecting the line connecting the centroid of the other defective area and the centroid of the surface defective area in the thermal road surface map to the road driving direction to obtain a deformation difference segment; Step 1057, for the same road surface area, based on the deformation difference, determining a comparison weight between the other defect areas and the surface defect areas in the thermal road surface map; Step 1058, determining the possibility that the other defect area is an internal defect area based on the time series variation factor of the other defect area, the time series variation factor of the surface defect area in the thermal pavement map, and the comparison weight between the other defect area and the surface defect area in the thermal pavement map; Step 1059, taking the other defect areas whose probability of being internal defect areas is greater than the second threshold as internal defect areas in the thermal road surface map, to obtain internal defect areas in the thermal road surface map.

4. The road surface nondestructive flaw detection method for highway engineering according to claim 2, characterized in that: The possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area is determined according to the following formula: ； In the formula, Indicates the possibility that the area in the optical road surface map corresponding to the suspected surface defect area in the thermal road surface map is a surface defect area, an eigenvector of a gray-level co-occurrence matrix representing a suspected surface defect region in the optical pavement image and the thermal pavement image, represents the eigenvector of the gray level co-occurrence matrix of the normal area in the optical road map, Represents the cosine function.

5. The road surface nondestructive flaw detection method for highway engineering according to claim 3, characterized in that: For The first The other defect areas The timing change factor of the heat dissipation sequence is calculated according to the following formula: ； In the formula, Indicates The first The other defect areas The timing variation factor of a heat dissipation sequence, Indicates that it contains The other defect areas The total number of frames of the thermal road map for the cooling sequence, Indicates that it contains The other defect areas The frame number of the thermal road map of the heat dissipation sequence, and Respectively represent Frame and Frame No. 1 in the thermal road map The other defect areas A cooling sequence, Indicates Frame and Frame No. 1 in the thermal road map The other defect areas The DTW distance between the heat dissipation sequences, Indicates the number of cooling sequences.

6. The road surface nondestructive flaw detection method for highway engineering according to claim 5, characterized in that: The timing variation factor of the other defective areas is determined according to the following formula: ； In the formula, represents the timing variation factor of other defect areas, Indicates that for The first The other defect areas The angle between the direction of the line segment corresponding to the heat dissipation sequence and the direction of the road travel, represents the sine function, Indicates the total number of cooling sequences.

7. The road surface nondestructive flaw detection method for highway engineering according to claim 6, characterized in that: The comparison weights of the other defect areas and the surface defect areas in the thermal pavement map are determined according to the following formula: ； In the formula, represents the comparison weight between the other defective areas and the surface defective areas in the thermal pavement map, represents the length of the deformation difference segment, represents the total length of the road surface area in the thermal road surface map, represents the distance from the midpoint of the deformation difference segment to the lower edge of the road surface area in the thermal road surface map, is an exponential function.

8. The road surface nondestructive flaw detection method for highway engineering according to claim 7, characterized in that: The possibility that the other defective areas are internal defective areas is determined according to the following formula: ； In the formula, Indicates The possibility that the other defect areas are internal defect areas, Indicates The temporal variation factor of the surface defect area, Indicates The surface defect area and the The contrast weight between other defect areas, Indicates Timing variation factors for other defect areas, Represents the S-type function, which is used for normalization operation. Indicates the number of other defect areas. Indicates the total number of surface defect areas.

9. The road surface nondestructive flaw detection method for highway engineering according to claim 1, characterized in that: In step 102, threshold segmentation is performed on the thermal road surface map using the Otsu threshold method to obtain suspected defect areas in the thermal road surface map.

10. A road surface nondestructive flaw detection system for highway engineering, characterized in that: The system comprises: An acquisition module, used to acquire a thermal road surface map and an optical road surface map, wherein the thermal road surface map is a multi-frame thermal image of a road surface area taken by a thermal imager, and the optical road surface map is a multi-frame optical image of a road surface area taken by an optical camera, wherein the multi-frame thermal images correspond to the multi-frame optical images one by one, and the road surface area taken by a certain frame of thermal image and its corresponding frame of optical image are the same road surface area; A segmentation module, used to perform threshold segmentation on the thermal pavement map to obtain a suspected defect area in the thermal pavement map; A first determination module is used to determine the surface defect area in the optical road surface image based on the difference in texture features between the surface defect area and the normal road surface area in the optical road surface image, and to use the corresponding area of ​​the suspected defect area in the thermal road surface image in the optical road surface image as the suspected surface defect area, wherein the suspected defect area in the thermal road surface image includes a friction high temperature area and a real defect area, and the real defect area includes a surface defect area and an internal defect area; A second determination module is used to determine the surface defect area in the thermal road surface map according to the optical road surface map; A third determination module is used to determine the internal defect area in the thermal road surface map according to the heat change of multiple consecutive frames of thermal images in the thermal road surface map; The fourth determination module is used to determine that the surface defect area and the internal defect area in the thermal road surface map are the road surface damage areas.

Images