An automated data acquisition system for highway engineering inspection

Abstract

The present application discloses an automated data acquisition system for highway engineering inspection. It belongs to the technical field of data processing. An automated data acquisition system for highway engineering inspection includes a movable workbench; an image information collection device for collecting road surface information; a radar detection device for collecting radar scanning information below the road surface; an information fusion device for collecting road surface information and radar scanning information and fusing the road surface information with the radar information to obtain fusion information; and a correlation evaluation device for collecting each piece of fusion information and calculating the quality score of the road surface based on the correlation between the road surface information and the radar information in the fusion information. In the technical solution provided by the present application, not only road surface information is collected and used to evaluate the construction quality of highway engineering. Instead, road surface information and radar scanning information are collected and mutually fused. Therefore, in this way, the construction quality of the road surface can be accurately evaluated.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and more specifically, to an automated data acquisition system for highway engineering inspection. Background Art

[0002] Highway acceptance is an important link for comprehensively reviewing the entire process of highway project construction to ensure that the project has been completed in accordance with relevant standards, specifications, and contract requirements and is ready for use. During highway acceptance, it is necessary to accurately evaluate the construction quality of the highway, and for this purpose, it is necessary to collect the inspection data of the highway.

[0003] Currently, in the highway engineering inspection system, generally, drones are used to patrol the ground to obtain the picture information of the highway pavement, and it is judged whether there are damages on the pavement according to the pictures, so as to complete the data collection work.

[0004] This method cannot accurately evaluate the actual construction quality of the project. Especially when the construction period is tight, the pavement is often laid urgently without ramming the roadbed, and then the pavement is made flat by adjusting the thickness of the asphalt (pavement). Therefore, when inspecting the pavement flatness for acceptance, the construction quality of the roadbed cannot be accurately evaluated. Summary of the Invention

[0005] The content part of this application is used to briefly introduce the concepts, which will be described in detail in the subsequent specific implementation part. The content part of this application is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] As the first aspect of this application, in order to solve the technical problem of the low standard of highway quality evaluation, an automated data acquisition system for highway engineering inspection is provided, including:

[0007] A movable workbench;

[0008] An image information collection device for collecting pavement information;

[0009] A radar detection device for collecting radar scanning information under the pavement;

[0010] An information fusion device that collects pavement information and radar scanning information and fuses the pavement information with the radar information to obtain fusion information;

[0011] A correlation evaluation device that collects each fusion information and calculates the quality score of the pavement based on the correlation between the pavement information and the radar information in the fusion information.

[0012] In the technical solution provided by this application, it not only collects road surface information and uses the road surface information to evaluate the construction quality of highway projects. Instead, it will collect road surface information and radar scanning information, and fuse the road surface information and radar scanning information. If there is a reasonable correlation between the road surface information and the radar scanning information, for example, the road surface above the hollow foundation is flat, or the roadbed bulges significantly upward, but the road surface does not show the corresponding density change. This is obviously caused by directly laying the road surface with asphalt after the foundation is not flattened. Therefore, in this way, the construction quality of the road surface can be accurately evaluated.

[0013] When collecting road surface information, if image processing technology is used to obtain the undulation of the road surface, it will consume a large amount of computing power and the accuracy is not high. Therefore, this application provides the following technical solution:

[0014] Furthermore, the image information collection device includes:

[0015] A camera, which is set with a fixed focal length to obtain image information within a fixed range;

[0016] A plurality of displacement sensors, which are used to calculate the closest distance between the camera and the ground to obtain distance information;

[0017] An image information generation unit, which obtains the image information and the distance information, adds the distance information to the image information to obtain the distance between each position in the image information and the camera, and generates road surface information.

[0018] In the technical solution provided by this application, the camera is used to obtain image information with a fixed focal length, and the distance between the camera and the image is also obtained. Therefore, not only can road surface information be obtained, but also the undulation of the road surface can be obtained, and further road surface information in more dimensions can be obtained.

[0019] Furthermore, the displacement sensors are arranged in an array, and the displacement sensors cover the width range of the camera during each photographing.

[0020] In this solution, only by continuously moving the movable platform, the undulation information of the road surface can be obtained.

[0021] Furthermore, the radar detection device includes:

[0022] A radar detection unit, which is used to emit detection signals downward and obtain a detection fault image below the radar;

[0023] A fault image preprocessing unit, which performs denoising processing on the detection fault image and obtains the fault information intercepted from the ground to a preset depth to obtain a fault picture;

[0024] The tomographic image fusion unit obtains tomographic images of each area and fuses all the tomographic images into radar scan information.

[0025] In the technical solution provided by this application, after obtaining the detected tomographic image, preprocessing is performed, so that the result of radar detection can be more accurately reflected. And because a preset depth is set, all the foundation information can be analyzed at the same level.

[0026] The acquisition methods of road surface information and the three-dimensional picture array are not synchronized. For this reason, the following technical solution is provided in this application:

[0027] Furthermore, the movable workbench further includes a control device, and the control device performs the following steps:

[0028] Step 1: The control device obtains the fixed range of the image information acquired by the camera to obtain discrete displacements;

[0029] Step 2: The control device controls the camera to obtain a clear image information;

[0030] Step 3: The control device controls the movable platform to move at a constant speed by discrete displacements; when the control device controls the movable platform to move at a constant speed, it controls the displacement sensor to obtain distance information and controls the radar detection device to obtain radar scan information.

[0031] In this way, the movable workbench in this solution can stably obtain the required road surface information and radar scan information.

[0032] Although the current high-precision radar has a high detection accuracy, relatively speaking, its imaging is still relatively rough, and the overall detection accuracy is not high, and it is easy to have the problem of missing feature information. For this reason, the following technical solution is provided in this application:

[0033] Furthermore, the information fusion device includes:

[0034] The radar information extraction unit extracts multi-scale information from each tomographic image in the radar scan information;

[0035] The road surface information regression unit regresses the multi-scale information into the foundation characterization information to generate road surface implicit information;

[0036] The siamese network unit obtains the correlation between the road surface information and the road surface implicit information to generate fusion information.

[0037] In the technical solution provided by this application, multi-scale information is extracted from radar scan information, which can extract more useful information from the radar scan information and avoid missing features. Moreover, the road surface information is regressed into the foundation characterization information to generate the characterization relationship between the road surface information and the foundation information, and then a siamese network unit is used for comparison. In this way, the correlation between the radar scan information and the road surface information can be accurately compared, and it can be accurately judged whether there is a reasonable technical correlation between the foundation construction and the road surface construction.

[0038] Generally speaking, although the obtained radar information has sufficient accuracy, due to the complexity of the electromagnetic wave reflection on the ground, there is a large amount of redundant information in the finally received and processed radar information. These redundant information affect each other, thereby reducing the connection between features in the tomographic image and making it difficult to find the connection between features. Therefore, this application provides the following technical solutions:

[0039] Furthermore, the radar information extraction unit includes:

[0040] An input layer for inputting the tomographic image X;

[0041] A preliminary feature extraction layer that performs preliminary feature extraction on the tomographic image X to obtain preliminary features F`;

[0042] F` = Conv 3×3 (X);

[0043] A multi-scale feature extraction layer that divides the preliminary features F` into three groups, namely F 1 `, F 2 `, F 3 `;

[0044] Perform feature extraction on F 1 ` using a standard 3×3 convolution Conv 3×3 to obtain the first feature map F 1 ;

[0045] F 1 = Conv 3×3 (F 1 `);

[0046] Perform multi-scale feature extraction on F 2 ` using the DRB module to obtain the second feature map F 2 ;

[0047] F 2 = DRB k1 (F 2 `);

[0048] Perform on F 3`The third feature map F is obtained by using the DRB module for multi-scale feature extraction 3 ;

[0049] F 3 = DRB k2 (F 3 `); DRB k1 and DRB k2 are the sizes of the DRB module respectively;

[0050] Feature fusion layer, which splices the first feature map F 1 , the second feature map F 2 , and the third feature map F 3 to obtain F fu ,

[0051] F fu = Concat(F 1 , F 2 , F 3 );

[0052] Downsampling processing layer, which performs feature learning on F fu to reduce the resolution of F fu and obtain feature information;

[0053] Fusion enhancement layer, which fuses and enhances the feature information to generate multi-scale information of the corresponding regions in each feature map;

[0054] Among them, the multi-scale information is the density and material of each region, and the material includes at least the foundation and road surface dressing.

[0055] In this solution, for the input fault image, after convolution processing to roughly reduce the image resolution, the preliminary feature map is divided into three groups. One group is further processed by convolution, and the other two groups are processed by the DRB module. The DRB module has a larger receptive field than the ordinary convolution module, so it can capture the feature dependence relationship at a farther distance. Thus, the three groups of features use different receptive fields for feature extraction and are finally fused, so that the features of the obtained feature map have a closer connection, and it is easier to find the connection between the features and the material and density in the fusion enhancement layer.

[0056] When performing feature grouping, if directly grouping according to the regions of the feature groups, for example, dividing the preliminary feature map into 3 different regions, during feature fusion, the features obtained from different regions lack continuity. For example, dividing an image into the upper part, the middle part, and the lower part, and using different convolution kernels for different parts to form the feature system will essentially result in any one region being able to represent only one way of feature division. As a result, the finally extracted features are only locally effective and not globally effective. To address this problem, the present application provides the following method:

[0057] Further,[[]]

[0058] F 1 `(i, j) = ∑ m，n X(2i + m, 2j + n)Ø(m, n)Ø(m, n);

[0059] F 2 `(i, j) = αF 2 `(i, j)` + βF 2 `(i, j)``;

[0060] F 2 `(i, j)` = ∑ m，n X(2i + m, 2j + n)Ø(m, n)ω(m, n);

[0061] F 2 `(i, j)`` = ∑ m，n X(2i + m, 2j + n)ω(m, n)Ø(m, n);

[0062] α + β = 1;

[0063] F 3 `(i, j) = ∑ m，n X(2i + m, 2j + n)ω(m, n)ω(m, n); i and j respectively represent the abscissa and ordinate in F1`, F2`, F3`, F1`(i, j) represents the eigenvalue at (i, j) in F1`, Ø represents the coefficient of the high-pass filter for extracting high-frequency components, ω represents the coefficient of the low-pass filter for extracting low-frequency components, α and β are weighted average coefficients for controlling the relative contribution after merging in the middle frequency band, F2`(i, j) represents the eigenvalue at (i, j) in F2`, F3`(i, j) represents the eigenvalue at (i, j) in F3`, F2`(i, j)` represents the first intermediate frequency component, F2`(i, j)`` represents the second intermediate frequency component, (i, j)` represents the coordinate of the first intermediate frequency component, (i, j)`` represents the coordinate of the second intermediate frequency component, X represents the input tomographic image, m represents the index of the filter in the vertical direction, and n represents the index of the filter in the horizontal direction.

[0064] In the technical solution provided by this application, when dividing the initial features, wavelet transform is used to divide the features into a low-frequency part, a middle-frequency part, and a high-frequency part respectively. Therefore, relatively speaking, F 1 `, F 2 `, F 3 ` have the same overall range, only the contribution ranges at different frequencies are highlighted. Thus, by adopting this solution, after extracting and fusing F 1 `, F 2 `, F 3 `, the extracted features will not be only locally effective. Moreover, in this solution, the frequencies of F 1 `, F 2 `, F 3 ` gradually increase. Therefore, relatively speaking, the change situation of the features in F 1 `, F 2 `, F 3 ` is from drastic to mild. Therefore, for F 1 `, using convolutional operation can highlight the feature change situation in the local area, while in F 2 ` and F 3 `, the DRB module can be used to analyze the image change situation in the low-frequency part with a larger receptive field.

[0065] Before the fusion enhancement layer processes the features, it is necessary to use the downsampling processing layer to reduce the resolution to improve the enhancement efficiency. Since the tomographic image X has undergone initial feature extraction and multi-scale feature extraction, the redundant information is limited. If the Conv module is directly used for downsampling, information loss will occur. Therefore, this application adopts the following solution:

[0066] Furthermore, the downsampling processing layer slices F fu in the spatial dimension at a ratio of 2 and divides it into 4 sub-regions of equal size. Among them, F fu = (S, S, C), where S is the size of F fu and C is the number of channels of F fu ;

[0067] The size of each sub-region is s×s, s = 1 / 2(S), where s is the side length of the downsampling processing layer;

[0068] Each sub-region is concatenated along the channel dimension to obtain the concatenated feature map F fu `, F fu ` = (s, s, 4C);

[0069] Perform a standard 3×3 convolution on the concatenated feature map F fu ` to obtain the feature information.

[0070] In this solution, downsampling of the feature map in the spatial dimension is achieved through slicing, splicing, and convolution operations, and the computational load is reduced by controlling the number of output channels of the convolutional layer. This processing method not only retains the important information of the feature map but also improves the computational efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings forming a part of this application are used to provide a further understanding of this application, making other features, objectives, and advantages of this application more obvious. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application.

[0072] In addition, throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the elements and components are not necessarily drawn to scale.

[0073] In the drawings:

[0074] Figure 1 is a schematic structural diagram of an automated data acquisition system for highway engineering inspection.

[0075] Figure 2 is a radar detection map obtained by a GPR radar. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0076] Embodiments of this application will be described in more detail below with reference to the drawings. Although some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand this application. It should be understood that the drawings and embodiments of this application are only for exemplary purposes and are not used to limit the protection scope of this application.

[0077] In addition, it should be noted that only parts related to the relevant invention are shown in the drawings for the sake of convenience of description. Without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0078] This application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0079] Refer to Figures 1-2, an automated data acquisition system for highway engineering detection includes: a movable workbench, an image information collection device, a radar detection device, an information fusion device, and a correlation evaluation device. Among them, the movable platform is a vehicle with moving ability, such as an engineering vehicle for construction. The radar detection device is a common GPR radar, mounted on the movable platform. After emitting electromagnetic waves downward, it can conduct underground detection based on the returned electromagnetic wave information. The image information collection device collects road surface information and radar scan information, and fuses the road surface information with the radar information to obtain fused information. The correlation evaluation device collects each fused information and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fused information.

[0080] In this solution, in fact, the movable platform carries the image collection device and the radar detection device to move along the road, collecting road surface information and radar scan information. The road surface information is used to characterize the undulation and flatness of the road surface, and the radar scan information is used to characterize the compaction of the foundation. Then, after fusing the road surface information and the radar scan information, the final quality score is obtained. The quality score in this solution also examines the relationship between the road surface laying situation and the foundation. If the correlation between the road surface and the foundation is high, it indicates that the detection result is reliable. If the correlation between the two is poor, it indicates that there are major problems in the construction process, such as laying the road surface without completing the treatment of the foundation, or illegally flattening the road surface instead of flattening the foundation for construction.

[0081] Specifically: The image information collection device includes: a camera, a displacement sensor, and an image information generation unit. The camera is set with a fixed focal length to obtain image information within a fixed range; the camera is located directly below the movable platform, so the camera can obtain a standard picture in one shot. And because the height of the movable platform is fixed, plus the fixed focal length of the camera, the length of the corresponding real world in each picture obtained by the camera is equal. That is, each picture obtains image information within a fixed range. Then, the movable platform only needs to use the width of this range as the moving scale and move such a long distance each time to obtain all the road surface information. Of course, each picture obtained by the camera is processed separately.

[0082] The displacement sensor is provided with multiple ones, used to calculate the closest distance between the camera and the ground to obtain distance information. So when the movable platform moves, the displacement sensor can obtain the distances of each area within the range that the camera can photograph from the movable platform, and then use this distance to characterize the undulation of the road surface.

[0083] Specifically, the movable workbench further includes a control device, and the control device performs the following steps:

[0084] Step 1: The control device obtains a fixed range of the image information acquired by the camera to obtain discrete displacements.

[0085] Step 2: The control device controls the camera to acquire a clear image information.

[0086] Step 3: The control device controls the movable platform to move at a constant speed by the discrete displacement; when the control device controls the movable platform to move at a constant speed, it controls the displacement sensor to acquire distance information and controls the radar detection device to acquire radar scan information.

[0087] Substantially, it is that the control device will control the movable platform to take a picture when it moves to a position. The visual field range of this picture is the fixed range. Then the movable platform will move at a constant speed when it moves the length of the fixed range, so that the distance between each point on the road surface and the movable platform can be obtained. Furthermore, according to the length of the distance, the undulation change of the road surface within the fixed range can be roughly described.

[0088] The image information generation unit obtains the image information and the distance information, adds the distance information to the image information to obtain the distance between each position in the image information and the camera, and generates road surface information.

[0089] The addition method here is to mark the distances of each area in the image information, and judge the flatness of the road surface according to the more precise distances.

[0090] In practice, this solution is only used to detect the relationship between the flatness of the road surface and the distance. Therefore, when evaluating, the pixel values in the image information can be directly converted into distance information, and then a distance matrix can be obtained.

[0091] The above is the acquisition method of the road surface information. In this solution, the road surface information is actually a distance matrix, which describes the height information of the road surface from the movable device within the distance matrix, so it actually characterizes the undulation of the corresponding road surface. The original image information can directly use the feature recognition model of the existing technology to identify the features of road surface damage, that is, as an evaluation index in the correlation evaluation device. This part can be directly processed by the existing technology. For example, the SIFT algorithm is used to extract the image features of the damaged position, and then the damaged positions are compared. If there are many similar features, it means that the road surface is severely damaged. The key of this solution is to calculate the correlation between the road surface information and the radar detection information.

[0092] Specifically, the radar detection device includes: a radar detection unit, a tomographic image preprocessing unit, and a tomographic image fusion unit. Among them, the radar detection unit is used to emit detection signals downward to obtain tomographic images below the radar. The tomographic image preprocessing unit performs denoising processing on the detected tomographic images and obtains tomographic information intercepted from the ground to a preset depth to obtain tomographic pictures; the tomographic picture fusion unit obtains tomographic pictures of each area and fuses all the tomographic pictures into radar scan information.

[0093] The vertical tomographic pictures of each area below the ground are obtained by the radar detection device. For each tomographic picture, it shows the scanning condition below the ground at the current position.

[0094] Therefore, it is necessary to first extract features from each tomographic picture to obtain key information, and then perform subsequent fusion and comparison work after obtaining the key information.

[0095] Specifically, the information fusion device includes:

[0096] A radar information extraction unit that extracts multi-scale information from each tomographic picture in the radar scan information;

[0097] A road surface information regression unit that regresses the multi-scale information into the foundation characterization information to generate road surface implicit information;

[0098] A siamese network unit that obtains the correlation between the road surface information and the road surface implicit information to generate fusion information.

[0099] In this solution, the multi-scale information is actually the information after fusing each tomographic picture. For example, after obtaining the road surface information within a range of 3m×3m, it is necessary to obtain all the tomographic pictures within this range, compress the tomographic picture group actually represented by 3D information into 2D information, and then input it together with the road surface implicit information into the siamese network unit for the calculation of fusion information.

[0100] Therefore, it is necessary to first extract key features from the tomographic pictures. Specifically:

[0101] The radar information extraction unit includes:

[0102] An input layer for inputting the tomographic picture X;

[0103] A preliminary feature extraction layer that performs preliminary feature extraction on the tomographic picture X to obtain preliminary features F`;

[0104] F` = Conv 3×3 (X);

[0105] A multi-scale feature extraction layer that divides the preliminary features F` into three groups, namely F 1 `, F2 `, F 3 `;

[0106] F 1 `(i, j) = ∑ m，n X(2i + m, 2j + n)Ø(m, n)Ø(m, n);

[0107] F 2 `(i, j) = αF 2 `(i, j)` + βF 2 `(i, j)``;

[0108] F 2 `(i, j)` = ∑ m，n X(2i + m, 2j + n)Ø(m, n)ω(m, n);

[0109] F 2 `(i, j)`` = ∑ m，n X(2i + m, 2j + n)ω(m, n)Ø(m, n);

[0110] α + β = 1;

[0111] F 3 `(i, j) = ∑ m，n X(2i + m, 2j + n)ω(m, n)ω(m, n);

[0112] i and j respectively represent the abscissa and ordinate in F1`, F2`, F3`. F1`(i, j) represents the eigenvalue at (i, j) in F1`. Ø represents the coefficient of the high-pass filter for extracting high-frequency components. ω represents the coefficient of the low-pass filter for extracting low-frequency components. α and β are weighted average coefficients for controlling the relative contributions after merging in the middle frequency band. F2`(i, j) represents the eigenvalue at (i, j) in F2`. F3`(i, j) represents the eigenvalue at (i, j) in F3`. F2`(i, j)` represents the first intermediate frequency component. F2`(i, j)`` represents the second intermediate frequency component. (i, j)` represents the coordinates of the first intermediate frequency component. (i, j)`` represents the coordinates of the second intermediate frequency component. X represents the input tomographic image. m represents the index of the filter in the vertical direction. n represents the index of the filter in the horizontal direction.

[0113] For F 1 `, use the standard 3×3 convolution Conv 3×3 to perform feature extraction to obtain the first feature map F 1 ;

[0114] F 1 = Conv 3×3 (F 1 `);

[0115] For F 2 `Use the DRB module to perform multi-scale feature extraction to obtain the second feature map F 2 ;

[0116] F 2 = DRB k1 (F 2 `);

[0117] For F 3 `Use the DRB module to perform multi-scale feature extraction to obtain the third feature map F 3 ;

[0118] F 3 = DRB k2 (F 3 `); DRB k1 and DRB k2 are the sizes of the DRB module respectively;

[0119] Feature fusion layer, which concatenates the first feature map F 1 , the second feature map F 2 , and the third feature map F 3 to obtain F fu ,

[0120] F fu = Concat(F 1 , F 2 , F 3 );

[0121] Downsampling processing layer, which performs feature learning on F fu to reduce the resolution of F fu to obtain feature information;

[0122] The downsampling processing layer slices F fu in the spatial dimension at a ratio of 2 and divides it into 4 sub-regions of equal size. Among them, F fu = (S, S, C), where S is the size of F fu , and C is the number of channels of F fu ;

[0123] The size of each sub-region is s×s, s = 1 / 2(S), where s is the side length of the downsampling processing layer;

[0124] Concatenate each sub-region along the channel dimension to obtain the concatenated feature map F fu `, F fu ` = (s, s, 4C);

[0125] For the concatenated feature map F fuPerform a standard 3×3 convolution to obtain feature information.

[0126] The fusion enhancement layer fuses and enhances the feature information to generate multi-scale information for corresponding regions in each feature map.

[0127] Among them, the multi-scale information is the density and material of each region, and the material includes at least the foundation and road surface dressing.

[0128] Therefore, the feature map finally extracted by the fusion enhancement layer in this solution will represent the multi-scale information of the corresponding region, that is, the density and material at the corresponding position.

[0129] In this way, only by fusing and normalizing the density and material and representing them with an array can it be reduced from two dimensions to one dimension.

[0130] For example, in the feature map, if it is calculated that there is a hollow below, its label is defined as 0, and if the density below is less than a certain threshold, it is defined as 1, and so on. Then, the two-dimensional feature map can be reduced to a one-dimensional sequence, and then all the two-dimensional feature maps are combined together to obtain the matrix information corresponding to the road surface information.

[0131] The above is the process of the road surface information regression device generating the hidden road surface information.

[0132] Specifically, the road surface information regression device includes an information stripping unit and an information regression unit. The information stripping unit strips the foundation information according to the material information.

[0133] Because the dressing materials of the foundation and the road surface are completely different, the foundation and the road surface dressing can be directly distinguished. Then, the foundation information of the thickness change and density change of the foundation is obtained.

[0134] The information regression unit is to regress the foundation information to the hidden road surface information. In fact, the information regression unit is a pre-trained neural network model, whose input is the foundation information under standard construction conditions and the label is the road surface information.

[0135] That is, if during the road surface construction, the normal construction process is adopted for paving the road surface, and if the foundation is not tamped, then during the road surface compaction, due to the inconsistent density of the foundation, the road surface will show the characteristic of unevenness.

[0136] Therefore, the information regression unit is used to find the correlation between the foundation information and the road surface information under standard construction conditions. In this way, after inputting the foundation information into the information regression unit, the road surface information under ideal conditions can be roughly obtained. Although this information is not the actual road surface information, it represents the possible construction situation of the road surface under normal procedures.

[0137] The model built into the information regression unit can be an LSTM. When its input and output are completely determined, the specific model structure and training method are not described here.

[0138] In this solution, the reason for using an LSTM is that it can solve the problems of gradient disappearance and gradient explosion in the training of traditional RNNs. By introducing a gating mechanism, the model can autonomously select which information needs to be retained and which needs to be forgotten. Thus, when processing adjacent matrices in the ground information, it can accurately judge the connection and dependence relationship between adjacent information, and accurately use the ground information to restore the hidden information of the road surface.

[0139] The siamese network unit obtains the correlation between the road surface information and the hidden information of the road surface to generate fusion information.

[0140] The fusion information is actually used to describe the correlation between the road surface information and the hidden information of the road surface. If the correlation between the two is high, it means that the credibility of the construction process is high; otherwise, it means that the credibility of the construction process is poor. The hidden information of the road surface and the road surface information are essentially matrix representations of the road surface undulation, with the same data structure and data type. Therefore, a siamese network unit is used for correlation evaluation. This is more accurate than directly using the similarity calculation method.

[0141] Specifically:

[0142] The information regression unit includes two sub-network groups (CNNs) with the same structure and weight sharing. The road surface information and the hidden information of the road surface are respectively input into the two sub-networks. (Of course, preprocessing may be required before input). Then, the similarity score between the two input data is output.

[0143] For example, the road surface information is normalized to generate a standard distance matrix H related to the road surface undulation 1 ;

[0144] Then the hidden information of the road surface is normalized to generate a hidden distance matrix H 2 ;

[0145] Then the standard distance matrix H 1 and the hidden distance matrix H 2 are respectively input into the two sub-networks (CNNs). Each sub-network will independently process the standard distance matrix H 1 and the hidden distance matrix H 2 , extract features through the convolutional layer of the sub-network, then generate feature vectors through the fully connected layer of the sub-network, and then the output layer calculates the similarity of the feature vectors output by the two sub-networks.

[0146] For example: the feature vector of the standard distance matrix H 1 is A;

[0147] Implicit distance matrix H 2 The eigenvector of is B;

[0148]

[0149] The output layer calculates the Euclidean distance d between the two eigenvectors and uses d as the similarity;

[0150] ; h and q are the horizontal and vertical coordinate indices of the eigenvector elements respectively; A hq represents the eigenvalue at the coordinate (h, q) in the eigenvector A, and B hq represents the eigenvalue at the coordinate (h, q) in the eigenvector B; The similarity d between the example eigenvector A and the eigenvector B is approximately 16.55.

[0151] In this way, in this solution, two input data are processed by two sub-networks with the same structure and weight sharing, and the similarity score between them is output. During the training process, the Siamese network uses a specific loss function to optimize the network parameters, so that the network can more accurately judge the similarity between the input data.

[0152] The above is the specific structure in the information regression unit, and its model performance is mainly related to the selected sub-network group. However, in order to increase the accuracy, the present application provides the following loss function.

[0153] L = max(d(a, p) - d(a, n) + margin, 0)

[0154] L is the loss value;

[0155] d(a, p) represents the distance between the anchor point a and the positive sample p;

[0156] d(a, n) represents the distance between the anchor point a and the negative sample;

[0157] margin is a hyperparameter used to control the minimum gap between the distances of the positive and negative samples,

[0158] max(a, p) means taking the larger value between the value in the parentheses and 0 to ensure that the loss value is non-negative.

[0159] By introducing positive and negative samples, this loss function enables the CNN network to learn the dependence relationship between spaces, so that similar samples are pulled closer and different samples are pushed farther apart, thereby effectively increasing the discrimination ability of the model.

[0160] The correlation evaluation device collects each piece of fusion information and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fusion information. Of course, when calculating the correlation, the flatness of the road surface and the compaction condition of the density in the road surface can also be considered. The correlation evaluation device is essentially a device for comprehensively processing the previous information. It mainly conducts a comprehensive scoring evaluation on the fusion information, the foundation information, and the road surface. For example, the fusion information accounts for 10 points, the road surface information accounts for 10 points, and the foundation information accounts for 10 points. The specific evaluation method and evaluation indicators are prior art and will not be elaborated here.

[0161] The above description is only some preferred embodiments of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the embodiments of the present application.

Claims

1. A highway engineering inspection automated data acquisition system, characterized by: include: Movable workbench; An image information collection device, used to collect road surface information; A radar detection device for collecting radar scan information from beneath the road surface; An information fusion device collects road surface information and radar scanning information, and fuses the road surface information with the radar information to obtain fused information; A correlation evaluation device collects each fused information and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fused information; in: The information fusion device includes: A radar information extraction unit extracts multi-scale information from each tomographic image in the radar scanning information; The multi-scale information is the density and material of each area, and the material includes at least the foundation and pavement covering; The pavement information regression unit regresses the multi-scale information into the ground representation information to generate the pavement implicit information; The twin network unit obtains the correlation between road surface information and road surface implicit information to generate fusion information; The pavement information regression unit includes an information stripping unit and an information regression unit, and the information stripping unit strips out the foundation information according to the material; The information regression unit has a built-in LSTM model, which outputs implicit road surface information based on the input foundation information; The twin network unit consists of two sub-network groups with the same structure and shared weights. The road surface information and road surface implicit information are input into the two sub-networks respectively. The loss function of the sub-network is: L=max(d(a,p)-d(a,n)+margin,0) L is the loss value; d(a,p) represents the distance between the anchor point, a, and the positive sample p; d(a,n) represents the distance between the anchor point, a, and the negative sample; Margin is a hyperparameter used to control the minimum distance between positive samples and negative samples. max(·,0) means taking the larger value between the value in the brackets and 0 to ensure that the loss value is non-negative.

2. The highway engineering detection automated data acquisition system according to claim 1 is characterized by: The image information collecting device comprises: A camera is set with a fixed focal length to obtain image information within a fixed range; A plurality of displacement sensors are provided, and are used to calculate the shortest distance between the camera and the ground to obtain distance information; The image information generating unit acquires the image information and the distance information, and adds the distance information to the image information to obtain the distance between each position in the image information and the camera, so as to generate the road surface information.

3. The highway engineering detection automated data acquisition system according to claim 2 is characterized in that: The displacement sensors are arranged in an array, and the displacement sensors cover the width range of the camera each time it takes a picture.

4. The highway engineering detection automated data acquisition system according to claim 1 is characterized by: The radar detection device includes: A radar detection unit, used to transmit detection signals downward to obtain a detection fault image below the radar; A tomographic image preprocessing unit performs denoising on the detected tomographic image and obtains the tomographic information from the ground to a preset depth to obtain a tomographic image; The tomographic image fusion unit obtains the tomographic images of each area and fuses all the tomographic images into radar scanning information.

5. The highway engineering detection automated data acquisition system according to claim 4 is characterized by: The movable workbench also includes a control device, which performs the following steps: Step 1: The control device obtains a fixed range of image information obtained by the camera to obtain discrete displacement; Step 2: The control device controls the camera to obtain a clear image information; Step 3: The control device controls the movable platform to move discrete displacements at a uniform speed; When controlling the movable platform to move at a constant speed, the control device controls the displacement sensor to obtain distance information and controls the radar detection device to obtain radar scanning information.

6. The highway engineering inspection automated data acquisition system according to claim 1 is characterized by: The radar information extraction unit includes: Input layer, used to input the tomographic image X; The preliminary feature extraction layer performs preliminary feature extraction on the tomographic image X to obtain preliminary features F`; F`=Conv 3×3 (X); The multi-scale feature extraction layer divides the preliminary features F' into three groups: F1', F2', and F3'; Use standard 3×3 convolution Conv for F1` 3×3 Perform feature extraction to obtain a first feature map F1; F1=Conv 3×3 (F1`); Use the DRB module to perform multi-scale feature extraction on F2` to obtain the second feature map F2; <h2 style=";text-align:left;direction:ltr">F2=DRB<h2 style=";text-align:left;direction:ltr"> k1 <h2 style=";text-align:left;direction:ltr"> (F2`) Use the DRB module to perform multi-scale feature extraction on F3` to obtain the third feature map F3; F3=DRB k2 (F3`);DRB k1 and DRB k2 are the sizes of the DRB modules respectively; The feature fusion layer combines the first feature map F1, the second feature map F2, and the third feature map F3 to obtain F fu , F fu =Concat(F1、F2、F3); Downsampling processing layer, for F fu Perform feature learning to reduce F fu The resolution of , obtains the feature information; The fusion enhancement layer fuses and enhances the feature information to generate multi-scale information of the corresponding area in each feature map; The multi-scale information includes the density and material of each area, and the material includes at least the foundation and pavement covering.

7. The highway engineering inspection automated data acquisition system according to claim 6 is characterized by: ； ； ； ； α+β=1； ; i and j represent the horizontal and vertical coordinates in F1`, F2`, and F3` respectively, and F1`(i, j) represents the eigenvalue at (i, j) in F1`. Represents the coefficients of the high-pass filter, which is used to extract high-frequency components. represents the coefficient of the low-pass filter, which is used to extract the low-frequency component. α and β are weighted average coefficients, which are used to control the relative contribution of the merged mid-band. F2`(i, j) represents the eigenvalue at (i, j) in F2`, F3`(i, j) represents the eigenvalue at (i, j) in F3`, F2`(i, j)` represents the first intermediate frequency component, F2`(i, j)`` represents the second intermediate frequency component, (i, j)` represents the coordinates of the first intermediate frequency component, (i, j)`` represents the coordinates of the second intermediate frequency component, X represents the input tomographic image, m represents the index of the filter in the vertical direction, and n represents the index of the filter in the horizontal direction.

8. The highway engineering inspection automated data acquisition system according to claim 7 is characterized by: The downsampling layer converts F fu Slice the space dimension with a ratio of 2 and divide it into 4 sub-regions of equal size, where F fu = (S, S, C), where S is F fu Dimensions, C is F fu The number of channels; The size of each sub-region is s×s, s=1 / 2(S), where s is the side length of the downsampling processing layer; Concatenate each sub-region along the channel dimension to obtain the concatenated feature map F fu `, F fu `=(s,s,4C); Concatenate feature map F fu `Perform a 3×3 standard convolution to obtain feature information.

Images