Automatic data acquisition system for highway engineering detection

Abstract

The invention discloses an automatic data acquisition system for highway engineering detection. Belongs to the technical field of data processing. An automatic data acquisition system for highway engineering detection comprises a movable workbench; the image information collecting device is used for collecting road surface information; the radar detection device is used for collecting radar scanning information below the road surface; the information fusion device is used for collecting the road surface information and the radar scanning information and fusing the road surface information and the radar information to obtain fusion information; and the correlation evaluation device is used for collecting each piece of fusion information and calculating the quality score of the road surface based on the correlation between the road surface information and the radar information in the fusion information. According to the technical scheme provided by the invention, not only is the road surface information collected, but also the road surface information is used for evaluating the construction quality of the highway engineering. According to the method, the road surface information and the radar scanning information are collected, and are mutually fused, so that the construction quality of the road surface can be accurately evaluated in the mode.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an automated data acquisition system for highway engineering inspection. Background Art

[0002] Highway acceptance is an important part of the overall review of the entire process of highway project construction to ensure that the project has been completed in accordance with relevant standards, specifications and contract requirements and is ready for use. During highway acceptance, it is necessary to accurately assess the construction quality of the highway, and for this purpose, it is necessary to collect highway inspection data.

[0003] The current highway engineering inspection system generally uses drones to patrol the ground, obtain image information of the highway surface, and determine whether there is any damage to the road surface based on the images, thereby completing the data collection work.

[0004] This method cannot accurately evaluate the actual construction quality of the project, especially when the construction period is tight. The road surface is often urgently paved without compacting the roadbed, and then the road surface is made smooth by adjusting the thickness of the asphalt (pavement). Therefore, when inspecting the road surface flatness for acceptance, it is impossible to accurately evaluate the construction quality of the roadbed. Summary of the invention

[0005] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] As a first aspect of the present application, in order to solve the technical problem that the highway quality evaluation standard is not high, a highway engineering inspection automatic data collection system is provided, comprising: Movable workbench; An image information collection device, used to collect road surface information; A radar detection device for collecting radar scan information from beneath the road surface; An information fusion device collects road surface information and radar scanning information, and fuses the road surface information with the radar information to obtain fused information; The correlation evaluation device collects each fusion information and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fusion information.

[0007] The technical solution provided by this application not only collects pavement information, but also uses pavement information to evaluate the construction quality of highway projects. Instead, pavement information and radar scanning information are collected and integrated with each other. If there is a reasonable correlation between the pavement information and the radar scanning information, for example, the pavement above the hollow foundation is flat, or the roadbed is obviously bulging upward, but the pavement does not reflect the corresponding density change, this is obviously caused by the direct use of asphalt to pave the pavement after the foundation is not flattened. For this reason, this method can accurately evaluate the construction quality of the pavement.

[0008] When collecting road surface information, if image processing technology is used to obtain the undulations of the road surface, a lot of computing power is required and the accuracy is not high. To this end, this application provides the following technical solutions: Furthermore, the image information collecting device includes: A camera is set with a fixed focal length to obtain image information within a fixed range; A displacement sensor, which is provided in plurality and is used to calculate the shortest distance between the camera and the ground to obtain distance information; The image information generating unit acquires the image information and the distance information, and adds the distance information to the image information to obtain the distance between each position in the image information and the camera, so as to generate the road surface information.

[0009] In the technical solution provided in the present application, a camera is used to obtain image information with a fixed focal length, and the distance between the camera and the image is also obtained, so that not only road surface information can be obtained, but also the undulation of the road surface can be obtained, thereby obtaining road surface information in more dimensions.

[0010] Furthermore, the displacement sensors are arranged in an array, and the displacement sensors cover the width range of the camera each time it takes a picture.

[0011] In this solution, the undulation information of the road surface can be obtained only by continuously moving the movable platform.

[0012] Further, the radar detection device comprises: A radar detection unit, used to transmit detection signals downward to obtain a detection fault image below the radar; A tomographic image preprocessing unit performs denoising on the detected tomographic image and obtains the tomographic information from the ground to a preset depth to obtain a tomographic image; The tomographic image fusion unit obtains the tomographic images of each area and fuses all the tomographic images into radar scanning information.

[0013] In the technical solution provided in the present application, after the detection tomographic image is acquired, preprocessing is performed, so that the result of radar detection can be more accurately reflected, and because a preset depth is set, all ground-based information can be analyzed at the same level.

[0014] The road surface information and the three-dimensional image array are not acquired synchronously. To this end, this application provides the following technical solutions: Furthermore, the movable workbench further includes a control device, which performs the following steps: Step 1: The control device obtains a fixed range of image information obtained by the camera to obtain discrete displacement; Step 2: The control device controls the camera to obtain a clear image information; Step 3: The control device controls the movable platform to move at a constant speed with discrete displacements; when controlling the movable platform to move at a constant speed, the control device controls the displacement sensor to obtain distance information and controls the radar detection device to obtain radar scanning information.

[0015] In this way, the movable workbench in this solution can stably obtain the required road surface information and radar scanning information.

[0016] Although the current high-precision radar has high detection accuracy, its imaging is relatively rough, the overall detection accuracy is not high, and it is easy to miss feature information. To this end, this application provides the following technical solutions: Furthermore, the information fusion device comprises: A radar information extraction unit extracts multi-scale information from each tomographic image in the radar scanning information; The pavement information regression unit regresses the multi-scale information into the ground representation information to generate the pavement implicit information; The twin network unit obtains the correlation between road surface information and road surface implicit information to generate fusion information.

[0017] In the technical solution provided by the present application, multi-scale information is extracted from radar scanning information, and more useful information can be extracted from radar scanning information to avoid missing features. In addition, the road surface information is regressed to the foundation representation information to generate a representation relationship between the road surface information and the foundation information, and then compared with the twin network unit, so that the correlation between the radar scanning information and the road surface information can be accurately compared, and whether there is a reasonable technical correlation between the foundation construction and the road surface construction can be accurately judged.

[0018] Generally speaking, although the radar information obtained has sufficient accuracy, due to the complexity of electromagnetic wave reflection on the ground, there is a lot of redundant information in the radar information finally received and processed. These redundant information affect each other, thereby reducing the connection between features in the fault image, making it difficult to find the connection between features. To this end, the present application provides the following technical solutions: Further, the radar information extraction unit includes: Input layer, used to input the tomographic image X; The preliminary feature extraction layer performs preliminary feature extraction on the tomographic image X to obtain preliminary features F`; F`=Conv 3×3 (X); The multi-scale feature extraction layer divides the preliminary features F' into three groups: F1', F2', and F3'; Use standard 3×3 convolution Conv for F1` 3×3 Perform feature extraction to obtain a first feature map F1; F1=Conv 3×3 (F1`); Use the DRB module to perform multi-scale feature extraction on F2` to obtain the second feature map F2; F2=DRB k1 (F2`); Use the DRB module to perform multi-scale feature extraction on F3` to obtain the third feature map F3; F3=DRB k2 (F3`);DRB k1 and DRB k2 are the sizes of the DRB modules respectively; The feature fusion layer combines the first feature map F1, the second feature map F2, and the third feature map F3 to obtain F fu , F fu =Concat(F1, F2, F3); Downsampling processing layer, for F fu Perform feature learning to reduce F fu The resolution of , get the feature information; The fusion enhancement layer fuses and enhances the feature information to generate multi-scale information of the corresponding area in each feature map; The multi-scale information includes the density and material of each area, and the material includes at least the foundation and pavement covering.

[0019] In this scheme, the input tomographic image is first subjected to convolution processing to roughly reduce the resolution of the image, and then the preliminary feature map is divided into three groups, one of which is further processed by convolution, and the other two groups are processed by the DRB module. Compared with the ordinary convolution module, the DRB module has a larger receptive field, so it can capture feature dependencies at a longer distance. In this way, the three groups of features use different receptive fields to extract features and are finally fused, which can make the features of the acquired feature maps have a closer connection, and it is easier to find the connection between features and materials and density in the fusion enhancement layer.

[0020] When performing feature grouping, if you group directly according to the area of ​​the feature group, for example, divide the preliminary feature map into 3 different areas, the features obtained in different areas will lack continuity when the features are fused. For example, if you divide a picture into the upper part, the middle part, and the lower part, and use different convolution kernels to perform feature systems on different parts, it will actually result in any area being able to represent only one feature division method, which will result in the final extracted features being only valid locally and not globally. To address this problem, this application provides the following methods: Furthermore, F1`(i,j)=∑ m，n X(2i+m,2j+n)Ø(m,n)Ø(m,n); F2`(i,j)=αF2`(i,j)`+βF2`(i,j)``; F2`(i,j)`=∑ m，n X(2i+m,2j+n)Ø(m,n)ω(m,n); F2`(i,j)``=∑ m，n X(2i+m,2j+n)ω(m,n)Ø(m,n); α+β=1; F3`(i,j)=∑ m，nX(2i+m, 2j+n)ω(m, n)ω(m, n); i and j represent the horizontal and vertical coordinates in F1`, F2`, and F3` respectively, F1`(i, j) represents the eigenvalue at (i, j) in F1`, Ø represents the coefficient of the high-pass filter, which is used to extract high-frequency components, ω represents the coefficient of the low-pass filter, which is used to extract low-frequency components, α and β are weighted average coefficients, which are used to control the relative contribution of the merged mid-band, F2`(i, j) represents the eigenvalue at (i, j) in F2`, F3`(i, j) represents the eigenvalue at (i, j) in F3`, F2`(i, j)` represents the first intermediate frequency component, F2`(i, j)`` represents the second intermediate frequency component, (i, j)`` represents the coordinates of the first intermediate frequency component, (i, j)`` represents the coordinates of the second intermediate frequency component, X represents the input tomographic image, m represents the index of the filter in the vertical direction, and n represents the index of the filter in the horizontal direction.

[0021] In the technical solution provided by the present application, when performing preliminary feature division, the wavelet transform is used to divide the features into low-frequency part, medium-frequency part and high-frequency part respectively, so relatively speaking, the overall range of F1`, F2`, and F3` is the same, which is only used to highlight the contribution range at different frequencies. In this way, after the feature extraction and fusion of F1`, F2`, and F3` by using this solution, the extracted features will not be only valid locally. Moreover, in this solution, the frequencies of F1`, F2`, and F3` gradually increase, so relatively speaking, the changes in the features in F1`, F2`, and F3` are from drastic to not drastic. For this reason, the convolution operation for F1` can highlight the feature changes in the local area, and the DRB module in F2` and F3` can use a larger receptive field to analyze the image changes in the low-frequency part.

[0022] Before the fusion enhancement layer processes the features, it is necessary to use a downsampling processing layer to reduce the resolution to improve the enhancement efficiency. Because the tomographic image X has undergone preliminary feature extraction and multi-scale feature extraction, the redundant information is limited. If the Conv module is directly used for downsampling, information loss will occur. To this end, this application adopts the following solution: Furthermore, the downsampling layer converts F fu Slice the space dimension with a ratio of 2 and divide it into 4 sub-regions of equal size, where F fu = (S, S, C), where S is F fu Dimensions, C is F fu The number of channels; The size of each sub-region is s×s, s=1 / 2(S), where s is the side length of the downsampling processing layer; Concatenate each sub-region along the channel dimension to obtain the concatenated feature map Ffu `, F fu `=(s,s,4C); Concatenate feature map F fu `Perform a 3×3 standard convolution to obtain feature information.

[0023] In this solution, the feature map is downsampled in the spatial dimension through slicing, splicing and convolution operations, and the amount of calculation is reduced by controlling the number of output channels of the convolution layer. This processing method not only retains the important information of the feature map, but also improves the computational efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The illustrative embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application.

[0025] In addition, throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the components and elements are not necessarily drawn to scale.

[0026] In the attached picture: Figure 1 This is a schematic diagram of the structure of the automated data acquisition system for highway engineering inspection.

[0027] Figure 2 Get radar detection map for GPR radar. DETAILED DESCRIPTION

[0028] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms, and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0029] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0030] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0031] Reference Figure 1~2, an automated data collection system for highway engineering inspection includes: a movable workbench, an image information collection device, a radar detection device, an information fusion device, and a correlation evaluation device. Among them, the movable platform is a small vehicle with mobility, such as an engineering vehicle used for construction. The radar detection device is a common GPR radar, which is mounted on a movable platform. After emitting electromagnetic waves downward, it can perform underground detection based on the returned electromagnetic wave information. The image information collection device collects road surface information and radar scanning information, and fuses the road surface information with the radar information to obtain fused information. The correlation evaluation device collects each fused information, and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fused information.

[0032] In this solution, a mobile platform carrying an image collection device and a radar detection device moves along the road to collect road surface information and radar scanning information. The road surface information is used to characterize the undulation and flatness of the road surface, and the radar scanning information is used to characterize the compaction of the foundation. The road surface information and the radar scanning information are then integrated to obtain the final quality score. The quality score in this solution will also examine the correlation between the road surface paving and the foundation. If the correlation between the road surface and the foundation is high, it means that the detection result is reliable. If the correlation between the two is poor, it means that there are major problems in the construction process, such as paving the road before the foundation is processed, or illegally leveling the road surface without leveling the foundation.

[0033] Specifically: The image information collection device includes: a camera, a displacement sensor and an image information generation unit. The camera is set with a fixed focal length to obtain image information within a fixed range; the camera is located directly below the movable platform, so the camera can get a standard picture after taking one shot, and because the height of the movable platform is fixed, and the focal length of the camera is fixed, the length of the corresponding real world in each picture obtained by the camera is equal. That is, each picture obtains image information within a fixed range, and then the mobile platform only needs to use the width of this range as a moving scale, and each time it moves such a long distance, it can obtain all road information. Of course, each picture obtained by the camera is processed separately.

[0034] There are multiple displacement sensors, which are used to calculate the shortest distance between the camera and the ground to obtain distance information. Therefore, when the movable platform moves, the displacement sensor can obtain the distance between each area within the range that the camera can shoot and the movable platform, and then use the distance to characterize the undulation of the road surface.

[0035] Specifically, the movable workbench further includes a control device, and the control device performs the following steps: Step 1: The control device obtains a fixed range of image information obtained by the camera to obtain discrete displacement; Step 2: The control device controls the camera to obtain a clear image information; Step 3: The control device controls the movable platform to move at a constant speed with discrete displacements; when controlling the movable platform to move at a constant speed, the control device controls the displacement sensor to obtain distance information and controls the radar detection device to obtain radar scanning information.

[0036] In essence, the control device controls the movable platform to move to a position and take a picture. The field of view of this picture is a fixed range. Then the movable platform will move at a constant speed when moving the length of the fixed range, and then the distance between each point on the road surface and the movable platform can be obtained. Then, according to the length of the distance, the ups and downs of the road surface within the fixed range can be roughly described.

[0037] The image information generating unit acquires the image information and the distance information, and adds the distance information to the image information to obtain the distance between each position in the image information and the camera, so as to generate the road surface information.

[0038] The method of adding here is to mark the distance of each area in the image information and judge the flatness of the road surface based on a more precise distance.

[0039] In practice, this scheme is only used to detect the relationship between the flatness of the road surface and the distance. Therefore, when evaluating, the pixel values ​​in the image information can be directly converted into distance information to obtain a distance matrix.

[0040] The above is a method for obtaining road surface information. In this solution, the road surface information is actually a distance matrix, which describes the height information of the road surface from the movable device, so it actually represents the undulation of the corresponding road surface. The original image information can directly use the feature recognition model of the existing technology to identify the characteristics of road damage, that is, it is used as an evaluation indicator in the correlation evaluation device. This part can be directly processed using existing technology, such as using the SIFT algorithm to extract the image features of the damaged position, and then compare the damaged positions. If there are many similar features, it means that the road surface is severely damaged. The key to this solution is to calculate the correlation between road surface information and radar detection information.

[0041] Specifically: the radar detection device includes: a radar detection unit, a tomographic image preprocessing unit, and a tomographic image fusion unit. The radar detection unit is used to transmit a detection signal downward to obtain a detection tomographic image below the radar; the tomographic image preprocessing unit performs denoising on the detection tomographic image and obtains tomographic information from the ground to a preset depth to obtain a tomographic image; the tomographic image fusion unit obtains tomographic images of each area and fuses all tomographic images into radar scanning information.

[0042] The radar detection device obtains vertical tomographic images of various areas below the ground. Each tomographic image shows the scanning status below the ground at the current position.

[0043] To this end, it is necessary to first extract features from each tomographic image to obtain key information from it. After obtaining the key information, subsequent fusion and comparison work can be carried out.

[0044] Specifically: The information fusion device includes: A radar information extraction unit extracts multi-scale information from each tomographic image in the radar scanning information; The pavement information regression unit regresses the multi-scale information into the ground representation information to generate the pavement implicit information; The twin network unit obtains the correlation between road surface information and road surface implicit information to generate fusion information.

[0045] The multi-scale information in this scheme is actually the information after the fusion of each fault image. For example, after obtaining the road surface information within the range of 3m×3m, it is necessary to obtain all the fault images within this range, compress this fault image group represented by 3D information into 2D information, and then input it into the twin network unit together with the road surface implicit information for calculation of fusion information.

[0046] To this end, it is necessary to first extract key features from the tomographic image. Specifically: The radar information extraction unit includes: Input layer, used to input the tomographic image X; The preliminary feature extraction layer performs preliminary feature extraction on the tomographic image X to obtain preliminary features F`; F`=Conv 3×3 (X); The multi-scale feature extraction layer divides the preliminary features F' into three groups: F1', F2', and F3'; F1`(i,j)=∑ m，n X(2i+m,2j+n)Ø(m,n)Ø(m,n); F2`(i,j)=αF2`(i,j)`+βF2`(i,j)``; F2`(i,j)`=∑ m，n X(2i+m,2j+n)Ø(m,n)ω(m,n); F2`(i,j)``=∑ m，n X(2i+m,2j+n)ω(m,n)Ø(m,n); α+β=1; F3`(i,j)=∑ m，n X(2i+m,2j+n)ω(m,n)ω(m,n); i and j represent the horizontal and vertical coordinates in F1`, F2`, and F3`, respectively. F1`(i, j) represents the eigenvalue at (i, j) in F1`. Ø represents the coefficient of the high-pass filter, which is used to extract high-frequency components. ω represents the coefficient of the low-pass filter, which is used to extract low-frequency components. α and β are weighted average coefficients, which are used to control the relative contribution of the merged mid-band. F2`(i, j) represents the eigenvalue at (i, j) in F2`. F3`(i, j) represents the eigenvalue at (i, j) in F3`. F2`(i, j)` represents the first intermediate frequency component. F2`(i, j)`` represents the second intermediate frequency component. (i, j)` represents the coordinates of the first intermediate frequency component. (i, j)`` represents the coordinates of the second intermediate frequency component. X represents the input tomographic image. m represents the index of the filter in the vertical direction. n represents the index of the filter in the horizontal direction.

[0047] Use standard 3×3 convolution Conv for F1` 3×3 Perform feature extraction to obtain a first feature map F1; F1=Conv 3×3 (F1`); Use the DRB module to perform multi-scale feature extraction on F2` to obtain the second feature map F2; F2=DRB k1 (F2`); Use the DRB module to perform multi-scale feature extraction on F3` to obtain the third feature map F3; F3=DRB k2 (F3`);DRB k1 and DRB k2 are the sizes of the DRB modules respectively; The feature fusion layer combines the first feature map F1, the second feature map F2, and the third feature map F3 to obtain F fu , F fu =Concat(F1, F2, F3); Downsampling processing layer, for F fu Perform feature learning to reduce F fuThe resolution of , get the feature information; The downsampling layer converts F fu Slice the space dimension with a ratio of 2 and divide it into 4 sub-regions of equal size, where F fu = (S, S, C), where S is F fu Dimensions, C is F fu Number of channels The size of each sub-region is s×s, s=1 / 2(S), where s is the side length of the downsampling processing layer; Concatenate each sub-region along the channel dimension to obtain the concatenated feature map F fu `, F fu `=(s,s,4C); Concatenate feature map F fu `Perform a 3×3 standard convolution to obtain feature information.

[0048] The fusion enhancement layer fuses and enhances the feature information to generate multi-scale information of the corresponding area in each feature map; The multi-scale information includes the density and material of each area, and the material includes at least the foundation and pavement covering.

[0049] To this end, the feature map finally extracted by the fusion enhancement layer in this scheme will represent the multi-scale information of the corresponding area, that is, the density and material of the corresponding position.

[0050] In this way, we only need to merge and normalize the density and material and represent them with an array to reduce it from two dimensions to one dimension.

[0051] For example, in the feature map, if it is calculated that there is a hollow below, its label is defined as 0, and if the density below is less than a certain threshold, it is defined as 1. Similarly, the two-dimensional feature map can be reduced to a one-dimensional series, and then all the two-dimensional feature maps can be combined together to obtain the matrix information corresponding to the road surface information.

[0052] The above is the process of the road surface information regression device generating implicit road surface information.

[0053] Specifically, the road surface information regression device includes an information stripping unit and an information regression unit, and the information stripping unit strips out the foundation information according to the material information.

[0054] Because the dressing materials of the foundation and the road are completely different, the dressing materials of the foundation and the road can be directly distinguished, and then the foundation information of the thickness change and density change of the foundation can be obtained.

[0055] The information regression unit regresses the foundation information to the implicit road surface information. In fact, the information regression unit is a pre-trained neural network model, whose input is the foundation information under standard construction conditions and the label is the road surface information.

[0056] That is, if the normal construction process is used to pave the road surface, but if the foundation is not compacted, when the road surface is compacted, the road surface will show an undulating characteristic due to the inconsistent density of the foundation.

[0057] To this end, the information regression unit is used to find the correlation between the foundation information and the road surface information under standard construction conditions. In this way, after the foundation information is input into the information regression unit, the road surface information under ideal conditions can be roughly obtained. Although this information is not the actual road surface information, it represents the possible construction conditions of the road surface under normal procedures.

[0058] The model built into the information regression unit may be LSTM. When its input and output are completely determined, its specific model structure and training method are no longer parameters.

[0059] The reason why LSTM is used in this solution is that it can solve the gradient vanishing and gradient exploding problems of traditional RNN in long sequence training. It introduces a gating mechanism to enable the model to autonomously select which information needs to be retained and which needs to be forgotten. Therefore, when processing adjacent matrices in the foundation information, it can accurately judge the connection and dependency between adjacent information, and accurately use the foundation information to restore the implicit information of the road surface.

[0060] The twin network unit obtains the correlation between road surface information and road surface implicit information to generate fusion information.

[0061] Fusion information is actually used to describe the correlation between road surface information and road surface implicit information. If the correlation between the two is high, it means that the credibility of the construction process is high, otherwise it means that the credibility of the construction process is poor. The road surface implicit information and road surface information are essentially matrix representations of road surface fluctuations, and their data structure and data type are the same. In this way, twin network units are used for correlation evaluation. This is more accurate than directly using similarity calculation.

[0062] Specific: The information regression unit includes two sub-network groups (CNN) with the same structure and shared weights. The road surface information and road surface implicit information are input into the two sub-networks respectively. (Of course, preprocessing may be required before input). Then, the similarity score between the two input data is output.

[0063] For example, the road surface information is normalized to generate a standard distance matrix H1 related to the road surface undulation; Then the road surface implicit information is normalized to generate the implicit distance matrix H2; Then the standard distance matrix H1 and the implicit distance matrix H2 are input into two sub-networks (CNN) respectively. Each sub-network will process the standard distance matrix H1 and the implicit distance matrix H2 independently, extract features through the convolutional layer of the sub-network, and then generate feature vectors through the fully connected layer of the sub-network. Then the output layer calculates the similarity of the feature vectors output by the two sub-networks.

[0064] For example: The eigenvector of the standard distance matrix H1 is A; The eigenvector of the implicit distance matrix H2 is B;

[0065] The output layer calculates the Euclidean distance d between the two feature vectors and uses d as the similarity; ; h and q are the horizontal and vertical coordinate indices of the feature vector elements respectively; A hq Indicates that the eigenvector is the eigenvalue at coordinate (h, q) in A, B hq It means that the eigenvector is the eigenvalue at the coordinate (h, q) in B; the similarity between the eigenvector A and the eigenvector B of the example is d≈16.55.

[0066] In this way, in this scheme, two sub-networks with the same structure and shared weights are used to process two input data and output the similarity score between them. During the training process, the twin network uses a specific loss function to optimize the network parameters so that the network can more accurately judge the similarity between the input data.

[0067] The above is the specific structure of the information regression unit, and its model performance is mainly related to the selected sub-network group. However, in order to increase accuracy, this application provides the following loss function.

[0068] L=max(d(a,p)-d(a,n)+margin,0) L is the loss value; d(a,p) represents the distance between the anchor point, a, and the positive sample p; d(a,n) represents the distance between the anchor point, a, and the negative sample; Margin is a hyperparameter used to control the minimum distance between positive samples and negative samples. max(a,p) means taking the larger value between the value in the brackets and 0 to ensure that the loss value is non-negative.

[0069] This loss function introduces positive and negative samples, which enables the CNN network to learn the dependencies between spaces, so that similar samples are brought closer and different samples are pushed further away, which can effectively increase the resolution of the model.

[0070] The correlation evaluation device collects each fused information, and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fused information. Of course, the flatness of the road surface and the compaction of the density in the road surface can also be considered when calculating the correlation. The correlation evaluation device is essentially a device that comprehensively processes the previous information. It mainly conducts a comprehensive scoring evaluation on the fused information, foundation information and road surface. For example, the fused information accounts for 10 points, the road surface information accounts for 10 points, and the foundation information accounts for 10 points. The specific evaluation method and evaluation indicators are existing technologies and will not be elaborated here.

[0071] The above description is only some preferred embodiments of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present application to form a technical solution.

Claims

1. A highway engineering inspection automated data acquisition system, characterized by: include: Movable workbench; An image information collection device, used to collect road surface information; A radar detection device for collecting radar scan information from beneath the road surface; An information fusion device collects road surface information and radar scanning information, and fuses the road surface information with the radar information to obtain fused information; The correlation evaluation device collects each fusion information and calculates the quality score of the road surface based on the correlation between the road surface information and the radar information in the fusion information.

2. The highway engineering detection automated data acquisition system according to claim 1 is characterized by: The image information collecting device comprises: A camera is set with a fixed focal length to obtain image information within a fixed range; A plurality of displacement sensors are provided, and are used to calculate the shortest distance between the camera and the ground to obtain distance information; The image information generating unit acquires the image information and the distance information, and adds the distance information to the image information to obtain the distance between each position in the image information and the camera, so as to generate the road surface information.

3. The highway engineering detection automated data acquisition system according to claim 2 is characterized by: The displacement sensors are arranged in an array, and the displacement sensors cover the width range of the camera each time it takes a picture.

4. The highway engineering detection automated data acquisition system according to claim 1 is characterized by: The radar detection device includes: A radar detection unit, used to transmit detection signals downward to obtain a detection fault image below the radar; A tomographic image preprocessing unit performs denoising on the detected tomographic image and obtains the tomographic information from the ground to a preset depth to obtain a tomographic image; The tomographic image fusion unit obtains the tomographic images of each area and fuses all the tomographic images into radar scanning information.

5. The highway engineering inspection automated data acquisition system according to claim 4 is characterized by: The movable workbench also includes a control device, which performs the following steps: Step 1: The control device obtains a fixed range of image information obtained by the camera to obtain discrete displacement; Step 2: The control device controls the camera to obtain a clear image information; Step 3: The control device controls the movable platform to move discrete displacements at a uniform speed; When controlling the movable platform to move at a constant speed, the control device controls the displacement sensor to obtain distance information and controls the radar detection device to obtain radar scanning information.

6. The highway engineering detection automated data acquisition system according to claim 4 is characterized by: The information fusion device includes: A radar information extraction unit extracts multi-scale information from each tomographic image in the radar scanning information; The pavement information regression unit regresses the multi-scale information into the ground representation information to generate the pavement implicit information; The twin network unit obtains the correlation between road surface information and road surface implicit information to generate fusion information.

7. The highway engineering inspection automated data acquisition system according to claim 6 is characterized by: The radar information extraction unit includes: Input layer, used to input the tomographic image X; The preliminary feature extraction layer performs preliminary feature extraction on the tomographic image X to obtain preliminary features F`; F`=Conv 3×3 (X); The multi-scale feature extraction layer divides the preliminary features F' into three groups: F1', F2', and F3'; Use standard 3×3 convolution Conv for F1` 3×3 Perform feature extraction to obtain a first feature map F1; F1=Conv 3×3 (F1`); Use the DRB module to perform multi-scale feature extraction on F2` to obtain the second feature map F2; <h2 style=";text-align:left;direction:ltr">F2=DRB<h2 style=";text-align:left;direction:ltr"> k1 <h2 style=";text-align:left;direction:ltr"> (F2`) Use the DRB module to perform multi-scale feature extraction on F3` to obtain the third feature map F3; F3=DRB k2 (F3`);DRB k1 and DRB k2 are the sizes of the DRB modules respectively; The feature fusion layer combines the first feature map F1, the second feature map F2, and the third feature map F3 to obtain F fu , F fu =Concat(F1、F2、F3); Downsampling processing layer, for F fu Perform feature learning to reduce F fu The resolution of , get the feature information; The fusion enhancement layer fuses and enhances the feature information to generate multi-scale information of the corresponding area in each feature map; The multi-scale information includes the density and material of each area, and the material includes at least the foundation and pavement covering.

8. The highway engineering inspection automated data acquisition system according to claim 7 is characterized by: F1`(i,j)=∑ m，n X(2i+m,2j+n)Ø(m,n)Ø(m,n); F2`(i,j)=αF2`(i,j)`+βF2`(i,j)``; F2`(i,j)`=∑ m，n X(2i+m,2j+n)Ø(m,n)ω(m,n); F2`(i,j)``=∑ m，n X(2i+m,2j+n)ω(m,n)Ø(m,n); α+β=1； F3`(i,j)=∑ m，n X(2i+m,2j+n)ω(m,n)ω(m,n); i and j represent the horizontal and vertical coordinates in F1`, F2`, and F3`, respectively. F1`(i, j) represents the eigenvalue at (i, j) in F1`. Ø represents the coefficient of the high-pass filter, which is used to extract high-frequency components. ω represents the coefficient of the low-pass filter, which is used to extract low-frequency components. α and β are weighted average coefficients, which are used to control the relative contribution of the merged mid-band. F2`(i, j) represents the eigenvalue at (i, j) in F2`. F3`(i, j) represents the eigenvalue at (i, j) in F3`. F2`(i, j)` represents the first intermediate frequency component. F2`(i, j)`` represents the second intermediate frequency component. (i, j)` represents the coordinates of the first intermediate frequency component. (i, j)`` represents the coordinates of the second intermediate frequency component. X represents the input tomographic image. m represents the index of the filter in the vertical direction. n represents the index of the filter in the horizontal direction.

9. The highway engineering inspection automated data acquisition system according to claim 8, characterized in that: The downsampling layer converts F fu Slice the space dimension with a ratio of 2 and divide it into 4 sub-regions of equal size, where F fu = (S, S, C), where S is F fu Dimensions, C is F fu The number of channels; The size of each sub-region is s×s, s=1 / 2(S), where s is the side length of the downsampling processing layer; Concatenate each sub-region along the channel dimension to obtain the concatenated feature map F fu `, F fu `=(s,s,4C); Concatenate feature map F fu `Perform a 3×3 standard convolution to obtain feature information.

Images