Asphalt pavement binocular infrared speckle three-dimensional reconstruction method

Abstract

The application discloses a kind of asphalt pavement binocular infrared speckle three-dimensional reconstruction methods, comprising: S1, using three-dimensional acquisition system to target area projection random or quasi-random distribution infrared speckle, and after collecting target area left target image and right target image with speckle characteristics are preprocessed;S2, after being preprocessed, using multi-scale adaptive feature fusion stereo matching method to target area left target image and right target image are carried out disparity estimation;S3, based on the disparity estimation result under different scales obtained is fused across scale, form multi-scale adaptive feature fusion stereo matching result and get fusion disparity map, the disparity optimization is carried out to the fusion disparity map, and final disparity map is obtained;S4, according to the imaging geometry relationship and stereo calibration parameter of left target image and right target image in S1, the final disparity map obtained in S3 is converted into three-dimensional coordinate information and is point cloud processed, and finally the three-dimensional point cloud reconstruction result of target area is obtained.

Description

Technical Field

[0001] This application relates to the field of road engineering inspection and machine vision 3D measurement technology, specifically to a binocular infrared speckle 3D reconstruction method for asphalt pavement. Background Technology

[0002] Asphalt pavement is composed of mineral aggregates, asphalt binder, and surface voids. Its surface color is dark, with relatively small local grayscale differences, while its natural texture varies significantly under different wear conditions. When reconstructing using ordinary passive binocular vision, corresponding points rely on the pavement's own texture for stereo matching, which easily leads to problems such as insufficient feature points, discontinuous disparity, mismatches, and matching gaps. These problems are even more pronounced for pavements with similar grayscale areas and obvious repetitive distribution of fine particles. Outdoor inspection is also affected by sunlight angle, shadows, reflections, and subtle changes in camera posture, making it difficult to reliably obtain high-quality disparity maps relying solely on natural texture. Among existing technologies, single-line laser scanning technology is relatively mature and can obtain reliable elevation data, but it typically uses a line-by-line scanning method for data acquisition, and its detection efficiency is limited by scanning speed, walking mechanism, and sampling interval. When rapid inspection of large pavement areas is required, this type of method often suffers from long acquisition times and high requirements for system motion control. Therefore, it is necessary to propose a binocular 3D reconstruction scheme that is more suitable for actual pavement environments, enhances matching features in areas with weak texture, and improves the stability of disparity estimation. Summary of the Invention

[0003] The purpose of this application is to provide a binocular infrared speckle three-dimensional reconstruction method for asphalt pavement, the specific technical solution of which is as follows: A method for binocular infrared speckle 3D reconstruction of asphalt pavement includes: S1, projecting randomly or quasi-randomly distributed infrared speckles onto the target area using a 3D acquisition system, and acquiring left and right target images with speckle features in the target area, followed by preprocessing; S2, based on the preprocessed left and right target images in S1, using a multi-scale adaptive feature fusion stereo matching method to estimate the disparity of the left and right target images in the target area; S3, performing cross-scale fusion based on the disparity estimation results at different scales obtained in S2 to form a multi-scale adaptive feature fusion stereo matching result and obtain a fused disparity map, and optimizing the disparity map to obtain a final disparity map; S4, according to the imaging geometric relationship between the left and right target images in S1 and the binocular stereo calibration parameters, converting the final disparity map obtained in S3 into 3D coordinate information and performing point cloud processing to finally obtain the 3D point cloud reconstruction result of the target area.

[0004] The 3D acquisition system in S1 includes: a support mechanism; a binocular industrial camera, comprising a left industrial camera, a right industrial camera, and a synchronization triggering unit, wherein the left and right industrial cameras are fixedly mounted on the support mechanism, and the synchronization triggering unit is used to control the left and right industrial cameras to acquire left and right target images at the same time; an infrared speckle projection device, wherein the projection angle of the infrared speckle projection device maintains a certain angle with the optical axis of the binocular camera, and is used to actively project infrared speckle onto the target area; and a circular hollow opaque acrylic plate, which is attached to the surface of the road surface to be measured, and is used to define the target area and form a clearly defined effective area, so as to facilitate the automatic extraction of the target area in the future.

[0005] The preprocessing in S1 includes: S1.1, generating an initial mask for the measurement area based on the inner boundary contour of the circular hollow opaque acrylic plate, and inputting the initial mask as prompt information into the Segment Anything Model 2 (SAM2) segmentation model, which automatically generates a target mask for the effective measurement area of ​​the asphalt pavement in the input image; automatically extracting the target area image based on the target mask, and removing the background, edge occlusion areas, and irrelevant reflection areas outside the target area; S1.2, performing binocular calibration on the binocular industrial camera to obtain the intrinsic parameters of the left and right cameras and the extrinsic parameter matrix between the binocular cameras, and then performing distortion correction and stereo correction on the left and right target images based on the calibration parameters to obtain coplanar and epipolar aligned corrected images.

[0006] S2 includes the following steps for disparity estimation of the left and right target images in the target region: S2.1 Constructing multi-scale image pyramids for the preprocessed left and right target images from S1 to express road surface texture features and speckle features at different resolution levels; S2.2 Calculating grayscale differences, local structural similarities, and gradient change information at each scale based on the multi-scale image pyramids of the left and right target images from S2.1, and forming a comprehensive matching cost function through weighted fusion; performing guided filtering aggregation on the comprehensive matching cost to improve the stability of the matching cost in weak texture regions and the ability to preserve edge regions; S2.3 Constructing an adaptive regularization weight map based on pixel gradients, local contrast, and texture richness from S2.2 to dynamically adjust the regularization intensity as local road surface features change.

[0007] In S3, a nonlinear fusion method based on Softmax normalization weighting is adopted for cross-scale fusion. The local matching confidence, texture response intensity and edge preservation degree of disparity candidate results at different scales are calculated respectively, and the weight of each scale result in the final disparity estimation is adaptively determined. The cross-scale fusion process includes: S3.1, determining the initial disparity range with low-resolution scale results; S3.2, performing local refinement in high-resolution scale images; S3.3, continuing to perform left-right consistency checks, occlusion repair, hole filling and sub-pixel interpolation optimization on the fused disparity map to obtain a dense, continuous disparity map with good edge preservation.

[0008] When converting the disparity map into three-dimensional coordinate information in S4, the disparity map obtained in S3.3 is converted into three-dimensional coordinate information based on the binocular imaging geometry and the stereo calibration parameters of the binocular camera, thus completing the three-dimensional point cloud reconstruction of the target area of ​​the asphalt pavement. The binocular imaging geometry includes the target point depth Z being calculated as Z=fB / d, where f is the camera focal length, B is the binocular baseline length, and d is the disparity value corresponding to the point. The pixel coordinates are converted into spatial coordinates by combining the camera principal point coordinates.

[0009] Point cloud processing in S4 includes coordinate unification, local reference plane correction, outlier removal, and statistical filtering of the reconstructed point cloud to remove outliers caused by camera pose deviation, edge occlusion, local reflection, and mismatch.

[0010] The beneficial effect of this application lies in the fact that by projecting randomly or quasi-randomly distributed infrared speckle patterns onto the road surface to be tested, relatively stable artificial texture features are formed on the road surface, enhancing the local grayscale changes and gradient response in weakly textured areas. This makes the correspondence in the binocular images no longer entirely dependent on the natural texture of the asphalt road surface. After the binocular cameras simultaneously acquire left and right images, combined with stereo correction, multi-scale adaptive feature fusion matching, and triangulation principles, a relatively dense and continuous 3D point cloud of the road surface within the target measurement area can be obtained. Compared with passive binocular stereo vision methods that rely solely on natural textures, this application is beneficial for improving the stereo matching stability and 3D point cloud reconstruction quality in weakly textured road surface areas. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the application process. Figure 2 This is a schematic diagram of the three-dimensional acquisition system in this application; Figure 3 This is a comparison diagram of the disparity results at different stages of stereo matching in this application; Figure 4 This is a three-dimensional point cloud reconstruction result of the asphalt pavement in this application; Among them, 1-target area surface, 2-infrared speckle projection device, 3-left industrial camera, 4-right industrial camera, 5-synchronous trigger unit, 6-circular hollow opaque acrylic sheet. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this application. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0013] like Figure 1 As shown, a binocular infrared speckle three-dimensional reconstruction method for asphalt pavement includes: S1. A three-dimensional acquisition system projects randomly or quasi-randomly distributed infrared speckle patterns onto the target area, and acquires left and right target images with speckle characteristics, followed by preprocessing. Specifically: like Figure 2 As shown, the 3D acquisition system includes: a support mechanism; a binocular industrial camera, comprising a left industrial camera 3, a right industrial camera 4, and a synchronization triggering unit 5, wherein the left industrial camera 3 and the right industrial camera 4 are fixedly mounted on the support mechanism, and the synchronization triggering unit 5 is used to control the left industrial camera 3 and the right industrial camera 4 to acquire left target images and right target images at the same time; an infrared speckle projection device 2, wherein the projection angle of the infrared speckle projection device 2 maintains a certain angle with the optical axis of the binocular camera, and is used to actively project infrared speckles onto the target area; and a circular hollow opaque acrylic plate 6, which is attached to the surface 1 of the target area to define the target area and form a clearly defined effective area. During acquisition, the circular hollow opaque acrylic plate 6 is attached to the surface of the asphalt pavement to be measured, so that the inner hole of the plate corresponds to the preset measurement area. The field of view of the binocular camera is adjusted to completely cover the preset measurement area, and the infrared speckle projection device 2 is turned on to form random or quasi-random speckle features within the measurement area. The size and density of the speckle feature should be adapted to the camera resolution, shooting distance, and road surface particle size to ensure distinguishability between adjacent speckles and avoid excessively dense speckles that could cause local texture adhesion. The projection angle of the infrared speckle projection device 2 should maintain a certain angle with the optical axis of the binocular camera, ensuring the speckle pattern covers the target area and avoids strong local reflections. The binocular camera can be used with an infrared narrowband filter to reduce the influence of visible light background on speckle imaging. During acquisition, the exposure time and gain can be adjusted according to the road surface reflection intensity to prevent overexposure or underexposure in the speckle area and maintain a relatively consistent brightness distribution between the left and right images.

[0014] The preprocessing includes: S1.1, based on the mask information provided by the inner boundary contour of the circular hollow opaque acrylic plate 6, the target region in the image is automatically extracted by the SAM2 segmentation model, and the background region outside the target region is removed; S1.2, the binocular industrial camera is calibrated to obtain the intrinsic parameters of the left and right cameras and the extrinsic parameter matrix between the binocular cameras, and then the distortion correction and stereo correction of the left target image and the right target image are performed based on the calibration parameters to obtain a coplanar and epipolar aligned corrected image.

[0015] S2. Based on the preprocessed left and right target images from S1, a multi-scale adaptive feature fusion stereo matching method is used to estimate the disparity of the left and right target images in the target region. Specifically, the disparity estimation for the left and right target images in the target region includes: S2.1. Constructing multi-scale image pyramids for the preprocessed left and right target images from S1, respectively, to enable the images to express road surface texture features and speckle features at different resolution levels. S2.2. Based on the multi-scale image pyramids of the left and right target images from S2.1, calculating gray-level differences, local structural similarities, and gradient change information at each scale, and forming a comprehensive matching cost function through weighted fusion; the comprehensive matching cost is subjected to guided filtering aggregation to improve the stability of the matching cost in weak texture regions and the ability to preserve edge regions. S2.3. Constructing an adaptive regularization weight map based on pixel gradients, local contrast, and texture richness from S2.2, to enable the regularization intensity to be dynamically adjusted according to changes in local road surface features. In applications, for regions with weak textures, insignificant grayscale changes, or weak speckle response, neighborhood consistency constraints are increased to reduce mismatches caused by random noise and repetitive textures; for texture edges, grain boundaries, or regions with abrupt changes in local elevation, excessive smoothing constraints are reduced to avoid over-smoothing of real undulation details, thereby balancing parallax continuity and the ability to preserve local details.

[0016] like Figure 3As shown, in step S3, based on the disparity estimation results obtained in step S2 at different scales, cross-scale fusion is performed to form a multi-scale adaptive feature fusion stereo matching result and obtain a fused disparity map. The fused disparity map is then optimized to obtain the final disparity map. Specifically, when performing cross-scale fusion, a nonlinear fusion method based on Softmax normalization weighting is adopted. The local matching confidence, texture response intensity, and edge preservation degree of the disparity candidate results at different scales are calculated respectively, and the weight of each scale result in the final disparity estimation is adaptively determined. The cross-scale fusion process includes: S3.1, determining the initial disparity range based on the low-resolution scale results; S3.2, performing local refinement in the high-resolution scale image; S3.3, the fused disparity map continues to undergo left-right consistency checks, occlusion repair, hole filling, and sub-pixel interpolation optimization to obtain a dense, continuous disparity map with good edge preservation. In applications, for stable but insufficiently detailed disparity results at low-resolution scales, global disparity constraints can be provided and the disparity search range at high-resolution scales can be narrowed. For disparity results with clear edges and sufficient local texture response at high-resolution scales, their fusion weights are increased to preserve pavement particle boundaries and subtle undulation information. For disparity results exhibiting isolated anomalies, local voids, or significant differences from adjacent scales at a certain scale, their weights are reduced to minimize the impact of anomalous scales on the final disparity map. Through the aforementioned multi-scale adaptive feature fusion stereo matching process, matching stability can be improved in weakly textured regions, and local geometric details can be preserved at texture edges and in areas of abrupt elevation changes, thereby improving the continuity, accuracy, and surface detail restoration capability of asphalt pavement 3D point cloud reconstruction.

[0017] like Figure 4 As shown, in step S4, based on the imaging geometry of the left and right target images in S1 and the binocular stereo calibration parameters, the final disparity map obtained in S3 is converted into three-dimensional coordinate information and processed into point cloud, ultimately yielding the three-dimensional point cloud reconstruction result of the target area. When converting the disparity map into three-dimensional coordinate information, based on the binocular imaging geometry and the binocular camera stereo calibration parameters, the disparity map obtained in S3.3 is converted into three-dimensional coordinate information, completing the three-dimensional point cloud reconstruction of the asphalt pavement target area. The binocular imaging geometry includes the target point depth Z being calculated as Z=fB / d, where f is the camera focal length, B is the binocular baseline length, and d is the disparity value corresponding to that point. The pixel coordinates are converted into spatial coordinates by combining the camera principal point coordinates. Point cloud processing includes coordinate unification, local reference plane correction, outlier removal, and statistical filtering of the reconstructed point cloud to remove abnormal points caused by camera pose deviation, edge occlusion, local reflection, and mismatch. In the application, the 3D point cloud in S4 is converted to the local coordinate system of the road surface, and standardized point cloud data or regularized elevation data is output according to the boundary of the target area, providing a data basis for road surface texture parameter extraction, construction depth calculation or reference point cloud comparison.

[0018] This application integrates active infrared speckle enhancement, physical measurement area limitation, and multi-scale stereo matching at the system level to form a complete 3D reconstruction technology solution for weak texture scenarios of asphalt pavement. This method features non-contact, full-field acquisition, and single-image acquisition of 3D information, ensuring measurement accuracy while meeting the needs of on-site deployment and rapid detection. Infrared speckle enhancement of pavement surface features enables dark asphalt pavement to obtain more stable and discriminative local textures in binocular images. This design reduces the dependence of disparity estimation on natural textures and reduces matching ambiguities in weak and repetitive texture areas. A circular hollow opaque acrylic plate (6) defines the measurement area, providing a clear boundary for the on-site acquisition area. This structure reduces interference from background, stray reflections, and edge-irrelevant regions outside the measurement area on the matching process and facilitates subsequent point cloud cropping, measurement area reproduction, and comparison with other detection methods within the same region. A multi-scale adaptive feature fusion stereo matching strategy is introduced, improving the matching stability of weak texture areas through multi-scale image representation, adaptive regularization constraints, and reliability weight fusion, while preserving local geometric details at particle edges and areas of abrupt elevation changes. After left-right consistency checks, occlusion repair, hole filling, and sub-pixel optimization, a disparity map with good continuity and fewer outliers can be obtained, thereby improving the depth resolution and surface detail restoration capabilities of the reconstructed point cloud. Following 3D reconstruction, coordinate unification, local plane correction, filtering and denoising, and region clipping are performed on the point cloud. The output 3D road surface point cloud data is more suitable for subsequent texture parameter extraction, depth assessment, or accuracy verification with reference measurement results. It does not require line-by-line scanning by the inspection vehicle or platform, nor does it require multi-step phase-shifting projection or complex phase unwrapping. After calibration, the system can perform a single synchronous acquisition of a defined measurement area and complete disparity calculation and point cloud reconstruction on the software side, making it more suitable for scenarios such as rapid on-site deployment, sample block detection, and re-measurement of local road measurement areas.

[0019] In summary, this application does not rely solely on a single projection pattern or a single matching cost for improvement. Instead, it combines infrared speckle enhancement, physical measurement area limitation, multi-scale adaptive feature fusion matching, and point cloud standardization processing to form a technical solution that is more suitable for three-dimensional reconstruction of on-site measurement areas of asphalt pavement.

Claims

1. A method for three-dimensional reconstruction of asphalt pavement using binocular infrared speckle, characterized in that, include: S1. A three-dimensional acquisition system is used to project randomly or quasi-randomly distributed infrared speckle patterns onto the target area, and the left and right target images with speckle features in the target area are acquired and then preprocessed. S2. Based on the preprocessed left and right target images in S1, a multi-scale adaptive feature fusion stereo matching method is used to estimate their disparity. S3. Based on the disparity estimation results at different scales obtained in S2, cross-scale fusion is performed to form a multi-scale adaptive feature fusion stereo matching result and a fused disparity map is obtained. The fused disparity map is then optimized to obtain the final disparity map. S4. Based on the imaging geometric relationship between the left and right target images in S1 and the stereo calibration parameters, the final disparity map obtained in S3 is converted into three-dimensional coordinate information and point cloud processing is performed to finally obtain the three-dimensional point cloud reconstruction result of the target area.

2. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 1, characterized in that, The three-dimensional acquisition system in S1 includes: Supporting institutions; A binocular industrial camera, comprising a left industrial camera, a right industrial camera, and a synchronization triggering unit, wherein the left industrial camera and the right industrial camera are respectively fixedly mounted on the support mechanism, and the synchronization triggering unit is used to control the left industrial camera and the right industrial camera to acquire left target image and right target image at the same time; An infrared speckle projection device, wherein the projection angle of the infrared speckle projection device is at a certain angle to the optical axis of the binocular camera, and is used to actively project infrared speckle onto the target area; A circular hollow opaque acrylic sheet is attached to the surface of the road surface to be tested, which is used to define the target area and form a clearly defined effective area so as to facilitate the automatic extraction of the target area in the future.

3. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 2, characterized in that, The preprocessing in S1 includes: S1.

1. Generate an initial mask for the measurement area based on the inner boundary contour of the circular hollow opaque acrylic plate, and input the initial mask as prompt information into the Segment Anything Model 2 (SAM2) segmentation model. The SAM2 segmentation model automatically generates a target mask for the effective measurement area of ​​the asphalt pavement in the input image. Automatically extract the target area image based on the target mask, and remove the background, edge occlusion areas, and irrelevant reflection areas outside the target area. S1.

2. Perform binocular calibration on the binocular industrial camera to obtain the intrinsic parameters of the left and right cameras and the extrinsic parameter matrix between the binocular cameras. Then, based on the calibration parameters, perform distortion correction and stereo correction on the left and right target images to obtain coplanar and epipolar aligned corrected images.

4. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 3, characterized in that, The disparity estimation of the left and right target images of the target region in S2 includes: S2.1 Construct multi-scale image pyramids for the preprocessed left and right target images in S1, respectively, to enable the images to express road surface texture features and speckle features at different resolution levels; S2.2 Based on the multi-scale image pyramid of the left and right target images in S2.1, grayscale differences, local structural similarities, and gradient change information are calculated at each scale, and a comprehensive matching cost function is formed by weighted fusion; the comprehensive matching cost is subjected to guided filtering aggregation to improve the stability of the matching cost in weak texture regions and the ability to preserve edge regions; S2.3 Construct an adaptive regularization weight map based on the pixel gradient, local contrast and texture richness in S2.2, so that the regularization intensity can be dynamically adjusted as the local features of the road surface change.

5. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 4, characterized in that, When performing cross-scale fusion in S3, a nonlinear fusion method based on Softmax normalization weighting is adopted to calculate the local matching confidence, texture response intensity and edge preservation degree of disparity candidate results at different scales, and adaptively determine the weight of each scale result in the final disparity estimation. The cross-scale fusion process includes: S3.1 Determine the initial parallax range using low-resolution scale results; S3.2, Perform local refinement on high-resolution scale images; S3.

3. The fused disparity map continues to undergo left-right consistency checks, occlusion repair, hole filling, and subpixel interpolation optimization to obtain a dense, continuous disparity map with well-preserved edges.

6. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 5, characterized in that, In step S4, when converting the disparity map into three-dimensional coordinate information, the disparity map obtained in step S3.3 is converted into three-dimensional coordinate information based on the binocular imaging geometry and the stereo calibration parameters of the binocular camera, thus completing the three-dimensional point cloud reconstruction of the target area of ​​the asphalt pavement. The binocular imaging geometry includes the following: the depth Z of the target point is calculated as Z=fB / d, where f is the camera focal length, B is the binocular baseline length, and d is the disparity value corresponding to the point. The pixel coordinates are converted into spatial coordinates by combining the camera principal point coordinates.

7. The binocular infrared speckle three-dimensional reconstruction method for asphalt pavement as described in claim 6, characterized in that, The point cloud processing in S4 includes coordinate unification, local reference plane correction, outlier removal, and statistical filtering of the reconstructed point cloud to remove abnormal points caused by camera attitude deviation, edge occlusion, local reflection, and mismatch.

Images