Subway Tunnel 3D Panoramic Image Submillimeter-Level Spatial Measurement System and Method

Abstract

This invention relates to a sub-millimeter-level spatial measurement system and method for 3D panoramic images of subway tunnels. Through joint adjustment of point cloud data acquisition and image information acquisition, a novel design combining 3D holographic modeling with traditional point cloud acquisition is proposed. This achieves high-resolution, large field-of-view, and full-domain high-precision 3D holographic modeling of tunnels, ensuring sub-millimeter-level accuracy in the 3D reconstruction and image coordinate assignment of high-definition subway tunnel images. It solves the problem of uneven accuracy and poor robustness in point cloud stitching caused by the limited features of point clouds in subway tunnels when using traditional laser scanning. Furthermore, by combining the features of high-definition subway tunnel images and point cloud data, mileage and attitude information are further optimized. Machine learning algorithms are used to remove non-tunnel fixed objects from the 3D reconstruction model of the tunnel, achieving sub-millimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence machine vision and navigation positioning technology, and relates to a sub-millimeter level spatial measurement system and method for three-dimensional panoramic imaging of subway tunnels. Background Art

[0002] As a crucial underground public transportation infrastructure, the efficient and safe operating environment of subway tunnels is paramount, necessitating intelligent safety monitoring throughout their entire lifecycle. Breaking away from traditional Leica robot point-based monitoring technology, the development of next-generation high-speed, wide-view, and large-area panoramic imaging technology has become the mainstream direction in this field. However, these images still lack three-dimensional spatial coordinate information of the tunnel, requiring the use of 3D laser point clouds to assign 3D values ​​to high-definition images. Currently, the algorithm for laser point cloud and image assignment still faces the challenge of achieving sub-millimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels through spatial measurement technology. Summary of the Invention

[0003] To address the problems existing in the above-mentioned traditional methods, this invention proposes a sub-millimeter-level spatial measurement system and a sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels, which can realize sub-millimeter-level reconstruction and assignment of three-dimensional panoramic images of subway tunnels.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] On the one hand, a sub-millimeter level spatial measurement system for three-dimensional panoramic imaging of subway tunnels is provided, including a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port, and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit, and the signal transmission unit are mounted on the mobile detection platform.

[0006] The point cloud data acquisition unit is used to acquire 3D laser point clouds of the subway tunnel at each inspection location; the mileage information acquisition unit is used to acquire high-precision mileage information of the mobile detection platform operating in the subway tunnel; the image information acquisition unit is used to acquire high-definition images of the subway tunnel at each inspection location; the mobile detection platform is used to move the units in the subway tunnel to conduct inspections in a multi-site, multi-station data acquisition manner; the signal transmission unit is used to wirelessly transmit the acquired information from the acquisition units to the cloud server for storage in real time; the digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operating status of the mobile detection platform; and the data processing fusion port is used to transmit the acquired information from the cloud server to the integrated data processing terminal.

[0007] The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit, and a data fusion unit. The target recognition unit identifies target information in high-definition images and 3D laser point clouds of the subway tunnel and then removes non-tunnel structures. The data preprocessing unit preprocesses the target information and the collected information. The data processing unit performs joint adjustment on the preprocessed high-definition images and 3D laser point clouds of the subway tunnel, constructs a dynamic control field, estimates the pose of the moving detection platform relative to global coordinates, and generates a global high-resolution 3D holographic image of the tunnel. The data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel. The data fusion unit uses a trained machine learning model to perform data fusion based on the global high-resolution 3D holographic image, 3D laser point cloud, and high-precision mileage information, and removes non-tunnel fixed objects.

[0008] On the other hand, a sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels is also provided, applied to the aforementioned sub-millimeter-level spatial measurement system for three-dimensional panoramic images of subway tunnels. This sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels includes the following steps:

[0009] The system acquires 3D laser point clouds of the subway tunnel at each inspection location using a point cloud data acquisition unit, high-precision mileage information of the mobile detection platform operating in the subway tunnel using a mileage information acquisition unit, and high-definition images of the subway tunnel at each inspection location using an image information acquisition unit.

[0010] The signal transmission unit is used to wirelessly transmit the information collected by the acquisition unit to the cloud server for storage in real time;

[0011] After identifying target information in high-definition images and 3D laser point clouds of subway tunnels by the target recognition unit, non-tunnel structures are eliminated.

[0012] The target information and collected information are preprocessed through the data preprocessing unit;

[0013] The data processing unit performs joint adjustment on the preprocessed high-definition images of the subway tunnel and the 3D laser point cloud to construct a dynamic control field and estimate the pose of the moving detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel; the data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel.

[0014] The data fusion unit uses a trained machine learning model to fuse data based on global high-resolution 3D holographic images, 3D laser point clouds, and high-precision mileage information, and removes non-tunnel fixed objects.

[0015] One of the above technical solutions has the following advantages and beneficial effects:

[0016] The aforementioned sub-millimeter-level spatial measurement system and method for 3D panoramic images of subway tunnels proposes a novel design that combines 3D holographic modeling with traditional point cloud acquisition. This design utilizes a 3D holographic stitching algorithm based on a panoramic camera array and a 3D laser scanner, achieving high-resolution, large field-of-view, and full-domain high-precision 3D holographic modeling of tunnels. This ensures sub-millimeter-level accuracy in the 3D reconstruction and coordinate assignment of high-definition subway tunnel images. It addresses the problem of uneven accuracy and poor robustness in point cloud stitching caused by the limited features of point clouds in subway tunnels, a problem inherent in traditional laser scanning. Furthermore, by combining the features of high-definition subway tunnel images and point cloud data, mileage and attitude information are further optimized, thus avoiding the issues of untimely mileage information acquisition and uneven accuracy in subway tunnel sections without signal coverage. The system employs a combination of machine vision and big data technologies to achieve dynamic target segmentation in the control field. Machine learning algorithms are used to remove non-tunnel fixed objects such as people and vehicles from the 3D reconstruction model. Global optimization through parallax mapping ensures the continuity of depth information from different viewpoints, reducing errors in parallax stitching. Ultimately, this achieves sub-millimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the overall architecture of a sub-millimeter-level spatial measurement system for 3D panoramic imaging of a subway tunnel in one embodiment.

[0019] Figure 2 This is a schematic diagram of the basic structure of a mobile detection platform in one embodiment;

[0020] Figure 3 This is a schematic diagram of the relative pose relationship of the motion detection platform in one embodiment;

[0021] Figure 4 This is a schematic diagram of the joint adjustment of a reference transfer camera array and a 3D laser scanner in one embodiment;

[0022] Figure 5 This is a schematic diagram of platform positioning and orientation using a combination of a panoramic camera array and a 3D laser scanner in one embodiment;

[0023] Figure 6This is a schematic diagram illustrating the process of joint optimization of localization and pose determination using a panoramic camera array, a 3D laser scanner, and an IMU in one embodiment.

[0024] Figure 7 This is a flowchart illustrating a sub-millimeter spatial measurement method for three-dimensional panoramic images of a subway tunnel in one embodiment.

[0025] The components include: 1. Panel controller; 2. Transmission reference camera; 3. Surround view camera; 4. 3D laser scanner; 5. Battery box; 6. Guide wheel; 7. Tensioning component; 8. Brake pad; 9. Running wheel; 10. Handle; 11. Cable. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0027] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0028] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0029] In one embodiment, such as Figure 1 As shown, a sub-millimeter level spatial measurement system for three-dimensional panoramic imaging of subway tunnels is provided, including a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port, and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit, and the signal transmission unit are mounted on the mobile detection platform.

[0030] The point cloud data acquisition unit is used to acquire 3D laser point clouds of the subway tunnel at each inspection location; the mileage information acquisition unit is used to acquire high-precision mileage information of the mobile detection platform operating in the subway tunnel; the image information acquisition unit is used to acquire high-definition images of the subway tunnel at each inspection location; the mobile detection platform is used to move the units in the subway tunnel to conduct inspections in a multi-site, multi-station data acquisition manner; the signal transmission unit is used to wirelessly transmit the acquired information from the acquisition units to the cloud server for storage in real time; the digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operating status of the mobile detection platform; and the data processing fusion port is used to transmit the acquired information from the cloud server to the integrated data processing terminal.

[0031] The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit, and a data fusion unit. The target recognition unit identifies target information in high-definition images and 3D laser point clouds of the subway tunnel and then removes non-tunnel structures. The data preprocessing unit preprocesses the target information and the collected information. The data processing unit performs joint adjustment on the preprocessed high-definition images and 3D laser point clouds of the subway tunnel, constructs a dynamic control field, estimates the pose of the moving detection platform relative to global coordinates, and generates a global high-resolution 3D holographic image of the tunnel. The data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel. The data fusion unit uses a trained machine learning model to perform data fusion based on the global high-resolution 3D holographic image, 3D laser point cloud, and high-precision mileage information, and removes non-tunnel fixed objects.

[0032] Understandable, such as Figure 2 The diagram shows the mobile inspection platform and its various acquisition units. The panel controller 1 can be used for on-site operation of the mobile inspection platform. The reference camera 2 and the surround-view camera 3 together form the image information acquisition unit. The panel controller 1, the image information acquisition unit, and the 3D laser scanner 4 can all be powered by the batteries built into the battery box 5. The running wheels 9 and guide wheels 6 are the movement mechanisms for the mobile inspection platform during inspections. The tensioning component 7 and brake pads 8 are used for braking. The handle 10 can be used to manually move the mobile inspection platform. The cable 11 is used for electrical connections between the various unit structures.

[0033] The image information acquisition unit may include a surround-view camera array and a reference transfer camera array. The reference transfer camera array includes fixed-mounted front-view and rear-view cameras, used to transfer control information from both ends of the tunnel to the survey area. The surround-view camera array includes at least two high-resolution cameras with overlapping fields of view, used to acquire high-definition images of the subway tunnel at each inspection location. The data processing and fusion port may include a platform data return port, a fiber optic grating return port, and a 3D laser scanning port, used to transmit the acquired information stored in the cloud server to the integrated data processing terminal.

[0034] Specifically, the target information identified by the target recognition unit refers to the structural information of various parts belonging to the tunnel, as opposed to non-tunnel structures such as electrical boxes, supports, and signal lights. The target recognition unit can perform preliminary screening and elimination of these non-tunnel structures. The data preprocessing unit performs preprocessing on the target information and the collected information. For example, it can include simple background weakening processing on structures that may not belong to the subway tunnel in the high-definition images of the subway tunnel, eliminating some images whose resolution does not meet the inspection requirements, and performing common preprocessing operations such as denoising, filtering, and enhancement on the remaining information. For 3D laser point clouds, the SOR statistical filtering algorithm can be used for denoising and outlier removal, and the extended Kalman filter algorithm can be used for denoising the collected high-precision mileage information.

[0035] The data preprocessing unit first synchronizes and registers information such as 3D laser point clouds, high-precision mileage information, and high-definition images of subway tunnels, ensuring that all acquisition units use a unified timestamp reference (this can be achieved using a time synchronization trigger). Simultaneously, it unifies the reference transfer camera, surround-view camera, 3D laser scanner in the point cloud data acquisition unit, and positioning sensors in the mileage information acquisition unit to the platform coordinate system through external parameter calibration or manual conversion. Next, it filters, denoises, and normalizes the 3D laser point clouds and high-precision mileage information to prevent noisy data from affecting subsequent reconstruction quality. Then, it performs denoising, distortion correction, and illumination unification operations on the high-definition images of the subway tunnels, preparing for subsequent feature extraction, image stitching, and point cloud mapping.

[0036] The data processing unit performs joint adjustment on the preprocessed high-resolution subway tunnel images and 3D laser point clouds to obtain a global high-resolution 3D holographic image. The detailed implementation will be described later. The data fusion unit can train a machine learning model and combine it with big data processing techniques to divide the entire dynamic control field space into a 3D voxel grid, achieving target segmentation and the construction of a global illumination field. The machine learning model can employ existing deep learning models suitable for data fusion and target recognition, and can be pre-trained using existing tunnel inspection databases in this field.

[0037] Adjustment is a mathematical processing method based on statistics and error theory. Its core purpose is to eliminate or reduce the influence of various errors in measurement data. Essentially, it involves analyzing redundant observation data, rationally allocating errors, and ensuring that the measurement results meet certain accuracy requirements and logical consistency, thereby obtaining more reliable and reasonable measurement results and providing accurate data support for tunnel deformation analysis and safety assessment.

[0038] The aforementioned sub-millimeter-level spatial measurement system for 3D panoramic imaging of subway tunnels proposes a novel design that combines 3D holographic modeling with traditional point cloud acquisition. This design utilizes a 3D holographic stitching algorithm, achieved through image information acquisition units and a 3D laser scanner, to combine adjustment. This enables high-resolution, large-field-of-view, and high-precision 3D holographic modeling of the tunnel, ensuring sub-millimeter-level accuracy in the 3D reconstruction and coordinate assignment of high-definition subway tunnel images. It addresses the problem of uneven accuracy and poor robustness in point cloud stitching caused by the limited features of point clouds in subway tunnels, a common issue with traditional laser scanning. Furthermore, by employing a combined navigation system of IMU (Inertial Measurement Unit), GNSS (Global Navigation Satellite System), and DMI (Odometer), the system ensures that the mileage and attitude information of the mobile detection platform are accurately recorded during its operation between adjacent key locations within the subway tunnel. This guarantees sub-millimeter-level accuracy in the constructed 3D holographic image of the tunnel. Simultaneously, by combining the features of high-definition subway tunnel images and point cloud data, the system further optimizes the mileage and attitude information, thus avoiding the problems of untimely mileage information acquisition and uneven accuracy in subway tunnel sections without signal coverage. By combining machine vision and big data technologies, target segmentation in a dynamic control field can be achieved. Machine learning algorithms are used to remove non-tunnel fixed objects such as people and vehicles from the 3D reconstruction model of the tunnel. Through global optimization of parallax mapping, the continuity of depth information under different perspectives is ensured, and errors in parallax stitching are reduced. Ultimately, sub-millimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels are achieved.

[0039] Compared to traditional technologies, this technology solves the problem that it is difficult to accurately locate and detect cracks with geometric dimensions less than 1 mm and tunnel segment damage in subway tunnels due to the difficulty in spatial coordinate positioning of high-definition images and the inability to achieve sub-millimeter accuracy. It meets the needs of detecting and statistically analyzing apparent cracks and fissures in subway tunnels while ensuring sub-millimeter positioning of high-definition images.

[0040] It should be noted that there is a certain overlap between the high-resolution cameras in the surround-view camera array. During the inspection process, the mobile inspection platform ensures that each camera in the surround-view camera array and the 3D laser scanner have a certain overlap between adjacent inspection positions. After completing one inspection, based on the image corresponding points in the high-resolution surround-view images of the surround-view camera array at adjacent inspection positions, the constraints between the image corresponding points and the 3D point cloud corresponding points in the point cloud data, and combined with the fixed constraints of the reference transfer camera array and the 3D laser scanner, the 3D displacement of key measuring points in the survey area relative to the global coordinate system is quickly estimated through joint adjustment of the reference transfer camera array and the 3D laser scanner under the condition of limited external control information, thereby constructing a high-precision dynamic control field with known positions. Finally, based on the constructed dynamic control field, the pose of the measurement platform at adjacent inspection positions relative to the global coordinate system is quickly estimated, thereby realizing the rapid stitching of the 3D point cloud. Combined with the fixed constraints of the surround-view cameras and the 3D laser scanner, the texture of the surround-view cameras is rapidly mapped, thus obtaining a global high-resolution 3D holographic image of the tunnel.

[0041] Meanwhile, a GNSS / INS / DMI integrated navigation algorithm is used to improve overall positioning accuracy between adjacent key locations in the dynamic control field. Feature points in high-resolution images of the subway tunnel are identified as control points (control point coordinates can be obtained through total station marking or other high-precision 3D measurement equipment). Centered on the central pixel location, the surrounding area is extracted as the target image region. A CNN (Convolutional Neural Network) algorithm is then applied to the high-resolution subway tunnel images to automatically detect and extract feature points and generate feature descriptions. This allows for the synchronization of timestamps and the use of machine learning models to compensate for defects in the reconstructed model and mileage positioning accuracy during subsequent mileage positioning assignment.

[0042] By combining machine learning models and big data processing technologies, the target segmentation of the entire dynamic control field is completed. For example, semantic segmentation networks (such as DeepLabv3+ models) are used to segment the image, identify the "tunnel structure region" and "non-tunnel fixed object region" (such as electrical boxes, brackets, signal lights and pipelines) in the global high-resolution three-dimensional holographic image, and remove the non-tunnel structure region. Based on the segmentation mask, the points belonging to the tunnel region in the three-dimensional laser point cloud are extracted, and the non-tunnel fixed objects are removed.

[0043] In one embodiment, the data fusion unit is further used to extract the image features of each branch of the global high-resolution three-dimensional holographic image from different perspectives through a multi-branch convolutional neural network structure, and then use a feature weighting mechanism to fuse the image features of each branch. The fused feature vector is then reconstructed by a decoder into a global high-resolution three-dimensional holographic image with uniform illumination and coherent texture.

[0044] It is understood that in this embodiment, the data fusion unit can also be used to adjust the global illumination field. Global high-resolution 3D holographic images from different viewpoints are obtained from the target recognition unit and constructed into multiple independent branches. Each branch extracts the illumination features, color information, and texture structure of the corresponding viewpoint image. Each branch can employ a lightweight convolutional neural network (CNN) to achieve efficient feature extraction and low-resource model operation, constructing a multi-branch CNN structure (based on the CNN structure). The image features extracted by each branch neural network are fused, and a feature weighting mechanism (such as an existing channel attention module or spatial attention module) is introduced to assign differentiated weights to features from different viewpoints, thereby enhancing the information expression of the main viewpoint or key areas. Subsequently, the weighted and fused feature vectors are input into the decoder to reconstruct an image with uniform illumination and coherent texture. Global optimization processing is performed on the fused and reconstructed global high-resolution 3D holographic image to improve the brightness consistency, color balance, and detail fidelity of the image. A pre-trained image enhancement model is used to perform brightness smoothing, color correction, and contrast enhancement on each frame of the global high-resolution 3D holographic image, reducing visual inconsistencies caused by exposure differences and shadow interference in multi-view images. The image sequences that have undergone illumination correction and enhancement are synthesized in chronological order to generate a continuous image stream or video sequence, ensuring a consistent overall visual style.

[0045] Subsequently, combining laser point cloud data, the corrected global high-resolution 3D holographic image was used for 3D texture mapping and structural modeling to verify the effectiveness of the illumination field adjustment. The impact of image processing on the modeling results was evaluated by comparing the changes in surface brightness, texture consistency, and transition smoothness of the stitched areas before and after illumination adjustment. If the final 3D tunnel structure model shows no obvious shadow jumps, color breaks, or illumination artifacts, then the global illumination field adjustment was considered complete.

[0046] The stitching effect of the above images can be tested using 3D holographic image interaction software (such as Unity or Unreal Engine). If the images remain continuous and consistent when the user adjusts the viewing angle, and the inspection path and real-time position of the mobile detection platform can be dynamically updated, then it can be determined that the sub-millimeter-level reconstruction and assignment of the 3D panoramic image of the subway tunnel has been completed.

[0047] It is understandable that in the aforementioned process, a multi-branch convolutional neural network structure is constructed, and a feature weighting mechanism is introduced to fuse the features of each branch, thereby enhancing the expressive power of the main viewpoint or key regions. The fused feature vectors are then reconstructed by the decoder into a globally high-resolution 3D holographic image with uniform illumination and coherent texture.

[0048] Furthermore, pre-trained existing image enhancement models can be used to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution 3D holographic images, completing global illumination optimization. The processed global high-resolution 3D holographic image sequence is then synthesized into a continuous image stream or video in chronological order to ensure consistent overall illumination style. Finally, the corrected global high-resolution 3D holographic images, 3D laser point clouds, and high-precision mileage information are fused for image 3D texture mapping and structural modeling. The effectiveness of the illumination field adjustment can be verified by comparing the performance of the subway tunnel 3D model before and after adjustment in terms of brightness, texture, and stitching smoothness, ultimately achieving sub-millimeter-level reconstruction and assignment of 3D panoramic images of the subway tunnel.

[0049] In one embodiment, the data processing unit divides the space of the dynamic control field into voxel grids of the same size, assigns grid values ​​using a nearest neighbor matching algorithm or an interpolation algorithm, identifies tunnel structure regions and non-tunnel fixed object regions in the image using a semantic segmentation network, masks non-tunnel regions using a mask generated by segmentation, generates a three-dimensional texture using the tunnel structure region, and performs texture mapping; each voxel grid is used to represent a tunnel segment.

[0050] It is understandable that the data processing unit can divide the entire dynamic control field space into voxel grids of the same size, with each voxel grid representing a tunnel segment to facilitate the positioning of the tunnel segment. Point cloud information and odometer information are assigned to the corresponding grids based on the coordinates of each point, using nearest neighbor matching or interpolation algorithms. After grid assignment, a semantic segmentation network (DeepLabv3+ model) is used to perform pixel-level semantic classification of the image, identifying "tunnel structure regions" and "non-tunnel fixed object regions." The non-tunnel regions in the global high-resolution 3D holographic image are masked using the segmented mask image, and further point cloud data corresponding to non-target regions are removed in 3D space through image-point cloud projection registration. Subsequently, a 3D texture is generated using the tunnel structure regions retained in the global high-resolution 3D holographic image, and combined with camera extrinsic parameters and the pose information of the moving detection platform, this 3D texture is accurately mapped onto the grid surface of the tunnel 3D structure model. Since the dynamic control field has achieved real-time positioning and orientation of the vehicle's operating status, it can ensure that the mapping accuracy reaches the sub-millimeter level during the texture mapping process, avoiding texture errors, blurring or misalignment.

[0051] In one embodiment, when the point cloud data acquisition unit acquires a 3D laser point cloud, it acquires the relative coordinates, distance, and timestamp of the measurement location; when the mileage information acquisition unit acquires high-precision mileage information, it acquires the mileage, motion trajectory, attitude, and timestamp of the mobile detection platform; and when the image information acquisition unit acquires high-definition images of the subway tunnel, it also records the timestamp of the image.

[0052] Understandably, during data acquisition, the intrinsic, extrinsic, and distortion parameters of the panoramic camera array and 3D laser scanner are first calibrated. An IMU is used to ensure that the panoramic camera array, 3D laser scanner, and other sensors maintain time and position synchronization. A mobile detection platform drives each acquisition unit to repeatedly collect information from multiple stations within the subway tunnel, obtaining more complete and accurate high-definition images, 3D laser point clouds, and high-precision mileage information. This facilitates the subsequent construction of a dynamic control field and the 3D holographic stitching of the subway tunnel. Specifically, acquiring the 3D laser point cloud of the subway tunnel includes collecting the relative coordinates, distance, and timestamp of the measurement location. Acquiring high-precision mileage information includes collecting the mileage, motion trajectory, attitude, and timestamp of the mobile detection platform. When acquiring high-definition images of the subway tunnel, the timestamp of the image is also recorded. This acquired information is transmitted in real-time to the cloud server via wireless transmission technology (such as Wi-Fi / 5G networking).

[0053] During the data processing stage, the data processing unit extracts features from the high-resolution images of the subway tunnel and establishes feature descriptions to ensure reliable feature matching for each frame, providing support for image and point cloud registration. By extracting geometrically significant points (such as corner points and planes) from the 3D laser point cloud through the data processing unit, the high-resolution images of the subway tunnel and the 3D laser point cloud can be accurately registered in space. To achieve better reconstruction of the 3D holographic image from the high-resolution images of the subway tunnel and the 3D laser point cloud, a 3D holographic stitching algorithm based on joint adjustment of a panoramic camera array and a 3D laser scanner will be used.

[0054] However, prior to this, it is necessary to align the high-resolution images of the subway tunnel with the dynamic coordinate system of the 3D laser point cloud, mapping the multi-source data uniformly to the platform coordinate system of the mobile detection platform. This ensures a good match between the subsequent spatial generation of the control tensor and point cloud modeling. After processing with a 3D holographic stitching algorithm using joint adjustment of a panoramic camera array and a 3D laser scanner, a dynamic control field for the camera array and the 3D laser scanner, as well as the platform positioning and orientation of the mobile detection platform, are constructed. At key locations in the dynamic control field, adjacent locations are positioned using a GNSS / INS / DMI integrated navigation algorithm for progress mileage positioning, ensuring high accuracy between adjacent key locations within the subway tunnel.

[0055] In one embodiment, the three-dimensional holographic stitching algorithm jointly adjusted by the image information acquisition unit and the three-dimensional laser scanner may include the following processing steps:

[0056] Offline calibration of panoramic camera arrays and 3D laser scanners on mobile inspection platforms; the panoramic camera array includes a surround-view camera array and a reference transfer camera array.

[0057] The mobile inspection platform is moved to each inspection position, and a high-resolution panoramic image of the tunnel at each inspection position is obtained by using a panoramic camera array. The reference transfer camera array is used to take pictures and extract the center coordinates of the measurement points in the survey area. A three-dimensional laser scanner is used to collect the three-dimensional laser point cloud in the survey area.

[0058] Based on the center coordinates of the measurement points in the survey area and the control information at both ends of the tunnel, the benchmark transfer camera array and three-dimensional laser scanner at different inspection positions are jointly adjusted to construct a dynamic control field with known displacement.

[0059] Based on the dynamic control field, using three or more dynamic control points observed by the surround-view camera array and the 3D laser scanner, the pose of the measurement platform relative to the global coordinate system is estimated, and then the 3D point cloud obtained by the 3D laser scanner is stitched together to the global coordinate system.

[0060] Based on the fixed constraint between the surround-view camera array and the 3D laser scanner, the high-resolution surround-view image is mapped to the 3D point cloud data in the global coordinate system to obtain a global high-resolution 3D holographic image of the tunnel.

[0061] Specifically, a dynamic control field is constructed based on the joint adjustment of the reference transfer camera array and the 3D laser scanner in the panoramic camera array. Using path planning software, the cameras and 3D laser scanner on the moving detection platform (including inertial navigation and accelerometer components) can be controlled to repeat images at designated locations. However, due to the enclosed environment inside the tunnel, even when the platform is stationary, the positioning and orientation of the moving detection platform will still have errors. To meet the needs of high-precision measurement, this embodiment constructs a high-precision robust estimation scheme for the relative pose of the moving platform using the reference transfer camera array and the 3D laser scanner.

[0062] like Figure 3 As shown, the relative pose of the moving detection platform refers to the platform coordinate system at the current time t1. Relative to the platform coordinate system at initial time t0 position (Use rigid body transformation matrices to represent pose). Figure 3 Unlabeled dots represent other spatial points. For spatial points... The imaging processes at time t1 and time t0 can be described as follows:

[0063] (1)

[0064] (2)

[0065] in, The depth factor of space at time t0. The depth factor of the space at time t1. This represents the homogeneous coordinates of the image point corresponding to the spatial point at time t0. This represents the homogeneous coordinates of the image point corresponding to the spatial point at time t1. Represents the intrinsic parameter matrix, Represents a three-row, one-column zero vector. This represents the pose transformation from the moving detection platform to the camera coordinate system. Represents the world coordinate system Pose transformation of the platform coordinate system at time t0.

[0066] It's understandable that in actual engineering surveying, a total station can be used to construct control fields (i.e., control information at both ends of the tunnel) at both ends of the tunnel. If the total station has been leveled, the origin of the platform coordinate system can be set at the center of the station, and the coordinate axes of the platform coordinate system can be parallel to the coordinate axes of the total station. Subsequent measurement results will then be referenced to the total station coordinate system. Given the camera's intrinsic parameters, existing PnP (Perspective-n-Point, a method for estimating camera pose) or NPnP (Non-perspective PnP, a method for multi-camera pose estimation) methods require first calculating the transformation from the world coordinate system W (control coordinate system) to the platform coordinate system at time t1 and t0, respectively. and Then, the relative pose of the moving detection platform is solved. This method belongs to indirect relative pose estimation.

[0067] Unlike existing design approaches, this embodiment constructs a relative pose estimation difference model, transforming the indirect relative pose estimation problem into a direct pose estimation problem. In actual engineering surveying, control points are far from the measurement area, and the change in the spatial point depth factor caused by the shaking of the measurement platform is relatively small. Therefore, the assumption of depth factor invariance can be introduced, i.e. Thus, the relative pose estimation difference model can be obtained as follows:

[0068] (3)

[0069] in, This represents a 4x4 identity matrix. Represents the platform coordinate system at time t0. The spatial points. Because the platform position and attitude of the inspection vehicle change little at the two locations where it takes photos before and after, an exponential form can be used to describe it. as follows:

[0070] (4)

[0071] in, Represents velocity and angular velocity parameters. Let represent the basis matrix of the Lie algebra for rigid body transformation.

[0072] For large engineering structures such as tunnels, control points are typically selected from stable structures far from the measurement area. This requires multiple levels of (camera) stations to transmit measurements and transfer control point information to the measurement area. For example... Figure 4 As shown, the reference transfer camera array and 3D laser scanner of the mobile inspection platform at different locations (such as inspection location 1 to inspection location 3) can be regarded as a joint network. For adjacent inspection locations, the common field of view between the reference transfer camera arrays can establish the connection between the reference transfer camera arrays at different inspection locations. In this way, joint adjustment can be achieved. Figure 4 In the diagram, P0 to P2 represent each point set, and each point set includes multiple measurement points.

[0073] Compared to the initial state, the target points within the measurement area may undergo three-dimensional displacement. Therefore, the model described by formula (3) can be modified as follows:

[0074] (5)

[0075] in, Let be the displacement of the target point in the initial platform coordinate system. If this point is a control point, then... .

[0076] Similarly, for 3D laser scanners:

[0077] (6)

[0078] in, This provides the position information of the target point in the 3D laser scanner coordinate system L at the initial time t0. Let L be the rigid body transformation matrix between the platform coordinate system and the 3D laser scanner coordinate system L at the initial position. To obtain the position information of the target point in the 3D laser scanner coordinate system L at time t1, a dynamic control field is constructed, which involves constructing a transformation between the three coordinate systems to ensure high accuracy.

[0079] The differential measurement model can be further constructed as follows:

[0080] (7)

[0081] Considering the three-dimensional displacement of the measuring point, a measurement equation similar to formula (5) can be further constructed as follows:

[0082] (8)

[0083] Equations (5) and (8) are the measurement equations for the joint adjustment of the reference transfer camera array and the 3D laser scanner. The observation constraints available in the dynamic network can be divided into two main categories: different target points observed by the reference transfer camera array at the same inspection position, and different target points observed by the 3D laser scanner, corresponding to the same relative pose parameters; and the same target point observed by the reference transfer camera array and the 3D laser scanner at different inspection positions, corresponding to the same displacement parameters. Based on these two types of observation constraints, a set of equations concerning the relative attitude parameters of the moving detection platform and the displacement of the target point at different inspection positions can be established.

[0084] Platform positioning and attitude determination based on a surround-view camera array and a 3D laser scanner can achieve the transfer of peripheral control information to the survey area through joint adjustment of the reference transfer camera array and the 3D laser scanner. The displacement of the measuring points within the survey area is measured, and the measuring points with known displacements can be used as dynamic control points to construct a dynamic control field. Based on this dynamic control field, high-precision positioning and attitude determination of the mobile inspection platform at each inspection location can be achieved.

[0085] By using only a surround-view camera array and providing three corresponding points (the sum of the number of dynamic control points observed by the surround-view camera array), the NPnP method can be used to linearly and analytically solve for the platform's pose relative to the global coordinate system, thus achieving platform positioning and orientation. Furthermore, by combining high-precision local coordinate data of the dynamic control points measured by a 3D laser scanner, providing only three points (the sum of the number of dynamic control points observed by the surround-view camera array and the number observed by the 3D laser scanner) allows for even more precise positioning and orientation of the platform.

[0086] Specifically, the imaging model of a camera array system composed of multiple panoramic cameras can be modeled using a non-perspective generalized camera system, such as... Figure 5 As shown, P 1 to P 4 represents each dynamic control point. It is a point in space. C 1 to C 4 represents each of the surround-view cameras. This indicates the bias of the 3D laser scanner in a multi-camera system. Represents the global coordinate system (i.e., the world coordinate system). R , T These represent the rotation and translation from the global coordinate system to the platform coordinate system, corresponding to the platform's pose and position in the global coordinate system. In this model, the relationship between 3D coordinates and their corresponding image point coordinates can be expressed as:

[0087] (9)

[0088] in, , , This represents the normalized direction vector from the camera to the 3D point. This indicates the camera offset in the platform coordinate system. and Let represent the rotation matrix and translation vector from the platform coordinate system to the camera coordinate system, respectively. This represents the intrinsic parameter matrix of the camera. is the mathematical symbol for a special orthogonal group. This represents the coefficient matrix from the camera to the 3D points. to These represent the specific components of the rotation matrix. to They represent x, y, z The translation vector of coordinates from the global coordinate system to the platform coordinate system. , and They represent x, y, z The offset position of the coordinates.

[0089] for n The correspondence between 2D and 3D points can be obtained from formula (9):

[0090] (10)

[0091] Formula (10) can be rewritten as:

[0092] (11)

[0093] Among them, each matrix , , , , In formula (10) I Represents the identity matrix. , and These represent the three-dimensional offset positions of the camera in the global coordinate system.

[0094] For the case where control points are coplanar, the default coordinate of the 3D point in the Z direction is 0. Substituting this into formula (11) results in a rotation matrix. R The third column is eliminated. Therefore, with the control points coplanar, the matrix... , b and v Dimensions unchanged, matrix U Become 3 n ×6 matrix, matrix xIt becomes a 6-dimensional vector.

[0095] Given a matrix x The matrix can be solved using the least squares method. b :

[0096] (12)

[0097] in, express The false inverse, that is Substituting formula (12) into formula (11) yields only the result regarding... x The system of linear equations:

[0098] (13)

[0099] in, , .

[0100] Due to the presence of noise, equation (13) generally cannot be satisfied. Therefore, it is transformed into a least squares problem by minimizing the sum of squared errors:

[0101] (14)

[0102] To facilitate global optimization through a polynomial system solution, the rotation matrix is ​​represented by a unit quaternion ( a , b , c , d )express:

[0103] (15)

[0104] in, Substituting formula (15) into formula (14) yields the cost equation:

[0105] (16)

[0106] in, , M It is about 2 n ×11 matrix.

[0107] Similarly, for a 3D laser scanner, the 3D coordinates in the global coordinate system and their corresponding laser scanning point coordinates can be written as:

[0108] (17)

[0109] in, , and These represent the rotation matrix and translation vector from the platform coordinate system to the 3D laser scanner coordinate system, respectively. The rotation matrix representing the 3D laser scanner's coordinate system relative to the global coordinate system. This represents the three-dimensional coordinate information of any point in space within the coordinate system of a 3D laser scanner. It is the rotation matrix from the global coordinate system to the platform coordinate system. It is the three-dimensional coordinate of any point in space in the global coordinate system.

[0110] The cost equation of the form of formula (16) can also be constructed by combining formula (9) and formula (17).

[0111] The cost equation shown in formula (16) can be solved using null space analysis to obtain the unit quaternion corresponding to the rotation of the mobile detection platform relative to the global coordinate system. Substituting it into formula (14) yields the rotation matrix of the mobile detection platform relative to the global coordinate system. Further substituting it into formula (8) yields the translation vector of the mobile detection platform relative to the global coordinate system. Based on the linear analytical solution, a nonlinear optimization error function can be further constructed to iteratively optimize the rotation matrix and translation vector, thereby achieving high-precision positioning and attitude determination of the mobile detection platform.

[0112] By jointly adjusting a panoramic camera array and a 3D laser scanner, a dynamic control field can be constructed, and analytical solutions for the position and attitude of the mobile detection platform can be obtained. Based on this, a nonlinear optimization model can be constructed using an IMU (Integrated Measurement Unit) to achieve high-precision estimation of the position and attitude of the mobile detection platform. The high-precision positioning and attitude fusion process of the platform is as follows: Figure 6 As shown. To reduce computational complexity, the sliding window BA algorithm described below is used for the solution. The sliding window is only responsible for optimizing a certain number of frames of data. After the number of frames in the window reaches a set value, each newly added frame is added to the first frame in the window. However, the first frame is not discarded directly because it contains valid observation data and information. Here, the previously observed information needs to be integrated into a priori information and included in the subsequent data optimization. The VIO optimization equation based on the sliding window is as follows:

[0113] (18)

[0114] Among them, the variables to be optimized , These are the position and attitude parameters of the mobile detection platform, including i Position of the real-time motion detection platform in the global coordinate system ,attitude ,speed And IMU acceleration and angular velocity bias estimation Here n and m These are the state variables of the mobile detection platform and the start time of the dynamic control point within the sliding window, i.e., the starting time of the sliding window from the [number]th [time]. n Frame number m The starting point is a road sign. N This represents the number of keyframes in the sliding window. M This represents the number of dynamic control points observed across all keyframes within the sliding window. The optimal pose is estimated by minimizing the sum of these state variable residuals.

[0115] Describes a robust function for prior terms; express n The estimated position and attitude of the motion detection platform at time -1, and the inverse transformation of the state of the motion detection platform within the sliding window. n Reverse prediction at time -1. The coefficient matrix representing the prior constraints; express Robust function of the term; A robust function representing an image item; Describes the robust function of the laser term; Indicates IMU measurement, This represents the covariance of IMU measurements; Representing image features, Represents the covariance of image observations; Indicates laser measurement, Represents the covariance of laser measurements; Indicates the motion measurement error of the IMU. Indicates image observation error. This indicates the error in laser measurement.

[0116] Taking any panoramic camera as an example, the error equation is: j The reprojection error of the dynamic control points observed by the time-lapse camera is specifically defined as follows:

[0117] (19)

[0118] in, This represents the image coordinates of the corresponding spatial point extracted from the image. If the corresponding spatial point is not observed in the image captured by the panoramic camera array, the corresponding error term is set to zero. These are the homogeneous coordinates corresponding to the three-dimensional coordinates of the dynamic control points in the global coordinate system. This represents the rigid body transformation matrix corresponding to the position and orientation of the mobile detection platform. This is the camera calibration transformation matrix. In the first Midpoint of the image captured by the camera Image plane observation x coordinate, In the first Midpoint of the image captured by the camera Image plane observation y coordinate, In the first Midpoint of the image captured by the camera Image plane observation z coordinate.

[0119] For a 3D laser scanner, the error equation is: j The coordinate transformation error of the dynamic control points scanned by the 3D laser scanner is defined as follows:

[0120] (20)

[0121] in, This refers to the observed coordinates of the corresponding spatial point extracted from the 3D laser scanner. If the corresponding point is not observed in the 3D laser scanner, the corresponding error term is set to zero. This represents the pose transformation matrix between the 3D laser scanner coordinate system and the platform coordinate system. , and They represent the first The corresponding spatial point x , y , z Global coordinates, represented in homogeneous form.

[0122] For IMUs, the measurement model is as follows:

[0123] (twenty one)

[0124] The superscript g indicates a gyroscope. a Indicates accelerometer, This is the acceleration due to gravity; the actual value from the IMU is... and The measured value is and According to the IMU dynamics equations, the derivatives of position P (osition), velocity v (elocity), and attitude q (uaternion) with respect to time between adjacent observations can be written as:

[0125] (twenty two)

[0126] in, The time interval between two observations This is four-element multiplication. This represents the position vector from the global coordinate system to the platform coordinate system at the current moment. Let be the attitude quaternion at the current moment, and represent the rotation matrix from the platform coordinate system to the global coordinate system. For the bias term of the accelerometer, This is the bias term for the gyroscope. , and These represent the changes in displacement, velocity, and attitude, respectively, and the specific calculations are as follows:

[0127] (twenty three)

[0128] Because the IMU has a high sampling frequency, i and i There will be many discrete observations within time +1, and the continuous integral expressed in the above formula can be converted into a summation operation using the mean value integration method.

[0129] Furthermore, by taking the changes in displacement, velocity, and attitude calculated according to formula (23) as observed values ​​and the changes in displacement, velocity, and attitude derived according to formula (22) as predicted values, the error equation of the IMU can be constructed as follows:

[0130] (twenty four)

[0131] In one embodiment, the data processing unit employs a GNSS / INS / DMI integrated navigation algorithm between two adjacent key locations to ensure that the reconstructed 3D imagery maintains high accuracy. The GNSS / INS / DMI integrated navigation algorithm includes the following processing steps:

[0132] The measurement model of the GNSS / INS / DMI integrated navigation algorithm is used to perform data fusion of GNSS, INS and DMI measurement data and attitude prediction of the mobile detection platform.

[0133] Perform measurement error analysis on control points;

[0134] Data fusion analysis is performed based on the coordinate measurements and measurement errors of the control points;

[0135] Error compensation for all control points is performed using a least-squares configuration.

[0136] Specifically, data fusion and attitude prediction are performed first. The position and velocity deviations of GNSS and IMU are combined as shown in formula (25). Since the mobile detection platform runs on the track platform, the Z-axis and Y-axis directions are constrained. Therefore, the velocity components on the two coordinate axes are specified to be close to 0, i.e., formula (26). Both are then substituted into the extended Kalman filter (EKF), i.e., formulas (27) to (31):

[0137] (25)

[0138] (26)

[0139] in, The location of the mobile platform is calculated based on the IMU. The location of the mobile platform is calculated based on GNSS. For mobile platform speed components calculated based on IMU, This refers to the velocity components of the mobile platform calculated based on GNSS. Prediction formula:

[0140] (27)

[0141] (28)

[0142] Update the model formula:

[0143] (29)

[0144] (30)

[0145] (31)

[0146] in, and Let represent the plane coordinates perpendicular to the direction of travel of the mobile detection platform along the track platform; F is the state transition matrix. Q Let H be the process noise covariance matrix, and H be the observation matrix. R To observe the noise covariance matrix, u Let P be the control variable, and P be the error covariance matrix. Kalman gain is used to balance the error between prediction and measurement. For the updated state estimate, The updated covariance matrix represents the uncertainty of the current estimate. From this, the expression for the motion velocity of the mobile detection platform in the platform coordinate system V can be derived as formula (32), and the calculation velocity error model in the platform coordinate system can be expressed as formula (33):

[0147] (32)

[0148] (33)

[0149] in, For the velocity error of the platform coordinate system, For the velocity error of the global coordinate system, This is the coordinate transformation matrix from the global coordinate system to the platform coordinate system; The cross product of velocity vectors in the global coordinate system; This is the attitude error vector; This is the estimated velocity vector in the global coordinate system; The direction cosine for transforming from the global coordinate system to the platform's standard coordinate system; To estimate the velocity representation in platform coordinates, the measurement model of the GNSS / INS / DMI integrated navigation algorithm is constructed as follows (34):

[0150] (34)

[0151] Because the constraints imposed by the subway tunnel track on the mobile detection platform prevent abrupt changes in its heading, roll, and pitch angles, these motion characteristics can be utilized to achieve high measurement accuracy for the entire system. However, the absolute accuracy of IMU / GNSS model dynamic positioning is only at the centimeter level, which is insufficient for high-precision requirements. Therefore, it is necessary to combine DMI technology to improve the accuracy of the subway tunnel track spatial coordinates to the sub-millimeter level.

[0152] Next, error analysis is performed. Several control points with high measurement accuracy are selected. A 3D laser scanner is used to rotate and observe the control points, obtaining the distance between the control point target and the 3D laser scanner. This distance is then fused with the position and attitude data of the moving detection platform to obtain the point cloud data of the control point target. Based on the different intensities of the point cloud data, the coordinate data of all laser points on the target are extracted and fitted to obtain the coordinate measurement values. These coordinate measurement values ​​are then compared one by one with the high-precision measurement values ​​of the control point to determine the measurement error of the control point.

[0153] Then, data fusion analysis is performed. After obtaining the coordinate measurements and measurement errors of the control points, the position and attitude of the entire moving detection platform's trajectory are adjusted and corrected using the control point errors. Comprehensive data is obtained by fusing the distance data obtained from the 3D laser scanner with the position and attitude data of the moving detection platform. T The calculation process for 1 is as follows: Formula (35):

[0154] (35)

[0155] in, It is the coordinate information of the control points in the platform coordinate system. It is the rotation matrix from the platform coordinate system to the global coordinate system. It is a rotation matrix from the 3D laser scanner coordinate system to the global coordinate system. Here is the coordinate vector of the 3D laser scanner. Let be the translation vector from the 3D laser scanner coordinate system to the platform coordinate system. The above formula (35) is processed, and an error term is added as shown in formula (36):

[0156] (36)

[0157] By simplifying the two equations, the final error model (37) can be obtained:

[0158] (37)

[0159] If the transformation parameters from the laser scanning coordinate system to the platform coordinate system can be accurately calibrated, then the two terms on the right side of the above formula (38) are treated as random noise. It can be shown that formula (36) can be simplified to formula (38):

[0160] (38)

[0161] in, It is a positioning error Mid-trend item The coefficient matrix of the error, yes Error of the random term, It is attitude determination error The random part, yes The coefficient matrix of the term, yes The coefficient matrix of the term, at this time This is the difference in coordinates between the control point and its corresponding point in the point cloud. Here... This represents the difference in rotation matrix between the platform coordinate system and the global coordinate system caused by the corresponding coordinate differences. This represents the difference in rotation matrix between the 3D laser scanner coordinate system and the global coordinate system caused by the corresponding coordinate differences. This refers to the difference in coordinate vectors within the coordinate system of the 3D laser scanner caused by the corresponding coordinate differences. This refers to the translation vector difference between the 3D laser scanner coordinate system and the platform coordinate system caused by the corresponding coordinate difference.

[0162] Next, error compensation is performed. All data are unified to the global coordinate system, and least squares configuration is used to correct and compensate for the errors of all control points. In the platform coordinate system, the nth-order polynomial formula (39) for the position error trend term in the three coordinate directions as a function of time is as follows:

[0163] (39)

[0164] in, , C It is its corresponding coefficient matrix that changes over time. , and These represent the specific components of the error along the X-axis with respect to time. , and These are the specific time components of the error in the Y-axis direction. , and These are the specific time components of the error along the Z-axis. Combining formula (39), formula (38) can be rewritten as formula (40):

[0165] (40)

[0166] in, According to the least squares configuration, when observed n When there are +1 control points, the error correction parameters for the random term and the trend term are calculated using the following formulas (41) and (42), respectively:

[0167] (41)

[0168] (42)

[0169] in, It is a random item Prior covariance matrix, yes The covariance matrix, It is the covariance propagation matrix between the control point time and other times. It is the covariance matrix between the control point time and the random term errors at other times, and the random errors at other times. The correction amount is calculated using formula (43):

[0170] (43)

[0171] Finally, the solution is substituted. After analyzing, calculating, and compensating for the errors, the measurement errors of the control points are substituted into the adjustment equation to obtain the error compensation value. This value is then used to compensate the machine learning model between adjacent key locations in the dynamic control field, achieving more accurate sub-millimeter-level spatial reconstruction and assignment of 3D holographic images of subway tunnels.

[0172] In one embodiment, such as Figure 7 As shown, a sub-millimeter spatial measurement method for three-dimensional panoramic images of subway tunnels is provided, applicable to any of the aforementioned three-dimensional panoramic image sub-millimeter spatial measurement systems for subway tunnels. This sub-millimeter spatial measurement method for three-dimensional panoramic images of subway tunnels may include the following processing steps S10 to S20:

[0173] S10 collects three-dimensional laser point clouds of the subway tunnel at each inspection location through the point cloud data acquisition unit, collects high-precision mileage information of the mobile detection platform running in the subway tunnel through the mileage information acquisition unit, and obtains high-definition images of the subway tunnel at each inspection location through the image information acquisition unit.

[0174] S12, using the signal transmission unit to wirelessly transmit the collected information from the acquisition unit to the cloud server for storage in real time;

[0175] S14, after identifying target information in the high-definition image and three-dimensional laser point cloud of the subway tunnel by the target recognition unit, non-tunnel structures are eliminated;

[0176] S16, The target information and collected information are preprocessed by the data preprocessing unit;

[0177] S18: The data processing unit performs joint adjustment on the pre-processed high-definition image of the subway tunnel and the 3D laser point cloud to construct a dynamic control field and estimate the pose of the moving detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel; the data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel.

[0178] S20 uses a data fusion unit to perform data fusion based on global high-resolution 3D holographic images, 3D laser point clouds, and high-precision mileage information, and uses a trained machine learning model to remove non-tunnel fixed objects.

[0179] The aforementioned sub-millimeter-level spatial measurement method for 3D panoramic images of subway tunnels proposes a novel design combining 3D holographic modeling with traditional point cloud acquisition. This design utilizes a 3D holographic stitching algorithm based on a combined image information acquisition unit and a 3D laser scanner, achieving high-resolution, large field-of-view, and full-domain high-precision 3D holographic modeling of the tunnel. This ensures sub-millimeter-level accuracy in the 3D reconstruction and coordinate assignment of high-definition subway tunnel images, solving the problem of uneven accuracy and poor robustness in point cloud stitching caused by the limited features of point clouds in traditional laser scanning within subway tunnels. Furthermore, by employing a combined IMU, GNSS, and DMI navigation system, the method ensures that the mileage and attitude information of the mobile detection platform are well recorded during its operation between adjacent key locations within the subway tunnel. This guarantees that the constructed 3D holographic image of the tunnel achieves sub-millimeter-level accuracy. Simultaneously, by combining the features of high-definition subway tunnel images and point cloud data, the method further optimizes the mileage and attitude information, thereby avoiding the problems of untimely mileage information acquisition and uneven accuracy in subway tunnel sections without signal coverage. By combining machine vision and big data technologies, target segmentation in a dynamic control field can be achieved. Machine learning algorithms are used to remove non-tunnel fixed objects such as people and vehicles from the 3D reconstruction model of the tunnel. Through global optimization of parallax mapping, the continuity of depth information under different perspectives is ensured, and errors in parallax stitching are reduced. Ultimately, sub-millimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels are achieved.

[0180] Compared to traditional technologies, this technology solves the problem that it is difficult to accurately locate and detect cracks with geometric dimensions less than 1 mm and tunnel segment damage in subway tunnels due to the difficulty in spatial coordinate positioning of high-definition images and the inability to achieve sub-millimeter accuracy. It meets the needs of detecting and statistically analyzing apparent cracks and fissures in subway tunnels while ensuring sub-millimeter positioning of high-definition images.

[0181] In one embodiment, the above-described sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels may further include the following steps:

[0182] The data fusion unit extracts the image features of each branch of the global high-resolution 3D holographic image from different perspectives through a multi-branch convolutional neural network structure. Then, it uses a feature weighting mechanism to fuse the image features of each branch. The fused feature vector is then reconstructed by the decoder into a global high-resolution 3D holographic image with uniform illumination and coherent texture.

[0183] In one embodiment, the above-described sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels may further include the following steps:

[0184] The data fusion unit utilizes a pre-trained image enhancement model to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution 3D holographic image, thereby completing the global illumination optimization of the global high-resolution 3D holographic image.

[0185] It is understandable that the specific limitations of the sub-millimeter spatial measurement method for 3D panoramic images of subway tunnels can be found in the corresponding limitations of the sub-millimeter spatial measurement system for 3D panoramic images of subway tunnels mentioned above, and will not be repeated here.

[0186] It should be understood that, although Figure 7 The steps are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Figure 7 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0187] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of protection of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and all such modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels, characterized in that, It includes a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port, and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit, and the signal transmission unit are mounted on the mobile detection platform. The point cloud data acquisition unit is used to acquire three-dimensional laser point clouds of the subway tunnel at each inspection location. The mileage information acquisition unit is used to acquire high-precision mileage information of the mobile detection platform running in the subway tunnel. The image information acquisition unit is used to acquire high-definition images of the subway tunnel at each inspection location. The mobile detection platform is used to move each unit in the subway tunnel to conduct inspections in a multi-site data acquisition manner. The signal transmission unit is used to wirelessly transmit the information collected by the acquisition unit to the cloud server for storage in real time. The digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operation status of the mobile detection platform. The data processing fusion port is used to transmit the information collected in the cloud server to the integrated data processing terminal. The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit, and a data fusion unit. The target recognition unit identifies target information in high-definition images and 3D laser point clouds of the subway tunnel and then removes non-tunnel structures. The data preprocessing unit preprocesses the target information and the collected information. The data processing unit performs joint adjustment on the preprocessed high-definition images and 3D laser point clouds of the subway tunnel, constructs a dynamic control field, estimates the pose of the moving detection platform relative to global coordinates, and generates a global high-resolution 3D holographic image of the tunnel. The data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel. The data fusion unit uses a trained machine learning model to perform data fusion based on the global high-resolution 3D holographic image, 3D laser point cloud, and high-precision mileage information, and removes non-tunnel fixed objects. The data processing unit uses a 3D holographic stitching algorithm, which combines the image information acquisition unit and the 3D laser scanner, for joint adjustment. The 3D holographic stitching algorithm includes the following steps: Offline calibration of panoramic camera arrays and 3D laser scanners on mobile inspection platforms; the panoramic camera array includes a surround-view camera array and a reference transfer camera array. The mobile inspection platform is moved to each inspection position, and a high-resolution panoramic image of the tunnel at each inspection position is obtained by using a panoramic camera array. The reference transfer camera array is used to take pictures and extract the center coordinates of the measurement points in the survey area. A three-dimensional laser scanner is used to collect the three-dimensional laser point cloud in the survey area. Based on the center coordinates of the measurement points in the survey area and the control information at both ends of the tunnel, the benchmark transfer camera array and three-dimensional laser scanner at different inspection positions are jointly adjusted to construct a dynamic control field with known displacement. Based on the dynamic control field, using three or more dynamic control points observed by the surround-view camera array and the 3D laser scanner, the pose of the measurement platform relative to the global coordinate system is estimated, and then the 3D point cloud obtained by the 3D laser scanner is stitched together to the global coordinate system. Based on the fixed constraint between the surround-view camera array and the 3D laser scanner, the high-resolution surround-view image is mapped to the 3D point cloud data in the global coordinate system to obtain a global high-resolution 3D holographic image of the tunnel.

2. The sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels according to claim 1, characterized in that, The data fusion unit is also used to extract the image features of each branch of the global high-resolution 3D holographic image from different perspectives through a multi-branch convolutional neural network structure, and then use a feature weighting mechanism to fuse the image features of each branch. The fused feature vector is then reconstructed by the decoder into a global high-resolution 3D holographic image with uniform illumination and coherent texture.

3. The sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels according to claim 2, characterized in that, The data fusion unit is also used to perform brightness smoothing, color correction and contrast enhancement on the reconstructed global high-resolution 3D holographic image using a pre-trained image enhancement model, thereby completing the global illumination optimization of the global high-resolution 3D holographic image.

4. The sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels according to claim 1, characterized in that, The data processing unit divides the space of the dynamic control field into voxel grids of the same size, assigns grid values ​​using nearest neighbor matching or interpolation algorithms, identifies tunnel structure regions and non-tunnel fixed object regions in the image using a semantic segmentation network, masks non-tunnel regions using a mask generated by segmentation, generates three-dimensional textures using tunnel structure regions and performs texture mapping; each voxel grid is used to represent a tunnel segment.

5. The sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels according to claim 1, characterized in that, When the point cloud data acquisition unit acquires 3D laser point clouds, it acquires the relative coordinates, distance, and timestamp of the measurement location; when the mileage information acquisition unit acquires high-precision mileage information, it acquires the mileage, motion trajectory, attitude, and timestamp of the mobile detection platform; when the image information acquisition unit acquires high-definition images of the subway tunnel, it also records the timestamp of the image.

6. The sub-millimeter-level spatial measurement system for three-dimensional panoramic imaging of subway tunnels according to claim 1, characterized in that, The GNSS / INS / DMI integrated navigation algorithm includes the following steps: The measurement model of the GNSS / INS / DMI integrated navigation algorithm is used to perform data fusion of GNSS, INS and DMI measurement data and attitude prediction of the mobile detection platform. Perform measurement error analysis on control points; Data fusion analysis is performed based on the coordinate measurements and measurement errors of the control points; Error compensation for all control points is performed using a least-squares configuration.

7. A method for sub-millimeter-level spatial measurement of three-dimensional panoramic images of subway tunnels, characterized in that, The method for sub-millimeter-level spatial measurement of three-dimensional panoramic images of subway tunnels, as described in any one of claims 1 to 6, comprises the following steps: The system acquires 3D laser point clouds of the subway tunnel at each inspection location using a point cloud data acquisition unit, high-precision mileage information of the mobile detection platform operating in the subway tunnel using a mileage information acquisition unit, and high-definition images of the subway tunnel at each inspection location using an image information acquisition unit. The signal transmission unit is used to wirelessly transmit the information collected by the acquisition unit to the cloud server for storage in real time; After identifying target information in high-definition images and 3D laser point clouds of subway tunnels by the target recognition unit, non-tunnel structures are eliminated. The target information and collected information are preprocessed through the data preprocessing unit; The data processing unit performs joint adjustment on the preprocessed high-definition images of the subway tunnel and the 3D laser point cloud to construct a dynamic control field and estimate the pose of the moving detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel; the data processing unit uses a GNSS / INS / DMI integrated navigation algorithm for joint positioning of adjacent key locations in the tunnel. The data fusion unit uses a trained machine learning model to fuse data based on global high-resolution 3D holographic images, 3D laser point clouds, and high-precision mileage information, and removes non-tunnel fixed objects.

8. The sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels according to claim 7, characterized in that, It also includes the following steps: The data fusion unit extracts the image features of each branch of the global high-resolution 3D holographic image from different perspectives through a multi-branch convolutional neural network structure. Then, it uses a feature weighting mechanism to fuse the image features of each branch. The fused feature vector is then reconstructed by the decoder into a global high-resolution 3D holographic image with uniform illumination and coherent texture.

9. The sub-millimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels according to claim 7, characterized in that, It also includes the following steps: The data fusion unit utilizes a pre-trained image enhancement model to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution 3D holographic image, thereby completing the global illumination optimization of the global high-resolution 3D holographic image.

Images