Subway tunnel three-dimensional panoramic image submillimeter-level space measurement system and method

Abstract

The invention relates to a submillimeter-level spatial measurement system and method for a three-dimensional panoramic image of a subway tunnel, and proposes a new design of combining three-dimensional holographic modeling and traditional point cloud collection through joint adjustment of point cloud data collection and image information collection, thereby achieving high-resolution, large-view-field and global high-precision three-dimensional holographic modeling of the tunnel. The submillimeter precision can be ensured in three-dimensional reconstruction and image coordinate assignment of the high-definition image of the subway tunnel, and the problems of non-uniform precision and poor robustness during point cloud splicing caused by few point cloud features in the subway tunnel in traditional laser scanning are solved. Meanwhile, mileage and attitude information is further optimized in combination with subway tunnel high-definition images and point cloud data features, non-tunnel fixed objects existing in a tunnel three-dimensional reconstruction model are removed with the help of a machine learning algorithm, and submillimeter reconstruction and assignment of subway tunnel three-dimensional panoramic images are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial intelligent machine vision and navigation positioning technology, and relates to a three-dimensional panoramic image submillimeter space measurement system and method for a subway tunnel. Background Art

[0002] As crucial underground public transportation infrastructure, subway tunnels require efficient and safe operating environments, necessitating intelligent safety monitoring throughout their entire lifecycle. Breaking through traditional Leica robotic point-based monitoring technology, the development of a new generation of high-speed, wide-angle, and large-area panoramic imaging technology has become a mainstream trend in monitoring technology within this field. However, these images lack the tunnel's three-dimensional spatial coordinate information, requiring the use of three-dimensional laser point clouds to assign three-dimensional values ​​to high-definition images. Currently, algorithms for assigning laser point clouds and images remain challenged with achieving submillimeter-level spatial measurement for the reconstruction and assignment of three-dimensional panoramic images of subway tunnels. Summary of the Invention

[0003] In response to the problems existing in the above-mentioned traditional methods, the present invention proposes a submillimeter-level spatial measurement system for three-dimensional panoramic images of subway tunnels and a submillimeter-level spatial measurement method for three-dimensional panoramic images of subway tunnels, which can realize submillimeter-level reconstruction and assignment of three-dimensional panoramic images of subway tunnels.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: On the one hand, a three-dimensional panoramic imaging submillimeter spatial measurement system for subway tunnels is provided, comprising a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port, and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit, and the signal transmission unit are mounted on the mobile detection platform. The point cloud data acquisition unit is used to collect three-dimensional laser point clouds at each inspection location in the subway tunnel. The mileage information acquisition unit is used to collect high-precision mileage information of the mobile detection platform operating in the subway tunnel. The image information acquisition unit is used to obtain high-definition images of the subway tunnel at each inspection location. The mobile detection platform is used to move each unit to conduct inspections in the subway tunnel using a multiple, multi-site data collection method. The signal transmission unit is used to wirelessly transmit the collected information of the collection unit to the cloud server in real time for storage. The digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operating status of the mobile detection platform. The data processing fusion port is used to transmit the collected information in the cloud server to the integrated data processing terminal. The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit and a data fusion unit. The target recognition unit is used to identify target information in the high-definition images and three-dimensional laser point clouds of subway tunnels and eliminate non-tunnel structures. The data preprocessing unit is used to preprocess the target information and collected information. The data processing unit is used to jointly adjust the preprocessed high-definition images and three-dimensional laser point clouds of subway tunnels, construct a dynamic control field and estimate the position of the mobile detection platform relative to the global coordinates, and generate a global high-resolution three-dimensional holographic image of the tunnel; the data processing unit uses the GNSS / INS / DMI combined navigation algorithm to jointly locate adjacent key positions in the tunnel; the data fusion unit is used to use the trained machine learning model to perform data fusion and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

[0005] On the other hand, a method for submillimeter-level spatial measurement of a subway tunnel 3D panoramic image is also provided, which is applied to the above-mentioned subway tunnel 3D panoramic image submillimeter-level spatial measurement system. The method for submillimeter-level spatial measurement of a subway tunnel 3D panoramic image comprises the following steps: The point cloud data acquisition unit collects 3D laser point clouds at each inspection location in the subway tunnel, the mileage information acquisition unit collects high-precision mileage information of the mobile inspection platform operating in the subway tunnel, and the image information acquisition unit is used to obtain high-definition images of the subway tunnel at each inspection location; The signal transmission unit is used to wirelessly transmit the collected information of the collection unit to the cloud server in real time for storage; The target recognition unit identifies the target information in the subway tunnel high-definition image and 3D laser point cloud and eliminates non-tunnel structures; Preprocessing the target information and the collected information through a data preprocessing unit; The data processing unit performs a joint adjustment of the pre-processed high-definition subway tunnel images and 3D laser point clouds, constructs a dynamic control field, and estimates the position of the mobile detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel. The data processing unit also uses a GNSS / INS / DMI integrated navigation algorithm to jointly locate adjacent key locations in the tunnel. The data fusion unit uses the trained machine learning model to fuse data and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

[0006] One of the above technical solutions has the following advantages and beneficial effects: The aforementioned system and method for submillimeter spatial measurement of 3D panoramic images of subway tunnels utilizes a 3D holographic stitching algorithm based on combined adjustment of a panoramic camera array and a 3D laser scanner. This system proposes a novel design that combines 3D holographic modeling with traditional point cloud acquisition. This design achieves high-resolution, large-field-of-view, and global high-precision 3D holographic modeling of tunnels, ensuring submillimeter accuracy in 3D reconstruction and image coordinate assignment of high-definition subway tunnel images. This system addresses the issue of traditional laser scanning, which suffers from the lack of feature-rich point clouds in subway tunnels, resulting in uneven accuracy and poor robustness during point cloud stitching. Furthermore, the system optimizes mileage and attitude information by combining the features of high-definition subway tunnel images and point cloud data, thereby avoiding the issues of delayed mileage information acquisition and uneven accuracy in signal-free sections of subway tunnels. The system utilizes a combination of machine vision and big data technologies to achieve target segmentation in dynamic control fields. Machine learning algorithms are used to remove non-tunnel fixed objects, such as people and vehicles, from the 3D tunnel reconstruction model. Global optimization of parallax mapping ensures the continuity of depth information from different perspectives, reducing errors in parallax stitching and ultimately achieving submillimeter reconstruction and assignment of 3D panoramic images of subway tunnels. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0008] Figure 1 A schematic diagram of the overall architecture of a submillimeter-level spatial measurement system for 3D panoramic imaging of a subway tunnel in one embodiment; Figure 2 A schematic diagram of the basic structure of a mobile detection platform in one embodiment; Figure 3 Schematic diagram of the relative position relationship of the mobile detection platform in one embodiment; Figure 4 A schematic diagram of a combined adjustment of a datum transfer camera array and a 3D laser scanner in one embodiment; Figure 5 A schematic diagram of platform positioning and posture determination for a combination of a panoramic camera array and a 3D laser scanner in one embodiment; Figure 6 A schematic diagram of a process for jointly optimizing positioning and attitude determination using a panoramic camera array, a 3D laser scanner, and an IMU in one embodiment; Figure 7 The figure is a flow chart of a method for sub-millimeter spatial measurement of three-dimensional panoramic images of a subway tunnel in one embodiment.

[0009] Among them, 1. Panel controller; 2. Transfer reference camera; 3. Surround view camera; 4. 3D laser scanner; 5. Battery box; 6. Guide wheel; 7. Tensioning member; 8. Brake pad; 9. Travel wheel; 10. Handle; 11. Cable. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0011] It should be noted that, when referred to in this document as an "embodiment", it means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The presentation of this phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be understood by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in this document refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0012] The following describes the implementation of the present invention in detail with reference to the accompanying drawings in the embodiments of the present invention.

[0013] In one embodiment, Figure 1 As shown, a subway tunnel 3D panoramic imaging submillimeter-level spatial measurement system is provided, which includes a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port, and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit, and the signal transmission unit are mounted on the mobile detection platform. The point cloud data acquisition unit is used to collect three-dimensional laser point clouds at each inspection location in the subway tunnel. The mileage information acquisition unit is used to collect high-precision mileage information of the mobile detection platform operating in the subway tunnel. The image information acquisition unit is used to obtain high-definition images of the subway tunnel at each inspection location. The mobile detection platform is used to move each unit to conduct inspections in the subway tunnel using a multiple, multi-site data collection method. The signal transmission unit is used to wirelessly transmit the collected information of the collection unit to the cloud server in real time for storage. The digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operating status of the mobile detection platform. The data processing fusion port is used to transmit the collected information in the cloud server to the integrated data processing terminal. The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit and a data fusion unit. The target recognition unit is used to identify target information in the high-definition images and three-dimensional laser point clouds of subway tunnels and eliminate non-tunnel structures. The data preprocessing unit is used to preprocess the target information and collected information. The data processing unit is used to jointly adjust the preprocessed high-definition images and three-dimensional laser point clouds of subway tunnels, construct a dynamic control field and estimate the position of the mobile detection platform relative to the global coordinates, and generate a global high-resolution three-dimensional holographic image of the tunnel; the data processing unit uses the GNSS / INS / DMI combined navigation algorithm to jointly locate adjacent key positions in the tunnel; the data fusion unit is used to use the trained machine learning model to perform data fusion and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

[0014] It is understandable that Figure 2 The figure shows a mobile inspection platform and its associated acquisition units. A panel controller 1 is used for on-site control of the mobile inspection platform. The transfer reference camera 2 and surround-view camera 3 together form the image information acquisition unit. The panel controller 1, image information acquisition unit, and 3D laser scanner 4 are all powered by batteries housed in a battery compartment 5. Running wheels 9 and guide wheels 6 are the mechanism for the mobile inspection platform to move and steer during inspections. A tensioning member 7 and brake pads 8 provide braking. A handle 10 allows for manual movement of the mobile inspection platform. Cables 11 provide electrical connections between the various units.

[0015] The image information acquisition unit may include a surround-view camera array and a reference transfer camera array. The reference transfer camera array includes rigidly mounted forward and rear-view cameras, which transmit control information from both ends of the tunnel to the measurement area. The surround-view camera array includes at least two high-resolution cameras with overlapping fields of view, which are used to obtain high-definition images of the subway tunnel at each inspection location. The data processing and fusion ports may include platform data return ports, fiber Bragg grating return ports, and 3D laser scanning ports, which are used to transmit collected information stored in the cloud server to the integrated data processing terminal.

[0016] Specifically, the target information identified by the target recognition unit refers to the structural information of various parts belonging to the tunnel, which is relative to non-tunnel structures such as electrical boxes, brackets, and signal lights. The target recognition unit can perform preliminary screening and elimination of these non-tunnel structures. The data preprocessing unit preprocesses the target information and collected information. For example, this may include simple background weakening of structures and facilities that may not belong to the subway tunnel that may be captured in the subway tunnel high-definition image, elimination of some images whose resolution does not meet inspection requirements, and common preprocessing operations such as denoising, filtering, and enhancement of the remaining information. The SOR statistical filtering algorithm can be used to denoise and remove outliers in the three-dimensional laser point cloud, and the extended Kalman filter algorithm can be used to denoise the collected high-precision mileage information.

[0017] The data preprocessing unit first synchronizes and aligns the 3D laser point cloud, high-precision mileage information, and high-definition images of subway tunnels, ensuring that all acquisition units share a unified timestamp reference (this can be achieved with the aid of time synchronization triggers). Simultaneously, the reference transfer camera, surround-view camera, 3D laser scanner in the point cloud data acquisition unit, and positioning sensor in the mileage information acquisition unit are unified into the platform coordinate system through external calibration or manual conversion. Next, the 3D laser point cloud and high-precision mileage information are filtered, denoised, and normalized to prevent noise from affecting subsequent reconstruction quality. Finally, the high-definition images of subway tunnels are subjected to denoising, dedistortion, and illumination normalization, paving the way for subsequent feature extraction, image stitching, and point cloud mapping.

[0018] The data processing unit performs a joint adjustment of the preprocessed subway tunnel high-definition images and 3D laser point clouds to obtain a global high-resolution 3D holographic image. The detailed implementation will be discussed later. The data fusion unit trains a machine learning model and combines it with big data processing techniques to divide the entire dynamic control field into a 3D voxel grid, achieving target segmentation and constructing a global illumination field. The machine learning model can utilize existing deep learning models suitable for data fusion and target recognition, and can be pre-trained using existing tunnel inspection databases in the field.

[0019] Adjustment is a mathematical processing method based on statistics and error theory. Its core purpose is to eliminate or reduce the impact of various errors in measurement data. In essence, it is to analyze redundant observation data, reasonably distribute errors, and make the measurement results meet certain accuracy requirements and logical consistency, thereby obtaining more reliable and reasonable measurement results and providing accurate data support for tunnel deformation analysis and safety assessment.

[0020] The aforementioned subway tunnel 3D panoramic imaging submillimeter spatial measurement system utilizes an image acquisition unit and a 3D laser scanner for combined adjustment and 3D holographic stitching algorithm. This system proposes a novel design that combines 3D holographic modeling with traditional point cloud acquisition. This system achieves high-resolution, large-field-of-view, and global high-precision 3D holographic modeling of tunnels, ensuring submillimeter accuracy in 3D reconstruction and coordinate assignment of high-definition subway tunnel images. This system addresses the issue of traditional laser scanning, which suffers from the lack of feature-rich point clouds in subway tunnels, resulting in uneven accuracy and poor robustness in point cloud stitching. By utilizing a combined navigation system comprised of an IMU (Inertial Navigation Unit), a GNSS (Global Navigation Satellite System), and a DMI (Distance Metering) system, the system ensures that the mobile inspection platform's mileage and attitude information is accurately recorded as it moves between key locations in the subway tunnel. This ensures that the constructed 3D holographic images of the tunnel achieve submillimeter accuracy. Furthermore, the system optimizes mileage and attitude information by combining the features of the subway tunnel's high-definition images and point cloud data, thereby avoiding issues such as delayed mileage information acquisition and uneven accuracy in sections of the subway tunnel with no signal. The combination of machine vision and big data technology can achieve target segmentation in the dynamic control field. With the help of machine learning algorithms, non-tunnel fixed objects such as people and vehicles in the tunnel's 3D reconstruction model are eliminated. Through global optimization of parallax mapping, the continuity of depth information under different perspectives is ensured, and the error in parallax stitching is reduced, ultimately achieving submillimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels.

[0021] Compared with traditional technologies, this method solves the problem that the spatial coordinate positioning of high-definition images of subway tunnels is difficult and the accuracy cannot reach the sub-millimeter level, which makes it difficult to accurately locate and detect cracks with geometric dimensions less than 1mm and damage to tunnel segments in subway tunnels. While ensuring the sub-millimeter positioning of high-definition images, it meets the needs of detecting and counting apparent cracks and fissure diseases in subway tunnels.

[0022] It should be noted that the high-resolution cameras in the surround-view camera array have a certain overlap area. During the inspection process, the mobile inspection platform ensures that each camera in the surround-view camera array and the 3D laser scanner have a certain overlap area at adjacent inspection locations. After completing an inspection, based on the image key points in the high-resolution surround-view images of the surround-view camera array at adjacent inspection locations, and the constraints between the image key points and the 3D point cloud key points in the point cloud data, combined with the fixed-link constraints of the fiducial transfer camera array and the 3D laser scanner, a joint adjustment of the fiducial transfer camera array and the 3D laser scanner is performed. Under the condition of limited peripheral control information, the 3D displacement of key measurement points in the survey area relative to the global coordinate system is rapidly estimated, thereby constructing a high-precision dynamic control field with known positions. Finally, based on the constructed dynamic control field, the position of the measurement platform at adjacent inspection locations relative to the global coordinate system is rapidly estimated, thereby achieving rapid splicing of the 3D point cloud. Combined with the fixed-link constraints of the surround-view camera and the 3D laser scanner, the surround-view camera texture is quickly mapped, resulting in a global high-resolution 3D holographic image of the tunnel.

[0023] At the same time, a combined GNSS / INS / DMI navigation algorithm is used between adjacent key locations in the dynamic control field to improve overall positioning accuracy. By identifying characteristic points in the subway tunnel's high-definition imagery as control points (control point coordinates can be obtained using a total station or other high-precision 3D measurement equipment), the surrounding area, centered around the central pixel, is captured as the target image area. A CNN (convolutional neural network) algorithm is then applied to the subway tunnel's HD imagery to automatically detect and extract characteristic points and generate feature descriptions. This allows for synchronization of timestamps during the subsequent mileage positioning process, as well as the use of machine learning models to compensate for defects in the reconstructed model and mileage positioning accuracy.

[0024] By combining machine learning models with big data processing technologies, the target segmentation of the entire dynamic control field is completed. For example, a semantic segmentation network (such as the DeepLabv3+ model) is used to segment the image, identify the "tunnel structure area" and "non-tunnel fixed object area" (such as electrical boxes, brackets, traffic lights and pipelines) in the global high-resolution three-dimensional holographic image, and eliminate the non-tunnel structure area. The points belonging to the tunnel area in the three-dimensional laser point cloud are extracted according to the segmentation mask, and the non-tunnel fixed objects are eliminated.

[0025] In one embodiment, the data fusion unit is also used to extract the image features of each branch of the global high-resolution three-dimensional holographic image from different perspectives through a multi-branch convolutional neural network structure, and then use a feature weighting mechanism to fuse the image features of each branch. The fused feature vector is reconstructed into a global high-resolution three-dimensional holographic image with uniform illumination and coherent texture through a decoder.

[0026] It is understood that in this embodiment, the data fusion unit can also be used to adjust the global illumination field. The target recognition unit obtains global high-resolution 3D holographic image inputs from different viewpoints and constructs them into multiple independent branches. Each branch independently extracts the illumination features, color information, and texture structure of the corresponding viewpoint image. Each branch can utilize a lightweight convolutional neural network (CNN) to achieve efficient feature extraction and low-resource model operation, constructing a multi-branch convolutional neural network structure (based on the CNN structure). The image features extracted by each branch neural network are fused, and a feature weighting mechanism (such as an existing channel attention module or spatial attention module) is introduced to assign differential weights to features from different viewpoints, thereby enhancing the information representation of the primary viewpoint or key areas. The weighted fused feature vector is then input into the decoder to reconstruct an image with uniform illumination and coherent texture. The fused and reconstructed global high-resolution 3D holographic image is then globally optimized to improve brightness consistency, color balance, and detail fidelity. A pre-trained image enhancement model is used to perform brightness smoothing, color correction, and contrast enhancement on each frame of the global high-resolution 3D holographic image, reducing visual disharmony caused by exposure differences, shadow interference, and other factors in the multi-view images. The image sequences that have been corrected and enhanced for illumination are synthesized in chronological order to generate a continuous image stream or video sequence, ensuring a unified overall visual style.

[0027] The corrected global high-resolution 3D holographic image was then used for 3D texture mapping and structural modeling, combined with laser point cloud data, to verify the effectiveness of the illumination field adjustment. The impact of image processing on the modeling results was evaluated by comparing surface brightness changes, texture consistency, and transition smoothness of the spliced ​​areas before and after illumination adjustment. If the final 3D tunnel structural model showed no noticeable shadow jumps, color discontinuities, or illumination artifacts, the global illumination field adjustment was considered complete.

[0028] The stitching effect of the above images can be tested through 3D holographic imaging interactive software (such as Unity or Unreal Engine). When the user adjusts the viewing angle, the image remains continuous and consistent, and the inspection path and real-time position of the mobile inspection platform can be dynamically updated. This confirms that the submillimeter-level reconstruction and assignment of the subway tunnel's three-dimensional panoramic image are complete.

[0029] As you can understand, in the aforementioned process, a multi-branch convolutional neural network structure is constructed, and a feature weighting mechanism is introduced to fuse the features of each branch, enhancing the expressiveness of the main perspective or key areas. The fused feature vector is then reconstructed by the decoder into a global high-resolution 3D holographic image with uniform illumination and coherent texture.

[0030] Furthermore, the pre-trained existing image enhancement model can be used to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution three-dimensional holographic image, thereby completing the global illumination optimization of the global high-resolution three-dimensional holographic image. The processed global high-resolution three-dimensional holographic image sequence is synthesized into a continuous image stream or video in chronological order to ensure the consistency of the overall illumination style. Finally, the corrected global high-resolution three-dimensional holographic image, three-dimensional laser point cloud, and high-precision mileage information are fused for image three-dimensional texture mapping and structural modeling. The effectiveness of the illumination field adjustment can be verified by comparing the brightness, texture, and splicing smoothness of the three-dimensional subway tunnel model before and after the adjustment, ultimately achieving submillimeter-level reconstruction and assignment of the three-dimensional panoramic image of the subway tunnel.

[0031] In one embodiment, the data processing unit divides the space of the dynamic control field into voxel grids of the same size, uses a nearest neighbor matching algorithm or an interpolation algorithm to assign grid values, uses a semantic segmentation network to identify the tunnel structure area and the non-tunnel fixed object area in the image, shields the non-tunnel area with a mask image generated by segmentation, and uses the tunnel structure area to generate a three-dimensional texture and perform texture mapping; each voxel grid is used to represent a tunnel segment.

[0032] As can be understood, the data processing unit can divide the entire dynamic control field space into voxel grids of equal size, each representing a tunnel segment to facilitate segment positioning. Point cloud and mileage information are assigned to the corresponding grid based on the coordinates of each point, using either a nearest neighbor matching algorithm or an interpolation algorithm. After grid assignment, a semantic segmentation network (DeepLabv3+ model) is used to perform pixel-level semantic classification on the image, identifying "tunnel structure regions" and "non-tunnel fixed object regions." The mask generated by segmentation is used to mask non-tunnel regions in the global high-resolution 3D holographic image. Image-point cloud projective registration is then used to further remove point cloud data corresponding to non-target regions from the 3D space. Subsequently, a 3D texture is generated using the tunnel structure regions retained in the global high-resolution 3D holographic image. This texture is then precisely mapped to the mesh surface of the 3D tunnel structure model by combining camera extrinsic parameters with the posture information of the mobile detection platform. Since the dynamic control field has achieved real-time positioning and attitude determination of the vehicle's operating status, the mapping accuracy can be ensured to reach sub-millimeter level during the texture mapping process, avoiding mapping errors, blurring or misalignment.

[0033] In one embodiment, when the point cloud data acquisition unit collects three-dimensional laser point clouds, it includes collecting the relative coordinates, distance and timestamp of the measurement position; when the mileage information acquisition unit collects high-precision mileage information, it includes collecting the mileage, motion trajectory, posture and timestamp of the mobile detection platform; when the image information acquisition unit collects high-definition images of subway tunnels, it also records the timestamp of the image.

[0034] As you can understand, during data collection, the intrinsic, extrinsic, and distortion parameters of the panoramic camera array and 3D laser scanner are first calibrated. An IMU is then used to ensure time and position synchronization between the panoramic camera array, 3D laser scanner, and other sensors. A mobile detection platform then drives each acquisition unit to collect information from multiple locations within the subway tunnel, obtaining more complete and accurate high-definition images, 3D laser point clouds, and high-precision mileage information. This facilitates the subsequent construction of dynamic control fields and the 3D holographic stitching of the subway tunnel. The acquisition of the subway tunnel's 3D laser point cloud includes the 3D point cloud coordinates, including the relative coordinates, distance, and timestamp of the measurement position. High-precision mileage information is collected by the mobile detection platform, including its mileage, trajectory, posture, and timestamp. When acquiring high-definition images of the subway tunnel, the image timestamp is also recorded. This information is transmitted to a cloud server in real time via wireless transmission technologies (such as Wi-Fi / 5G networks).

[0035] During the data processing phase, the data processing unit extracts features from the subway tunnel HD images and creates a feature description, ensuring reliable feature matching for each frame and supporting image and point cloud registration. By extracting points with distinct geometric features (such as corners and planes) from the 3D laser point cloud, the data processing unit enables precise spatial registration of the subway tunnel HD images and the 3D laser point cloud. To effectively reconstruct a 3D holographic image from the subway tunnel HD images and the 3D laser point cloud, this is achieved using a 3D holographic stitching algorithm based on the combined adjustment of a panoramic camera array and a 3D laser scanner.

[0036] However, prior to this, it was necessary to align the dynamic coordinate systems of the subway tunnel's high-definition images with the 3D laser point cloud, uniformly mapping the multi-source data into the mobile inspection platform's platform coordinate system to ensure optimal alignment between the subsequent spatial generation of control tensors and point cloud modeling. Using a 3D holographic stitching algorithm that combines the panoramic camera array and 3D laser scanner for adjustment, a dynamic control field for the camera array and 3D laser scanner, as well as platform positioning and attitude determination for the mobile inspection platform, was constructed. Within key locations within the dynamic control field, adjacent locations were positioned using a GNSS / INS / DMI integrated navigation algorithm for mileage tracking, ensuring high accuracy between adjacent key locations within the subway tunnel.

[0037] In one embodiment, the three-dimensional holographic stitching algorithm using the image information acquisition unit and the three-dimensional laser scanner for joint adjustment may include the following processing steps: Offline calibration of the panoramic camera array and 3D laser scanner on the mobile inspection platform; the panoramic camera array includes a surround view camera array and a reference transfer camera array; The mobile detection platform moves to each inspection location, and uses a surround-view camera array to obtain high-resolution surround-view images of the tunnel at each inspection location. The fiducial transfer camera array takes photos and extracts the center coordinates of the measurement points in the measurement area. The 3D laser scanner collects the 3D laser point cloud in the measurement area. Based on the center coordinates of the measurement points in the survey area and the control information at both ends of the tunnel, the camera array and the 3D laser scanner are transferred to perform joint adjustment of different inspection positions to construct a dynamic control field with known displacement. Based on the dynamic control field, the position of the measurement platform relative to the global coordinates is estimated using the surround-view camera array and three or more dynamic control points observed by the 3D laser scanner. The 3D point cloud obtained by the 3D laser scanner is then stitched into the global coordinate system. According to the fixed connection constraint between the surround-view camera array and the 3D laser scanner, the high-resolution surround-view image is mapped to 3D point cloud data in the global coordinate system to obtain a global high-resolution 3D holographic image of the tunnel.

[0038] Specifically, a dynamic control field is constructed based on the combined adjustment of the reference transfer camera array and 3D laser scanner within the panoramic camera array. Path planning software, based on the mobile detection platform's IMU (consisting of inertial navigation and accelerometer components), controls the camera and 3D laser scanner on the mobile detection platform to repeat images at designated locations. However, due to the enclosed environment of the tunnel, positioning and attitude determination of the mobile detection platform can still exhibit errors, even when parked. To meet the requirements of high-precision measurement, this embodiment constructs a high-precision robust estimation scheme for the relative position of the reference transfer camera array and 3D laser scanner on the moving platform.

[0039] like Figure 3 As shown, the relative position of the mobile detection platform refers to the platform coordinate system at the current time t1 Relative to the platform coordinate system at the initial time t0 Posture (Use rigid body transformation matrix to represent pose). Figure 3 The unmarked dots are other spatial points. , the imaging process at time t1 and time t0 can be described as: (1) (2) in, represents the depth factor of the space at time t0, represents the depth factor of the space at time t1, Represents the homogeneous coordinates of the image point corresponding to the spatial point at time t0, Represents the homogeneous coordinates of the image point corresponding to the spatial point at time t1, represents the intrinsic parameter matrix, represents three rows and one column of zero vectors, Represents the pose transformation from the mobile detection platform to the camera coordinate system, Represents the world coordinate system The pose transformation to the platform coordinate system at time t0.

[0040] It can be understood that in actual engineering measurements, a total station can be used to construct a control field at both ends of the tunnel (i.e., control information at both ends of the tunnel). If the total station is leveled, the origin of the platform coordinate system can be built at the center of the measuring station, and the coordinate axes of the platform coordinate system are parallel to the coordinate axes of the total station. In this way, subsequent measurement results are based on the total station coordinate system. When the intrinsic parameters of the camera are known, the existing PnP (Perspective-n-Point, a method for estimating camera pose) or NPnP (Non-perspective PnP, a method for multi-camera pose estimation) needs to first calculate the transformation from the world coordinate system W (control coordinate system) to the platform coordinate system at time t1 and time t0 respectively. and Then solve the relative pose of the mobile detection platform , this method belongs to indirect relative pose estimation.

[0041] Different from the existing design ideas, this embodiment constructs a relative pose estimation differential model to transform the indirect relative pose estimation problem into a direct pose estimation problem. In actual engineering measurement, the control point is far away from the measurement area, and the change of the depth factor of the spatial point caused by the shaking of the measurement platform is relatively small. Therefore, the depth factor invariance assumption can be introduced, that is, , so the relative pose estimation differential model can be obtained as follows: (3) in, represents the identity matrix with 4 rows and 4 columns, Indicates the platform coordinate system at time t0 Because the inspection vehicle's platform position and posture change little at the two retake positions, it can be described in exponential form. as follows: (4) in, represents the velocity and angular velocity parameters, Represents the Lie algebra basis matrix of rigid body transformation.

[0042] For large engineering structures such as tunnels, control points are usually selected from stable structures far away from the survey area, and multi-level (camera) station transfer measurements are required to transfer control point information to the survey area. Figure 4As shown in the figure, the reference transfer camera arrays and 3D laser scanners at different positions of the mobile inspection platform (such as inspection position 1 to inspection position 3) can be regarded as a joint network. For adjacent inspection positions, the common field of view area between the reference transfer camera arrays can establish a connection between the reference transfer camera arrays at different inspection positions. In this way, joint adjustment can be achieved. Figure 4 In , P0 to P2 represent point sets respectively, and each point set includes multiple measuring points.

[0043] Relative to the initial state, the target point in the measurement area may produce a three-dimensional displacement, so the model described by formula (3) can be modified as follows: (5) in, is the displacement of the target point in the initial platform coordinate system. If the point is a control point, then .

[0044] Similarly, for 3D laser scanners there are: (6) in, is the position information of the target point in the 3D laser scanner coordinate system L at the initial time t0, is the rigid body change matrix between the platform coordinate system and the 3D laser scanner coordinate system L at the initial position, For the position information of the target point in the 3D laser scanner coordinate system L at time t1, constructing a dynamic control field is to construct the transformation between the three coordinate systems to ensure higher accuracy.

[0045] The differential measurement model can be further constructed as follows: (7) Considering the three-dimensional displacement of the measuring point, a measurement equation similar to formula (5) can be further constructed as follows: (8) Equations (5) and (8) are the measurement equations for the joint adjustment of the fiducial transfer camera array and the 3D laser scanner. The observation constraints that can be used in dynamic networks can be divided into two categories: one is that different target points observed by the fiducial transfer camera array and different target points observed by the 3D laser scanner at the same inspection location correspond to the same relative pose parameters; the other is that the same target point observed by the fiducial transfer camera array and the 3D laser scanner at different inspection locations corresponds to the same displacement parameters. Based on these two types of observation constraints, a set of equations for the relative pose parameters of the mobile detection platform and the displacement of the target points at different inspection locations can be established.

[0046] The platform positioning and attitude determination based on the surround-view camera array and 3D laser scanner can be achieved through the joint adjustment of the reference transfer camera array and the 3D laser scanner to transfer peripheral control information to the measurement area. The displacement of the measurement points within the measurement area is measured. The measurement points with known displacement can be used as dynamic control points to construct a dynamic control field. Based on this dynamic control field, high-precision positioning and attitude determination of the mobile inspection platform can be achieved at each inspection location.

[0047] Using only the surround-view camera array, given three corresponding points (the sum of the dynamic control points observed by the surround-view camera array), the NPnP method can be used to linearly solve the platform's position relative to the global coordinate system, achieving platform positioning and attitude determination. However, by combining the high-precision local coordinate data of the dynamic control points measured by a 3D laser scanner, even higher-precision positioning and attitude determination can be achieved by giving three corresponding points (the sum of the dynamic control points observed by the surround-view camera array and the dynamic control points observed by the 3D laser scanner).

[0048] Specifically, the imaging model of the camera array system composed of multiple surround-view cameras can be modeled using a non-perspective generalized camera system, such as Figure 5 As shown, P 1 to P 4 represent each dynamic control point, is a point in space, C 1 to C 4 represent the surround view cameras, represents the bias of the 3D laser scanner in a multi-camera system, Represents the global coordinate system (i.e., the world coordinate system), R , T They represent the rotation and translation from the global coordinate system to the platform coordinate system, corresponding to the posture and position of the platform in the global coordinate system. In this model, the relationship between the 3D coordinates and their corresponding image point coordinates can be expressed as: (9) in, , , Represents the normalized direction vector from the camera to the 3D point, Indicates the offset of the camera in the platform coordinate system, and They represent the rotation matrix and translation vector from the platform coordinate system to the camera coordinate system, represents the intrinsic parameter matrix of the camera, is the mathematical symbol for a special orthogonal group. Represents the coefficient matrix from camera to 3D point, to Represent the specific components of the rotation matrix, to Respectively x, y, z The translation vector of the coordinate from the global coordinate system to the platform coordinate system, 、 and Respectively x, y, z The offset position of the coordinate.

[0049] for n The 2D-3D point correspondences are obtained from formula (9): (10) Rewrite formula (10) as: (11) Among them, each matrix , , , , In formula (10) I represents the identity matrix, 、 and They represent the three-dimensional offset position of the camera in the global coordinate system.

[0050] For the coplanar control point situation, the default coordinate of the 3D point in the Z direction is 0. Substituting it into formula (11) and the rotation matrix is R The third column is eliminated. Therefore, in the configuration where the control points are coplanar, the matrix 、 b and v The dimension remains unchanged, the matrix U becomes 3 n ×6 matrix, matrix x becomes a 6-dimensional vector.

[0051] If the given matrix x , the matrix can be solved by the least squares method b : (12) in, express The pseudo-inverse of Substituting formula (12) into formula (11) yields the following equation: x The linear equations for : (13) in, , .

[0052] Due to the presence of noise, Equation (13) cannot be satisfied in general, so it is converted into a least squares problem by minimizing the sum of squares of the errors: (14) In order to facilitate global optimization by solving polynomial systems, the rotation matrix is ​​represented by the unit quaternion ( a , b , c , d )express: (15) in, , substituting formula (15) into formula (14) yields the cost equation: (16) in, , M It's about 2 n ×11 matrix.

[0053] Similarly, for a 3D laser scanner, the 3D coordinates in the global coordinate system and the corresponding laser scanning point coordinates can be written as: (17) in, , and They represent the rotation matrix and translation vector from the platform coordinate system to the 3D laser scanner coordinate system respectively. Represents the rotation matrix between the 3D laser scanner coordinate system and the global coordinate system, Represents the three-dimensional coordinate information of any spatial point in the three-dimensional laser scanner coordinate system, is the rotation matrix from the global coordinate system to the platform coordinate system, It is the three-dimensional coordinate of any spatial point in the global coordinate system.

[0054] Combining Formula (9) and Formula (17) can also construct the cost equation in the form of Formula (16).

[0055] The cost equation shown in formula (16) can be solved using null space analysis to obtain the unit quaternion corresponding to the rotation of the mobile detection platform relative to the global coordinate system. Substituting it into formula (14) can obtain the rotation matrix of the mobile detection platform relative to the global coordinate system, and further substituting it into formula (8) can obtain the translation vector of the mobile detection platform relative to the global coordinate system. Based on the linear analytical solution, a nonlinear optimization error function can be further constructed to iteratively optimize the rotation matrix and translation vector, thereby achieving high-precision positioning and attitude determination of the mobile detection platform.

[0056] Through the joint adjustment of panoramic camera array and 3D laser scanner, a dynamic control field can be constructed and the position and attitude analytical solution of the mobile detection platform can be obtained. On this basis, a nonlinear optimization model can be constructed in combination with IMU to achieve high-precision estimation results of the position and attitude of the mobile detection platform. The high-precision positioning and attitude fusion process of the platform is as follows: Figure 6 As shown. In order to reduce the amount of calculation, the sliding window BA algorithm is used for solution. The moving window is only responsible for optimizing the data of a certain number of frames. After the number of frames in the window reaches the set value, each new frame is thrown into the front frame of the window. However, the front frame is not directly discarded because it contains valid observation data and information. Here, the previously observed information needs to be integrated into a priori information and put into the subsequent data optimization. The VIO optimization equation based on the sliding window is as follows: (18) Among them, the variables to be optimized , are the position and attitude parameters of the mobile detection platform, including i The position of the mobile detection platform in the global coordinate system at all times ,attitude ,speed , and IMU acceleration and angular velocity bias estimation Here n and m They are the state quantity of the mobile detection platform and the starting time of the dynamic control point in the sliding window, that is, the sliding window starts from the n Frame No. m The starting point is a signpost. N is the number of key frames in the sliding window, M is the number of dynamic control points observed by all key frames in the sliding window. The optimal pose is estimated by minimizing the sum of the residual values ​​of these state quantities.

[0057] represents the robust function of the prior term; express n -1 moment the estimated value of the position and attitude of the mobile detection platform, the state of the mobile detection platform in the sliding window is reversely converted to n Backward prediction at time -1. The coefficient matrix representing the prior constraints; express Robust function of the term; A robust function representing an image term; represents the robust function of the laser term; represents IMU measurement, represents the covariance of IMU measurements; Represents image features, represents the covariance of image observations; Indicates laser measurement, represents the covariance of the laser measurement; represents the IMU motion measurement error, represents the image observation error, Indicates the laser measurement error.

[0058] Taking any surround view camera as an example, the error equation is j The reprojection error of the dynamic control points observed by the camera at each moment is defined as follows: (19) in, is the image coordinate observation value of the corresponding spatial point extracted from the image. If the corresponding spatial point is not observed in the image taken by the surround camera array, the corresponding error term is set to zero. is the homogeneous coordinate corresponding to the three-dimensional coordinate of the dynamic control point in the global coordinate system, is the rigid body transformation matrix corresponding to the position and posture of the mobile detection platform, is the camera's calibration transformation matrix. For the The camera captures the midpoint of the image Image plane observation x coordinate, For the The camera captures the midpoint of the image Image plane observation y coordinate, For the The camera captures the midpoint of the image Image plane observation z coordinate.

[0059] For a 3D laser scanner, the error equation is j The coordinate conversion error of the dynamic control point scanned by the 3D laser scanner at the moment is specifically defined as follows: (20) in, is the coordinate observation value of the corresponding spatial point extracted from the 3D laser scanner. If the corresponding point is not observed in the 3D laser scanner, the corresponding error term is set to zero. Represents the pose change matrix between the 3D laser scanner coordinate system and the platform coordinate system, 、 and Respectively represent The space points corresponding to x 、 y、 z Global coordinates, expressed in homogeneous form.

[0060] For IMU, the measurement model is as follows: (twenty one) The superscript g indicates a gyroscope. a represents the accelerometer, is the gravity acceleration value, and the true value of IMU is and , the measured value is and According to the IMU dynamic equation, the time derivatives of position P(osition), velocity v(elocity), and attitude q(uaternion) between adjacent observations can be written as: (twenty two) in, is the time interval between two observations, is a four-element multiplication, is the position vector from the global coordinate system to the platform coordinate system at the current moment, is the current attitude quaternion, which represents the rotation matrix from the platform coordinate system to the global coordinate system. is the bias term of the accelerometer, is the bias term of the gyroscope. , and Represent the changes in displacement, velocity and posture respectively. The specific calculations are as follows: (twenty three) Because the IMU sampling frequency is high, i and i There will be many discrete observations within the +1 moment, and the continuous integral expressed in the above formula can be converted into the median integral method for summation operation.

[0061] Furthermore, the displacement, velocity, and attitude changes calculated according to formula (23) are used as observation values, and the displacement, velocity, and attitude changes derived according to formula (22) are used as predicted values, and the error equation of the IMU can be constructed as follows: (twenty four) In one embodiment, the data processing unit uses a GNSS / INS / DMI integrated navigation algorithm between two adjacent key positions to ensure that the reconstructed three-dimensional image can maintain a high accuracy requirement. The GNSS / INS / DMI integrated navigation algorithm includes the following processing steps: The measurement model of the GNSS / INS / DMI integrated navigation algorithm is used to fuse the measurement data of GNSS, INS and DMI and predict the attitude of the mobile detection platform; Conduct measurement error analysis of control points; Perform data fusion analysis based on the coordinate measurement values ​​and measurement errors of the control points; The least squares configuration is used to compensate errors of all control points.

[0062] Specifically, data fusion and attitude prediction are first performed. The position and velocity deviations of the GNSS and IMU are combined as shown in the following formula (25). Since the mobile detection platform operates on a track platform, the Z-axis and Y-axis directions are constrained. Therefore, the velocity components on the two coordinate axes are required to be close to 0, which is formula (26). Both are then introduced into the extended Kalman filter (EKF), which is formulas (27) to (31): (25) (26) in, is the position of the mobile platform calculated based on the IMU, is the position of the mobile platform calculated based on GNSS, is the velocity component of the mobile platform calculated based on the IMU, is the velocity component of the mobile platform calculated based on GNSS. Prediction formula: (27) (28) Update model formula: (29) (30) (31) in, and They represent the plane coordinates of the mobile detection platform along the vertical direction of the track platform; F is the state transfer matrix, Q is the process noise covariance matrix, H is the observation matrix, R is the observation noise covariance matrix, u is the control quantity, P is the error covariance matrix, is the Kalman gain, which is used to balance the error between prediction and measurement. is the updated state estimate, is the updated covariance matrix, which represents the uncertainty of the current estimate. From this, we can derive the expression of the motion speed of the mobile detection platform in the platform coordinate system V as shown in formula (32), and the calculation speed error model in the platform coordinate system can be expressed as formula (33): (32) (33) in, is the velocity error of the platform coordinate system, is the velocity error in the global coordinate system, is the coordinate transformation matrix from the global coordinate system to the platform coordinate system; is the cross product of the velocity vector in the global coordinate system; is the attitude error vector; is the estimated velocity vector in the global coordinate system; is the direction cosine transformed from the global coordinate system to the platform's center coordinate system; is the estimated velocity in platform coordinates. The measurement model of the GNSS / INS / DMI integrated navigation algorithm is constructed as follows: (34) Because the subway tunnel's track constrains the mobile detection platform, its heading, roll, and pitch angles do not change suddenly. These motion characteristics can be exploited to achieve high measurement accuracy for the entire system. However, the absolute accuracy of dynamic positioning using IMU / GNSS models is only at the centimeter level, which does not meet high-precision requirements. Therefore, DMI technology is required to improve the spatial coordinate accuracy of subway tunnel track to submillimeter levels.

[0063] Next, error analysis is performed. Several control points with high measurement accuracy are selected. A 3D laser scanner is used to rotate and observe the control points, obtaining the distance between the control point target and the 3D laser scanner. This distance is then integrated with the position and posture data of the mobile detection platform to obtain point cloud data for the control point target. Based on the intensity of the point cloud data, the coordinate data of all laser points on the target is extracted and fitted to obtain coordinate measurement values. These coordinate measurement values ​​are then compared one by one with the high-precision measurement values ​​of the control point to determine the measurement error of the control point.

[0064] Then, data fusion analysis is performed. After obtaining the coordinate measurement value and measurement error of the control point, the position and attitude of the trajectory of the entire mobile detection platform are adjusted using the error of the control point. The distance obtained by the 3D laser scanner is fused with the position and attitude data of the mobile detection platform to obtain the comprehensive data. T The calculation process of 1 is as follows: (35) in, is the coordinate information of the control point in the platform coordinate system, is the rotation matrix from the platform coordinate system to the global coordinate system, is the rotation matrix from the 3D laser scanner coordinate system to the global coordinate system, is the coordinate vector of the 3D laser scanner, where is the translation vector from the 3D laser scanner coordinate system to the platform coordinate system. The above formula (35) is processed and the error term is added to the following formula (36): (36) By simplifying the two equations, we can obtain the final error model (37): (37) Among them, the transformation parameters from the laser scanning coordinate system to the platform coordinate system can be accurately calibrated, and the two terms on the right side of the above formula (38) are regarded as random noise, and are expressed as Formula (36) can be simplified to formula (38): (38) in, is the positioning error Medium trend items The coefficient matrix of the error, yes Random error in the is the attitude error The random part of yes The coefficient matrix of the term, yes The coefficient matrix of the term, then It is the coordinate difference between the control point and its corresponding point in the point cloud. is the rotation matrix difference between the platform coordinate system and the global coordinate system caused by the corresponding coordinate difference, is the rotation matrix difference between the 3D laser scanner coordinate system and the global coordinate system caused by the corresponding coordinate difference, is the coordinate vector difference of the 3D laser scanner coordinate system caused by the corresponding coordinate difference, is the translation vector difference from the 3D laser scanner coordinate system to the platform coordinate system caused by the corresponding coordinate difference.

[0065] Then, error compensation is performed. All data are unified into the global coordinate system, and the least squares configuration is used to correct and compensate the errors of all control points. In the platform coordinate system, the nth-order polynomial formula (39) for the position error trend term of the three coordinate directions changing with time is as follows: (39) in, , C is the coefficient matrix corresponding to time variation, 、 and are the specific components of the error in the X-axis direction with respect to time, 、 and are the specific components of the error in the Y-axis direction with respect to time, 、 and are the specific components of the error in the Z-axis direction with respect to time. Combined with formula (39), formula (38) can be rewritten as formula (40): (40) in, According to the least squares configuration, when the observation n When there is +1 control point, the random error correction term and trend error correction parameters are calculated using the following formulas (41) and (42), respectively: (41) (42) in, Is a random item Prior covariance matrix, yes The covariance matrix of is the covariance propagation matrix between the control point time and other moments. is the covariance matrix between the control point time and the random errors at other times. The correction amount is calculated using formula (43): (43) Finally, the solution is substituted. After error analysis, calculation, and compensation, the measurement errors of the control points are substituted into the adjustment equation to obtain error compensation values. These values ​​are then applied to adjacent key positions in the dynamic control field of the machine learning model, achieving more accurate submillimeter-level spatial reconstruction and assignment of 3D holographic images of subway tunnels.

[0066] In one embodiment, Figure 7As shown, a method for submillimeter-level spatial measurement of a 3D panoramic image of a subway tunnel is provided, which is applied to any of the above-mentioned 3D panoramic image submillimeter-level spatial measurement systems for a subway tunnel. The method for submillimeter-level spatial measurement of a 3D panoramic image of a subway tunnel may include the following processing steps S10 to S20: S10, collecting a three-dimensional laser point cloud of the subway tunnel at each inspection location using a point cloud data acquisition unit, collecting high-precision mileage information of the mobile inspection platform operating in the subway tunnel using a mileage information acquisition unit, and obtaining a high-definition image of the subway tunnel at each inspection location using an image information acquisition unit; S12, using the signal transmission unit to wirelessly transmit the collected information of the collection unit to the cloud server in real time for storage; S14, identifying target information in the subway tunnel high-definition image and the three-dimensional laser point cloud through a target recognition unit and eliminating non-tunnel structures; S16, preprocessing the target information and the collected information by a data preprocessing unit; S18: The data processing unit performs a joint adjustment on the pre-processed high-definition subway tunnel image and the 3D laser point cloud, constructs a dynamic control field, and estimates the position of the mobile detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel. The data processing unit uses a GNSS / INS / DMI integrated navigation algorithm to perform joint positioning of adjacent key locations in the tunnel. S20, through the data fusion unit, uses the trained machine learning model to fuse data and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

[0067] The above-mentioned method for submillimeter spatial measurement of 3D panoramic images of subway tunnels utilizes a 3D holographic stitching algorithm based on combined adjustment of an image information acquisition unit and a 3D laser scanner. This method proposes a novel design that combines 3D holographic modeling with traditional point cloud acquisition. This method achieves high-resolution, large-field-of-view, and global high-precision 3D holographic modeling of tunnels, ensuring submillimeter accuracy in 3D reconstruction and image coordinate assignment of high-definition images of subway tunnels. This method addresses the issue of traditional laser scanning, which suffers from the lack of feature-rich point clouds in subway tunnels, resulting in uneven accuracy and poor robustness in point cloud stitching. By utilizing an integrated navigation system of IMU, GNSS, and DMI, the method ensures that the mileage and attitude information of the mobile inspection platform is accurately recorded as it moves between key locations in the subway tunnel, ensuring that the constructed 3D holographic images of the tunnel achieve submillimeter accuracy. Furthermore, the method further optimizes mileage and attitude information by combining the characteristics of the subway tunnel's high-definition images and point cloud data, thereby avoiding the issues of delayed mileage information acquisition and uneven accuracy in sections of the subway tunnel with no signal. The combination of machine vision and big data technology can achieve target segmentation in the dynamic control field. With the help of machine learning algorithms, non-tunnel fixed objects such as people and vehicles in the tunnel's 3D reconstruction model are eliminated. Through global optimization of parallax mapping, the continuity of depth information under different perspectives is ensured, and the error in parallax stitching is reduced, ultimately achieving submillimeter-level reconstruction and assignment of 3D panoramic images of subway tunnels.

[0068] Compared with traditional technologies, this method solves the problem that the spatial coordinate positioning of high-definition images of subway tunnels is difficult and the accuracy cannot reach the sub-millimeter level, which makes it difficult to accurately locate and detect cracks with geometric dimensions less than 1mm and damage to tunnel segments in subway tunnels. While ensuring the sub-millimeter positioning of high-definition images, it meets the needs of detecting and counting apparent cracks and fissure diseases in subway tunnels.

[0069] In one embodiment, the above-mentioned method for submillimeter spatial measurement of a subway tunnel 3D panoramic image may further include the following steps: The data fusion unit extracts the image features of each branch of the global high-resolution three-dimensional holographic image from different perspectives through a multi-branch convolutional neural network structure, and then fuses the image features of each branch using a feature weighting mechanism. The fused feature vector is reconstructed into a global high-resolution three-dimensional holographic image with uniform illumination and coherent texture through a decoder.

[0070] In one embodiment, the above-mentioned method for submillimeter spatial measurement of a subway tunnel 3D panoramic image may further include the following steps: The data fusion unit uses a pre-trained image enhancement model to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution three-dimensional holographic image, thereby completing the global illumination optimization of the global high-resolution three-dimensional holographic image.

[0071] It can be understood that for the specific limitations of the subway tunnel 3D panoramic image submillimeter space measurement method, please refer to the corresponding limitations of the subway tunnel 3D panoramic image submillimeter space measurement system above, which will not be repeated here.

[0072] It should be understood that although Figure 7 The steps in the diagram are shown in the order indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Figure 7 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0073] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The above embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention.

Claims

1. A three-dimensional panoramic imaging sub-millimeter spatial measurement system for subway tunnels, characterized by: It includes a point cloud data acquisition unit, a mileage information acquisition unit, an image information acquisition unit, a mobile detection platform, a signal transmission unit, a cloud server, a digital monitoring platform, a data processing fusion port and an integrated data processing terminal. The point cloud data acquisition unit, the mileage information acquisition unit, the image information acquisition unit and the signal transmission unit are mounted on the mobile detection platform. The point cloud data acquisition unit is used to collect three-dimensional laser point clouds at each inspection location in the subway tunnel. The mileage information acquisition unit is used to collect high-precision mileage information of the mobile inspection platform operating in the subway tunnel. The image information acquisition unit is used to obtain high-definition images of the subway tunnel at each inspection location. The mobile inspection platform is used to move each unit in the subway tunnel to conduct inspections in a multiple-time, multi-site data collection manner. The signal transmission unit is used to wirelessly transmit the collected information from the collection unit to the cloud server in real time for storage. The digital monitoring platform is used to monitor abnormal conditions in the subway tunnel and the operating status of the mobile detection platform. The data processing fusion port is used to transmit the collected information in the cloud server to the integrated data processing terminal. The integrated data processing terminal is equipped with a target recognition unit, a data preprocessing unit, a data processing unit and a data fusion unit. The target recognition unit is used to identify target information in the high-definition images and three-dimensional laser point clouds of subway tunnels and eliminate non-tunnel structures. The data preprocessing unit is used to preprocess the target information and collected information. The data processing unit is used to jointly adjust the preprocessed high-definition images and three-dimensional laser point clouds of subway tunnels, construct a dynamic control field and estimate the position of the mobile detection platform relative to the global coordinates, and generate a global high-resolution three-dimensional holographic image of the tunnel; the data processing unit uses the GNSS / INS / DMI combined navigation algorithm to jointly locate adjacent key positions in the tunnel; the data fusion unit is used to use the trained machine learning model to perform data fusion and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

2. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to claim 1 is characterized in that: The data fusion unit is also used to extract the image features of each branch of the global high-resolution three-dimensional holographic image from different perspectives through a multi-branch convolutional neural network structure, and then use the feature weighting mechanism to fuse the image features of each branch. The fused feature vector is reconstructed into a global high-resolution three-dimensional holographic image with uniform illumination and coherent texture through the decoder.

3. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to claim 2 is characterized in that: The data fusion unit is also used to use the pre-trained image enhancement model to perform brightness smoothing, color correction and contrast enhancement on the reconstructed global high-resolution three-dimensional holographic image, thereby completing the global illumination optimization of the global high-resolution three-dimensional holographic image.

4. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to any one of claims 1 to 3, characterized in that: The data processing unit uses a three-dimensional holographic stitching algorithm of the image information acquisition unit and the three-dimensional laser scanner for joint adjustment. The three-dimensional holographic stitching algorithm includes the following steps: Offline calibration of the panoramic camera array and 3D laser scanner on the mobile inspection platform; the panoramic camera array includes a surround view camera array and a reference transfer camera array; The mobile detection platform moves to each inspection location, and uses a surround-view camera array to obtain high-resolution surround-view images of the tunnel at each inspection location. The fiducial transfer camera array takes photos and extracts the center coordinates of the measurement points in the measurement area. The 3D laser scanner collects the 3D laser point cloud in the measurement area. Based on the center coordinates of the measurement points in the survey area and the control information at both ends of the tunnel, the camera array and the 3D laser scanner are transferred to perform joint adjustment of different inspection positions to construct a dynamic control field with known displacement. Based on the dynamic control field, the position of the measurement platform relative to the global coordinates is estimated using the surround-view camera array and three or more dynamic control points observed by the 3D laser scanner. The 3D point cloud obtained by the 3D laser scanner is then stitched into the global coordinate system. According to the fixed connection constraint between the surround-view camera array and the 3D laser scanner, the high-resolution surround-view image is mapped to 3D point cloud data in the global coordinate system to obtain a global high-resolution 3D holographic image of the tunnel.

5. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to claim 4 is characterized in that: After the data processing unit divides the space of the dynamic control field into voxel grids of the same size, it uses the nearest neighbor matching algorithm or interpolation algorithm to assign grid values. It uses a semantic segmentation network to identify the tunnel structure area and non-tunnel fixed object area in the image. After shielding the non-tunnel area with the mask image generated by segmentation, it uses the tunnel structure area to generate a three-dimensional texture and perform texture mapping; each voxel grid is used to represent the tunnel segment.

6. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to claim 4 is characterized in that: When the point cloud data acquisition unit collects three-dimensional laser point clouds, it includes collecting the relative coordinates, distance and timestamp of the measurement position; when the mileage information acquisition unit collects high-precision mileage information, it includes collecting the mileage, motion trajectory, posture and timestamp of the mobile detection platform; when the image information acquisition unit collects high-definition images of subway tunnels, it also records the timestamp of the image.

7. The subway tunnel 3D panoramic imaging submillimeter spatial measurement system according to claim 1 is characterized in that: The GNSS / INS / DMI integrated navigation algorithm includes the following steps: The measurement model of the GNSS / INS / DMI integrated navigation algorithm is used to fuse the measurement data of GNSS, INS and DMI and predict the attitude of the mobile detection platform; Conduct measurement error analysis of control points; Perform data fusion analysis based on the coordinate measurement values ​​and measurement errors of the control points; The least squares configuration is used to compensate errors of all control points.

8. A method for sub-millimeter spatial measurement of 3D panoramic images of subway tunnels, characterized in that: Applied to the subway tunnel 3D panoramic image submillimeter space measurement system according to any one of claims 1 to 7, the subway tunnel 3D panoramic image submillimeter space measurement method comprises the steps of: The point cloud data acquisition unit collects 3D laser point clouds at each inspection location in the subway tunnel, the mileage information acquisition unit collects high-precision mileage information of the mobile inspection platform operating in the subway tunnel, and the image information acquisition unit is used to obtain high-definition images of the subway tunnel at each inspection location; The signal transmission unit is used to wirelessly transmit the collected information of the collection unit to the cloud server in real time for storage; The target recognition unit identifies the target information in the subway tunnel high-definition image and 3D laser point cloud and eliminates non-tunnel structures; Preprocessing the target information and the collected information through a data preprocessing unit; The data processing unit performs a joint adjustment of the pre-processed high-definition subway tunnel images and 3D laser point clouds, constructs a dynamic control field, and estimates the position of the mobile detection platform relative to the global coordinates, generating a global high-resolution 3D holographic image of the tunnel. The data processing unit also uses a GNSS / INS / DMI integrated navigation algorithm to jointly locate adjacent key locations in the tunnel. The data fusion unit uses the trained machine learning model to fuse data and eliminate non-tunnel fixed objects based on the global high-resolution three-dimensional holographic image, three-dimensional laser point cloud and high-precision mileage information.

9. The method for submillimeter spatial measurement of 3D panoramic images of subway tunnels according to claim 8, characterized in that: Also includes the steps: The data fusion unit extracts the image features of each branch of the global high-resolution three-dimensional holographic image from different perspectives through a multi-branch convolutional neural network structure, and then fuses the image features of each branch using a feature weighting mechanism. The fused feature vector is reconstructed into a global high-resolution three-dimensional holographic image with uniform illumination and coherent texture through a decoder.

10. The method for sub-millimeter spatial measurement of 3D panoramic images of subway tunnels according to claim 8, characterized in that: Also includes the steps: The data fusion unit uses a pre-trained image enhancement model to perform brightness smoothing, color correction, and contrast enhancement on the reconstructed global high-resolution three-dimensional holographic image, thereby completing the global illumination optimization of the global high-resolution three-dimensional holographic image.

Images