A 3D imaging reconstruction method and system based on artificial intelligence recognition

By implementing SLAM algorithms and artificial intelligence recognition models on mobile robots, the problem of localization and reconstruction for autonomous exploration of mobile robots in environments without satellite signals was solved, achieving efficient and accurate 3D imaging reconstruction and model generation.

CN123118909APending Publication Date: 2026-11-17QINGDAO HIGHREN WATER SUPPLY GRP CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611333849.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-31
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In environments without satellite signals, mobile robots lack high-precision positioning and attitude compensation during autonomous exploration, resulting in inaccurate registration of two-dimensional scanning sequences, distortion of the reconstruction model, and low efficiency of full offline processing. Ignoring the electromagnetic wave physical propagation model leads to inaccurate reconstruction results.

Method used

A 3D imaging reconstruction method based on artificial intelligence recognition is adopted. The world coordinate system is established by executing the SLAM algorithm through a visual sensor and an inertial measurement unit. The global pose of the scanning probe is calculated by combining forward kinematics and hand-eye calibration. The artificial intelligence recognition model is used to perform real-time feature reasoning and physical inversion to generate a 3D model.

Benefits of technology

It achieves high-precision 3D imaging reconstruction in environments without satellite signals, compensates for carrier motion in real time, improves processing efficiency and imaging accuracy, and generates 3D models that can be used for visualization and engineering analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118909A_ABST
    Figure CN123118909A_ABST
Patent Text Reader

Abstract

This invention relates to a three-dimensional imaging reconstruction method and system based on artificial intelligence recognition, belonging to the field of three-dimensional imaging technology. It includes acquiring scanned image data and spatial pose data of a scanning probe generated during scanning on a mobile platform; inputting the scanned image data into a pre-trained artificial intelligence recognition model to infer and recognize features of a target object of a preset category in the scanned image data; generating a three-dimensional reconstruction trigger signal based on the recognition result and recording the trigger time; extracting scanned image data and spatial pose data within a preset time window corresponding to the trigger time; converting wave propagation effect features in the extracted scanned image data into a physical scattering point set corresponding to the target object; mapping the physical scattering point set to spatial coordinates to generate a three-dimensional model of the target object; and achieving high-precision, high-efficiency, and physically realistic three-dimensional real-time imaging and reconstruction of concealed targets in environments without satellite signals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and specifically to a three-dimensional imaging reconstruction method and system based on artificial intelligence recognition. Background Technology

[0002] In environments where satellite navigation signals are unavailable, such as indoor spaces and underground utility tunnels, the use of mobile robots for autonomous inspection and concealed target detection has become an urgent need. This requires the system to not only utilize visual positioning technology to achieve high-precision positioning of the robot itself, but also to simultaneously process the scanning imaging of concealed structures by equipment such as ground-penetrating radar.

[0003] In existing technologies, electromagnetic wave scanning equipment such as ground-penetrating radar is usually mounted on mobile robots to continuously scan the work area to obtain a sequence of two-dimensional profile images. Operators need to rely on professional experience to manually interpret and extract features from the collected full image data offline, and then generate a rough three-dimensional model of the target through a simple geometric mapping method.

[0004] However, in environments without satellite signals, existing technologies cannot accurately register 2D scan sequences in 3D space if the robot lacks high-precision self-localization synchronized with the scanned data. Secondly, without real-time compensation for pose changes during robot movement, geometric distortions in the reconstructed model will directly occur. Furthermore, full offline processing is inefficient and ignores the simple mapping of electromagnetic wave propagation models, resulting in inaccurate positioning and blurred details in the reconstructed results. Therefore, achieving high-precision fusion of non-visual sensor data and self-localization is a pressing technical challenge in the field of autonomous mobile robot exploration. Summary of the Invention

[0005] In view of the shortcomings of the related technologies, the purpose of this invention is to provide a three-dimensional imaging reconstruction method and system based on artificial intelligence recognition, so as to solve the technical problems of positioning loss, motion-induced imaging distortion, low efficiency of full offline processing and inaccurate reconstruction caused by ignoring physical effects in the absence of satellite signals.

[0006] This invention provides a three-dimensional imaging reconstruction method based on artificial intelligence recognition, comprising: Data acquisition steps: Acquire the scanning image data generated by the scanning probe on the mobile carrier platform during the scanning process, as well as the spatial pose data of the scanning probe; Reasoning and recognition steps: Input the scanned image data into a pre-trained artificial intelligence recognition model, perform reasoning and recognition on the features of target objects of a preset category in the scanned image data, obtain recognition results, generate a three-dimensional reconstruction trigger signal based on the recognition results, and record the trigger time; Data capture steps: In response to the three-dimensional reconstruction trigger signal, based on the trigger time, capture the scanned image data within a preset time window corresponding to the trigger time, as well as the spatial pose data aligned with the acquisition time of the scanned image data; Physical inversion steps: Based on the physical wave propagation mechanism, the intercepted scanned image data is inverted to convert the wave propagation effect characteristics in the intercepted scanned image data into a set of physical scattering points corresponding to the target body; 3D reconstruction steps: Based on the physical scattering point set and the extracted spatial pose data, the physical scattering point set is mapped to spatial coordinates to generate a 3D model of the target object.

[0007] By integrating the orientation of the scanning probe with the images scanned by the scanning probe, a fully automated reconstruction of the entire chain from raw data to a 3D model is achieved. Among them, the system analyzes image data in real time based on an artificial intelligence recognition model and generates reconstruction trigger signals, enabling the system to start the computationally intensive 3D reconstruction process only when target features are detected. This avoids redundant processing of all data and effectively solves the problem of low efficiency caused by offline processing of all data in traditional methods. At the same time, through spatiotemporal synchronization and physical inversion, the system significantly improves the real-time performance and positional accuracy of imaging, and achieves efficient and accurate 3D visualization of concealed targets.

[0008] In some embodiments of the present invention, the data acquisition step specifically includes: The SLAM algorithm is executed by a vision sensor and an inertial measurement unit installed on the mobile carrier platform. The world coordinate system is established using the initial physical position of the vision sensor or the inertial measurement unit when executing the SLAM algorithm as the origin and the right-hand rule. A base coordinate system is predefined on the base of the mobile support platform; The pose transformation of the base coordinate system relative to the world coordinate system is obtained by using the SLAM algorithm, thus obtaining the first attitude data of the base of the mobile carrier platform.

[0009] By using a visual sensor and an inertial measurement unit to execute the SLAM algorithm, and establishing a world coordinate system with well-defined directions based on the right-hand rule during SLAM initialization, a stable and unified absolute spatial reference datum is provided for the entire system. Even in environments where GNSS signals are denied, the mobile platform can still obtain accurate first attitude data of its base relative to the world coordinate system. This overcomes the key defect of traditional methods, which cannot accurately register two-dimensional images in three-dimensional space due to the lack of global positioning information. This lays a crucial foundation for the spatial fusion of all subsequent sensor data and high-precision three-dimensional reconstruction.

[0010] In some embodiments of the present invention, the data acquisition step further includes: An end coordinate system is predefined at the end of the pose adjustment mechanism on the mobile bearing platform; Based on the joint state data, the pose transformation relationship between the end-effector coordinate system and the base coordinate system is calculated using a forward kinematics algorithm to obtain the second pose data.

[0011] By acquiring the joint state data of the pose adjustment mechanism and using the forward kinematics algorithm to calculate the second pose data of the end-effector coordinate system relative to the base coordinate system, a precise mathematical description of the motion state of the robotic arm or similar adjustment mechanism is achieved. The system can acquire the actual spatial pose of the scanning probe mounting base in real time and quantitatively, providing key input for subsequent fusion of the overall motion of the platform and the local micro-motion of the probe, and performing precise attitude chain synthesis. This effectively compensates for the scanning trajectory deviation caused by the shaking of the carrier, and suppresses the geometric distortion of the imaging from the source.

[0012] In some embodiments of the present invention, the data acquisition step specifically includes: A probe coordinate system is predefined at the phase center of the scanning probe; The probe hand-eye matrix is ​​obtained by a hand-eye calibration method; wherein, the probe hand-eye matrix is ​​used to represent the pose transformation relationship of the probe coordinate system relative to the end-effector coordinate system; Based on the first posture data, the second posture data, and the probe hand-eye matrix, the global pose data of the probe coordinate system in the world coordinate system is calculated by homogeneous transformation matrix chain multiplication.

[0013] By explicitly defining the probe coordinate system at the phase center of the scanning probe and obtaining the probe hand-eye matrix relative to the end coordinate system using hand-eye calibration, and then combining the first and second attitude data to calculate the global pose data of the scanning probe in the world coordinate system through homogeneous transformation chain multiplication, the system mathematically ensures that no matter how the carrier platform moves or how the adjustment mechanism operates, the system can track the precise position and orientation of the phase center of the scanning probe in absolute space in real time. This provides a core guarantee for achieving high-precision binding of subsequent scanned image data with the real physical space.

[0014] In some embodiments of the present invention, the data acquisition step further includes: Using the acquisition time of each frame of the image data as a reference, the global pose data is interpolated temporally to obtain the spatial pose data that is time-aligned with each frame of the image.

[0015] By using the acquisition time of each image frame as a reference, the calculated global pose data is interpolated over time to obtain spatial pose data that is strictly time-aligned with each image frame. This solves the time drift problem caused by the asynchrony of sampling frequency and clock between the image sensor and the pose estimation system, ensuring that each scanned image can be associated with the real spatial position of the probe at the moment of acquisition. It achieves sub-sampling period-level synchronization of the data stream in the time dimension, eliminating a key source of time error for accurate mapping of spatial coordinates in subsequent physical inversion and 3D reconstruction.

[0016] In some embodiments of the present invention, the reasoning and recognition step specifically includes: The artificial intelligence recognition model is used to infer and recognize the features of target objects of a preset category, and the confidence score of the features is output. If the confidence score exceeds a preset threshold, it is determined that the target body has been detected, the three-dimensional reconstruction trigger signal is generated, and the trigger time is recorded.

[0017] By using an artificial intelligence recognition model to output a confidence score for the identification of target features, and only when the score exceeds a preset threshold is it determined that a target has been detected and a 3D reconstruction is triggered. This allows the system to mimic expert experience, automatically filter out interference signals and invalid data, and only start the subsequent computationally intensive reconstruction process when there is a high probability of detecting a target. This optimizes the allocation of edge computing resources and improves the processing efficiency and practicality of the entire system.

[0018] In some embodiments of the present invention, the physical inversion step specifically includes: Based on preset medium wave velocity parameters, the captured scanned image data is processed by a back projection algorithm or a synthetic aperture focusing algorithm to convert the diffraction features or discrete features in the captured scanned image data into a physical scattering point set of the target object.

[0019] By processing the captured scanned image data based on preset medium wave velocity parameters using a back projection algorithm or a synthetic aperture focusing algorithm, the diffraction or discrete features are converged and focused into a set of physical scattering points of the target body. This effectively restores the true spatial location of scattering points underground or inside the medium, fundamentally solving the core problem of inaccurate physical location and distorted target shape in the three-dimensional reconstruction results caused by ignoring beam scattering and diffraction effects.

[0020] In some embodiments of the present invention, the three-dimensional reconstruction step specifically includes: Based on the captured spatial pose data, the rotation matrix of the probe coordinate system relative to the world coordinate system and the translation vector of the origin of the probe coordinate system in the world coordinate system are obtained. Based on the coordinates of the physical scattering point set in the probe coordinate system, the rotation matrix, and the translation vector, the physical scattering point set is back-projected into the world coordinate system through spatial coordinate transformation to generate a sparse three-dimensional point cloud of the target object.

[0021] By combining the local coordinates of the physical scattering point set in the probe coordinate system with the second global pose at the corresponding moment, the points are back-projected to the world coordinate system through spatial coordinate transformation, thereby generating a sparse three-dimensional point cloud of the target in world space. This completes the key transformation from the probe's local observation coordinate system to the global unified world coordinate system, so that all the inverted scattering points are placed under the same absolute spatial reference frame, laying a direct data foundation for accurately representing the three-dimensional geometry of the target, performing multi-view data fusion, and subsequent point cloud processing.

[0022] In some embodiments of the present invention, the three-dimensional reconstruction step further includes: Based on the sparse 3D point cloud, the spatial envelope of the target body is calculated by principal component analysis, and the surface is reconstructed to obtain the 3D model of the target body.

[0023] By using principal component analysis to calculate the spatial envelope of the target object and reconstruct its surface from sparse 3D point clouds, the conversion from discrete point sets to continuous and complete 3D solid models is realized. It can automatically estimate the main orientation, approximate range and geometric contour of the target from scattered point cloud data, and reconstruct its visually coherent and structurally reasonable surface model, generating a standard 3D digital model that can be directly used for visualization, dimensional measurement or engineering analysis, greatly improving the usability and engineering value of the reconstruction results.

[0024] Some embodiments of the present invention further provide a three-dimensional imaging reconstruction system based on artificial intelligence recognition, comprising: Mobile carrier platform; A motion execution subsystem, which includes a pose adjustment mechanism mounted on the mobile carrier platform; A multi-source sensing subsystem, comprising: A vision sensor, which is mounted on the mobile carrier platform; An inertial measurement unit, which is installed on the mobile carrier platform, is used to acquire the first attitude data of the base of the mobile carrier platform in conjunction with the vision sensor; A scanning probe, installed at the end of the pose adjustment mechanism, is used to scan the area to be tested and acquire scanned image data; The computational control subsystem is communicatively connected to the motion execution subsystem and the multi-source sensing subsystem. The computational control subsystem is configured to execute the aforementioned three-dimensional imaging reconstruction method based on artificial intelligence recognition.

[0025] By combining the mobile carrier platform, motion execution subsystem, multi-source sensing subsystem, and computing control subsystem into a complete integrated hardware and software system, the engineering and productization of the 3D imaging reconstruction method is realized. The functional division and data flow of each hardware module in the system are clarified. The computing control subsystem is configured to execute the complete algorithm flow of the above method, ensuring that the method can run stably on an independent, deployable physical device. Thus, the innovative method flow is transformed into reliable technical equipment that can be practically applied to field testing. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating a three-dimensional imaging reconstruction method based on artificial intelligence recognition, provided as an embodiment of the present invention; Figure 2 A flowchart for calculating global pose data is provided in an embodiment of the present invention; Figure 3 A data flow diagram for reasoning, recognition, and data interception of an artificial intelligence model provided in this embodiment of the invention; Figure 4 This is a schematic diagram of a two-dimensional image before offset imaging, provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a three-dimensional image after offset imaging, provided in an embodiment of the present invention; Figure 6 This is a structural diagram of a three-dimensional imaging reconstruction system based on artificial intelligence recognition, provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0029] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0030] When detecting and imaging concealed spaces such as underground cavities, building interiors, and hidden structures behind tunnel walls, electromagnetic wave-based scanning technology is the primary method; for example, using ground-penetrating radar (GPR) for scanning. However, existing scanning imaging technologies mainly rely on manual interpretation of two-dimensional cross-sectional images, which presents the following technical bottlenecks in digital three-dimensional reconstruction; In environments where there is no global navigation satellite system signal, such as indoor spaces, underground utility tunnels, or tunnels, mobile scanning devices have difficulty obtaining accurate global location information, which makes it impossible to accurately register the acquired two-dimensional images in three-dimensional space. When using mobile robots or wearable platforms for scanning, the movement and shaking of the carrier can cause the scanning trajectory to deviate. Without fine kinematic posture compensation, the final generated image will suffer severe geometric distortion. Traditional methods mostly involve full offline processing, and they simply map the pixels of two-dimensional ground-penetrating radar images to three-dimensional space. This processing method ignores the scattering characteristics of radar beams and hyperbolic diffraction effects, resulting in three-dimensional reconstruction results that are not only computationally intensive but also have inaccurate physical positions, failing to meet the operational requirements of real-time scanning and precise imaging.

[0031] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] The technical solution of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0033] like Figure 1 As shown, the present invention provides a three-dimensional imaging reconstruction method based on artificial intelligence recognition, comprising: Data acquisition step S1: Acquire the scanning image data generated by the scanning probe on the mobile carrier platform during the scanning process, as well as the spatial pose data of the scanning probe; In some embodiments, the data acquisition step S1 specifically includes: The SLAM algorithm is executed by a vision sensor and an inertial measurement unit mounted on a mobile carrier platform. The world coordinate system is established using the initial physical position of the vision sensor or the inertial measurement unit when executing the SLAM algorithm as the origin and the right-hand rule. A base coordinate system is predefined on the base of the mobile platform; The pose transformation relationship between the base coordinate system and the world coordinate system is obtained by using the SLAM algorithm, thus obtaining the first attitude data of the base of the mobile carrier platform.

[0034] By using a visual sensor and an inertial measurement unit to execute the SLAM algorithm, and establishing a world coordinate system with well-defined directions based on the right-hand rule during SLAM initialization, a stable and unified absolute spatial reference datum is provided for the entire system. Even in environments where GNSS signals are denied, the mobile platform can still obtain accurate first attitude data of its base relative to the world coordinate system. This overcomes the key defect of traditional methods, which cannot accurately register two-dimensional images in three-dimensional space due to the lack of global positioning information. This lays a crucial foundation for the spatial fusion of all subsequent sensor data and high-precision three-dimensional reconstruction.

[0035] Furthermore, such as Figure 2 As shown, establish a world coordinate system. World coordinate system The initial position of the mobile platform during the execution of the SLAM algorithm is taken as the origin of the coordinate system, and defined as follows: The direction opposite to the direction of gravity is taken as the Z-axis direction, and defined as follows: The initial forward direction of the mobile platform is taken as the X-axis direction, and defined as follows: The direction of the Y-axis is determined according to the right-hand rule and defined as follows: ; Furthermore, establish a base coordinate system. Base coordinate system The origin of the coordinate system is defined as the geometric center of the mobile platform or the installation center of the inertial measurement unit. The Z-axis direction is defined as the direction perpendicular to the platform surface and pointing upwards. The X-axis is defined as the direction directly in front of the mobile platform. The direction of the Y-axis is determined by the right-hand rule and defined as follows: ; The SLAM algorithm is executed by a vision sensor and an inertial measurement unit installed on the mobile platform to acquire the first attitude data of the mobile platform's base in the world coordinate system in real time. .

[0036] In some embodiments, the data acquisition step S1 further includes: A predefined end coordinate system is defined at the end of the pose adjustment mechanism on the mobile carrier platform; Based on joint state data, the pose transformation relationship between the end-effector coordinate system and the base coordinate system is calculated using a forward kinematics algorithm to obtain the second pose data.

[0037] By acquiring the joint state data of the pose adjustment mechanism and using the forward kinematics algorithm to calculate the second pose data of the end-effector coordinate system relative to the base coordinate system, a precise mathematical description of the motion state of the robotic arm or similar adjustment mechanism is achieved. The system can acquire the actual spatial pose of the scanning probe mounting base in real time and quantitatively, providing key input for subsequent fusion of the overall motion of the platform and the local micro-motion of the probe, and performing precise attitude chain synthesis. This effectively compensates for the scanning trajectory deviation caused by the shaking of the carrier, and suppresses the geometric distortion of the imaging from the source.

[0038] Furthermore, establish the end coordinate system. End coordinate system The origin of the coordinate system is defined as the center of the flange plane at the end of the posture adjustment mechanism, and is further defined as follows: The Z-axis direction is defined as the direction perpendicular to the flange plane and pointing outwards. The zero position of the pose adjustment mechanism is determined as the X-axis direction, and is defined as follows: The direction of the Y-axis is determined according to the right-hand rule and defined as follows: ; Based on joint state data, and combined with forward kinematics algorithms, the pose transformation relationship between the end-effector coordinate system and the base coordinate system is calculated to obtain the second pose data. .

[0039] Optionally, the pose adjustment mechanism can be a multi-degree-of-freedom robotic arm.

[0040] In some embodiments, the data acquisition step S1 further includes: A probe coordinate system is predefined at the phase center of the scanning probe; The probe hand-eye matrix is ​​obtained through a hand-eye calibration method; the probe hand-eye matrix is ​​used to represent the pose transformation relationship between the probe coordinate system and the end effector coordinate system. Based on the first attitude data, the second attitude data, and the probe hand-eye matrix, the global pose data of the probe coordinate system in the world coordinate system is calculated through homogeneous transformation matrix chain multiplication.

[0041] By explicitly defining the probe coordinate system at the phase center of the scanning probe and obtaining the probe hand-eye matrix relative to the end coordinate system using hand-eye calibration, and then combining the first and second attitude data to calculate the global pose data of the scanning probe in the world coordinate system through homogeneous transformation chain multiplication, the system mathematically ensures that no matter how the carrier platform moves or how the adjustment mechanism operates, the system can track the precise position and orientation of the phase center of the scanning probe in absolute space in real time. This provides a core guarantee for achieving high-precision binding of subsequent scanned image data with the real physical space.

[0042] Furthermore, the active motion compensation of the pose adjustment mechanism to the moving support platform ensures the stability of the coupling posture of the scanning probe relative to the measured surface.

[0043] Establish probe coordinate system Probe coordinate system The origin of the coordinate system is defined as the phase center of the scanning probe, and it is further defined as follows: The Z-axis is defined as the main emission direction of electromagnetic waves. The polarization direction parallel to the scanning probe is defined as the X-axis direction. The direction of the Y-axis is determined according to the right-hand rule and defined as follows: ; Obtain the probe's hand-eye matrix using a hand-eye calibration method. ; Based on the first attitude data Second attitude data and probe hand-eye matrix The global pose data of the probe coordinate system in the world coordinate system is calculated by chain multiplication of homogeneous transformation matrices. The computational model for global pose data is as follows:

[0044] in, This is global pose data.

[0045] Optionally, the scanning probe can be a ground-penetrating radar (GPR), which scans the area to be measured and acquires a GPR B-Scan image data stream.

[0046] In some embodiments, such as Figure 3 As shown, data acquisition step S1 further includes: Using the acquisition time of each frame of the image data as a reference, the global pose data is interpolated temporally to obtain spatial pose data that is time-aligned with each frame of the image.

[0047] By using the acquisition time of each image frame as a reference, the calculated global pose data is interpolated over time to obtain spatial pose data that is strictly time-aligned with each image frame. This solves the time drift problem caused by the asynchrony of sampling frequency and clock between the image sensor and the pose estimation system, ensuring that each scanned image can be associated with the real spatial position of the probe at the moment of acquisition. It achieves sub-sampling period-level synchronization of the data stream in the time dimension, eliminating a key source of time error for accurate mapping of spatial coordinates in subsequent physical inversion and 3D reconstruction.

[0048] Furthermore, the acquisition timestamp of each frame in the ground-penetrating radar B-Scan image data stream is used. As a baseline; for each time step In the global pose data stream, find the two temporally nearest pose points, and use a time interpolation algorithm to calculate the pose points that strictly correspond to the time interval. Spatial pose data ; Each frame of image is bound to its corresponding spatial pose data to form a time-aligned synchronous data stream, which is then stored in a circular buffer.

[0049] Alternatively, the time difference algorithm can employ a spherical linear interpolation algorithm.

[0050] Inference and recognition step S2: Input the scanned image data into the pre-trained artificial intelligence recognition model, perform inference and recognition on the features of the target objects of the preset category in the scanned image data, obtain the recognition result, generate a three-dimensional reconstruction trigger signal based on the recognition result, and record the trigger time; In some embodiments, the reasoning and discrimination step S2 specifically includes: The AI ​​recognition model infers and judges the features of target objects in a preset category, and outputs the confidence score of the features. If the confidence score exceeds the preset threshold, it is determined that a target body has been detected, a 3D reconstruction trigger signal is generated, and the trigger time is recorded.

[0051] By using an artificial intelligence recognition model to output a confidence score for the identification of target features, and only when the score exceeds a preset threshold is it determined that a target has been detected and a 3D reconstruction is triggered. This allows the system to mimic expert experience, automatically filter out interference signals and invalid data, and only start the subsequent computationally intensive reconstruction process when there is a high probability of detecting a target. This optimizes the allocation of edge computing resources and improves the processing efficiency and practicality of the entire system.

[0052] Furthermore, the real-time collected ground-penetrating radar B-Scan image data stream is continuously input into a pre-trained artificial intelligence recognition model for online reasoning and analysis; The core task of the artificial intelligence recognition model is to intelligently identify whether an image contains a target of a preset category, and to output a quantitative confidence score for the recognition result of each frame of the image. This confidence score reflects the probability that the artificial intelligence recognition model determines that the current image features belong to a specific target. Among them, the target objects in the preset categories include the feature textures of holes or heterogeneous objects; When the confidence score of a frame exceeds a preset threshold, a valid target is detected, and a 3D reconstruction trigger signal is immediately generated, while the current trigger moment is precisely recorded. .

[0053] By using the quantitative indicator of confidence level, the system can simulate expert experience to achieve intelligent initial screening of massive scanning data, automatically filter out a large number of interference signals and invalid data segments, and ensure that subsequent computationally intensive physical inversion and 3D reconstruction processes are only initiated in image areas where targets are highly likely to exist.

[0054] Data capture step S3: In response to the 3D reconstruction trigger signal, based on the trigger time, capture the scanned image data within the preset time window corresponding to the trigger time, as well as the spatial pose data aligned with the acquisition time of the scanned image data.

[0055] Furthermore, in response to the generated 3D reconstruction trigger signal, the trigger moment is precisely recorded. Centered on the circular buffer, backtrack and extract a continuous data segment within a preset time window, which covers the area... Scope; The extracted continuous data segments include data blocks of continuous ground-penetrating radar B-Scan image data within the preset time window, which necessarily contain the trigger time. The corresponding image frame and several adjacent frames ensure the contextual integrity of the target features in the scanning sequence. The extracted continuous data segments also include a sequence of second global poses that are strictly aligned with each of the above-mentioned frames through time interpolation, ensuring that each image is bound to the absolute position and orientation of the probe in three-dimensional space at the moment of acquisition.

[0056] Physical inversion step S4: Based on the physical wave propagation mechanism, the intercepted scanned image data is inverted and processed to convert the wave propagation effect characteristics in the intercepted scanned image data into the physical scattering point set corresponding to the target body; In some embodiments, the physical inversion step S4 specifically includes: Based on preset medium wave velocity parameters, the intercepted scanned image data is processed by a back projection algorithm or a synthetic aperture focusing algorithm to focus the diffraction features or discrete features in the intercepted scanned image data into a set of physical scattering points of the target object.

[0057] By processing the captured image data based on preset medium wave velocity parameters using a back projection algorithm or a synthetic aperture focusing algorithm, the diffraction or discrete features are converged and focused into a set of physical scattering points of the target object. This effectively restores the true spatial location of scattering points underground or inside the medium, fundamentally solving the core problem of inaccurate physical location and distorted target shape in the three-dimensional reconstruction results caused by ignoring beam scattering and diffraction effects.

[0058] Furthermore, the extracted data blocks undergo core physical inversion imaging and feature focusing processing based on preset probe medium wave velocity parameters. And use either the back projection algorithm or the synthetic aperture focusing algorithm for processing; The focusing or restoration process involves converting the image texture representing the differences in electromagnetic wave propagation paths into a three-dimensional spatial point set that directly characterizes the geometric position of the target object through this physical inversion process. This lays a precise and physically sound foundation for the subsequent transformation of local observation coordinates to the global world coordinate system. like Figure 4 As shown, in the two-dimensional image before offset, i.e., the B-Scan image data of the bottom-penetrating radar, the response of a point target is represented by a diffraction hyperbola on the image; the vertex of the diffraction hyperbola corresponds to the apparent position of the target in the horizontal direction, but due to the travel time effect of electromagnetic wave propagation, the target in the image is stretched in the vertical direction and its spatial position is blurred. like Figure 5 As shown, after processing, the energy on the diffraction hyperbola is back-projected to its true physical location, resulting in a three-dimensional space. The focus is concentrated into a single point, corresponding to the actual physical location of the target object.

[0059] Step S5 of 3D reconstruction: Based on the physical scattering point set and the extracted spatial pose data, perform spatial coordinate mapping on the physical scattering point set to generate a 3D model of the target object.

[0060] In some embodiments, the three-dimensional reconstruction step S5 further includes: Based on the captured spatial pose data, the rotation matrix of the probe coordinate system relative to the world coordinate system and the translation vector of the origin of the probe coordinate system in the world coordinate system are obtained. Based on the coordinates, rotation matrix, and translation vector of the physical scattering point set in the probe coordinate system, the physical scattering point set is back-projected into the world coordinate system through spatial coordinate transformation to generate a sparse three-dimensional point cloud of the target object.

[0061] By combining the local coordinates of the physical scattering point set in the probe coordinate system with the spatial pose data at the corresponding time, the points are back-projected to the world coordinate system through spatial coordinate transformation, thereby generating a sparse three-dimensional point cloud of the target in world space. This completes the key transformation from the probe's local observation coordinate system to the global unified world coordinate system, so that all the inverted scattering points are placed under the same absolute spatial reference frame. This lays a direct data foundation for accurately representing the three-dimensional geometry of the target, performing multi-view data fusion, and subsequent point cloud processing.

[0062] Furthermore, based on the coordinates of the physical discrete point set in the probe coordinate system And combined with spatial pose data Among them, spatial pose data This includes defining the orientation of the probe coordinate system relative to the world coordinate system. Rotation matrix And the position of the origin of the probe coordinate system in the world coordinate system. Translation vector ; By applying the spatial coordinate transformation formula, each local physical scattering point is accurately transformed to a unified world coordinate system, generating a three-dimensional sparse point cloud representing the spatial distribution of the target body.

[0063] in, The three-dimensional coordinates of the physical scattering point set in the world coordinate system .

[0064] In some embodiments, the three-dimensional reconstruction step S5 further includes: Based on sparse 3D point clouds, the spatial envelope of the target body is calculated by principal component analysis, and the surface is reconstructed to obtain a 3D model of the target body.

[0065] By using principal component analysis to calculate the spatial envelope of the target object and reconstruct its surface from sparse 3D point clouds, the conversion from discrete point sets to continuous and complete 3D solid models is realized. It can automatically estimate the main orientation, approximate range and geometric contour of the target from scattered point cloud data, and reconstruct its visually coherent and structurally reasonable surface model, generating a standard 3D digital model that can be directly used for visualization, dimensional measurement or engineering analysis, greatly improving the usability and engineering value of the reconstruction results.

[0066] Furthermore, a density-based spatial clustering algorithm is used to process the 3D sparse point cloud to classify points and filter out noise, thereby separating point sets belonging to the same target body and improving data quality to obtain a clean point cloud; Principal component analysis (PCA) is used to calculate the main spatial orientation and geometric envelope of the target body. Based on the clean point cloud, a continuous and complete three-dimensional mesh surface model is generated through surface reconstruction algorithm, and finally a standardized three-dimensional model that can be widely used in the engineering field is output. For example, it can be exported as an IFC format file for building information modeling system integration, or directly transmitted to an augmented reality terminal to achieve real-time, accurate virtual-real fusion visualization of the reconstruction results and the real scene, thereby completing the entire process of transformation from raw detection data to digital results that can be directly used for analysis, decision-making and display.

[0067] Alternatively, the density-based spatial clustering algorithm can employ the DBSCAN algorithm.

[0068] Based on the above-mentioned 3D imaging reconstruction method, by integrating the posture of the scanning probe with the image scanned by the scanning probe, the entire chain of automated reconstruction from raw data to 3D model is realized. Among them, the system analyzes image data in real time based on an artificial intelligence recognition model and generates reconstruction trigger signals, enabling the system to start the computationally intensive 3D reconstruction process only when target features are detected. This avoids redundant processing of all data and effectively solves the problem of low efficiency caused by offline processing of all data in traditional methods. At the same time, through spatiotemporal synchronization and physical inversion, the system significantly improves the real-time performance and positional accuracy of imaging, and achieves efficient and accurate 3D visualization of concealed targets.

[0069] Preferably, the above-mentioned three-dimensional imaging reconstruction method based on artificial intelligence recognition is applied to the underground pipeline maintenance scenario; Among them, the target body of the preset category specifically refers to underground pipelines and water-bearing anomalies formed by their leakage; It uses an exoskeleton as a mobile support platform, with a binocular vision camera installed on its back as a vision sensor, and a high-precision inertial measurement unit also installed on its back. A robotic arm is used as the posture adjustment mechanism, and a 400MHz ground-penetrating radar antenna is mounted at the end of the robotic arm as a scanning probe. YOLO series object detection models are used as pre-trained artificial intelligence recognition models; Inspection personnel donned the integrated exoskeleton system to enter the underground utility tunnel work area. After power-on initialization, the system immediately began executing the SLAM algorithm via a binocular vision camera and inertial measurement unit on its back, calculating and outputting the pose matrix of the exoskeleton's back base in real time, thus obtaining the first pose data. ; Meanwhile, the system continuously reads the encoder values ​​of each joint of the robotic arm to obtain joint status data; The real-time transformation matrix of the robotic arm's end effector relative to the exoskeleton's back base is calculated using the forward kinematics model of the robotic arm to obtain the second pose data. ; Combined with the probe hand-eye matrix, which was obtained in advance through hand-eye calibration and describes the relationship between the phase center of the ground penetrating radar antenna and the end flange of the robotic arm, The system uses the matrix chain multiplication rule. Real-time calculation of global pose data of radar antenna phase center ; During the scanning process, the 400MHz ground-penetrating radar continuously probes the area below the utility tunnel, generating B-Scan image data streams; The B-Scan image data stream is input in real time into a YOLO series object detection model that has been pre-trained using a large number of pipe and leak images; When the YOLO series target detection model identifies features representing the outline of a pipe or bright clumps representing a leakage area in an image, and its confidence score exceeds a set threshold, the system immediately generates a 3D reconstruction trigger signal.

[0070] In response to the 3D reconstruction trigger signal, the system extracts a sequence of image data blocks and corresponding spatial pose data from the past second, centered on the trigger moment. The captured image patch data is processed based on the preset detection medium wave velocity, and then the diffraction features or discrete features are focused into a set of physical scattering points using a back projection algorithm or a synthetic aperture focusing algorithm. Subsequently, the physical scattering point set is back-projected onto the world coordinate system using the corresponding spatial pose data to generate a sparse 3D point cloud.

[0071] The DBSCAN clustering algorithm is used to cluster sparse 3D point clouds, and a 3D model of the pipe layout and leaking water masses is rendered in AR glasses.

[0072] Optionally, the above-mentioned three-dimensional imaging reconstruction method based on artificial intelligence recognition can also be applied to automatic search and rescue and reconnaissance in collapsed ruins or dangerous areas that are difficult for humans to enter after disasters such as earthquakes and explosions. Among them, the mobile carrier platform is a quadruped robot with strong obstacle-crossing capabilities. The visual sensor and inertial measurement unit are integrated into the robot's torso head to execute the SLAM algorithm in real time in complex unstructured environments and output the first pose data of the robot's torso base in the world coordinate system. The pose adjustment mechanism is the robot's own bionic leg motion system. The system reads the motor encoder data of each joint and calculates the second pose data of the scanning probe mounting point mounted on the front or abdomen of the robot relative to the torso base in real time based on the whole-body kinematic model of the quadruped robot. In this way, the kinematic data of the robot body is used to compensate for the scanning motion in real time. The scanning probe can be an ultra-wideband radar suitable for detecting vital signs or a ground-penetrating radar used to detect cavity structures. The quadruped robot autonomously enters the ruins area to perform a grid-based scanning task; the scanning probe continuously collects image data streams and inputs them into a pre-trained artificial intelligence recognition model in real time; The artificial intelligence recognition model is configured to identify micro-motion signal features representing vital signs or hole and cavity reflection features representing living space. Once the recognition confidence score exceeds the threshold, the system immediately generates a three-dimensional reconstruction trigger signal. The system captures the scanned image data and the synchronously obtained global pose data after robot kinematics compensation, with the trigger moment as the center. By physical inversion and three-dimensional spatial back projection, a sparse three-dimensional point cloud is generated to represent the space where life might be located or the hidden cavity structure inside the ruins. After processing, the internal perspective three-dimensional model under the ruins is finally reconstructed.

[0073] The 3D model can be transmitted to the rear command system in real time, providing search and rescue personnel with intuitive and accurate internal structural information, accurately locating the target to be rescued, greatly improving search and rescue efficiency and safety, and enabling machines to replace humans in entering dangerous environments for intelligent and visual detection.

[0074] like Figure 6 As shown, this embodiment of the invention also provides a three-dimensional imaging reconstruction system based on artificial intelligence recognition, comprising: Mobile carrier platform; The motion execution subsystem includes a pose adjustment mechanism mounted on a mobile carrier platform; A multi-source sensing subsystem, comprising: A vision sensor, mounted on a mobile platform; An inertial measurement unit, mounted on a mobile carrier platform, is used in conjunction with a vision sensor to acquire the first attitude data of the base of the mobile carrier platform. The scanning probe, installed at the end of the pose adjustment mechanism, is used to scan the area to be measured and acquire scanned image data; The computational control subsystem is communicatively connected to the motion execution subsystem and the multi-source sensing subsystem. The computational control subsystem is configured to execute the aforementioned three-dimensional imaging reconstruction method based on artificial intelligence recognition.

[0075] By combining the mobile carrier platform, motion execution subsystem, multi-source sensing subsystem, and computing control subsystem into a complete integrated hardware and software system, the engineering and productization of the 3D imaging reconstruction method is realized. The functional division and data flow of each hardware module in the system are clarified. The computing control subsystem is configured to execute the complete algorithm flow of the above method, ensuring that the method can run stably on an independent, deployable physical device. Thus, the innovative method flow is transformed into reliable technical equipment that can be practically applied to field testing.

[0076] Furthermore, mobile carrier platforms include exoskeleton-style platforms, wheeled robots, multi-legged robots, or unmanned aerial vehicles used to carry equipment.

[0077] Furthermore, the pose adjustment mechanism can be a lightweight robotic arm mounted on a back support frame to assist operators in maintaining stable scanning of the probe while moving.

[0078] In some embodiments, the 3D imaging reconstruction system further includes a spatiotemporal synchronization module. The spatiotemporal synchronization module is used to unify the time reference of each sensor, ensuring that the timestamps of the scanned image data and the global pose data are strictly aligned.

[0079] In some embodiments, the 3D imaging reconstruction system further includes a visualization interactive terminal. The visualization interactive terminal includes a tablet computer or augmented reality glasses, used to receive and display the reconstructed 3D model, achieving a fusion of virtual and real-world display.

[0080] It should be noted that the above is a reference method and system for three-dimensional imaging reconstruction based on artificial intelligence recognition, and the present invention is not limited thereto.

[0081] The embodiments of the present invention realize high-precision, high-efficiency, and physically realistic three-dimensional real-time imaging and reconstruction of concealed targets in the absence of satellite signals. It solves the technical problems of existing technologies in the absence of satellite signals, such as positioning loss, imaging distortion caused by motion, and low efficiency of full offline processing and inaccurate reconstruction due to neglect of physical effects.

[0082] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0083] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A three-dimensional imaging reconstruction method based on artificial intelligence recognition, characterized in that, include: Data acquisition steps: Acquire the scanning image data generated by the scanning probe on the mobile carrier platform during the scanning process, as well as the spatial pose data of the scanning probe; Reasoning and recognition steps: Input the scanned image data into a pre-trained artificial intelligence recognition model, perform reasoning and recognition on the features of target objects of a preset category in the scanned image data, obtain recognition results, generate a three-dimensional reconstruction trigger signal based on the recognition results, and record the trigger time; Data capture steps: In response to the three-dimensional reconstruction trigger signal, based on the trigger time, capture the scanned image data within a preset time window corresponding to the trigger time, as well as the spatial pose data that is time-aligned with the scanned image data; Physical inversion steps: Based on the physical wave propagation mechanism, the intercepted scanned image data is inverted to convert the wave propagation effect characteristics in the intercepted scanned image data into a set of physical scattering points corresponding to the target body; 3D reconstruction steps: Based on the physical scattering point set and the extracted spatial pose data, the physical scattering point set is mapped to spatial coordinates to generate a 3D model of the target object.

2. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 1, characterized in that, The data acquisition steps are as follows: The SLAM algorithm is executed by a vision sensor and an inertial measurement unit installed on the mobile carrier platform. The world coordinate system is established using the initial physical position of the vision sensor or the inertial measurement unit when executing the SLAM algorithm as the origin and the right-hand rule. A base coordinate system is predefined on the base of the mobile support platform; The pose transformation relationship between the base coordinate system and the world coordinate system is obtained by using the SLAM algorithm, thereby obtaining the first attitude data of the base of the mobile carrier platform.

3. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 2, characterized in that, The data acquisition steps also include: An end coordinate system is predefined at the end of the pose adjustment mechanism on the mobile bearing platform; Based on the joint state data, the pose transformation relationship between the end-effector coordinate system and the base coordinate system is calculated using a forward kinematics algorithm to obtain the second pose data.

4. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 3, characterized in that, The data acquisition steps also include: A probe coordinate system is predefined at the phase center of the scanning probe; The probe hand-eye matrix is ​​obtained by a hand-eye calibration method; wherein, the probe hand-eye matrix is ​​used to represent the pose transformation relationship of the probe coordinate system relative to the end-effector coordinate system; Based on the first posture data, the second posture data, and the probe hand-eye matrix, the global pose data of the probe coordinate system in the world coordinate system is calculated by homogeneous transformation matrix chain multiplication.

5. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 4, characterized in that, The data acquisition steps also include: Using the acquisition time of each frame in the scanned image data as a reference, the global pose data is interpolated over time to obtain the spatial pose data that is time-aligned with each frame.

6. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to any one of claims 1 to 5, characterized in that, The reasoning and recognition steps are specifically as follows: The artificial intelligence recognition model is used to infer and recognize the features of target objects of a preset category, and the confidence score of the features is output. If the confidence score exceeds a preset threshold, it is determined that the target body has been detected, the three-dimensional reconstruction trigger signal is generated, and the trigger time is recorded.

7. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 4, characterized in that, The physical inversion steps are as follows: Based on preset medium wave velocity parameters, the captured scanned image data is processed by a back projection algorithm or a synthetic aperture focusing algorithm to convert the diffraction features or discrete features in the captured scanned image data into a physical scattering point set of the target object.

8. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 7, characterized in that, The specific steps of the three-dimensional reconstruction are as follows: Based on the captured spatial pose data, the rotation matrix of the probe coordinate system relative to the world coordinate system and the translation vector of the origin of the probe coordinate system in the world coordinate system are obtained. Based on the coordinates of the physical scattering point set in the probe coordinate system, the rotation matrix, and the translation vector, the physical scattering point set is back-projected into the world coordinate system through spatial coordinate transformation to generate a sparse three-dimensional point cloud of the target object.

9. The three-dimensional imaging reconstruction method based on artificial intelligence recognition according to claim 8, characterized in that, The three-dimensional reconstruction step also includes: Based on the sparse 3D point cloud, the spatial envelope of the target body is calculated by principal component analysis, and the surface is reconstructed to obtain the 3D model of the target body.

10. A three-dimensional imaging reconstruction system based on artificial intelligence recognition, characterized in that, include: Mobile carrier platform; A motion execution subsystem, which includes a pose adjustment mechanism mounted on the mobile carrier platform; A multi-source sensing subsystem, comprising: A vision sensor, which is mounted on the mobile carrier platform; An inertial measurement unit, which is installed on the mobile carrier platform, is used to acquire the first attitude data of the base of the mobile carrier platform in conjunction with the vision sensor; A scanning probe, installed at the end of the pose adjustment mechanism, is used to scan the area to be tested and acquire scanned image data; A computational control subsystem, communicatively connected to the motion execution subsystem and the multi-source sensing subsystem, is configured to execute the three-dimensional imaging reconstruction method based on artificial intelligence recognition as described in claims 1 to 9.