Large-range deformation measurement based on multi-camera space-time consistency under complex working condition of AI + vision + IMU fusion

Abstract

The invention relates to the technical field of deformation monitoring, and discloses a large-range deformation measurement method and device based on multi-camera space-time consistency under the complex working condition of AI + vision + IMU fusion. The method comprises the steps of multi-camera data synchronization, IMU data fusion, AI algorithm optimization, space-time consistency calculation and deformation compensation. The method comprises the following steps: enabling multiple cameras to jointly work through a space-time synchronization technology to obtain a multi-view image, recording motion information by using an IMU (Inertial Measurement Unit) to compensate visual errors, fusing and optimizing data by using an AI deep learning model, constructing a measurement model through space-time consistency analysis, and carrying out deformation compensation by combining visual and IMU information. The method avoids the measurement limitation of a single camera, effectively eliminates the measurement error of a complex environment, can achieve high-precision real-time dynamic deformation monitoring in a large range, can be widely applied to the fields of civil engineering, aerospace and the like, and has remarkable innovativeness and application value.

Description

Technical Field

[0001] This invention relates to the field of deformation monitoring technology, and in particular to an AI+vision+IMU fusion method and device for measuring large-scale deformation of large civil structures, tunnels, bridges and other structures under complex working conditions. It is especially suitable for high-precision monitoring systems that require accurate capture of structural deformation or displacement. Background Technology

[0002] Currently, with the rapid development of civil engineering, infrastructure construction, and high-precision measurement, the demand for deformation monitoring is increasing. While traditional deformation measurement methods (such as total stations and GPS) offer high accuracy, they still have limitations in large-scale, complex environments. For example, in large-scale civil structure deformation monitoring, traditional measuring equipment typically requires a large number of measurement points and cannot provide real-time results; in dynamic environments, optical equipment such as total stations, limited by fixed viewing angles and fields of view, often fail to comprehensively and accurately reflect structural deformation.

[0003] Furthermore, existing visual measurement methods (such as single-camera or multi-camera arrays) also face problems such as poor measurement accuracy over large areas, poor synchronization, and data instability in dynamic environments. In order to achieve high-precision structural deformation measurement in large-scale, dynamic environments, new, efficient, and stable measurement methods are urgently needed. Summary of the Invention

[0004] This invention aims to solve the problems of low accuracy and inability to guarantee efficient synchronization in large-scale structural deformation measurement under complex working conditions in existing technologies. By integrating AI algorithms, computer vision and IMU (inertial measurement unit) technologies, a new method for large-scale deformation measurement based on multi-camera spatiotemporal consistency is proposed.

[0005] To achieve the above objectives, this application provides the following technical solution: large-scale deformation measurement based on multi-camera spatiotemporal consistency under complex working conditions using AI + vision + IMU fusion, including the following steps: Multi-camera data synchronization: Multiple visual measurement cameras work together through spatiotemporal synchronization technology, with the camera array covering the entire monitoring area to acquire image data from multiple different perspectives; IMU data fusion: Using IMU devices to record the pose and motion information of the measurement platform, providing motion data compensation for visual measurement, and solving the error caused by motion in visual measurement; AI algorithm optimization: The AI ​​deep learning model is used to fuse multi-dimensional data from visual cameras and IMUs, and the measurement data is optimized through spatiotemporal consistency constraints to reduce image errors and improve measurement accuracy; Spatiotemporal consistency calculation: Analyze image sequences captured by multiple cameras and displacement information provided by IMU, and use spatiotemporal consistency analysis technology to construct a large-scale deformation measurement model to accurately estimate structural deformation and displacement; Deformation compensation: The measurement data is processed for deformation compensation. By combining visual data and IMU motion information, large-scale and high-precision structural deformation compensation is achieved, and the final high-precision deformation data is output.

[0006] Preferably, in the multi-camera data synchronization step, multiple cameras synchronously capture image data through internal timestamps to ensure data consistency in time and space.

[0007] Preferably, in the IMU data fusion step, the IMU device monitors and records motion information such as acceleration and angular velocity of the measurement platform in real time to compensate for displacement errors in visual measurement in high-speed, dynamically changing scenarios.

[0008] Preferably, in the AI ​​algorithm optimization step, the AI ​​deep learning model automatically processes noise and data mismatch issues during the measurement process and optimizes the measurement data through spatiotemporal consistency constraints.

[0009] Preferably, in the spatiotemporal consistency calculation step, the large-scale deformation measurement model constructed based on image sequences captured by multiple cameras and displacement information provided by the IMU can comprehensively process multi-view data in a dynamic measurement environment and accurately estimate structural deformation and displacement.

[0010] Preferably, in the deformation compensation step, the measurement data is deformed by combining visual data and IMU motion information to output high-precision deformation data.

[0011] Preferably, it includes multiple visual measurement cameras, an IMU device, and a data processing module; the multiple visual measurement cameras are used to acquire image data from multiple different perspectives; the IMU device is used to record the pose and motion information of the measurement platform; the data processing module is used to perform multi-camera data synchronization, IMU data fusion, AI algorithm optimization, spatiotemporal consistency calculation, and deformation compensation steps.

[0012] In summary, the technical effects and advantages of this invention are as follows: High-precision measurement: Through multi-camera spatiotemporal consistency analysis and AI algorithm optimization, it can accurately capture the deformation and displacement of the structure, effectively solving the problem of low accuracy of traditional measurement methods in large-scale measurements, and providing high-precision measurement results.

[0013] Reduced hardware requirements: By combining multiple vision sensors and IMUs, the problem of needing a large number of control points to be deployed, as required by traditional measurement methods, is avoided, thus reducing the number and cost of hardware devices used.

[0014] Enhance system robustness: Introducing AI algorithms to fuse IMU and visual data can effectively eliminate measurement errors caused by factors such as platform movement and environmental interference, enabling the system to maintain high efficiency and stability even under complex working conditions and dynamic environments.

[0015] Wide range of applications: It can be applied to multiple fields such as civil engineering, aerospace, manufacturing, and intelligent transportation, and has broad market value and application prospects. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the process for large-scale deformation measurement based on multi-camera spatiotemporal consistency under complex conditions involving the fusion of AI, vision, and IMU in this application embodiment. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," "top," and "bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] It should also be noted that all standard parts used in this application are commercially available, and can be custom-made according to the description and drawings. Unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances, and unless explicitly limited, machinery, parts, and equipment can all adopt conventional models in the prior art.

[0021] In this document, the term "comprising" is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] The present invention proposes a method and apparatus for measuring large-scale deformation under complex working conditions based on multi-camera spatiotemporal consistency through the fusion of AI, vision, and IMU. The specific technical solution is as follows: Multi-camera data synchronization: Multiple visual measurement cameras work together using spatiotemporal synchronization technology, allowing the camera array to cover the entire monitoring area and acquire image data from multiple different perspectives. Multiple cameras simultaneously capture image data and synchronize via internal timestamps, ensuring data consistency in time and space, thereby comprehensively capturing information about the monitoring area.

[0023] IMU data fusion: IMU devices are used to record the pose and motion information of the measurement platform. The IMU can monitor and record motion information such as acceleration and angular velocity of the measurement platform in real time, providing motion data compensation for visual measurements. Especially in high-speed, dynamically changing scenarios, it resolves errors caused by motion in visual measurements and improves the accuracy of displacement tracking.

[0024] AI algorithm optimization: A deep learning AI model is used to fuse multi-dimensional data from visual cameras and IMUs. Measurement data is optimized through spatiotemporal consistency constraints, automatically handling noise and data mismatch issues during the measurement process, reducing image errors, further improving measurement accuracy, and enhancing system robustness.

[0025] Spatiotemporal consistency calculation: By analyzing image sequences captured by multiple cameras and displacement information provided by IMU, a large-scale deformation measurement model is constructed using spatiotemporal consistency analysis technology. In a dynamic measurement environment, this model accurately estimates the deformation and displacement of the structure through comprehensive processing of multi-view data.

[0026] Deformation compensation: Deformation compensation processing is performed on the measurement data. By combining visual data and IMU motion information, large-scale, high-precision structural deformation compensation is achieved, and the final high-precision deformation data is output.

[0027] Working principle Data acquisition phase: Multiple visual measurement cameras simultaneously capture image data and synchronize them using internal timestamps to obtain image information from multiple different perspectives of the monitored area. Simultaneously, the IMU device monitors and records motion information such as acceleration and angular velocity of the measurement platform in real time, providing data support for subsequent motion compensation.

[0028] Data preprocessing stage: Distortion correction and image registration are performed on the image data to eliminate inherent image distortion and positional deviations. IMU data is filtered to remove high-frequency noise and extract effective displacement information, preparing for subsequent data fusion and analysis.

[0029] Data fusion and optimization phase: AI deep learning models are used to fuse multi-camera image data and IMU data. Spatiotemporal consistency constraints are applied to optimize the fused data, reducing image errors and data mismatch issues, and improving data accuracy and reliability.

[0030] Deformation Measurement and Compensation Stage: Based on the spatiotemporal consistency analysis results, a large-scale deformation measurement model is constructed to accurately estimate the deformation and displacement of the structure. The measured structural deformation results are compared with IMU data to further compensate for errors, ultimately outputting high-precision deformation data.

[0031] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for measuring large-scale deformation under complex working conditions based on multi-camera spatiotemporal consistency, characterized by: AI + vision + IMU fusion. Includes the following steps: Multi-camera data synchronization: Multiple visual measurement cameras work together through spatiotemporal synchronization technology, with the camera array covering the entire monitoring area to acquire image data from multiple different perspectives; IMU data fusion: Using IMU devices to record the pose and motion information of the measurement platform, providing motion data compensation for visual measurement, and solving the error caused by motion in visual measurement; AI algorithm optimization: The AI ​​deep learning model is used to fuse multi-dimensional data from visual cameras and IMUs, and the measurement data is optimized through spatiotemporal consistency constraints to reduce image errors and improve measurement accuracy; Spatiotemporal consistency calculation: Analyze image sequences captured by multiple cameras and displacement information provided by IMU, and use spatiotemporal consistency analysis technology to construct a large-scale deformation measurement model to accurately estimate structural deformation and displacement; Deformation compensation: The measurement data is processed for deformation compensation. By combining visual data and IMU motion information, large-scale and high-precision structural deformation compensation is achieved, and the final high-precision deformation data is output.

2. The method for measuring large-scale deformation under complex conditions based on multi-camera spatiotemporal consistency in AI + vision + IMU fusion as described in claim 1, characterized in that, In the multi-camera data synchronization step, multiple cameras synchronously capture image data through internal timestamps to ensure data consistency in time and space.

3. The method for measuring large-scale deformation under complex conditions based on multi-camera spatiotemporal consistency in AI + vision + IMU fusion as described in claim 1, characterized in that, In the IMU data fusion step, the IMU device monitors and records motion information such as acceleration and angular velocity of the measurement platform in real time to compensate for displacement errors in visual measurement in high-speed, dynamically changing scenarios.

4. The method for measuring large-scale deformation under complex conditions based on multi-camera spatiotemporal consistency in AI + vision + IMU fusion as described in claim 1, characterized in that, In the AI ​​algorithm optimization step, the AI ​​deep learning model automatically handles noise and data mismatch issues in the measurement process and optimizes the measurement data through spatiotemporal consistency constraints.

5. The method for measuring large-scale deformation under complex conditions based on multi-camera spatiotemporal consistency in AI + vision + IMU fusion as described in claim 1, characterized in that, In the spatiotemporal consistency calculation step, the large-scale deformation measurement model constructed based on image sequences captured by multiple cameras and displacement information provided by the IMU can comprehensively process multi-view data in a dynamic measurement environment and accurately estimate structural deformation and displacement.

6. The method for measuring large-scale deformation under complex conditions based on multi-camera spatiotemporal consistency in AI + vision + IMU fusion as described in claim 1, characterized in that, In the deformation compensation step, the measurement data is deformed by combining visual data and IMU motion information to output high-precision deformation data.

7. A large-scale deformation measurement device based on multi-camera spatiotemporal consistency under complex working conditions, implementing the method of any one of claims 1-6, characterized in that: It includes multiple visual measurement cameras, an IMU device, and a data processing module; the multiple visual measurement cameras are used to acquire image data from multiple different perspectives; the IMU device is used to record the pose and motion information of the measurement platform; the data processing module is used to perform multi-camera data synchronization, IMU data fusion, AI algorithm optimization, spatiotemporal consistency calculation, and deformation compensation steps.

Images