Three-dimensional attitude correction and cable rod three-dimensional modeling method for cable climbing robot
By integrating three-dimensional angle sensors and encoders on the cable robot, combined with deep learning and image stitching technology, the image sequence mismatch caused by pose changes in the cable robot when traveling on the complex cable structure is solved, achieving high accuracy The three-dimensional modeling of cable rods and data consistency improves detection efficiency and data processing efficiency.
Patent Information
- Application Number
- CN202411085769.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-06
AI Technical Summary
When the cable robot travels on a complex cable structure, the image sequence mismatch occurs due to posture changes, which affects the three-dimensional reconstruction accuracy and data consistency of the cable rod.
Three-dimensional angle sensors, encoders and deep learning technology are used, combined with image stitching methods, three-dimensional attitude correction and three-dimensional cable rod modeling are realized to ensure the consistency and continuity of the image sequence and generate high-precision three-dimensional cable rod model.
The accuracy and integrity of three-dimensional modeling of cable rods are improved, the reliability and detection efficiency of image analysis are ensured, and resource waste caused by attitude deviation is reduced.
Smart Images

Figure CN119091039A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automated detection and modeling, and in particular to a method for three-dimensional posture correction of a cable-climbing robot and three-dimensional modeling of a cable rod. Background Art
[0002] As an important tool for health monitoring of large structures such as bridges and cables, the cable-climbing robot often moves in a non-linear manner when performing inspection tasks due to the different shapes or irregular structures of the cables, causing the robot itself to rotate, affecting the consistency and continuity of the data collected by the onboard camera, and further affecting the 3D reconstruction accuracy of the cable-climbing entity. The existing technology has limitations in dealing with this type of image sequence mismatch problem caused by the robot's posture change. Therefore, it is necessary to design a method for 3D posture correction of the cable-climbing robot and 3D modeling of the cable. Summary of the invention
[0003] The purpose of the present invention is to provide a three-dimensional posture correction and cable rod three-dimensional modeling method for a cable-climbing robot, aiming at the three-dimensional posture control of the cable-climbing robot when it is moving on a complex cable rod structure and the three-dimensional modeling of the cable rod entity, aiming to improve the accuracy of the detection data and the effectiveness of the subsequent analysis.
[0004] By integrating three-dimensional angle sensors, encoders and adopting deep learning and image stitching technology, precise positioning in complex cable-rod environments and effective stitching of continuous image sequences can be achieved, thereby constructing a high-precision three-dimensional model of the cable-rod.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0006] A method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot, the method comprising the following steps:
[0007] Step 1: Read the processed video frames and sensor data at each moment;
[0008] Step 2: Combine the data from step 1 with the distance coordinates and the rotation angle; (1) After the rope-climbing robot is installed, start the power supply. At this time, the system reads the data of the current encoder and converts it into the distance coordinates and the initial value of the rotation angle of the three-dimensional angle sensor as the initial value of this crawling; (2) After starting to crawl, detect the changes in the encoder and the three-dimensional angle sensor, and record the time when the changes occur;
[0009] Step 3: Load the pre-trained deep learning-based 3D reconstruction model; (1) The initial value of step 1, the change value recorded in step 2, and the video image frames of the four cameras at the current moment are input into the trained 3D imaging model for 3D reconstruction; (2) The reconstruction logic is:
[0010] a. Using the processed video image data, stretch and splice them into an expanded image, that is, the images of the four cameras at the corresponding time are combined and spliced into an expanded large image;
[0011] b. Roll up the combined and spliced large image according to the trained empirical ratio parameters, and then output the cable-rod model at that moment;
[0012] c. Rotate the cable rod model accordingly based on the input data to ensure that the image data of each camera is as if the vision is always walking in a straight line.
[0013] Step 4: Generate a 3D solid model of the cable rod. By splicing the cable rod models at different times, a real-time dynamically changing cable rod model can be obtained.
[0014] Furthermore, in step 1, a three-dimensional angle sensor, a gyroscope and an accelerometer are installed on the cable-climbing robot to monitor and record the changes in the robot's rotation angle on the cable rod in real time, providing basic data for subsequent posture correction. The three-dimensional angle sensor needs to be installed in a strong and stable component to prevent interference data caused by shaking caused by the climbing robot.
[0015] Furthermore, in step 2, an encoder is installed on the driving wheel of the robot to accurately measure the linear displacement of each crawling step and fuse it with the three-dimensional angle sensor data to achieve dual tracking of the robot's absolute position and posture in space. The encoder is installed in one of the driven wheels, or two or more encoders can be installed to take the average value to improve the accuracy of the encoder data.
[0016] Furthermore, in step 3, a deep learning algorithm is used to analyze the sensor data and encoder data to predict and correct the changes in camera viewing angle caused by the rotation of the robot. At the same time, image stitching technology is applied to accurately stitch the continuously acquired image sequences in a unified coordinate system to eliminate image misalignment caused by posture changes and ensure view continuity.
[0017] Furthermore, in step 4, based on the corrected continuous image sequence, a 3D reconstruction algorithm is used to generate a high-precision cable-rod solid model, which contains structural details and potential defect information, providing intuitive and accurate data support for structural health assessment.
[0018] The present invention has the following beneficial effects due to the adoption of the above technical solution:
[0019] The present invention improves the accuracy of three-dimensional modeling of cables and rods under non-straight travel conditions, ensures the integrity and accuracy of the model, ensures the consistency of the acquired image sequence through real-time correction of the robot's posture, provides a reliable basis for subsequent image analysis and structural evaluation, improves detection efficiency and data processing efficiency, and reduces duplication of work and waste of resources caused by posture deviation. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is a flow chart of the method of the present invention;
[0021] Figure 2 It is a video frame processing flow chart at a certain moment of the present invention;
[0022] Figure 3 It is a schematic diagram of the device structure for implementing the method of the present invention.
[0023] In the attached drawings, 1-external frame, 2-cable rod, 3-automatic telescopic wheel supporting device, 4-crawling wheel, 5-camera. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are only for the purpose of enabling the reader to have a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be implemented even without these specific details.
[0025] like Figure 1-2 As shown, a method for three-dimensional posture correction of a cable-climbing robot and three-dimensional modeling of a cable rod comprises the following steps:
[0026] Step 1: Read the processed video frames and sensor data at each moment. Figure 2 As shown, the video frame at a certain moment is read, preprocessed and combined with a predetermined matching strategy for image matching, a transformation model is established and a unified reference coordinate system is established, fusion and reconstruction are performed, the current encoder and three-dimensional angle sensor values are read, converted into a distance coordinate and rotation angle, the current image and sensor data are saved, and the video frame processing at a certain moment is completed.
[0027] Step 2: Combine the data from step 1 with the distance coordinates and rotation angle. Install a three-dimensional angle sensor, a gyroscope and an accelerometer on the cable-climbing robot to monitor and record the changes in the robot's rotation angle on the cable rod in real time, providing basic data for subsequent posture correction. The three-dimensional angle sensor needs to be installed in a strong and stable component to prevent the shaking caused by the crawling of the cable-climbing robot from interfering with the data. An encoder is installed on the robot's driving wheel to accurately measure the linear displacement of each step of crawling, and integrate it with the three-dimensional angle sensor data to achieve dual tracking of the robot's absolute position and posture in space. The encoder is installed in one of the driven wheels, or two or more encoders can be installed to take the average value to improve the accuracy of the encoder data. After the cable-climbing robot is installed, start the power supply. At this time, the system reads the current encoder data and converts it into the distance coordinates and the initial value of the rotation angle of the three-dimensional angle sensor as the initial value of this crawling. After crawling begins, detect the changes in the encoder and the three-dimensional angle sensor, and record the time of the change.
[0028] Step 3: Load the pre-trained deep learning-based 3D reconstruction model. Use deep learning algorithms to analyze sensor data and encoder data, predict and correct changes in camera perspective caused by robot rotation, and apply image stitching technology to accurately stitch continuously acquired image sequences in a unified coordinate system to eliminate image misalignment caused by posture changes and ensure view continuity. The initial value of step 1, the change value recorded in step 2, and the video image frames of the four cameras at the current moment are input into the trained 3D imaging model for 3D reconstruction. The reconstruction process is:
[0029] a. Using the processed video image data, stretch and splice them into an expanded image, that is, the images of the four cameras at the corresponding time are combined and spliced into an expanded large image;
[0030] b. Roll up the combined and spliced large image according to the trained empirical ratio parameters, and then output the cable-rod model at that moment;
[0031] c. Rotate the cable rod model accordingly based on the input data to ensure that the image data of each camera is as if the vision is always walking in a straight line.
[0032] Step 4: Generate a 3D solid model of the cable rod. Based on the corrected continuous image sequence, a 3D reconstruction algorithm is used to generate a high-precision cable rod solid model. The model contains structural details and potential defect information, providing intuitive and accurate data support for structural health assessment. By splicing the cable rod models at different times, a real-time dynamically changing cable rod model can be obtained.
[0033] The three-dimensional posture control of the cable-climbing robot when it moves on a complex cable-rod structure and the three-dimensional modeling of the cable-rod entity are studied to improve the accuracy of the detection data and the effectiveness of the subsequent analysis.
[0034] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A three-dimensional posture correction method for a cable-climbing robot and a three-dimensional modeling method for a cable rod, characterized in that: The method comprises the following steps: Step 1: Read the processed video frames and sensor data at each moment; Step 2: Combine the data from step 1 with the distance coordinates and rotation angle; Step 3: Load the pre-trained deep learning-based 3D reconstruction model; Step 4: Generate a 3D solid model of the cable rod.
2. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 1, the video frame at a certain moment is read, preprocessed and image matching is performed in combination with a predetermined matching strategy, a transformation model is established and a unified reference coordinate system is established, fusion and reconstruction are performed, the current encoder and three-dimensional angle sensor values are read, converted into a distance coordinate and rotation angle, the current image and sensor data are saved, and the video frame processing at a certain moment is completed.
3. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 1, a three-dimensional angle sensor, a gyroscope and an accelerometer are installed on the cable-climbing robot to monitor and record the changes in the robot's rotation angle on the cable rod in real time, providing basic data for subsequent posture correction. The three-dimensional angle sensor needs to be installed in a strong and stable component to prevent the shaking caused by the climbing robot from causing interference to the data.
4. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 2, an encoder is installed on the robot's driving wheel to accurately measure the linear displacement of each crawling step and fuse it with the three-dimensional angle sensor data to achieve dual tracking of the robot's absolute position and posture in space. The encoder is installed in one of the driven wheels, or two or more encoders can be installed to take the average value to improve the accuracy of the encoder data.
5. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 3, deep learning algorithms are used to analyze sensor data and encoder data to predict and correct changes in camera viewing angles caused by robot rotation. Image stitching technology is also used to accurately stitch continuously acquired image sequences in a unified coordinate system to eliminate image misalignment caused by posture changes and ensure view continuity.
6. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 4, based on the corrected continuous image sequence, a 3D reconstruction algorithm is used to generate a high-precision cable-rod solid model. The model contains structural details and potential defect information, providing intuitive and accurate data support for structural health assessment.
7. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 2, after the rope-climbing robot is installed, the power is turned on. At this time, the system reads the data of the current encoder and converts it into distance coordinates and the initial value of the rotation angle of the three-dimensional angle sensor as the initial value of this crawling. After starting crawling, the changes in the encoder and the three-dimensional angle sensor are detected, and the time of the change is recorded.
8. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 3, the initial value of step 1, the change value recorded in step 2, and the video image frames of the four cameras at the current moment are input into the trained 3D imaging model for 3D reconstruction. The reconstruction process is: a. Using the processed video image data, stretch and splice them into an expanded image, that is, the images of the four cameras at the corresponding time are combined and spliced into an expanded large image; b. Roll up the combined and spliced large image according to the trained empirical ratio parameters, and then output the cable-rod model at that moment; c. Rotate the cable rod model accordingly based on the input data to ensure that the image data of each camera is as if it is always walking in a straight line.
9. The method for three-dimensional posture correction and cable rod three-dimensional modeling of a cable-climbing robot according to claim 1, characterized in that: In step 4, the cable-strut models at different times are spliced together to obtain a real-time dynamically changing cable-strut model.
