Automatic three-dimensional reconstruction experimental device based on multi-sensor fusion and motion control algorithm

Abstract

The invention discloses an automatic three-dimensional reconstruction experimental device based on multi-sensor fusion and a motion control algorithm, and mainly relates to the technical field of three-dimensional reconstruction, a mechanical part comprises a base, a gear motor, a controller, an encoder, a direct current power supply, an IMU (inertial measurement unit), a motor driver, a storage platform, a support arm and a mounting platform; the gear motor provides power for the whole device, the direct-current power source provides power for the whole device, and the controller is used for controlling movement of the device and communicates with the upper computer. And the encoder and the IMU are used for feeding back motion data of the device to the controller in real time so as to form closed-loop control of the device. According to the motion control algorithm part, a controller communicates with an upper computer through a serial port, the posture of the device is adjusted in real time according to an upper computer reconstruction algorithm, and combined control of the upper computer and a lower computer is formed through real-time detection of an encoder and an IMU. The method is suitable for algorithm research and verification evaluation of three-dimensional reconstruction and multi-sensor fusion.

Description

Technical Field

[0001] The patent of this invention relates to the field of three-dimensional reconstruction technology, specifically a set of three-dimensional reconstruction automation experimental equipment based on multi-sensor fusion and a joint motion control algorithm of the upper and lower computers. Background Art

[0002] With the rapid development of computer technology and automation devices, there is a great demand for environmental perception and scene reconstruction in technical fields such as driverless cars, intelligent robots, and automated industrial equipment. How to obtain the perception data of sensors such as radar, cameras, IMU, and GPS to efficiently, accurately, and quickly establish scene information around the equipment and even semantic information of objects is of great significance in the positioning, perception, and measurement of robots and driverless cars in the actual operating environment. The accuracy of the algorithm determines the safety and reliability of the corresponding products or equipment in actual use. Therefore, major technology companies and universities have increased their research investment in related fields accordingly.

[0003] During the research process, it is necessary to collect a large amount of experimental data through multiple sensors at calibrated relative positions to simulate and verify the feasibility and robustness of the corresponding algorithm. At present, in the research process of three-dimensional scene reconstruction, the main methods of collecting data to simulate actual working conditions are to manually fix multiple sensors at relative positions and conduct experiments by hand, or to collect data by controlling the operation of an electric car. However, there is no highly automated, accurate, efficient, fast and reliable dedicated experimental equipment and a lower-level device control method for the joint reconstruction algorithm to simulate actual working conditions for data collection and verification. The previous two methods cannot accurately quantify the moving distance and moving speed of the experimental equipment, so it is impossible to obtain the true value of the motion data of the experimental device, resulting in the inability to reliably and accurately quantify the reconstruction algorithm and subsequent cross-validation and evaluation.

[0004] Based on the existing research environment and research needs, the patent of this invention proposes an experimental device for automated three-dimensional reconstruction based on multi-sensor fusion and a joint control algorithm for the upper and lower computers. It is used to solve the needs of automated reconstruction in the process of three-dimensional scene reconstruction, as well as the problems of quantification and algorithm evaluation in the process of algorithm verification. It can realize the fixation of the relative positions of multiple sensors, and accurately control and quantify the moving distance, speed, and height of the equipment through encoders and IMUs; in the process of algorithm research, the accuracy of the algorithm itself is verified. Therefore, the information calculated by the algorithm itself must not be used to verify the algorithm during the verification process. Instead, other means are needed to obtain the true value information of the scene to be reconstructed and the accurate motion data during the experimental process of the experimental device to cross-validate the algorithm to ensure the objectivity of the algorithm evaluation. While the patent of this invention cooperates with the reconstruction algorithm to realize the automation of three-dimensional reconstruction, it can also help researchers to reliably cross-validate the reconstruction algorithm and evaluate the reliability and robustness of the algorithm in the future. This invention patent is suitable for most three-dimensional scene reconstruction, object reconstruction, and sensor fusion algorithm research and verification. Summary of the invention

[0005] This patent proposes an automated 3D reconstruction experimental device and joint control method based on multi-sensor fusion, which aims to solve the problem that the data of the moving distance, moving speed and moving height of the equipment cannot be accurately quantified and controlled during the reconstruction algorithm research process. This patent can accurately quantify and control the distance, speed and height during the reconstruction algorithm research process when the quantization accuracy is known, so as to cooperate with the reconstruction algorithm for automatic reconstruction without manual intervention. During the reconstruction process, accurate motion data can be obtained when the experimental device is running, which meets the purpose of algorithm verification and evaluation.

[0006] The specific contents of the patent of this invention are as follows:

[0007] The automated 3D reconstruction experimental device and joint control method based on multi-sensor fusion, the mechanical structure part includes a fixed base, a fixed flange, a synchronous belt, a planetary reduction DC brushless motor, an absolute encoder, a 220V to 24V DC power supply, bearings, gears, a storage platform, a universal wheel, a horizontal arm, a vertical arm, a locking screw, a sleeve, a mounting platform, an experimental module, an IMU, and a motor driver. The gear of the horizontal arm is connected to the central axis fixed to the base through a flange through a bearing, and the storage platform is connected to the central axis through a flange to keep it fixed. The encoder and the motor are connected to the gear of the horizontal arm through a synchronous belt, wherein the encoder is connected through a 1:1 gear ratio, the motor is connected through a 1:3 gear ratio, the motor is connected to the motor driver, and the controller controls the rotation of the motor by controlling the driver. The power supply of the whole device is provided by a 220V to 24V DC power supply under the base. The horizontal arm is equipped with a universal wheel with adjustable height to keep the arm horizontal and move at any angle, and the vertical arm is connected to the horizontal arm through a locking screw. The horizontal arm is connected to realize the adjustment of the vertical arm at any horizontal distance of the horizontal arm. The sleeve is connected to the vertical arm through a locking screw to realize the adjustment of the mounting platform at any height of the vertical arm. The mounting platform and the sleeve are connected through a rotatable locking structure to realize the adjustment of any pitch angle of the mounting platform. The IMU is installed on the mounting platform to synchronously obtain the motion state of the experimental device on the mounting platform, and together with the encoder, it provides the controller with high-precision motion feedback of the entire device to achieve accurate and fast motion control. The mounting platform can be replaced according to different experimental requirements to enhance the adaptability and practicality of the device. The controller of the device in this patent is connected to the host computer part of the algorithm through serial communication to realize the joint control of the host computer and the lower computer, as well as the high automation of the experiment.

[0008] Another part of this patent is the joint control algorithm of the system, which specifically includes the following steps:

[0009] In order to complete the automated acquisition and experimentation in the reconstruction algorithm research process, the joint control algorithm of this patent is divided into two parts, namely the reconstruction algorithm part of the host computer and the motion control part of the lower computer. The host computer is the computer used in our research process, and the lower computer is the controller described in this patent. The host computer and the lower computer complete the connection of the two algorithms through serial port communication. The specific implementation steps are as follows:

[0010] S1: Initialize the upper computer equipment and algorithm parameters, initialize the lower computer system, and obtain the initial position and posture of the device.

[0011] S2: The upper computer controls the sensor to start collecting data, and the lower computer keeps the position unchanged.

[0012] S3: If data collection is not completed, the collection state is maintained. If data collection is completed, the upper computer stops collecting and sends a control instruction containing motion requirements to the lower computer. The lower computer receives the instruction from the upper computer, and the controller sends the corresponding motion control signal to the motor driver to control the motor according to the control instruction of the upper computer. Otherwise, the device continues to maintain the limit state.

[0013] S4: The encoder and IMU collect the position status and motion information of the device in real time and feed it back to the controller of the device. The controller comprehensively determines whether the device has moved to the predetermined position based on the information fed back by the sensor.

[0014] S5: If the controller of the lower computer determines that the device has reached the predetermined position, it sends a limit signal to the motor driver, and the device stops moving and remains stationary. At the same time, the controller sends a task completion signal to the upper computer, so that the algorithm part of the upper computer continues to run.

[0015] S6: The upper computer receives the signal from the lower computer that it has reached the predetermined position, and continues to collect the data for the next frame. Continue to run from S2.

[0016] S7: After data collection is completed, the algorithm part performs reconstruction work, calculates the posture transformation relationship between each frame data, and completes the reconstruction process.

[0017] The patent of this invention has the following advantages:

[0018] 1. Through the design of dual motion sensors, namely, absolute encoder and 16-bit high-precision IMU, the motion state and motion information of the device can be collected and calculated in real time, which can greatly reduce the error caused by the operation of a single sensor, and realize the precise motion control of the whole device, and accurately quantify the motion information of the device during the movement process, accurately control the motion posture of the device, and provide a reliable data source for the verification and evaluation of the reconstruction algorithm. It can achieve the acquisition of data true value without relying on the reconstruction algorithm. Improve the authenticity, reliability and objectivity of the algorithm evaluation.

[0019] 2. Through the above-mentioned joint motion control algorithm, the reconstruction algorithm experimental process can be well automated, reducing human-caused errors, and accurately quantifying the experimental process, which is convenient for algorithm evaluation and cross-validation of algorithm accuracy.

[0020] 3. The power part of the patent of this invention adopts a planetary reduction brushless DC motor, which has a long life and large torque. It can undertake most of the experimental tasks based on multi-sensor 3D reconstruction. It is equipped with a dedicated motor driver with optical coupling isolation, which responds quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the overall mechanical structure of the patent of this invention;

[0022] Figure 2 A front view of the overall mechanical structure of the patented invention;

[0023] Figure 3 The left view of the overall mechanical structure of the patent of this invention;

[0024] Figure 4 Annotated diagram of the overall mechanical structure of the patented invention;

[0025] Figure 5 Annotated side view of the overall mechanical structure of the patented invention;

[0026] Figure 6 Annotated bottom view of the overall mechanical structure of the patented invention;

[0027] Figure 7 Flow chart of the patented control algorithm of the present invention;

[0028] The components include (1) a storage platform, (2) an encoder, (3) a motor, (4) a universal wheel, (5) a controller, (6) a synchronous belt, (7) a base, (8) a horizontal arm, (9) an arm assembly, (10) an IMU, (11) a vertical arm, (12) an arm sleeve, (13) an experimental device, (14) a mounting platform, (15) a connecting flange, (16) a synchronous gear, (17) a locking screw, (18) a lockable rotating structure, (19) a 220 V to 24 V DC power supply, and (20) a motor driver. DETAILED DESCRIPTION

[0029] The specific structure and implementation methods of the present invention are further described below in conjunction with the accompanying drawings.

[0030] An automatic three-dimensional reconstruction device and control method based on multi-sensor fusion, the mechanical structure of which is characterized by: Figure 4 As shown, the disc (1) is connected to the central axis through a connecting flange, the central axis is connected to the base (7) through a flange (15) and is fixed, the central axis is connected to the synchronous gear (16) through two bearings, the planetary reduction brushless DC motor (3) is connected to the gear (16) through a synchronous belt with a gear ratio of 1:3, providing rotational power for the entire device, and the absolute encoder (2) with an accuracy of 0.09 degrees is connected to the gear (16) through a synchronous belt with a gear ratio of 1:1, obtaining the rotation angle and rotation speed of the device, providing real-time high-precision motion state feedback for the control system, so as to help the system achieve a closed control loop. The middle gear (16) is connected to the support arm (8), the support arm (8) is connected to the universal wheel (4) through a flange, the support arm (8) can be equipped with a suitable length according to needs, the support arm (11) is connected to the support arm (8) through a screw (17), the support arm (11) can adjust its position on the support arm (8) by tightening (17), and the sleeve (12) is connected to the support arm (8) through a screw (17). Figure 5 The lockable rotating structure (18) is connected to the sensor mounting platform (14), so that the installed sensor can be adjusted to any pitch angle. The sensor (13) is fixed to the platform (14) through the mounting hole. Because the mounting platform (14) is designed to be detachable, different mounting platforms can be made according to the actual sensors used, which greatly increases the universality and practicality of the device. The sliding sleeve (12) can adjust the sensor mounting platform (14) to any height on the support arm (11). The 16-bit high-precision IMU sensor (10) is installed on the side of the platform (14). It can collect and obtain the instantaneous speed, acceleration, pitch angle, and relative height change data of the sensor during movement in real time and transmit it to the controller (5) through serial communication. Together with the encoder (2), it provides high-precision motion data feedback for the control system. The design of the dual motion sensor greatly enhances the control accuracy of the system and the robustness of the control process. The controller (5) is connected to the motor drive (20) under the base (7), such as Figure 6 The rotation of the motor is controlled by a motor driver (20) with an optical coupling isolation module, and a 220V to 24V direct current power supply (19) is used to power the entire device.

[0031] like Figure 7As shown, this patent also proposes a joint control method for automated three-dimensional reconstruction based on multi-sensor fusion, which specifically includes the following steps: S1: Initialize the upper computer equipment and algorithm parameters, initialize the lower computer system, and obtain the initial posture of the device. S2: The upper computer controls the sensor to start collecting data, and the lower computer keeps the posture unchanged. S3: If the data collection is not completed, the collection state is maintained. If the data collection is completed, the upper computer stops collecting and sends a control instruction containing motion requirements to the lower computer. The lower computer receives the instruction from the upper computer, and the controller sends the corresponding motion control signal to the motor driver to control the motor according to the control instruction of the upper computer. Otherwise, the limit state of the device continues to be maintained. S4: The encoder and IMU collect the position state and motion information of the device in real time, and feed it back to the controller of the device. The controller comprehensively judges whether the device has moved to the predetermined position based on the information fed back by the sensor. S5: If the controller of the lower computer determines that the device has reached the predetermined position, it sends a limit signal to the motor driver, the device stops moving and remains stationary, and the controller sends a task completion signal to the upper computer. The algorithm part of the upper computer continues to run. S6: The upper computer receives the signal from the lower computer that it has reached the predetermined position and continues to collect data for the next frame. Continue to run from S2. S7: After the data collection is completed, the algorithm part performs reconstruction work, calculates the posture transformation relationship between each frame of data, and completes the reconstruction process. In addition, in the subsequent algorithm evaluation, it is not necessary to rely on the reconstruction algorithm to obtain the true value of the data. The setting of the dual motion sensor in this patent can obtain high-precision motion quantification information, including motion angle, motion speed, relative height change during motion and other data, to improve the authenticity and reliability of the algorithm evaluation.

[0032] Finally, the overall mechanical structure of this embodiment is as follows Figure 1 , 2 , 3, the flow chart of the control algorithm is as follows Figure 7 shown.

[0033] The above specific embodiments further illustrate the technical problems, technical solutions and beneficial effects solved by the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automated 3D reconstruction experimental device and motion control method based on multi-sensor fusion, Features: The storage platform (1) is connected to the central axis via a connecting flange, the central axis is connected to the base (7) via a flange (15) and is fixed, the central axis is connected to the synchronous gear (16) via two bearings, the reduction motor (3) is connected to the gear (16) via a synchronous belt with a gear ratio of 1:3, and provides power for the entire device, the encoder (2) is connected to the gear (16) via a synchronous belt with a gear ratio of 1:1, and obtains the rotation angle and rotation speed of the device in real time, providing real-time high-precision motion state feedback for the control system, so as to help the system achieve a control closed loop. The gear (16) in the middle is connected to the transverse support arm (8), and the transverse support arm (8) is connected to the universal wheel (4) through a flange. The transverse support arm (8) can be equipped with a suitable length according to needs. The vertical support arm (11) is connected to the transverse support arm (8) through a tension screw (17). The vertical support arm (11) can adjust its position on the transverse support arm (8) through the tension screw (17). The sliding sleeve (12) is connected to the sensor mounting platform (14) through a lockable rotating structure (18) as shown in Figure 5, so that the installed sensor can be adjusted to any pitch angle. The experimental device (13) is fixed to the platform (14) through the mounting hole. Because the mounting platform (14) is designed to be detachable, different mounting platforms can be made according to the actual sensors used, which greatly enhances the universality and practicality of the device. The sliding sleeve (12) can adjust the vertical height of the sensor mounting platform (14) on the vertical support arm (11) to any desired height. The 16-bit high-precision IMU sensor (10) is installed on the side of the platform (14), which can collect and obtain the instantaneous speed, acceleration, pitch angle, and relative height change data of the sensor when it moves in real time and transmit it to the controller (5) through serial communication, and together with the encoder (2), provide high-precision motion data feedback for the control system. The design of the dual motion sensor greatly enhances the control accuracy and robustness of the system. The controller (5) is connected to the motor drive (20) under the base (7), and the motor drive (20) with an optical coupling isolation module is used to control the rotation of the motor. The 220V to 24V DC power supply (19) is used to power the entire device.

2. According to claim 1, an automated three-dimensional reconstruction experimental device and motion control method based on multi-sensor fusion, Features: The motor (3) drives the horizontal support arm to rotate through a synchronous belt, and drives the encoder to rotate at the same time to obtain angle change information, wherein the storage platform (1), the central axis and the base (7) remain stationary, the vertical support arm (11) can adjust the horizontal distance from the storage platform by the tightness of the locking screw (17), the installation platform (14) can arbitrarily adjust the pitch angle of the experimental device through a lockable rotating structure (18), and can adjust the vertical distance of the experimental device through a sliding sleeve (12).

3. An automated 3D reconstruction experimental device and motion control method based on multi-sensor fusion according to claim 2, It is characterized in that The method includes the following steps: first, the upper computer equipment and algorithm parameters are initialized, the lower computer system is initialized, and the initial posture of the device is obtained. Then, the upper computer controls the sensor to start collecting data, and the lower computer keeps the posture unchanged. If the data collection is not completed, the collection state is maintained. After the data collection is completed, the upper computer stops collecting and sends a control instruction containing motion requirements to the lower computer. The lower computer receives the instruction from the upper computer, and the controller sends the corresponding motion control signal to the motor driver to control the motor motion state according to the control instruction of the upper computer. Otherwise, the limit state of the device continues to be maintained. Then the encoder and IMU collect the position state and motion information of the device in real time, and feed it back to the controller of the device. The controller comprehensively judges whether the device has moved to the predetermined position based on the information fed back by the sensor. Then, if the controller of the lower computer determines that the device has reached the predetermined position, it sends a limit signal to the motor driver, the device stops moving and remains stationary, and the controller sends a task completion signal to the upper computer. The algorithm part of the upper computer continues to run. If the upper computer receives a signal from the lower computer that it has reached the predetermined position, it continues to collect the data of the next frame. Repeat the previous steps in a cycle. Finally, after data collection is completed, the algorithm part performs reconstruction work, calculates the posture transformation relationship between each frame data, and completes the reconstruction process.

Images