Three-dimensional reconstruction method, device, storage medium and electronic device for moving target

Abstract

The present invention relates to a three-dimensional reconstruction method, device, storage medium and electronic device for a moving target, comprising: controlling a first galvanometer component of a laser 3D reconstruction system to track a moving target so that the moving target is located at the center of a target image; obtaining a first parameter adjustment value of a second galvanometer component according to the galvanometer voltage and a first voltage conversion matrix calibrated by an active tracking subsystem and a dynamic camera subsystem; obtaining a second parameter adjustment value of a third galvanometer component according to the galvanometer voltage and a second voltage conversion matrix calibrated by an active tracking subsystem and a dynamic laser subsystem; adjusting the current galvanometer voltage of the third galvanometer component according to the second parameter adjustment value so that a laser scans the moving target; adjusting the current voltage of the second galvanometer component according to the first parameter adjustment value, triggering a second camera to shoot a moving target with laser stripes, and performing three-dimensional reconstruction of the moving target based on the moving target laser image. The three-dimensional reconstruction of a moving target object can be realized.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional technology, and in particular to a three-dimensional reconstruction method, device, storage medium and electronic equipment for a moving target. Background Art

[0002] Line laser vision measurement systems are widely used in a variety of scenarios, such as railway traffic monitoring, medical imaging, robotics, and industrial production, due to their high adaptability, high precision, and low cost. In related technologies, line laser vision measurement systems usually include a camera, a laser projector, and a mechanical scanning platform. The line laser output by the laser projector projects line laser stripes onto the surface of the target object. The camera captures the image of the target object with laser stripes by shooting, and then processes it through triangulation to obtain the three-dimensional (3D) information of the target object. By moving the laser stripes or the target object through the mechanical scanning platform, the three-dimensional information of different positions of the target object can be obtained, thereby completing the 3D reconstruction of the target object and obtaining a complete three-dimensional model.

[0003] However, when performing 3D reconstruction, the line laser vision measurement system mainly relies on the mechanical scanning platform to drive the line laser or target object output by the laser projector to achieve scanning and reconstruction of the target object. For example, the target object being scanned is moved by the mechanical scanning platform, or the laser projector is driven by the mechanical arm or pan / tilt of the mechanical scanning platform, and the laser stripe beam is driven by the galvanometer system of the laser projector, etc., which can only complete the 3D reconstruction of stationary target objects, and cannot achieve the 3D reconstruction of moving target objects.

[0004] Since the 3D reconstruction of moving target objects is widely used in industrial online detection, robot grasping navigation, motion analysis and other fields, achieving 3D reconstruction of moving target objects is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the present invention provides a method, device, storage medium and electronic device for three-dimensional reconstruction of a moving target.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] According to a first aspect of the present invention, a method for three-dimensional reconstruction of a moving target is provided, the method for three-dimensional reconstruction of a moving target comprising:

[0008] Constructing a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, the laser 3D reconstruction system includes: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser;

[0009] Controlling the first galvanometer assembly to track the moving target, so that the moving target is located at the center of the target image acquired by photographing the moving target by the first camera, and acquiring a galvanometer voltage of the first galvanometer assembly;

[0010] According to the galvanometer voltage of the first galvanometer assembly and a first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, a first parameter adjustment value of the second galvanometer assembly is obtained; according to the galvanometer voltage of the first galvanometer assembly and a second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem, a second parameter adjustment value of the third galvanometer assembly is obtained;

[0011] Adjusting the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly so that the laser stripes output by the laser scan the moving target;

[0012] The current voltage of the second galvanometer assembly is adjusted according to the first parameter adjustment value of the second galvanometer assembly, the second camera is triggered to capture a moving target laser image of the moving target carrying the laser stripes, and three-dimensional reconstruction of the moving target is performed based on the moving target laser image.

[0013] According to a second aspect of the present invention, a three-dimensional reconstruction device for a moving target is provided, the three-dimensional reconstruction device for a moving target comprising:

[0014] A system construction module, used to construct a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, wherein the laser 3D reconstruction system includes: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser;

[0015] a voltage acquisition module, used for controlling the first galvanometer assembly to track the moving target, so that in a target image captured by the first camera when the moving target is photographed, the moving target is located at the center of the target image, and acquiring a galvanometer voltage of the first galvanometer assembly;

[0016] a conversion matrix module, used to obtain a first parameter adjustment value of the second galvanometer component according to the galvanometer voltage of the first galvanometer component and a first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, and to obtain a second parameter adjustment value of the third galvanometer component according to the galvanometer voltage of the first galvanometer component and a second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem;

[0017] A target scanning module, used for adjusting the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly, so that the laser stripes output by the laser scan the moving target;

[0018] A three-dimensional reconstruction module is used to adjust the current voltage of the second galvanometer assembly according to the first parameter adjustment value of the second galvanometer assembly, trigger the second camera to capture a moving target laser image of the moving target carrying the laser stripes, and perform three-dimensional reconstruction of the moving target based on the moving target laser image.

[0019] According to a third aspect of the present invention, there is provided a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for three-dimensional reconstruction of a moving target in any possible implementation of the first aspect.

[0020] According to a fourth aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for three-dimensional reconstruction of a moving target in any possible implementation of the first aspect are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 A schematic diagram of a flow chart of a method for three-dimensional reconstruction of a moving target provided by an embodiment of the present invention;

[0024] Figure 2 Another schematic diagram of a flow chart of a method for three-dimensional reconstruction of a moving object provided by an embodiment of the present invention;

[0025] Figure 3A schematic diagram of the three-dimensional reconstruction and three-dimensional motion compensation results of a ceramic step at different speeds in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of the three-dimensional reconstruction and three-dimensional motion compensation results of a ceramic ball at different speeds in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0027] Figure 5 An error diagram between the actual distance and the fitted distance of a ceramic step in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0028] Figure 6 An error diagram between the actual diameter and the fitted diameter of a ceramic sphere in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of a three-dimensional reconstruction device for a moving target provided by an embodiment of the present invention;

[0030] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In the process of laser scanning 3D reconstruction, the movement of the target will cause the laser stripes to fail to be projected at the pre-calculated expected position, thereby introducing errors in the 3D reconstruction. In this embodiment, visual feedback is used to ensure that the moving target remains relatively still relative to the laser 3D reconstruction system.

[0033] In this embodiment, a line laser-based motion target tracking and active three-dimensional reconstruction method is proposed. By utilizing the collaborative control and mathematical modeling of three pre-built galvanometer systems, the tracking information (coordinate information) of the moving target is fed back to the dynamic three-dimensional reconstruction system, and 3D motion error compensation is completed, thereby achieving high-precision three-dimensional reconstruction of the moving target.

[0034] See also Figure 1 The embodiment of the present invention provides a method for three-dimensional reconstruction of a moving target, which may include the following steps:

[0035] S101, constructing a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, the laser 3D reconstruction system comprising: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem comprises: a first galvanometer component and a first camera, the dynamic camera subsystem comprises: a second galvanometer component and a second camera, and the dynamic laser subsystem comprises: a third galvanometer component and a laser;

[0036] In this embodiment, the laser 3D reconstruction system can realize moving target tracking and 3D reconstruction, and includes three subsystems: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem.

[0037] The active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser.

[0038] In this embodiment, as an optional embodiment, the first camera and the second camera include but are not limited to: complementary metal oxide semiconductor (CMOS) cameras, the laser includes but is not limited to: line lasers, and the galvanometer assembly includes but is not limited to: a dual-axis galvanometer assembly. The dynamic camera subsystem and the dynamic laser subsystem are combined to achieve dynamic three-dimensional reconstruction.

[0039] S102, controlling the first galvanometer assembly to track the moving target, so that the moving target is located at the center of the target image obtained by photographing the moving target by the first camera, and obtaining a galvanometer voltage of the first galvanometer assembly;

[0040] In this embodiment, as an optional embodiment, controlling the first galvanometer assembly to track a moving target so that the moving target is located at the center of a target image obtained by photographing the moving target by the first camera includes:

[0041] When the moving target is tracked for the first time, controlling the first galvanometer assembly to track the moving target;

[0042] triggering the first camera to shoot the moving target to obtain a target image;

[0043] Determine the moving target in the target image according to a preset target detection algorithm;

[0044] The galvanometer voltage of the first galvanometer assembly is adjusted so that in a target image captured by the first camera of the moving target, the moving target is located at the center of the target image.

[0045] In this embodiment, in the captured target image, the corresponding voltage when the moving target is located at the center of the target image is the acquired galvanometer voltage of the first galvanometer assembly.

[0046] In this embodiment, as another optional embodiment, the method further includes:

[0047] Controlling the first galvanometer assembly to continue tracking the moving target;

[0048] triggering the first camera to shoot the moving target to obtain a target image;

[0049] Determine the moving target in the target image according to a preset target detection algorithm;

[0050] Calculating a control voltage of the first galvanometer assembly according to the pixel coordinates of the moving target in the current frame and the pixel coordinates of the previous frame and a pre-calibrated offset coefficient of the first galvanometer assembly;

[0051] The galvanometer voltage of the first galvanometer assembly is adjusted according to the control voltage.

[0052] In this embodiment, the voltage of the first galvanometer assembly after adjusting the galvanometer voltage of the first galvanometer assembly according to the control voltage is the obtained galvanometer voltage of the first galvanometer assembly. As an optional embodiment, the current galvanometer voltage of the first galvanometer assembly is added with the control voltage to obtain the obtained galvanometer voltage of the first galvanometer assembly.

[0053] In this embodiment, as an optional embodiment, the control voltage is calculated using the following formula:

[0054]

[0055] In the formula, Δx pi ,Δy pi are the X-axis offset and Y-axis offset of the pixel coordinates of the center of the moving target in the previous and next frames respectively;

[0056] ΔU 1-pan is the voltage variation of the horizontal first mirror of the first galvanometer assembly in the front and rear frames (horizontal galvanometer control voltage), ΔU 1-tilt The voltage variation of the rotating first mirror of the first galvanometer assembly in the front and rear frames (vertical galvanometer control voltage);

[0057] α is the horizontal offset coefficient of the first galvanometer assembly, corresponding to the horizontal first lens, β is the vertical offset coefficient of the first galvanometer assembly, corresponding to the rotating first lens, and both the horizontal offset coefficient and the vertical offset coefficient are pre-calibrated.

[0058] In this embodiment, the first galvanometer assembly triggers the first camera to capture a target image of a preset moving target, and uses a preset target detection algorithm, such as a continuous frame difference detection method, a background difference detection method, an optical flow detection method, and a motion energy detection method, to identify the moving target in the target image, and calculates the pixel coordinates of the center of the moving target based on the identified moving target.

[0059] In this embodiment, the control voltage of the first galvanometer assembly includes: a horizontal galvanometer control voltage of the horizontal first lens of the first galvanometer assembly and a rotating galvanometer control voltage of the rotating first lens. According to the control voltage, the current galvanometer voltage of the first galvanometer assembly is adjusted to obtain the galvanometer voltage of the first galvanometer assembly.

[0060] In this embodiment, based on the pixel coordinate change (Δpixel) of the center of the moving target between consecutive frames of the target image, the control voltage of the first galvanometer assembly is calculated, and the orientation of the first galvanometer assembly is controlled according to the calculated control voltage to ensure that the moving target remains in the center of the target image, thereby achieving tracking.

[0061] S103, obtaining a first parameter adjustment value of the second galvanometer assembly according to the galvanometer voltage of the first galvanometer assembly and a first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, and obtaining a second parameter adjustment value of the third galvanometer assembly according to the galvanometer voltage of the first galvanometer assembly and a second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem;

[0062] In this embodiment, as an optional embodiment, the parameter adjustment value includes: a galvanometer voltage and a galvanometer rotation angle.

[0063] In this embodiment, the galvanometer voltage of the first galvanometer assembly of the active tracking subsystem and the galvanometer voltage of the second galvanometer assembly of the dynamic camera subsystem satisfy the following formula:

[0064]

[0065] Where U 1-pan is the horizontal galvanometer voltage value of the horizontal first lens in the first galvanometer assembly, U 1-tilt is the voltage value of the rotating galvanometer when the first lens is rotated in the first galvanometer assembly, U 2-pan is the horizontal galvanometer voltage value of the horizontal second lens in the second galvanometer assembly, U 2-tilt is the rotating galvanometer voltage value of rotating the second lens in the second galvanometer assembly, and T1 is the first voltage conversion matrix pre-calibrated between the active tracking subsystem (first galvanometer assembly) and the dynamic camera subsystem (second galvanometer assembly).

[0066]

[0067] In this embodiment, the galvanometer voltage of the first galvanometer assembly of the active tracking subsystem and the galvanometer voltage of the second galvanometer assembly of the dynamic laser subsystem satisfy the following formula:

[0068]

[0069] U 3-pan is the horizontal galvanometer voltage value of the third mirror in the third galvanometer assembly, U 3-tilt is the rotating galvanometer voltage value of rotating the third lens in the third galvanometer assembly, and T2 is a second voltage conversion coefficient matrix pre-calibrated between the first galvanometer assembly and the third galvanometer assembly.

[0070] S104, adjusting the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly, so that the laser stripes output by the laser scan the moving target;

[0071] In this embodiment, the second parameter adjustment value includes the galvanometer voltage and the galvanometer rotation angle. The current galvanometer voltage and rotation angle of the horizontal galvanometer and the vertical galvanometer in the third galvanometer assembly are adjusted according to the obtained parameter adjustment value so that the laser stripes output by the laser cover the moving target.

[0072] In this embodiment, based on the control voltage of the first galvanometer assembly, the calibration parameters of the active tracking subsystem and the dynamic laser subsystem (the first voltage conversion coefficient matrix), and the pixel coordinates of the center of the moving target, the control voltage of the third galvanometer assembly is calculated so that the laser stripes are projected onto the moving target and scanned periodically.

[0073] S105. Adjust the current voltage of the second galvanometer assembly according to the first parameter adjustment value of the second galvanometer assembly, trigger the second camera to capture a moving target laser image of the moving target carrying the laser stripes, and perform three-dimensional reconstruction of the moving target based on the moving target laser image.

[0074] In this embodiment, the galvanometer voltage of the second galvanometer assembly is calculated according to the control voltage of the first galvanometer assembly and the calibration parameters (second voltage conversion coefficient matrix) of the active tracking subsystem acquired in advance, and the orientation of the second galvanometer assembly is controlled according to the calculated galvanometer voltage, so that the moving target is in the center of the second image captured by the second camera. As an optional embodiment, the second galvanometer assembly adjusts the current voltage to the calculated galvanometer voltage and triggers the second camera to capture the moving target laser image carrying laser stripes of the moving target.

[0075] In this embodiment, as an optional embodiment, performing three-dimensional reconstruction of the moving target based on the moving target laser image includes:

[0076] Extracting the laser center contained in the moving target laser image;

[0077] Based on the calibration parameters of the dynamic camera subsystem and the first rotation angle of the galvanometer, a transformation matrix of the dynamic camera is obtained;

[0078] Based on the calibration parameters of the dynamic laser subsystem and the second rotation angle of the galvanometer, a dynamic laser plane is obtained;

[0079] According to the transformation matrix of the dynamic camera and the dynamic laser plane, the laser plane equation in the dynamic camera coordinate system is calculated;

[0080] 3D reconstruction of laser stripes is performed based on the laser plane equation and laser center;

[0081] Based on the 3D reconstructed 3D point cloud, coordinate fusion and error correction are performed;

[0082] Based on the calibration parameters of the active tracking subsystem and the dynamic camera subsystem, 3D motion compensation is performed on the coordinate information for error correction;

[0083] Based on the coordinate information of 3D motion compensation, three-dimensional reconstruction of the moving target is performed.

[0084] In this embodiment, the center of the laser stripe curve is extracted according to the moving target laser image to obtain the pixel coordinates (u, v) of the laser stripe.

[0085] In this embodiment, the rotation matrix of the dynamic camera and the plane equation of the dynamic laser are calculated according to the calibration parameters of the dynamic camera subsystem and the dynamic laser subsystem and the rotation angles of the second galvanometer assembly and the third galvanometer assembly. The laser plane equation in the dynamic camera coordinate system is expressed as VPlane. Combining VPlane and (u, v), 3D reconstruction of laser stripes can be achieved.

[0086] In this embodiment, the reconstructed laser stripes are integrated into the same coordinate system for error correction. The calibration parameters of the active tracking subsystem and the dynamic camera subsystem and the control voltage of the first galvanometer assembly are used to compensate for the three-dimensional motion errors of the moving target to obtain a complete three-dimensional point cloud of the moving target.

[0087] In this embodiment, the active tracking subsystem tracks the moving target and feeds back the position tracking information of the tracked moving target to the dynamic laser subsystem. The dynamic laser subsystem controls the laser projection onto the moving target based on the fed back position tracking information and periodically scans the moving target. The dynamic camera subsystem photographs the moving target to capture the target image with laser stripe information, and calculates the 3D information of the moving target based on the mathematical model of the laser 3D reconstruction system and the pre-calibrated parameters of the laser 3D reconstruction system and the captured target image and the voltage value of the second galvanometer assembly.

[0088] Figure 2 Another schematic flow chart of a method for three-dimensional reconstruction of a moving target provided by an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0089] S201, controlling the first galvanometer assembly to track a moving target;

[0090] S202, outputting a trigger signal to the first camera to trigger the first camera to shoot the moving target and obtain a target image;

[0091] S203, determining the moving target in the target image according to the target detection algorithm, and obtaining the target center pixel coordinates of the moving target;

[0092] S204, calculating the galvanometer voltage of the first galvanometer assembly according to the target center pixel coordinates and the offset coefficient of the first galvanometer assembly, and outputting the voltage to the dynamic camera subsystem and the dynamic laser subsystem;

[0093] In this embodiment, steps S201 to S204 are visual feedback.

[0094] S205, the dynamic camera subsystem obtains the galvanometer voltage and the first galvanometer rotation angle of the second galvanometer assembly according to the galvanometer voltage and the first voltage conversion matrix between the active tracking subsystem and the dynamic camera subsystem;

[0095] S206, the dynamic laser subsystem obtains the control voltage of the third galvanometer component and the second rotation angle of the galvanometer according to the galvanometer voltage and the second voltage conversion matrix between the active tracking subsystem and the dynamic laser subsystem;

[0096] In this embodiment, steps S205 to S206 are to perform projection and scanning to perform active three-dimensional reconstruction.

[0097] S207, adjusting the current voltage of the second galvanometer assembly according to the galvanometer voltage of the second galvanometer assembly;

[0098] S208, triggering the second camera to capture a moving target laser image of the moving target;

[0099] S209, extracting the laser center based on the moving target laser image;

[0100] S210, acquiring a transformation matrix of the dynamic camera based on the calibration parameters of the dynamic camera subsystem and the first rotation angle of the galvanometer;

[0101] S211, acquiring a dynamic laser plane based on the calibration parameters of the dynamic laser subsystem and the second rotation angle of the galvanometer;

[0102] S212, calculating the laser plane equation in the dynamic camera coordinate system according to the transformation matrix of the dynamic camera and the dynamic laser plane;

[0103] S213, performing 3D reconstruction of the laser stripes according to the laser plane equation and the laser center;

[0104] In this embodiment, a 3D point cloud is obtained by 3D reconstruction of the laser stripes. Steps S207 to S213 are three-dimensional reconstruction.

[0105] In this embodiment, as an optional embodiment, the relationship between the motion coordinates of the center of the moving target on the motion plane and the voltage of the galvanometer assembly is as follows:

[0106]

[0107] in,

[0108] x n ,y n is the motion coordinate of the center of the moving target on the motion plane, U pan-1 is the horizontal galvanometer voltage of the first horizontal mirror in the galvanometer assembly, U tilt-1 is the rotating galvanometer voltage of the first rotating mirror in the galvanometer assembly, S pan ,t pan , S tilt ,t tilt is the linear parameter of the galvanometer assembly.

[0109] The motion coordinates of the 3D point cloud and the moving target on the motion plane satisfy the following formula:

[0110]

[0111] In the formula, X n , Y n , Z n are the 3D point cloud coordinates, n x 、n y 、n z is the laser axis coordinate, and T3 is the transformation matrix.

[0112] S214, performing coordinate fusion and error correction based on the 3D reconstructed 3D point cloud;

[0113] S215, performing 3D motion compensation on the coordinate information for error correction based on calibration parameters of the active tracking subsystem and the dynamic camera subsystem;

[0114] S216: Perform three-dimensional reconstruction of the moving target based on the coordinate information of the 3D motion compensation.

[0115] In this embodiment, steps S214 to S216 are motion compensation.

[0116] The present embodiment is described in detail below:

[0117] In this embodiment, the pixel size captured by the camera (first camera, second camera) is 6.9 microns x 6.9 microns, the resolution is 720 pixels x 540 pixels, and the frame rate is 500 frames per second. Among them, the exposure time of the first camera is set to 5000 microseconds to capture and track the bright image of the moving target for moving target detection, and the exposure time of the second camera is set to 500 microseconds to capture the dark image of the laser with a wavelength of 650 nanometers for three-dimensional reconstruction. The control voltage range of the first galvanometer assembly, the second galvanometer assembly and the third galvanometer assembly is -10V to +10V, the scanning angle is ±20 degrees, the maximum scanning frequency is 1kHz, and the angular resolution is 0.0008.

[0118] In this embodiment, a ceramic step with a distance of 20 mm between the two surfaces and a sphere with a diameter of 20 mm are used to perform measurements at four different speeds. As an optional embodiment, in order to quantitatively analyze the movement speed of the sphere, the sphere is mounted on a ball screw slide, and the sphere is controlled to move at speeds of 100 mm / s, 200 mm / s, 500 mm / s, and 1000 mm / s, respectively. The active tracking subsystem is used to track the movement of the sphere, the dynamic laser subsystem controls the laser stripes to periodically scan the sphere, and the dynamic camera subsystem captures images for three-dimensional reconstruction.

[0119] In this embodiment, in order to verify the accuracy, ten positions of each target object are reconstructed repeatedly at each speed. After three-dimensional motion compensation, the three-dimensional point clouds of the step and the sphere are fitted, and the distance between the two planes of the fitted ceramic step and the diameter of the sphere are calculated.

[0120] Figure 3 A schematic diagram of the three-dimensional reconstruction and three-dimensional motion compensation results of a ceramic step at different speeds in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0121] Figure 4 A schematic diagram of the three-dimensional reconstruction and three-dimensional motion compensation results of a ceramic ball at different speeds in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0122] Figure 5 An error diagram between the actual distance and the fitted distance of a ceramic step in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention;

[0123] Figure 6The error diagram between the actual diameter and the fitted diameter of a ceramic sphere in a three-dimensional reconstruction method of a moving target provided by an embodiment of the present invention. From the results, it can be seen that at the same speed, the reconstruction error of the moving target remains stable, indicating the robustness of the method of this embodiment when measuring moving objects. In addition, the root mean square error (RMSE) of the moving target at different speeds remains below 0.5 mm, indicating that the method of this embodiment can measure moving objects with high precision.

[0124] In this embodiment, the faster the moving target moves, the greater the distance moved in one scanning cycle, and the greater the cumulative error in 3D motion compensation, which will reduce the overall 3D reconstruction accuracy. In order to mitigate the situation where the reconstruction error increases accordingly with the increase in movement speed, as an optional embodiment, the reconstruction accuracy can be improved by increasing the camera's acquisition frame rate.

[0125] The three-mirror based motion target tracking and three-dimensional reconstruction system of this embodiment realizes high-precision three-dimensional reconstruction of the motion target by feeding back the tracking information (coordinate information) of the motion target to the dynamic three-dimensional reconstruction system and completing 3D motion error compensation.

[0126] Based on the same inventive concept, Figure 7 As shown, an embodiment of the present invention further provides a three-dimensional reconstruction device for a moving target, the device comprising:

[0127] The system construction module 701 is used to construct a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, wherein the laser 3D reconstruction system includes: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser;

[0128] In this embodiment, as an optional embodiment, the first camera and the second camera include but are not limited to: a CMOS camera, the laser includes but is not limited to: a line laser, and the galvanometer assembly includes but is not limited to: a dual-axis galvanometer assembly.

[0129] The voltage acquisition module 702 is used to control the first galvanometer assembly to track the moving target, so that the moving target is located at the center of the target image obtained by the first camera shooting the moving target, and acquire the galvanometer voltage of the first galvanometer assembly;

[0130] In this embodiment, as an optional embodiment, the voltage acquisition module 702 includes:

[0131] A first tracking unit (not shown in the figure), used to control the first galvanometer assembly to track the moving target when the moving target is tracked for the first time;

[0132] A first shooting unit, used to trigger the first camera to shoot the moving target and obtain a target image;

[0133] A first detection unit, configured to determine a moving target in the target image according to a preset target detection algorithm;

[0134] The first voltage regulating unit is used to regulate the galvanometer voltage of the first galvanometer assembly so that in the target image captured by the first camera of the moving target, the moving target is located at the center of the target image.

[0135] In this embodiment, as another optional embodiment, the voltage acquisition module 702 further includes:

[0136] A second tracking unit, used for controlling the first galvanometer assembly to continue tracking the moving target;

[0137] A second shooting unit, used for triggering the first camera to shoot the moving target and obtain a target image;

[0138] A second detection unit, used to determine a moving target in the target image according to a preset target detection algorithm;

[0139] The second voltage regulating unit is used to calculate the control voltage of the first galvanometer assembly according to the pixel coordinates of the moving target in the current frame and the pixel coordinates of the previous frame and the pre-calibrated offset coefficient of the first galvanometer assembly; and to adjust the galvanometer voltage of the first galvanometer assembly according to the control voltage.

[0140] In this embodiment, as an optional embodiment, the control voltage is calculated using the following formula:

[0141]

[0142] In the formula, Δx pi ,Δy pi are the X-axis offset and Y-axis offset of the pixel coordinates of the center of the moving target in the previous and next frames respectively;

[0143] ΔU 1-pam is the voltage variation of the horizontal first mirror of the first galvanometer assembly in the front and rear frames, ΔU 1-tilt The voltage variation of the rotating first lens of the first galvanometer assembly in the front and rear frames;

[0144] α is the horizontal offset coefficient of the first galvanometer assembly, and β is the vertical offset coefficient of the first galvanometer assembly.

[0145] The conversion matrix module 703 is used to obtain the first parameter adjustment value of the second galvanometer component according to the galvanometer voltage of the first galvanometer component and the first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, and to obtain the second parameter adjustment value of the third galvanometer component according to the galvanometer voltage of the first galvanometer component and the second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem;

[0146] In this embodiment, as an optional embodiment, the control voltage of the first galvanometer assembly includes: a horizontal galvanometer control voltage of a horizontal first lens of the first galvanometer assembly and a rotating galvanometer control voltage of a rotating first lens.

[0147] In this embodiment, as an optional embodiment, the galvanometer voltage of the first galvanometer assembly of the active tracking subsystem and the galvanometer voltage of the second galvanometer assembly of the dynamic camera subsystem satisfy the following formula:

[0148]

[0149] Where U 1-pan is the horizontal galvanometer voltage value of the horizontal first lens in the first galvanometer assembly, U 1-tilt is the voltage value of the rotating galvanometer when the first lens is rotated in the first galvanometer assembly, U 2-pan is the horizontal galvanometer voltage value of the second mirror in the second galvanometer assembly, U 2-tilt is the rotating galvanometer voltage value of rotating the second lens in the second galvanometer assembly, and T1 is the first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem.

[0150] A target scanning module 704 is used to adjust the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly, so that the laser stripes output by the laser scan the moving target;

[0151] In this embodiment, based on the control voltage of the first galvanometer assembly, the first voltage conversion coefficient matrix and the pixel coordinates of the center of the moving target, the control voltage of the third galvanometer assembly is calculated, so that the laser stripes are projected onto the moving target and scanned periodically.

[0152] The three-dimensional reconstruction module 705 is used to adjust the current voltage of the second galvanometer assembly according to the first parameter adjustment value of the second galvanometer assembly, trigger the second camera to capture the moving target laser image of the moving target carrying the laser stripes, and perform three-dimensional reconstruction of the moving target based on the moving target laser image.

[0153] In this embodiment, as an optional embodiment, performing three-dimensional reconstruction of the moving target based on the moving target laser image includes:

[0154] Extracting the laser center contained in the moving target laser image;

[0155] Based on the calibration parameters of the dynamic camera subsystem and the first rotation angle of the galvanometer, a transformation matrix of the dynamic camera is obtained;

[0156] Based on the calibration parameters of the dynamic laser subsystem and the second rotation angle of the galvanometer, a dynamic laser plane is obtained;

[0157] According to the transformation matrix of the dynamic camera and the dynamic laser plane, the laser plane equation in the dynamic camera coordinate system is calculated;

[0158] 3D reconstruction of laser stripes is performed based on the laser plane equation and laser center;

[0159] Based on the 3D reconstructed 3D point cloud, coordinate fusion and error correction are performed;

[0160] Based on the calibration parameters of the active tracking subsystem and the dynamic camera subsystem, 3D motion compensation is performed on the coordinate information for error correction;

[0161] Based on the coordinate information of 3D motion compensation, three-dimensional reconstruction of the moving target is performed.

[0162] Based on the same inventive concept, an embodiment of the present invention further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the three-dimensional reconstruction method of a moving target in any possible implementation manner described above are implemented.

[0163] Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0164] Based on the same inventive concept, see Figure 8 The embodiment of the present invention further provides an electronic device, including a memory 101 (such as a non-volatile memory), a processor 102, and a computer program stored in the memory 101 and executable on the processor 102. When the processor 102 executes the program, the steps of the three-dimensional reconstruction method of the moving target in any possible implementation manner described above can be equivalent to the three-dimensional reconstruction device of the moving target as described above. Of course, the processor can also be used to process other data or operations. The electronic device can be a PC, a server, a terminal, and other devices.

[0165] like Figure 8 As shown, the electronic device may also generally include: a memory 103, a network interface 104, and an internal bus 105. In addition to these components, other hardware may also be included, which will not be described in detail.

[0166] It should be pointed out that the above-mentioned three-dimensional reconstruction device of the moving target can be realized by software. As a device in a logical sense, it is formed by the processor 102 of the electronic device in which it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0167] The embodiments of the subject matter and functional operations described in this specification may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules in computer program instructions encoded on a tangible non-temporary program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0168] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform the corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits), and the apparatus can also be implemented as special purpose logic circuits.

[0169] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or to transmit data to it, or both. However, the computer does not necessarily have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0170] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0171] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although the features may work as above in certain combinations and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0172] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0173] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0174] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0175] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A three-dimensional reconstruction method for a moving target, characterized in that: include: Constructing a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, the laser 3D reconstruction system includes: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser; Controlling the first galvanometer assembly to track the moving target, so that the moving target is located at the center of the target image acquired by photographing the moving target by the first camera, and acquiring a galvanometer voltage of the first galvanometer assembly; According to the galvanometer voltage of the first galvanometer component and the first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, a first parameter adjustment value of the second galvanometer component is obtained, and the first parameter adjustment value includes: the galvanometer voltage and the galvanometer rotation angle; according to the galvanometer voltage of the first galvanometer component and the second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem, a second parameter adjustment value of the third galvanometer component is obtained, and the second parameter adjustment value includes: the galvanometer voltage and the galvanometer rotation angle; Adjusting the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly so that the laser stripes output by the laser scan the moving target; The current voltage of the second galvanometer assembly is adjusted according to the first parameter adjustment value of the second galvanometer assembly, the second camera is triggered to capture a moving target laser image of the moving target carrying the laser stripes, and three-dimensional reconstruction of the moving target is performed based on the moving target laser image.

2. The three-dimensional reconstruction method of a moving target according to claim 1, characterized in that: The controlling the first galvanometer assembly to track the moving target so that the first camera captures the moving target and obtains a target image in which the moving target is located at the center of the target image includes: When the moving target is tracked for the first time, controlling the first galvanometer assembly to track the moving target; triggering the first camera to shoot the moving target to obtain a target image; Determine the moving target in the target image according to a preset target detection algorithm; The galvanometer voltage of the first galvanometer assembly is adjusted so that in a target image captured by the first camera of the moving target, the moving target is located at the center of the target image.

3. The three-dimensional reconstruction method of a moving object according to claim 2, characterized in that: The method further comprises: Controlling the first galvanometer assembly to continue tracking the moving target; triggering the first camera to shoot the moving target to obtain a target image; Determine the moving target in the target image according to a preset target detection algorithm; Calculating a control voltage of the first galvanometer assembly according to the pixel coordinates of the moving target in the current frame and the pixel coordinates of the previous frame and a pre-calibrated offset coefficient of the first galvanometer assembly; The galvanometer voltage of the first galvanometer assembly is adjusted according to the control voltage.

4. The three-dimensional reconstruction method of a moving object according to claim 3, characterized in that: The control voltage is calculated using the following formula: In the formula, Δx pi ,Δy pi are the X-axis offset and Y-axis offset of the pixel coordinates of the center of the moving target in the previous and next frames respectively; ΔU 1-pan is the voltage variation of the horizontal first mirror of the first galvanometer assembly in the front and rear frames, ΔU 1-tilt The voltage variation of the rotating first lens of the first galvanometer assembly in the front and rear frames; α is the horizontal offset coefficient of the first galvanometer assembly, and β is the vertical offset coefficient of the first galvanometer assembly.

5. The three-dimensional reconstruction method of a moving object according to claim 4, characterized in that: The control voltage of the first galvanometer assembly includes: a horizontal galvanometer control voltage for a horizontal first lens of the first galvanometer assembly and a rotational galvanometer control voltage for rotating the first lens.

6. The three-dimensional reconstruction method of a moving object according to any one of claims 1 to 4, characterized in that: The galvanometer voltage of the first galvanometer assembly of the active tracking subsystem and the galvanometer voltage of the second galvanometer assembly of the dynamic camera subsystem satisfy the following formula: Where U 1-pan is the horizontal galvanometer voltage value of the horizontal first lens in the first galvanometer assembly, U 1-tilt is the voltage value of the rotating galvanometer when the first lens is rotated in the first galvanometer assembly, U 2-pan is the horizontal galvanometer voltage value of the second mirror in the second galvanometer assembly, U 2-tilt is the rotating galvanometer voltage value of rotating the second lens in the second galvanometer assembly, and T1 is the first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem.

7. The three-dimensional reconstruction method of a moving object according to any one of claims 1 to 4, characterized in that: The three-dimensional reconstruction of the moving target based on the moving target laser image comprises: Extracting the laser center contained in the moving target laser image; Based on the calibration parameters of the dynamic camera subsystem and the first rotation angle of the galvanometer, a transformation matrix of the dynamic camera is obtained; Based on the calibration parameters of the dynamic laser subsystem and the second rotation angle of the galvanometer, a dynamic laser plane is obtained; According to the transformation matrix of the dynamic camera and the dynamic laser plane, the laser plane equation in the dynamic camera coordinate system is calculated; 3D reconstruction of laser stripes is performed based on the laser plane equation and laser center; Based on the 3D reconstructed 3D point cloud, coordinate fusion and error correction are performed; Based on the calibration parameters of the active tracking subsystem and the dynamic camera subsystem, 3D motion compensation is performed on the coordinate information for error correction; Based on the coordinate information of 3D motion compensation, three-dimensional reconstruction of the moving target is performed.

8. A three-dimensional reconstruction device for a moving object, characterized in that: The three-dimensional reconstruction device of the moving target comprises: A system construction module, used to construct a laser 3D reconstruction system for moving target tracking and three-dimensional reconstruction, wherein the laser 3D reconstruction system includes: an active tracking subsystem, a dynamic camera subsystem, and a dynamic laser subsystem, wherein the active tracking subsystem includes: a first galvanometer component and a first camera, the dynamic camera subsystem includes: a second galvanometer component and a second camera, and the dynamic laser subsystem includes: a third galvanometer component and a laser; a voltage acquisition module, used for controlling the first galvanometer assembly to track the moving target, so that in a target image captured by the first camera when the moving target is photographed, the moving target is located at the center of the target image, and acquiring a galvanometer voltage of the first galvanometer assembly; A conversion matrix module, used to obtain a first parameter adjustment value of the second galvanometer component according to the galvanometer voltage of the first galvanometer component and a first voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic camera subsystem, wherein the first parameter adjustment value includes: a galvanometer voltage and a galvanometer rotation angle; and to obtain a second parameter adjustment value of the third galvanometer component according to the galvanometer voltage of the first galvanometer component and a second voltage conversion matrix pre-calibrated between the active tracking subsystem and the dynamic laser subsystem, wherein the second parameter adjustment value includes: a galvanometer voltage and a galvanometer rotation angle; A target scanning module, used for adjusting the current galvanometer voltage of the third galvanometer assembly according to the second parameter adjustment value of the third galvanometer assembly, so that the laser stripes output by the laser scan the moving target; A three-dimensional reconstruction module is used to adjust the current voltage of the second galvanometer assembly according to the first parameter adjustment value of the second galvanometer assembly, trigger the second camera to capture a moving target laser image of the moving target carrying the laser stripes, and perform three-dimensional reconstruction of the moving target based on the moving target laser image.

9. A storage medium, characterized in that: The storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the three-dimensional reconstruction method of a moving target as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for three-dimensional reconstruction of a moving object according to any one of claims 1 to 7 are implemented.

Images