Binocular stereo vision dynamic measurement method and device based on double-mirror camera

Abstract

The application relates to the technical field of sensors, and provides a binocular stereo vision dynamic measurement method and device based on a double-mirror camera. The method realizes high-precision measurement of a target under a dynamic condition based on the high dynamic response characteristic of the mirror camera; a time-varying dynamic measurement model is constructed, a traditional static measurement model is introduced into a time dimension, time-varying dynamic vision measurement combining measurement and estimation is realized, an optimal visual angle control law is provided, continuous tracking of a dynamic target is realized at an optimal measurement visual angle, a target motion mode estimation module is further provided, motion compensation during exposure is realized, clear imaging is realized, and high-precision dynamic measurement is ensured in the three aspects of imaging, control and measurement.

Description

Technical Field

[0001] This application relates to the field of sensor technology, and in particular to a binocular stereo vision dynamic measurement method and device based on a dual-mirror camera. Background Technology

[0002] Visual measurement, characterized by its non-contact nature and high flexibility, has seen increasingly widespread application in fields such as autonomous driving and industrial automation in recent years. However, traditional visual measurement technologies largely focus on static measurement, assuming that the relative position between the target and the sensor remains constant, allowing for clear imaging and subsequent image analysis and geometric calculations. In reality, however, many scenarios involve dynamic measurement where there is relative motion between the target and the sensor, such as long-exposure scenes in low light or when the moving vehicle itself experiences strong vibrations. Compared to traditional static measurement, these scenarios can lead to problems such as the target easily slipping out of the field of view, motion blur, and decreased measurement accuracy. Therefore, achieving the transition from static to dynamic measurement is a crucial step in advancing visual measurement technology.

[0003] Existing dynamic measurement methods mainly focus on two aspects: addressing target detachment from the field of view and mitigating motion blur. The former utilizes pan-tilt-zoom (PTZ) cameras to continuously adjust the viewing angle to follow target movement, keeping the target within the field of view for extended periods. For example, some solutions use dual PTZ cameras in visual monitoring systems, simultaneously meeting the needs for target tracking, real-time close-up view acquisition, and 3D information acquisition; other solutions use dual PTZ to achieve continuous target tracking and cross-field-of-view integration. However, existing methods often simply place the target at the center of the field of view or maintain visibility, without discussing which viewing angle configuration achieves optimal measurement accuracy.

[0004] For motion blur, current methods mostly employ deblurring algorithms to remove blur from moving images. Some solutions combine traditional Wiener filtering with deep learning features to achieve a classic non-blind deblurring algorithm; others first estimate the blur kernel using deep learning and then remove the blur through deconvolution. However, most image deblurring algorithms are based on mathematical probability estimation, which cannot achieve pixel-level precise blur removal, thus limiting the accuracy of dynamic measurements.

[0005] It is evident that existing dynamic visual measurement methods mostly focus on single problems and fail to design dynamic visual measurement models and methods from a system-level macroscopic perspective, thus failing to achieve a deeper level of improvement in dynamic measurement accuracy. Summary of the Invention

[0006] In view of this, the present application provides a binocular stereo vision dynamic measurement method and apparatus based on a dual-galvanometer camera to solve the problem of insufficient accuracy in dynamic vision measurement in the prior art.

[0007] A first aspect of this application provides a binocular stereo vision dynamic measurement method based on a dual-galvanometer camera, comprising:

[0008] Obtain the mirror parameters of the galvanometer, and use the optimal viewing angle control law to control the mirror parameters to obtain the optimized mirror parameters;

[0009] The target is tracked using a dynamic compensation imaging algorithm based at least on the optimized mirror parameters to obtain the image point coordinates of the target in the view of the dual galvanometer camera; wherein the dual galvanometer camera includes a left galvanometer camera and a right galvanometer camera;

[0010] The time-varying dynamic measurement model is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment;

[0011] Based on the measurement results at this moment, target observation and motion estimation are performed, and the motion parameters and optimal view control law of the dynamic compensation imaging algorithm are updated based on the estimation results.

[0012] A second aspect of this application provides a binocular stereo vision dynamic measurement device based on a dual-galvanometer camera, comprising:

[0013] The optimization module is configured to acquire the mirror parameters of the galvanometer, control the mirror parameters using the optimal viewing angle control law, and obtain the optimized mirror parameters.

[0014] The tracking module is configured to use a dynamic compensation imaging algorithm to track the target at least based on the optimized mirror parameters, and obtain the image point coordinates of the target in the view of the dual galvanometer camera; wherein the dual galvanometer camera includes a left galvanometer camera and a right galvanometer camera;

[0015] The measurement module is configured to use a time-varying dynamic measurement model to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment.

[0016] The update module is configured to perform target observation and motion estimation based on the measurement results at the current moment, and update the motion parameters and optimal view control law of the dynamic compensation imaging algorithm based on the estimation results.

[0017] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0019] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment establishes a dynamic visual servo measurement framework for a galvanometer camera, which can achieve high-precision target measurement under dynamic conditions based on the high dynamic response characteristics of the galvanometer camera. This framework first includes a time-varying dynamic measurement model, which introduces a traditional static measurement model into the time dimension, realizing time-varying dynamic visual measurement that combines measurement and estimation. Second, it includes an optimal viewing angle control law, enabling continuous tracking of dynamic targets at the optimal measurement viewing angle. Finally, it includes a target motion pattern estimation module, which can achieve motion compensation during exposure, thereby achieving clear imaging. Under this unified framework, high-precision dynamic measurement is jointly guaranteed in terms of imaging, control, and measurement. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a binocular stereo vision dynamic measurement method based on a dual-mirror camera, provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the structure of the dual-mirror camera visual dynamic measurement system provided in the embodiments of this application.

[0023] Figure 3 This is a flowchart illustrating a method provided in this application for obtaining the measurement result at the current moment using a time-varying dynamic measurement model, based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment.

[0024] Figure 4 This is a system framework diagram for implementing the binocular stereo vision dynamic measurement method based on a dual-mirror camera provided in the embodiments of this application.

[0025] Figure 5 This is a schematic diagram of a binocular stereo vision dynamic measurement device based on a dual-mirror camera, provided in an embodiment of this application.

[0026] Figure 6 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0028] The following describes in detail, with reference to the accompanying drawings, a binocular stereo vision dynamic measurement method and apparatus based on a dual-galvanometer camera according to an embodiment of this application.

[0029] As mentioned above, existing dynamic measurement methods mainly improve measurement accuracy by addressing two issues: target detachment from the field of view and mitigating motion blur. Existing solutions for target detachment often involve simply placing the target at the center of the field of view or maintaining visibility, without discussing which viewing angle configuration achieves optimal measurement accuracy. Solutions for mitigating motion blur are mostly based on image deblurring algorithms, which are largely based on mathematical probability estimations and cannot achieve pixel-level precision in blur removal, thus limiting the accuracy of dynamic measurements.

[0030] In view of this, embodiments of this application provide a binocular stereo vision dynamic measurement method based on a dual-galvanometer camera, which establishes a dynamic visual servo measurement framework for the galvanometer camera. This framework, based on the high dynamic characteristics of the galvanometer camera, establishes a time-varying dynamic measurement model, proposes an optimal viewing angle control law, and implements a dynamic imaging compensation method. This achieves high-precision dynamic adjustment and visual servo control of the system's viewing angle, continuously controlling the line of sight to follow the target movement at the optimal viewing angle and performing dynamic compensation to obtain clear imaging, thus realizing high-precision dynamic measurement of the target.

[0031] Figure 1 This is a flowchart illustrating a binocular stereo vision dynamic measurement method based on a dual-galvanometer camera, provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0032] In step S101, the mirror parameters of the galvanometer are obtained, and the optimal viewing angle control law is used to control the mirror parameters to obtain the optimized mirror parameters.

[0033] In step S102, a dynamic compensation imaging algorithm is used to track the target based at least on the optimized mirror parameters to obtain the image point coordinates of the target in the view of the dual-mirror camera.

[0034] The dual-mirror camera system includes a left galvanometer camera and a right galvanometer camera.

[0035] In step S103, the time-varying dynamic measurement model is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment.

[0036] In step S104, target observation and motion estimation are performed based on the measurement results at this moment, and the motion parameters and optimal view control law of the dynamic compensation imaging algorithm are updated based on the estimation results.

[0037] In some embodiments of this application, the method may be executed by a server or by a terminal device with certain processing capabilities.

[0038] In some embodiments of this application, the mirror parameters of the galvanometer can be obtained first, and the mirror parameters can be controlled using an optimal viewing angle control law to obtain optimized mirror parameters. Then, a dynamic compensation imaging algorithm is used to track the target at least based on the optimized mirror parameters to obtain the image point coordinates of the target in the dual galvanometer camera view.

[0039] A galvanometer camera is a novel high-dynamic vision sensor composed of a two-dimensional galvanometer and an area array camera. By controlling the current, the galvanometer mirror can be rotated in a highly dynamic manner, thereby enabling highly dynamic adjustment of the camera's field of view. Figure 2 This is a schematic diagram of the structure of the dual-galvanometer camera visual dynamic measurement system provided in an embodiment of this application. Figure 2 As shown, the system includes a left galvanometer camera and a right galvanometer camera, which work together to focus on the target's position (i.e., the target point). Take measurements.

[0040] The left galvanometer camera may include a left camera and a left galvanometer, the left galvanometer having a vertical rotation angle of the mirror surface parameter. and horizontal corner The right galvanometer camera may include a right camera and a right galvanometer, and the right galvanometer also has a mirror parameter of vertical rotation angle. and horizontal corner .

[0041] In this embodiment, the galvanometer can be rotated. Directly control the object.

[0042] In some implementations, the optimized galvanometer rotation angle and the images acquired by the left and right cameras can be input together into a dynamic compensation imaging algorithm to obtain the image point coordinates of the target in the dual galvanometer camera view through target tracking.

[0043] Next, a time-varying dynamic measurement model can be used to obtain the measurement result at the current moment, based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment. The measurement result at the current moment can be a posterior estimate of the target's coordinates in the three-dimensional world coordinate system at the current moment, and the measurement result at the previous moment can be a posterior estimate of the target's coordinates in the three-dimensional world coordinate system at the previous moment.

[0044] Furthermore, embodiments of this application can also perform target observation and motion estimation based on the measurement results at the current moment, and update the motion parameters and optimal view control law of the dynamic compensation imaging algorithm based on the estimation results, thereby providing the motion parameters and optimal view control law of the dynamic compensation imaging algorithm updated in real time for the measurement at the next moment.

[0045] According to the technical solution provided in the embodiments of this application, high-precision target measurement under dynamic conditions is achieved based on the high dynamic response characteristics of the galvanometer camera. On the one hand, a time-varying dynamic measurement model is constructed, which introduces the traditional static measurement model into the time dimension to realize time-varying dynamic visual measurement that combines measurement and estimation. On the other hand, an optimal viewpoint control law is proposed to realize continuous tracking of dynamic targets with the optimal measurement viewpoint. Furthermore, a target motion pattern estimation module is provided to realize motion compensation during exposure, thereby achieving clear imaging. Thus, high-precision dynamic measurement is jointly guaranteed in the three aspects of imaging, control, and measurement.

[0046] Figure 3 This is a flowchart illustrating a method provided in this application for obtaining the measurement result at the current moment using a time-varying dynamic measurement model, based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment. For example... Figure 3 As shown, the method includes the following steps:

[0047] In step S301, the coordinates of the target in the three-dimensional world coordinate system are determined based on the image point coordinates of the target in the dual-mirror camera view.

[0048] In step S302, observation variables and state variables are constructed using the target's coordinates in the three-dimensional world coordinate system.

[0049] In step S303, Kalman filtering is performed on the state variables and observation variables to obtain the measurement results at the current moment, based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement results of the previous moment.

[0050] In some embodiments of this application, the time-varying dynamic measurement model can first determine the coordinates of the target in the three-dimensional world coordinate system based on the image point coordinates of the target in the view of the dual-mirror camera, then use the coordinates of the target in the three-dimensional world coordinate system to construct observation variables and state variables, and finally perform Kalman filtering on the state variables and observation variables to obtain the measurement result at the current moment.

[0051] Among them, the time-varying dynamic measurement model can be ; Let be any non-zero coefficient. for The coordinates of the target's image point in the left galvanometer camera view at any given time. for The coordinates of the target's image point in the right galvanometer camera view at any given time. These are the internal parameters of the left galvanometer camera. These are the internal parameters of the right galvanometer camera. This represents the augmented internal parameter matrix of the left galvanometer camera. This represents the augmented intrinsic parameter matrix of the right galvanometer camera. for The transformation matrix from the target to the left galvanometer camera coordinate system at any given time. for The transformation matrix from the target to the right galvanometer camera coordinate system at any given time.

[0052] Transformation matrix and The following method is used to determine:

[0053] ,and ;in, This refers to the vertical rotation angle of the left galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the left galvanometer in the mirror parameters. This refers to the vertical rotation angle of the right galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the right galvanometer in the mirror parameters; Here is the reflection transformation matrix. , It is a 3-order identity matrix. The normal vector to the mirror plane can be determined by the rotation angle of the galvanometer. It is the transpose symbol; , , Let be any point on the mirror surface.

[0054] This time-varying dynamic measurement model allows for the measurement of data based on target tracking. and Determine the target's coordinates in the three-dimensional world coordinate system The three-dimensional world coordinate system is set at the initial viewpoint of the left camera.

[0055] Compared to traditional time-invariant visual measurement models, the most significant feature of this model is the introduction of a time dimension, considering the relationship between measurement values ​​at different moments during dynamic measurement. Therefore, during dynamic measurement, not only is static calculation of the current frame performed based on the current mirror parameters, but also the temporal logic is unified through target observation methods such as Kalman filtering, and the current measurement result is corrected based on the results of previous calculations.

[0056] In other words, it is possible to utilize certain... Build Observed variables at time ,as well as State variables at time 1 ,in for The first derivative.

[0057] Then, through Based on the posterior state estimate at time t-1 Estimate the prior state at time t Then the measurement value at time t and The residuals are calculated by comparison, and finally the posterior state estimate at time t is obtained by using the residuals and Kalman coefficients. , which is the measurement result at time t.

[0058] in, For process noise, To measure noise, Here is the state transition matrix. Let be the time interval between time t and time t-1. It is the identity matrix. This is the observation matrix.

[0059] In dynamic visual measurement, to keep the target always visible, the viewing angle of the binocular stereo vision sensor needs to be adjusted in real time to follow the target. Different viewing angle configurations correspond to different structural parameters of the binocular stereo vision sensor, which in turn leads to different measurement accuracies. Therefore, how to design the control law to ensure that the measurement accuracy is always at the optimal level during the viewing angle adjustment process is a key factor determining the accuracy of dynamic measurement.

[0060] To achieve optimal measurement accuracy through viewing angle configuration, embodiments of this application provide an optimal viewing angle control law.

[0061] The optimal viewing angle control law can be determined as follows: during the control process, keep the left galvanometer camera always aligned with the target and keep the pitch angles of the left and right galvanometer cameras synchronized to determine the control law of the right galvanometer camera; determine the control law of the left galvanometer camera according to the center alignment principle; combine the control laws of the right galvanometer camera and the left galvanometer camera to obtain the optimal viewing angle control law.

[0062] In some embodiments of this application, the control law for the right galvanometer camera can be determined in the following manner:

[0063] First, based on the target's coordinates in the current coordinate system of the left galvanometer camera... Determine the target depth value ;in, .

[0064] Then, using the left galvanometer camera as the control reference, determine... ;in, for Baseline value at time, For the left galvanometer At the turning point of time, Right galvanometer At the turning point of time, It is the cotangent function. , for x-coordinate for The coordinates of the target's image point in the right galvanometer camera view at any given time. for focal length in the middle, These are the internal parameters of the right galvanometer camera.

[0065] Next based on Determine the reconstruction error ;in, for , , and Any parameter in, for Measurement error. There can be four different values, when When taking different values, for , , and Different parameters in. It means that it will be based on different and Calculated error value The squares are accumulated.

[0066] Based on this, it can be determined right The partial derivatives are Horizontal rotation angle of the right galvanometer at time At the same time, it can also be determined according to the principle of center alignment. Vertical rotation angle of the right galvanometer at time Finally, combining and The control law for the right galvanometer camera can then be obtained.

[0067] Furthermore, based on the center alignment principle, the control law for the left galvanometer camera can be determined as follows: and By combining the control laws of the right galvanometer camera and the left galvanometer camera, we can obtain... Optimal view control law at any given moment .

[0068] Based on the mirror rotation angle obtained from the optimal viewpoint control law, the target can be imaged at the optimal viewpoint. However, since the object itself is still in motion, static exposure will produce motion blur. In view of this, the embodiments of this application adopt a dynamic compensation imaging method, which realizes motion compensation of the target during the imaging process through dynamic exposure, thereby eliminating motion blur at the hardware level.

[0069] In some embodiments of this application, target tracking using a dynamic compensation imaging algorithm based at least on optimized mirror parameters may include: calculating mirror parameter compensation values ​​using a dynamic compensation imaging algorithm; the mirror parameter compensation values ​​are calculated with the target being controlled during exposure to keep the galvanometer mirror rotating to follow the target rotation so that the target is always imaged on the same image point; compensating the optimized mirror parameters using the mirror parameter compensation values; and tracking the target based at least on the compensated mirror parameters to obtain the image point coordinates of the target in the dual galvanometer camera view.

[0070] The calculation of mirror parameter compensation values ​​using a dynamic compensation imaging algorithm can be achieved by estimating the target motion pattern using target observation methods such as Kalman filtering in a time-varying dynamic measurement model to obtain the target's motion parameters. Then, based on these motion parameters, the mirror is controlled to continue following the target's rotation during exposure, thereby ensuring that the image is always projected onto the same image point, achieving both target motion compensation and blur removal.

[0071] After compensation imaging, the mirror parameters will change. This can be determined using the formula... The process shown compensates for the current mirror parameters, which are then used in subsequent measurements and feedback. Among these... For the compensated mirror parameters, This is the compensation value for the mirror parameters.

[0072] Figure 4This is a system framework diagram for implementing the binocular stereo vision dynamic measurement method based on a dual-galvanometer camera provided in the embodiments of this application. For example... Figure 4 As shown, the framework includes a control unit and an optimal viewing angle control law unit. The optimal viewing angle control law outputs the optimal viewing angle control law at the current moment. The control unit uses the optimal viewing angle control law at the current moment to control the current mirror parameters of the galvanometer to obtain optimized mirror parameters. It also includes a dynamic compensation imaging unit and a target tracking unit, which are used to track the target based at least on the optimized mirror parameters to obtain the image point coordinates of the target in the dual galvanometer camera view. It also includes a time-varying dynamic measurement model, which is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual galvanometer camera view and the measurement result at the previous moment.

[0073] The measurement results at this moment are output directly on one hand, and fed back to the target observation and motion estimation unit on the other hand, so as to update the motion parameters in the dynamic compensation imaging unit and the control law in the optimal viewpoint control law unit.

[0074] This framework, based on modern control theory, establishes closed-loop servo control with the galvanometer rotation angle as the direct control object. Following the established optimal viewpoint control law, the binocular galvanometer camera is controlled to consistently track the moving target at a viewpoint capable of optimal accuracy reconstruction. Simultaneously, based on target observation and motion estimation results, dynamic compensation imaging of the moving target is achieved during the exposure period, obtaining a clear image of the dynamic target. Furthermore, target tracking is performed to acquire the target position, providing a positional basis for subsequent target observation, estimation, and control. Finally, a proposed time-varying dynamic measurement model is combined to achieve high-precision measurement of the dynamic target. After outputting the target position, a closed loop is established through target observation, motion estimation, and the optimal viewpoint control law, ensuring that the system consistently performs tracking and measurement with high accuracy.

[0075] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0076] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0077] Figure 5 This is a schematic diagram of a binocular stereo vision dynamic measurement device based on a dual-galvanometer camera, provided in an embodiment of this application. Figure 5 As shown, the device includes:

[0078] The optimization module 501 is configured to acquire the mirror parameters of the galvanometer, control the mirror parameters using the optimal viewing angle control law, and obtain the optimized mirror parameters.

[0079] The tracking module 502 is configured to use a dynamic compensation imaging algorithm to track the target at least based on the optimized mirror parameters, and obtain the image point coordinates of the target in the view of the dual galvanometer camera; wherein the dual galvanometer camera includes a left galvanometer camera and a right galvanometer camera.

[0080] Measurement module 503 is configured to use a time-varying dynamic measurement model to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment.

[0081] The update module 504 is configured to perform target observation and motion estimation based on the measurement results at the current moment, and update the motion parameters of the dynamic compensation imaging algorithm and the optimal view control law based on the estimation results.

[0082] According to the technical solution provided in the embodiments of this application, high-precision target measurement under dynamic conditions is achieved based on the high dynamic response characteristics of the galvanometer camera. On the one hand, a time-varying dynamic measurement model is constructed, which introduces the traditional static measurement model into the time dimension to realize time-varying dynamic visual measurement that combines measurement and estimation. On the other hand, an optimal viewpoint control law is proposed to realize continuous tracking of dynamic targets with the optimal measurement viewpoint. Furthermore, a target motion pattern estimation module is provided to realize motion compensation during exposure, thereby achieving clear imaging. Thus, high-precision dynamic measurement is jointly guaranteed in the three aspects of imaging, control, and measurement.

[0083] In some implementations, the time-varying dynamic measurement model is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment. This includes: determining the coordinates of the target in the three-dimensional world coordinate system based on the image point coordinates of the target in the dual-mirror camera view; constructing observation variables and state variables using the coordinates of the target in the three-dimensional world coordinate system; and performing Kalman filtering on the state variables and the observation variables to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement result at the previous moment.

[0084] In some implementations, the coordinates of the target in the three-dimensional world coordinate system are determined based on the image point coordinates of the target in the view of the dual-galvanometer camera, including: constructing a measurement model. The measurement model is used to determine the target's coordinates in the three-dimensional world coordinate system. ;in, Let be any non-zero coefficient. for The coordinates of the target's image point in the left galvanometer camera view at any given time. for The coordinates of the target's image point in the right galvanometer camera view at any given time. These are the internal parameters of the left galvanometer camera. These are the internal parameters of the right galvanometer camera. This represents the augmented internal parameter matrix of the left galvanometer camera. This represents the augmented intrinsic parameter matrix of the right galvanometer camera. for The transformation matrix from the target to the left galvanometer camera coordinate system at any given time. for Transformation matrix from the target to the right galvanometer camera coordinate system at any given time; transformation matrix and The following method is used to determine: ,and ;in, This refers to the vertical rotation angle of the left galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the left galvanometer in the mirror parameters. This refers to the vertical rotation angle of the right galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the right galvanometer in the mirror parameters; Here is the reflection transformation matrix. , It is a 3-order identity matrix. The normal vector of the mirror plane is determined by the rotation angle of the galvanometer. It is the transpose symbol; , , Let be any point on the mirror surface.

[0085] In some implementations, Observed variables at time for , State variables at time 1 for ,in for The first derivative; Kalman filtering is performed on the state variable and the observed variable, including: constructing the Kalman filter equation. The state variable and the observed variable are filtered using the Kalman filter equation to obtain... The measurement results at time; among which, Let be the prior state estimate at time t. Let be the posterior state estimate at time t-1. For process noise, To measure noise, Here is the state transition matrix. Let be the time interval between time t and time t-1. It is the identity matrix. This is the observation matrix.

[0086] In some implementations, the optimal viewing angle control law is determined as follows: during the control process, the left galvanometer camera is kept always aligned with the target, and the pitch angles of the left and right galvanometer cameras are kept synchronized to determine the control law for the right galvanometer camera; the control law for the left galvanometer camera is determined according to the center alignment principle; and the optimal viewing angle control law is obtained by combining the control laws for the right and left galvanometer cameras.

[0087] In some implementations, the control law for the right galvanometer camera is determined as follows: based on the target's coordinates in the current coordinate system of the left galvanometer camera. Determine the target depth value ;in, Using the left galvanometer camera as the control reference, determine ;in, for Baseline value at time, For the left galvanometer At the turning point of time, Right galvanometer At the turning point of time, It is the cotangent function. , for x-coordinate for The coordinates of the target's image point in the right galvanometer camera view at any given time. for focal length in the middle, These are the internal parameters of the right galvanometer camera; based on Determine the reconstruction error ;in, for , , and Any parameter in, for Measurement error; Determine right The partial derivatives are Horizontal rotation angle of the right galvanometer at time Determined according to the principle of center alignment Vertical rotation angle of the right galvanometer at time , combined and The control law for the right galvanometer camera is obtained.

[0088] In some implementations, target tracking is performed using a dynamic compensation imaging algorithm based at least on the optimized mirror parameters, including: calculating a mirror parameter compensation value using the dynamic compensation imaging algorithm; the mirror parameter compensation value is calculated with the goal of controlling the galvanometer mirror to follow the target rotation during exposure to keep the target always imaged on the same image point; compensating the optimized mirror parameters using the mirror parameter compensation value; and performing target tracking based at least on the compensated mirror parameters to obtain the image point coordinates of the target in the dual galvanometer camera view.

[0089] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0090] Figure 6 This is a schematic diagram of the electronic device provided in an embodiment of this application. For example... Figure 6 As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module / unit in the various device embodiments described above.

[0091] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0092] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0093] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0096] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for dynamic measurement of binocular stereo vision based on a dual-galvanometer camera, characterized in that, include: The mirror parameters of the galvanometer are obtained, and the optimal viewing angle control law is used to control the mirror parameters to obtain the optimized mirror parameters. The target is tracked using a dynamic compensation imaging algorithm based at least on the optimized mirror parameters to obtain the image point coordinates of the target in the view of the dual galvanometer camera; wherein the dual galvanometer camera includes a left galvanometer camera and a right galvanometer camera; The time-varying dynamic measurement model is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment; Based on the measurement results at this moment, target observation and motion estimation are performed, and the motion parameters of the dynamic compensation imaging algorithm and the optimal view control law are updated based on the estimation results.

2. The binocular stereo vision dynamic measurement method based on a dual-galvanometer camera according to claim 1, characterized in that, The time-varying dynamic measurement model is used to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment, including: The coordinates of the target in the three-dimensional world coordinate system are determined based on the coordinates of the image points of the target in the view of the dual-mirror camera; Observation variables and state variables are constructed using the target's coordinates in the three-dimensional world coordinate system; Kalman filtering is applied to the state variables and the observation variables to obtain the measurement results at the current moment, based at least on the image point coordinates of the target in the dual-mirror camera view and the measurement results at the previous moment.

3. The binocular stereo vision dynamic measurement method based on a dual-galvanometer camera according to claim 2, characterized in that, Determining the target's coordinates in the 3D world coordinate system based on the image point coordinates of the target in the dual-galvanometer camera view includes: Build a measurement model ; The measurement model is used to determine the coordinates of the target in the three-dimensional world coordinate system. ; in, Let be any non-zero coefficient. for The coordinates of the target's image point in the left galvanometer camera view at any given time. for The coordinates of the target's image point in the right galvanometer camera view at any given time. These are the internal parameters of the left galvanometer camera. These are the internal parameters of the right galvanometer camera. This represents the augmented internal parameter matrix of the left galvanometer camera. This represents the augmented intrinsic parameter matrix of the right galvanometer camera. for The transformation matrix from the target to the left galvanometer camera coordinate system at any given time. for The transformation matrix from the target to the right galvanometer camera coordinate system at any given time; Transformation matrix and The following method is used to determine: ,and ;in, This refers to the vertical rotation angle of the left galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the left galvanometer in the mirror parameters. This refers to the vertical rotation angle of the right galvanometer in the mirror parameters. This refers to the horizontal rotation angle of the right galvanometer in the mirror parameters; Here is the reflection transformation matrix. , It is a 3-order identity matrix. The normal vector of the mirror plane is determined by the rotation angle of the galvanometer. It is the transpose symbol; , , Let be any point on the mirror surface.

4. The binocular stereo vision dynamic measurement method based on a dual-galvanometer camera according to claim 3, characterized in that, Observed variables at time for , State variables at time 1 for ,in for The first derivative; Performing Kalman filtering on the state variables and the observed variables includes: Constructing the Kalman filter equation ; The state variable and the observed variable are filtered using the Kalman filter equation to obtain... The measurement results at time; in, Let be the prior state estimate at time t. Let be the posterior state estimate at time t-1. For process noise, To measure noise, Here is the state transition matrix. Let be the time interval between time t and time t-1. It is the identity matrix. This is the observation matrix.

5. The binocular stereo vision dynamic measurement method based on a dual-galvanometer camera according to claim 1, characterized in that, The optimal viewpoint control law is determined in the following manner: During the control process, the left galvanometer camera is kept always pointed at the target, and the pitch angles of the left and right galvanometer cameras are kept synchronized. The control law of the right galvanometer camera is determined. The control law for the left galvanometer camera is determined based on the center alignment principle; The optimal viewing angle control law is obtained by combining the control laws of the right galvanometer camera and the left galvanometer camera.

6. The method according to claim 5, characterized in that, The control law for the right galvanometer camera is determined as follows: Based on the target's coordinates in the current coordinate system of the left galvanometer camera Determine the target depth value ;in, ; Using the left galvanometer camera as the control reference, determine ;in, for Baseline value at time, For the left galvanometer At the turning point of time, Right galvanometer At the turning point of time, It is the cotangent function. , for x-coordinate for The coordinates of the target's image point in the right galvanometer camera view at any given time. for focal length in the middle, These are the internal parameters of the right galvanometer camera; based on Determine the reconstruction error ;in, for , , and Any parameter in, for Measurement error; Sure right The partial derivatives are Horizontal rotation angle of the right galvanometer at time ; Determined according to the principle of center alignment Vertical rotation angle of the right galvanometer at time , combined and The control law for the right galvanometer camera is obtained.

7. The binocular stereo vision dynamic measurement method based on a dual-galvanometer camera according to claim 1, characterized in that, Target tracking using a dynamic compensation imaging algorithm, based at least on the optimized mirror parameters, includes: The mirror parameter compensation value is calculated using a dynamic compensation imaging algorithm. The mirror parameter compensation value is calculated with the goal of controlling the galvanometer mirror to follow the target rotation during the exposure process so that the target is always imaged on the same image point. The optimized mirror parameters are compensated using the aforementioned mirror parameter compensation values; Target tracking is performed based at least on the compensated mirror parameters to obtain the image point coordinates of the target in the view of the dual-mirror camera.

8. A binocular stereo vision dynamic measurement device based on a dual-galvanometer camera, characterized in that, include: The optimization module is configured to acquire the mirror parameters of the galvanometer, control the mirror parameters using the optimal viewing angle control law, and obtain the optimized mirror parameters. The tracking module is configured to use a dynamic compensation imaging algorithm to track the target at least based on the optimized mirror parameters, and obtain the image point coordinates of the target in the view of the dual galvanometer camera; wherein the dual galvanometer camera includes a left galvanometer camera and a right galvanometer camera; The measurement module is configured to use a time-varying dynamic measurement model to obtain the measurement result at the current moment based at least on the image point coordinates of the target in the dual-galvanometer camera view and the measurement result at the previous moment. The update module is configured to perform target observation and motion estimation based on the measurement results at the current moment, and update the motion parameters of the dynamic compensation imaging algorithm and the optimal view control law based on the estimation results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images