Camera imaging system and method based on dual-axis galvanometer

Abstract

A camera imaging system and method based on a two-axis galvanometer, the system comprising a camera and a two-axis galvanometer unit composed of a pan mirror and a tilt mirror, the pan mirror rotating along the axis of the camera coordinate system, and the tilt mirror rotating along the axis of the camera coordinate system; further comprising a galvanometer driving unit for generating a control signal to drive the pan mirror and the tilt mirror to rotate at a set angle; the camera realizes two-dimensional scanning of a target through the pan mirror and the tilt mirror, and the related parameters of the pan mirror and the tilt mirror are optimized according to a mathematical model constructed according to the imaging process. The parameter optimization method comprises setting a coordinate system and determining galvanometer control parameters; constructing a mathematical model according to the imaging process; constructing a re-projection error minimization objective function, and optimizing the parameters involved in the mathematical model. The present application improves the performance of wide-field imaging and high-speed target tracking by constructing an accurate geometric reflection model and jointly optimizing the mirror angle and installation error parameters.

Description

Technical Field

[0001] This invention belongs to the field of optical imaging and precision control technology, specifically relating to a camera imaging system and method based on a dual-axis galvanometer. Background Technology

[0002] With the development of optical imaging technology and machine vision, modern camera systems are increasingly widely used in fields such as industrial inspection, autonomous driving, aerial remote sensing, robot vision, 3D reconstruction, and dynamic target tracking. However, the imaging range and field-of-view switching capabilities of traditional camera systems are limited by lens focal length, sensor size, and mechanical motion structures, making it difficult to achieve wide field of view, large-scale, and high-speed target observation while ensuring imaging accuracy.

[0003] To address the aforementioned issues, galvanometer (Galvo Scanner) technology was introduced into camera systems. A galvanometer is a precision optical device that uses electromagnetic force to drive a mirror to rotate at high speed. By adjusting the mirror angle, the direction of the light path can be quickly changed, thereby achieving wide-range optical scanning and field-of-view switching. A dual-axis galvanometer system (including a horizontal pan mirror and a vertical tilt mirror) can achieve rapid two-dimensional scanning without moving the camera body, offering advantages such as wide field-of-view coverage, high scanning speed, and short dynamic response time.

[0004] Although galvanometer cameras have shown great potential in wide field-of-view acquisition, dynamic target tracking, and multi-angle imaging, the existing technology still has the following shortcomings: (1) Lack of accurate geometric modeling methods: The optical path of the galvanometer camera involves multiple mirror reflections. Small deviations in mirror position, installation error, and rotation angle will significantly affect the imaging accuracy. The existing technology does not have a perfect accurate geometric model for galvanometer imaging. (2) Difficulty in calibration and optimization: The installation deviation and digital-to-analog conversion error of the galvanometer system are difficult to measure directly. The existing calibration methods cannot obtain mirror angle parameters and installation error parameters at the same time, resulting in insufficient calibration accuracy of the galvanometer camera and affecting the imaging quality. (3) Limited dynamic applications: The demand for high-speed and large-scale dynamic target imaging places higher demands on the response speed, control accuracy, and model calculation capability of the galvanometer system. The existing solutions are difficult to meet the needs of large scale, high precision, and real-time performance. Summary of the Invention

[0005] The purpose of this invention is to address the problems in the prior art by providing a camera imaging system and method based on a dual-axis galvanometer. By constructing an accurate geometric reflection model and jointly optimizing the mirror angle and installation error parameters, the performance of wide field-of-view imaging and high-speed target tracking can be improved.

[0006] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, a camera imaging system based on a dual-axis galvanometer is provided, comprising a camera and a dual-axis galvanometer unit consisting of a pan mirror and a tilt mirror; wherein the pan mirror is aligned with the camera coordinate system. The axis rotates, and the tilt mirror moves along the camera coordinate system. The camera rotates along the axis; it also includes a galvanometer drive unit, which generates control signals to drive the pan mirror and tilt mirror to rotate at a set angle; the camera achieves two-dimensional scanning of the target through the pan mirror and tilt mirror, and constructs a mathematical model based on the imaging process to optimize the relevant parameters of the pan mirror and tilt mirror.

[0007] As a preferred approach, the camera imaging system based on a dual-axis galvanometer is calibrated using a calibration board. The control signals for driving the pan mirror and the tilt mirror are simultaneously adjusted to 0, so that the calibration board covers the entire observation range of the camera. Parameter optimization is performed by minimizing reprojection error.

[0008] As a preferred embodiment, the calibration board is surrounded by a QR code generated by the ArUco toolkit for augmented reality marker detection and pose estimation from the OpenCV open-source computer vision and image processing library. The QR code represents a number from 0 to 7 and is used for spatial point localization and acquisition of the spatial coordinates of the center point. By recording multiple sets of spatial points and capturing images, a nonlinear optimization method is used to optimize the parameters.

[0009] Secondly, a parameter optimization method for a camera imaging system based on a dual-axis galvanometer is provided, including: Set the coordinate system and determine the galvanometer control parameters; Based on the established coordinate system and galvanometer control parameters, a mathematical model is constructed according to the imaging process. An objective function for minimizing reprojection error is constructed for the mathematical model, and the parameters of the camera imaging system based on the dual-axis galvanometer involved in the mathematical model are optimized based on the objective function for minimizing reprojection error.

[0010] As a preferred embodiment, in the step of setting the coordinate system and determining the galvanometer control parameters, the coordinate system includes: World coordinate system ; Ideal camera coordinate system Assuming no installation deviation; Real camera coordinate system This includes rotation and translation errors related to camera mounting.

[0011] As a preferred embodiment, in the step of setting the coordinate system and determining the galvanometer control parameters, the galvanometer control parameters include: the actual rotation angle of the pan mirror. The actual rotation angle of the tilt mirror ; Digital voltage controlling the rotation of the pan mirror Digital voltage controlling the rotation of the tilt mirror ; The linear scaling factor of the digital-to-analog conversion error of the pan-mirror control signal The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror control signal It conforms to the following relationship: ,

[0012] Ideal camera coordinate system Distance to the pan mirror ; The orthogonal distance between the pan mirror and the tilt mirror ; Ideal camera coordinate system Below, the intersection of the camera's central optical path and the pan lens. The intersection of the camera's central optical path and the tilt mirror It conforms to the following relationship: ,

[0013] The unit normal vector of the pan mirror The unit normal vector of the tilt mirror. The initial unit normal vectors of the pan and tilt mirrors conform to the following relationship: ,

[0014] pan mirror along the coordinate axis The axis rotates clockwise and counterclockwise, and the tilt mirror moves along the coordinate axis. The axis rotates clockwise and counterclockwise. World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix ; Ideal camera coordinate system To the real camera coordinate system Camera mounting error rotation matrix Ideal camera coordinate system To the real camera coordinate system Camera installation error translation matrix ; World coordinate system coordinates of the points in space below Ideal camera coordinate system coordinates of the points in space below Real camera coordinate system coordinates of the points in space below ; Camera intrinsic parameter matrix ; Camera distortion coefficient .

[0015] As a preferred embodiment, the step of constructing a mathematical model based on the established coordinate system and galvanometer control parameters, according to the imaging process, includes: The world coordinate system is set according to the following formula. coordinates of the points in space below Transform to ideal camera coordinate system coordinates of the points in space below :

[0016] The mathematical model for representing the virtual image formed by reflections through a tilt mirror and a pan mirror sequentially is constructed as follows: Around the tilt mirror The axis rotates, and the pan mirror revolves. During the rotation of the axis, the unit normal vector of the tilt mirror surface The unit normal vector of the mirror surface of the pan mirror The rotation matrix is ​​as follows:

[0017]

[0018] The unit normal vector of the tilting mirror The unit normal vector of the mirror surface of the pan mirror The mathematical expression obtained by rotating the initial mirror unit normal vector is: ,

[0019] According to the specular reflection formula, the ideal camera coordinate system coordinates of the points in space below After reflection by the tilt mirror, a virtual image point is obtained under the tilt mirror. The mathematical expression is as follows:

[0020] After reflection by the pan-lens, the virtual image point under the pan-lens is obtained. The mathematical expression is as follows: .

[0021] As a preferred embodiment, the step of constructing a mathematical model based on the established coordinate system and galvanometer control parameters according to the imaging process further includes analyzing the virtual image points under the pan-lens. Corrected based on camera installation errors, using the true camera coordinate system. virtual image point Calculate using the following formula:

[0022] Using the camera's intrinsic parameter matrix And the camera's distortion coefficient Transform the real camera coordinate system virtual image point Projecting onto the pixel plane, the expression is as follows:

[0023] In the formula, This represents the camera projection model.

[0024] As a preferred embodiment, the step of constructing the objective function for minimizing the reprojection error for the mathematical model includes the following mathematical expression for the objective function:

[0025] In the formula, For the observation image point, This represents the complete imaging mapping function from the world coordinate system to the image coordinate system, specifically, Indicates the optimization parameters Below, spatial point Projection point in camera image ; Optimize parameters Including: the linear scaling factor of the digital-to-analog conversion error of the pan-mirror control signal. The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror control signal Ideal camera coordinate system Distance to the pan mirror The orthogonal distance between the pan mirror and the tilt mirror World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix Ideal camera coordinate system To the real camera coordinate system Camera mounting error rotation matrix and ideal camera coordinate system To the real camera coordinate system Camera installation error translation matrix .

[0026] As a preferred embodiment, the step of optimizing the parameters of the camera imaging system based on the dual-axis galvanometer in the mathematical model based on the objective function of minimizing reprojection error includes: Assuming the control voltages of both the pan and tilt mirrors are zero and there are no installation errors, the camera imaging process is simplified to a pinhole model. Using feature points on the calibration plate, the world coordinate system is obtained through the camera's extrinsic parameter calibration method. To the ideal camera coordinate system rotation matrix and world coordinate system To the ideal camera coordinate system Translation matrix And obtain initial values ​​for other optimization parameters through physical measurements; Based on the initial values ​​of the optimized parameters, a nonlinear optimization algorithm is used to perform joint iterative optimization of all optimized parameters until the calculation result of the objective function for minimizing the reprojection error converges to the set threshold.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects: Existing technologies for geometric modeling of the optical path of a dual-axis galvanometer rely heavily on simplification assumptions, failing to adequately consider installation deviations of the pan and tilt mirrors, mirror position errors, and angle control errors. This results in insufficient imaging accuracy, making it difficult to meet the demands of high-precision applications. This invention provides a camera imaging system based on a dual-axis galvanometer, comprising a camera and a dual-axis galvanometer unit composed of a pan mirror and a tilt mirror. The pan mirror is positioned along the camera coordinate system... The axis rotates, and the tilt mirror moves along the camera coordinate system. The camera, rotating along its axis, performs a two-dimensional scan of the target using a pan mirror and a tilt mirror. A mathematical model is constructed based on the imaging process to optimize the relevant parameters of the pan and tilt mirrors. By building a precise geometric reflection model, the limitations of traditional simplistic assumptions are eliminated, and various error factors that may occur during the installation and operation of the pan and tilt mirrors, such as installation deviations, positional errors, and control errors, are comprehensively and meticulously considered. This precise modeling method allows the imaging process to more realistically reflect the actual situation, greatly improving imaging accuracy and providing reliable assurance for high-precision applications. The analog-to-digital conversion error, mirror installation error, and camera extrinsic parameter error of the galvanometer system are difficult to measure directly, and existing calibration methods cannot accurately solve for these errors simultaneously, resulting in decreased calibration accuracy and imaging stability of the galvanometer camera. This invention constructs a reprojection error minimization objective function for the mathematical model and adopts a method of jointly optimizing mirror angle and installation error parameters, which can simultaneously and accurately solve for multiple errors that are difficult to measure directly. In this way, the calibration accuracy of the galvanometer camera is effectively improved, ensuring the accuracy and consistency of various parameters during the imaging process, thereby significantly improving imaging stability. In practical applications, whether for wide-field-of-view imaging in complex environments or for tracking high-speed dynamic targets, this method ensures stable and reliable image quality, reducing image distortion and blurring caused by inaccurate calibration, and providing a high-quality data foundation for subsequent image analysis and processing. Existing technologies lack a unified optimization framework for dual-axis galvanometer cameras, failing to achieve rapid convergence while maintaining calibration accuracy, thus limiting the real-time application capabilities of galvanometer cameras in high-speed dynamic imaging scenarios. The parameter optimization method proposed in this invention constructs a complete and efficient unified optimization framework. By reasonably setting the coordinate system and determining the galvanometer control parameters, a foundation is laid for subsequent optimization. Based on the set coordinate system and galvanometer control parameters, a mathematical model is constructed according to the imaging process, accurately describing various relationships during imaging. An objective function for minimizing reprojection error is constructed for the mathematical model, and this objective function is used to optimize the parameters of the dual-axis galvanometer-based camera imaging system involved in the mathematical model. This optimization method can achieve rapid convergence while ensuring calibration accuracy, significantly shortening the parameter optimization time and significantly enhancing the dynamic response capability of the galvanometer camera in high-speed dynamic imaging scenarios. This invention effectively solves the problems of inaccurate geometric modeling, low calibration accuracy, and insufficient dynamic response capability of galvanometer cameras in the prior art by constructing an accurate geometric reflection model, jointly optimizing mirror parameters, and providing an efficient parameter optimization method. It comprehensively improves the performance of wide field-of-view imaging and high-speed target tracking, and is suitable for scenarios such as aerial remote sensing, industrial inspection, and robot vision. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram of a camera imaging system based on a dual-axis galvanometer according to an embodiment of the present invention; Figure 2 A schematic diagram of the calibration plate in an embodiment of the present invention; Figure 3 Flowchart of the camera imaging system parameter optimization method based on dual-axis galvanometers in this embodiment of the invention. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] In traditional galvanometer camera systems, the accuracy of geometric modeling is significantly affected by mirror position deviations, installation errors, and minute changes in rotation angles, leading to a systematic deviation between optical path calculations and the actual imaging position. The calibration process struggles to simultaneously acquire mirror angle parameters and installation error parameters, resulting in accumulated reprojection errors and reduced imaging accuracy. Furthermore, in dynamic target tracking scenarios, the galvanometer system's response speed and control precision are insufficient, causing image continuity loss during large-scale high-speed scanning and impacting the system's reliability in real-time applications. To address the problems of inaccurate geometric modeling, low calibration accuracy, and insufficient dynamic response capabilities in existing galvanometer camera systems, this invention proposes a camera imaging system based on a dual-axis galvanometer and a parameter optimization method.

[0032] Please see Figure 1 This embodiment is a camera imaging system based on a dual-axis galvanometer, including a camera 1 and a dual-axis galvanometer unit composed of a pan mirror 21 and a tilt mirror 22; wherein, the pan mirror 21 is aligned with the coordinate system of the camera 1. The axis rotates, and the tilt mirror 22 rotates along the coordinate system of camera 1. The shaft rotates; it also includes a galvanometer drive unit for generating control signals. , The camera 1 drives the pan mirror 21 and tilt mirror 22 to rotate at a set angle. In some possible implementations, a parameter calibration and optimization module may also be included, used to construct a geometric model and optimize relevant parameters based on the observation data of the camera 1. In this embodiment, the camera 1 achieves two-dimensional scanning of the target through the pan mirror 21 and tilt mirror 22, and constructs a mathematical model based on the imaging process to optimize the relevant parameters of the pan mirror 21 and tilt mirror 22.

[0033] Please see Figure 2 In one possible implementation, this embodiment of the invention calibrates the camera imaging system based on a dual-axis galvanometer using a calibration board. The control signals driving the pan mirror 21 and tilt mirror 22 are simultaneously set to 0, returning the pan mirror 21 and tilt mirror 22 to their default mechanical stop positions. This ensures the calibration board covers the entire observation range of camera 1, and parameter optimization is performed by minimizing reprojection error. In this embodiment, the calibration board is surrounded by a ring of QR codes generated by the ArUco toolkit (OpenCV), an open-source computer vision and image processing library for augmented reality marker detection and pose estimation. These QR codes, ranging from 0 to 7, are used for spatial point localization and obtaining the spatial coordinates of the center point. The QR codes generated by the ArUco toolkit are visual markers with unique patterns and identifiers, designed to enable efficient and robust detection and recognition by computer vision algorithms. These QR codes can serve as feature points on the calibration board, providing accurate image coordinates. Parameter optimization is performed using a nonlinear optimization method by recording multiple sets of spatial points and capturing images. Specifically, the system places the calibration plate within different fields of view of camera 1 at different galvanometer angles (i.e., different rotational positions of pan mirror 21 and tilt mirror 22) and captures multiple images. For each image, the pixel coordinates of detected feature points (such as corner points or center points marked by ArUco) are recorded, and a correspondence is established between these feature points and their 3D spatial coordinates in the real world. Finally, nonlinear optimization methods are used for parameter optimization. Nonlinear optimization methods are iterative algorithms used to minimize a nonlinear objective function, such as reprojection error. In camera and galvanometer calibration, the goal is to find a set of camera intrinsic and extrinsic parameters and galvanometer parameters that minimizes the distance (i.e., reprojection error) between all observed image points and their corresponding 3D spatial points reprojected onto the image plane using the current parameters. Commonly used nonlinear optimization algorithms include the Levenberg-Marquardt algorithm or the Gauss-Newton method, which can effectively handle complex nonlinear models, thereby obtaining high-precision parameter estimates.

[0034] Please see Figure 3 Another embodiment of the present invention proposes a parameter optimization method for a camera imaging system based on a dual-axis galvanometer, which mainly includes the following steps: S1. Set the coordinate system and determine the galvanometer control parameters; S2. Based on the set coordinate system and galvanometer control parameters, construct a mathematical model according to the imaging process; S3. Construct a reprojection error minimization objective function for the mathematical model, and optimize the parameters of the camera imaging system based on the dual-axis galvanometer involved in the mathematical model based on the reprojection error minimization objective function.

[0035] In one possible implementation, the coordinate system set in step S1 includes: World coordinate system ; Ideal camera coordinate system Assuming no installation deviation; Real camera coordinate system This includes minute rotation and translation errors due to the installation of camera 1.

[0036] In one possible implementation, step S1 determines the following galvanometer control parameters: The actual rotation angle of pan mirror 21 The actual rotation angle of tilt mirror 22 ; Digital voltage controlling the rotation of pan mirror 21 Digital voltage controlling the rotation of tilt mirror 22 ; The linear proportionality coefficient of the digital-to-analog conversion error of the control signal of pan mirror 21 The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror 22 control signal It conforms to the following relationship: ,

[0037] Ideal camera coordinate system Distance to Pan Mirror 21 ; The orthogonal distance between pan mirror 21 and tilt mirror 22 ; Ideal camera coordinate system Below, the intersection of the central optical path of camera 1 and pan lens 21 The intersection of the central optical path of camera 1 and tilt mirror 22 It conforms to the following relationship: ,

[0038] The unit normal vector of the mirror surface of pan mirror 21 The unit normal vector of the mirror surface of tilt mirror 22 The initial unit normal vectors of the pan mirror 21 and the tilt mirror 22 conform to the following relationship: ,

[0039] pan mirror 21 along the coordinate axis The axis rotates clockwise and counterclockwise, and the tilt mirror 22 moves along the coordinate axis. The axis rotates clockwise and counterclockwise. World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix ; Ideal camera coordinate system To the real camera coordinate system Camera 1 installation error rotation matrix Ideal camera coordinate system To the real camera coordinate system Camera 1 installation error translation matrix ; World coordinate system coordinates of the points in space below Ideal camera coordinate system coordinates of the points in space below Real camera coordinate system coordinates of the points in space below ; Intrinsic parameter matrix of camera 1 ; Distortion coefficient of camera 1 .

[0040] In one possible implementation, step S2 constructs a mathematical model based on the imaging process, including: The world coordinate system is set according to the following formula. coordinates of the points in space below Transform to ideal camera coordinate system coordinates of the points in space below :

[0041] The mathematical model is constructed as follows to represent the virtual image formed by reflections sequentially through tilt mirror 22 and pan mirror 21: Around the tilt mirror 22 The axis rotates, and the pan mirror 21 rotates. During the rotation of the axis, the unit normal vector of the mirror surface of tilt mirror 22 The unit normal vector of the mirror surface of pan mirror 21 The rotation matrix is ​​as follows:

[0042]

[0043] The unit normal vector of the tilt mirror 22 The unit normal vector of the mirror surface of pan mirror 21 The mathematical expression obtained by rotating the initial mirror unit normal vector is: ,

[0044] According to the specular reflection formula, the ideal camera coordinate system coordinates of the points in space below First, the image is reflected by tilt mirror 22 to obtain the virtual image point under tilt mirror 22. The mathematical expression is as follows:

[0045] After reflection by pan mirror 21, a virtual image point is obtained under pan mirror 21. The mathematical expression is as follows: .

[0046] In one possible implementation, the virtual image formed after reflection by the pan mirror 21 and tilt mirror 22 is currently directly in front of the camera. However, considering installation errors, there are generally slight rotational and translational variations. Therefore, step S2 also includes adjusting the virtual image point under the pan mirror 21. Corrected based on camera 1 installation error, actual camera coordinate system. virtual image point Calculate using the following formula:

[0047] Using the intrinsic parameter matrix of camera 1 Distortion coefficients of camera 1 Transform the real camera coordinate system virtual image point Projecting onto the pixel plane, the expression is as follows: .

[0048] In the formula, This represents the camera projection model.

[0049] In one possible implementation, the objective function for minimizing the reprojection error constructed in step S3 is expressed as follows:

[0050] In the formula, For the observation image point, This represents the complete imaging mapping function from the world coordinate system to the image coordinate system, specifically, Indicates the optimization parameters Below, spatial point Projection point in camera image ; Optimize parameters Includes: linear scaling factor for the digital-to-analog conversion error of the control signal of pan mirror 21 The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror 22 control signal Ideal camera coordinate system Distance to Pan Mirror 21 The orthogonal distance between pan mirror 21 and tilt mirror 22 World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix Ideal camera coordinate system To the real camera coordinate system Camera 1 installation error rotation matrix and ideal camera coordinate system To the real camera coordinate system Camera 1 installation error translation matrix .

[0051] Furthermore, in step S3 of this embodiment of the invention, optimizing the parameters of the camera imaging system based on the dual-axis galvanometer in the mathematical model based on the objective function of minimizing reprojection error includes: With the control voltages of both pan mirror 21 and tilt mirror 22 at zero, assuming no installation error, the camera imaging process is simplified to a pinhole model; using the feature points of the calibration plate, the world coordinate system is obtained through the external parameter calibration method of camera 1. To the ideal camera coordinate system rotation matrix and world coordinate system To the ideal camera coordinate system Translation matrix And obtain initial values ​​for other optimization parameters through physical measurements; Based on the initial values ​​of the optimized parameters, nonlinear optimization algorithms such as the Levenberg-Marquardt method are used to jointly iteratively optimize all parameters until the calculation result of the objective function that minimizes the reprojection error converges to a set threshold. The Levenberg-Marquardt method is an optimization algorithm for solving nonlinear least squares problems, combining the advantages of gradient descent and Gauss-Newton methods. During the iteration process, it dynamically adjusts the search direction by introducing a damping factor (also called a damping parameter). When the damping factor is large, the algorithm approximates the gradient descent method and has good global convergence; when the damping factor is small, the algorithm approximates the Gauss-Newton method and has a faster local convergence speed.

[0052] Another embodiment of the present invention, a camera imaging system based on a dual-axis galvanometer, specifically includes: Camera 1: Model MV-CS020-10UC. MV-CS020-10UC is a 2-megapixel industrial area scan camera from Hikvision with a 1 / 1.7-inch CMOS sensor and USB 3.0 interface. It features high frame rate, low power consumption, and rich functionality, and is suitable for various industrial vision scenarios. The resolution is 1624×1240, the focal length is 55mm, and the intrinsic parameter matrix K is known.

[0053] Galvanometer unit: Composed of pan mirror 21 and tilt mirror 22, pan mirror 21 rotates around The axis rotates, and the tilt mirror 22 rotates. Axis rotation; Galvanometer drive unit: capable of outputting control voltage , ; Parameter calibration and optimization module: includes an external parameter calibration unit and a nonlinear optimization module.

[0054] The dual-axis galvanometer-based camera imaging system was calibrated using a calibration board. A QR code surrounding the calibration board, generated by the OpenCV ArUco toolkit, consists of numbers from 0 to 7 and is used for spatial point localization and obtaining the spatial coordinates of the center point. Furthermore, the parameter optimization method for the dual-axis galvanometer-based camera imaging system is as follows: Step 1: Adjust the galvanometer control voltage , At the same time, it is 0, so that the dot part falls completely within the camera's field of view.

[0055] The following can be calculated using PNP:

[0056]

[0057] Simultaneously, estimated initial values ​​for other parameters were obtained through measurement: d = 50 mm, e = 10 mm. , , .

[0058] Step 2: Move the calibration plate to cover as much of the galvanometer camera's observation range as possible, ensuring the dot image is clear and completely falls within the galvanometer camera's field of view. Record multiple sets of spatial points. And galvanometer images. Then, a nonlinear optimization method is used to minimize the reprojection error and optimize all parameters.

[0059] Step 3: The reprojection error is 2.6385 pixels.

[0060] This invention relates to a parameter optimization method for camera imaging systems based on dual-axis galvanometers. By combining geometric modeling and parameter optimization, it achieves high-precision, large field of view, and high-speed dynamic imaging, which is applicable to scenarios such as aerial remote sensing, industrial inspection, and robot vision.

[0061] The camera imaging system and parameter optimization method based on dual-axis galvanometers of this invention can effectively solve the problems of low geometric modeling and calibration accuracy and insufficient dynamic response capability in the existing technology, and realize high-precision, large field of view and high-speed dynamic imaging, which is suitable for scenarios such as airborne remote sensing, industrial inspection and robot vision.

[0062] Another embodiment of the present invention provides an electronic device including a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the parameter optimization method of the camera imaging system based on a dual-axis galvanometer described in the present invention.

[0063] Another embodiment of the present invention provides a computer-readable storage medium storing at least one instruction that, when executed by a processor, implements the parameter optimization method for the camera imaging system based on a dual-axis galvanometer as described in the present invention.

[0064] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals. For ease of explanation, the above content only shows the parts related to the embodiments of the present invention; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This computer-readable storage medium is non-transitory and can be stored in storage devices formed by various electronic devices, enabling the execution process described in the method of the embodiments of the present invention.

[0065] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0067] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0068] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A camera imaging system based on a dual-axis galvanometer, characterized in that, Includes a camera (1) and a dual-axis galvanometer unit consisting of a pan mirror (21) and a tilt mirror (22); wherein the pan mirror (21) is aligned with the coordinate system of the camera (1). The axis rotates, and the tilt mirror (22) moves along the coordinate system of the camera (1). The axis rotates; it also includes a galvanometer drive unit, which generates control signals to drive the pan mirror (21) and tilt mirror (22) to rotate at a set angle; the camera (1) realizes two-dimensional scanning of the target through the pan mirror (21) and tilt mirror (22), and constructs a mathematical model based on the imaging process to optimize the relevant parameters of the pan mirror (21) and tilt mirror (22).

2. The camera imaging system based on a dual-axis galvanometer according to claim 1, characterized in that, The camera imaging system based on the dual-axis galvanometer is calibrated by using a calibration plate. The control signals of the driving pan mirror (21) and tilt mirror (22) are adjusted to be 0 at the same time so that the calibration plate covers the entire observation range of the camera (1). The parameters are optimized by minimizing the reprojection error.

3. The camera imaging system based on a dual-axis galvanometer according to claim 1, characterized in that, The calibration board is surrounded by a QR code generated by the ArUco toolkit for augmented reality marker detection and pose estimation from the OpenCV open-source computer vision and image processing library. The QR code represents a number from 0 to 7 and is used for spatial point localization and acquisition of the spatial coordinates of the center point. By recording multiple sets of spatial points and capturing images, a nonlinear optimization method is used to optimize the parameters.

4. A parameter optimization method for a camera imaging system based on a dual-axis galvanometer as described in any one of claims 1 to 3, characterized in that, include: Set the coordinate system and determine the galvanometer control parameters; Based on the established coordinate system and galvanometer control parameters, a mathematical model is constructed according to the imaging process. An objective function for minimizing reprojection error is constructed for the mathematical model, and the parameters of the camera imaging system based on the dual-axis galvanometer involved in the mathematical model are optimized based on the objective function for minimizing reprojection error.

5. The parameter optimization method according to claim 4, characterized in that, In the step of setting the coordinate system and determining the galvanometer control parameters, the coordinate system includes: World coordinate system ; Ideal camera coordinate system Assuming no installation deviation; Real camera coordinate system , including the rotation and translation errors of the camera (1) installation.

6. The parameter optimization method according to claim 5, characterized in that, In the step of setting the coordinate system and determining the galvanometer control parameters, the galvanometer control parameters include: The actual rotation angle of the pan mirror (21) The actual rotation angle of the tilt mirror (22) ; Digital voltage controlling the rotation of pan mirror (21) Digital voltage controlling the rotation of tilt mirror (22) ; The linear proportionality coefficient of the digital-to-analog conversion error of the control signal of the pan mirror (21) The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror (22) control signal It conforms to the following relationship: ， Ideal camera coordinate system Distance to pan mirror (21) ; The orthogonal distance between the pan mirror (21) and the tilt mirror (22) ; Ideal camera coordinate system Below, the intersection of the central optical path of the camera (1) and the pan lens (21) The intersection of the central optical path of the camera (1) and the tilt mirror (22) It conforms to the following relationship: ， The unit normal vector of the pan mirror (21) The unit normal vector of the tilt mirror (22) The initial unit normal vectors of the pan mirror (21) and the tilt mirror (22) conform to the following relationship: ， pan mirror (21) along the coordinate axis The axis rotates clockwise and counterclockwise, and the tilt mirror (22) moves along the coordinate axis. The axis rotates clockwise and counterclockwise. World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix ; Ideal camera coordinate system To the real camera coordinate system Camera (1) Installation error rotation matrix Ideal camera coordinate system To the real camera coordinate system Camera (1) Installation error translation matrix ; World coordinate system coordinates of the points in space below Ideal camera coordinate system coordinates of the points in space below Real camera coordinate system coordinates of the points in space below ; Intrinsic parameter matrix of camera (1) ; Distortion coefficient of camera (1) .

7. The parameter optimization method according to claim 6, characterized in that, The steps of constructing a mathematical model based on the established coordinate system and galvanometer control parameters, according to the imaging process, include: The world coordinate system is set according to the following formula. coordinates of the points in space below Transform to ideal camera coordinate system coordinates of the points in space below : The mathematical model is constructed as follows to represent the virtual image formed by reflections through the tilt mirror (22) and the pan mirror (21) in sequence: Around the tilt mirror (22) The axis rotates, and the pan mirror (21) revolves. During the rotation of the axis, the unit normal vector of the tilt mirror (22) The unit normal vector of the mirror surface of the pan mirror (21) The rotation matrix is ​​as follows: The unit normal vector of the tilt mirror (22) The unit normal vector of the mirror surface of the pan mirror (21) The mathematical expression obtained by rotating the initial mirror unit normal vector is: ， According to the specular reflection formula, the ideal camera coordinate system coordinates of the points in space below The virtual image point under the tilt mirror (22) is obtained after reflection. The mathematical expression is as follows: The virtual image point under the pan mirror (21) is obtained after reflection. The mathematical expression is as follows: 。 8. The parameter optimization method according to claim 7, characterized in that, The step of constructing a mathematical model based on the set coordinate system and galvanometer control parameters according to the imaging process also includes the virtual image points under the pan mirror (21). Correction is performed based on the camera (1) installation error, and the actual camera coordinate system is used. virtual image point Calculate using the following formula: Using the intrinsic parameter matrix of camera (1) And the distortion coefficient of camera (1) Transform the real camera coordinate system virtual image point Projecting onto the pixel plane, the expression is as follows: In the formula, This represents the camera projection model.

9. The parameter optimization method according to claim 7, characterized in that, The step of constructing the objective function for minimizing the reprojection error for the mathematical model is described below. The mathematical expression of the objective function for minimizing the reprojection error is as follows: In the formula, For the observation image point, This represents the complete imaging mapping function from the world coordinate system to the image coordinate system, specifically, Indicates the optimization parameters Below, spatial point Projection point in camera image ; Optimize parameters Includes: linear scaling factor of the control signal digital-to-analog conversion error of the pan mirror (21) The linear proportionality coefficient of the digital-to-analog conversion error of the tilt mirror (22) control signal Ideal camera coordinate system Distance to pan mirror (21) The orthogonal distance between the pan mirror (21) and the tilt mirror (22) World coordinate system To the ideal camera coordinate system rotation matrix World coordinate system To the ideal camera coordinate system Translation matrix Ideal camera coordinate system To the real camera coordinate system Camera (1) Installation error rotation matrix and ideal camera coordinate system To the real camera coordinate system Camera (1) Installation error translation matrix .

10. The parameter optimization method according to claim 9, characterized in that, The steps for optimizing the parameters of the dual-axis galvanometer-based camera imaging system in the mathematical model based on the objective function of minimizing reprojection error include: Assuming that there is no installation error, the camera imaging process is simplified to a pinhole model when the control voltages of both the pan mirror (21) and the tilt mirror (22) are zero. Using the feature points of the calibration plate, the world coordinate system is obtained through the external parameter calibration method of the camera (1). To the ideal camera coordinate system rotation matrix and world coordinate system To the ideal camera coordinate system Translation matrix And obtain initial values ​​for other optimization parameters through physical measurements; Based on the initial values ​​of the optimized parameters, a nonlinear optimization algorithm is used to perform joint iterative optimization of all optimized parameters until the calculation result of the objective function for minimizing the reprojection error converges to the set threshold.

Images