Camera monitoring linear motion target compensation method based on galvanometer control

Abstract

The invention discloses a compensation method for monitoring a linear motion target by a camera based on galvanometer control. The method comprises the steps of performing initial angle adjustment on a galvanometer to enable a target to be located at a preset position in a camera view, then setting camera exposure time and continuously shooting target moving images; the image processor selects the first two images of the target moving to the specific area to perform block matching so as to obtain the angular velocity of the galvanometer; when the target moves to a specific area, sine wave signals are generated according to the angular velocity of the galvanometer and input to a galvanometer motion controller, and then the galvanometer is controlled to rotate through a galvanometer motor; the target continues to move linearly, and when the target moves into the specific area, a target moving image shot by the camera serves as a compensation image. According to the invention, a dynamic response sine of the galvanometer is used for scanning a target, the motion blur phenomenon caused by exposure time in a high-speed tracking image is solved, a sine wave signal is used for replacing an existing triangular wave signal to drive a galvanometer motor, and the smoothness and stability of rotation of the galvanometer are improved.

Description

Technical Field

[0001] The present invention relates to the field of galvanometer motion control, and particularly to a target motion compensation method based on galvanometer control. Background Art

[0002] When monitoring a target moving at high speed, traditional monitoring technologies such as cameras may not be able to quickly capture the target, and there may be a phenomenon of blurred images due to the size limitation of the sensor. Using high-precision monitoring equipment is costly and difficult to maintain. Therefore, a method is needed to compensate for target motion during monitoring and solve the difficulty of target motion blur in images.

[0003] For this reason, existing research has proposed compensating for blur by adding additional sensors and systems to follow the target, and by using an image sensor with time-delay integration to effectively extend the exposure time of the scanning camera. However, as the relative speed of the target increases, the cost of the sensor will gradually increase and the calculation is complex. Another method is to use the coupling optimization of magnetic field and machinery to improve the stability of the optical image, which is commonly used to prevent jitter caused by a handheld camera, but the compensation accuracy of this method is low and additional sensors are required for the system.

[0004] Existing methods for compensating blur through galvanometers usually adopt the method of presetting angles, that is, the galvanometer rotates to the corresponding angle in advance before shooting, rather than real-time dynamic adjustment. Such methods can only collect target information frame by frame and cannot effectively compensate for the continuous displacement of high-speed moving targets, resulting in motion blur still existing during the imaging process. Summary of the Invention

[0005] In order to solve the problems proposed in the background, the present invention proposes a compensation method for a camera to monitor a linearly moving target based on galvanometer control.

[0006] The present invention proposes that under real-time high-efficiency update, assuming that the one-dimensional motion velocity vector of the target remains unchanged, the system can ensure that the relative velocity between the target and the camera remains unchanged within the exposure time by the galvanometer moving at a certain angular velocity, so as to compensate for motion blur, improve the comprehensiveness of target state data, and is easy to deploy and maintain.

[0007] The technical solution adopted by the present invention is as follows: A compensation method for a camera to monitor a linearly moving target based on galvanometer control includes the following steps: S1. The light beam of the target is reflected by the galvanometer and incident on the camera for acquisition, and image processing is performed on the target motion image collected by the camera for tracking. At this time, the initial angle of the galvanometer is adjusted so that the target is in a preset position within the camera's field of view, and then the camera exposure time is set, and the camera continuously captures the target motion image; S2, the image processor selects the first two target motion images of the target moving to the specific area for block matching and then calculates the galvanometer angular velocity; S3, when the target moves to a specific area, a sine wave signal is generated according to the angular velocity of the galvanometer and input into the galvanometer motion controller, and the galvanometer motion controller controls the rotation of the galvanometer through the galvanometer motor; S4. The target continues to move in a straight line. When the target moves into a specific area, the target motion image captured by the camera is used as a compensation image. After the target moves out of the specific area, the galvanometer stops rotating and resets to the initial angle.

[0008] The imaging optical path of the camera is that the target enters the camera after being reflected by the galvanometer.

[0009] The camera exposure time in S1 is set according to the following formula: t ex <1 / 2f Among them, t ex represents the camera exposure time, and f represents the frequency of the galvanometer movement.

[0010] The S2 is specifically: S2.1. The image processor selects the first two target motion images when the target moves to a specific position and performs block matching according to the following formula to obtain the minimum offset pixel amount: R SSD (x d )=∑ j ∑ i (I 0 (i,j)-I 1 (i+x d ,j)) 2 x d min = argmin xd (R SSD (x d )) Among them, W ω Indicates the window width of block matching, I 0 (i,j) represents the grayscale value of the pixel in the i-th row and j-th column of the previous image. 1 (i+x d ,j) represents the i+xth d Gray value of pixel in row and column j, j=~1, i=~W ω -1, x d min Indicates the minimum offset pixel amount, x d Indicates the offset pixel amount, argmin xd ( ) indicates the x when the function takes the minimum value d The value of R SSD (xd ) represents the sum of the squares of the pixel differences between corresponding blocks of the previous image and the next image when the offset pixel amount is x d ; S2.2. Calculate the galvanometer angular velocity according to the minimum offset pixel amount according to the following formula: w m = 2tan -1 ((x d min tan(α / 2)) / (s w △t)) where w m represents the galvanometer angular velocity, s w represents the field of view width of the camera, x d min represents the minimum offset pixel amount, α represents the field of view angle of the camera, and △t represents the time interval between two images for performing block matching.

[0011] The sine wave signal in S3 is set according to the following formula: θ(t) = Asin(2πft) A = w m / (4f) where θ(t) represents the angle of the galvanometer at time t, w m represents the galvanometer angular velocity, A represents the amplitude of the sine wave, and f represents the frequency of the galvanometer movement.

[0012] The target moves in a straight line, and the specific area is the area on the preset target movement path where the camera is used to capture compensation pictures.

[0013] During the exposure time of the camera to capture the compensation image, the entire target is completely within the specific area.

[0014] The method adopts a galvanometer tracking compensation system, which includes an image processor, a camera, a galvanometer motion controller, a galvanometer motor, and a galvanometer. The galvanometer motion controller is electrically connected to the galvanometer motor, the galvanometer motor is connected to the rotation shaft of the galvanometer, the imaging optical path of the camera is that the target enters the camera after being reflected by the galvanometer, the camera is electrically connected to the image processor, and the image processor is used to process the target motion images captured by the camera to calculate the galvanometer angular velocity.

[0015] The beneficial effects of the present invention are: A target motion compensation method based on galvanometer control provided by the present invention uses a combination of a galvanometer and a camera to achieve real-time capture of a high-speed moving target, and uses the dynamic response of the galvanometer to sinusoidally scan the target, solving the motion blur phenomenon caused by the exposure time in the image of the high-speed moving target. Under this method, by performing a block matching method on consecutive images in the Bayer raw domain, calculating the pixel difference between adjacent frame block regions, and obtaining the amount of moving pixels of the target; calculating the relative angular velocity according to the camera field of view width and field of view angle, and updating it to the galvanometer angular velocity to ensure that the exposure timing is synchronized with the specified angle, and using a sine wave to control the galvanometer motion to monitor the target, solving the blur problem caused by the high-speed motion of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a linear motion target compensation method based on a galvanometer.

[0017] Figure 2 It is a flowchart of galvanometer control.

[0018] Figure 3 It is a schematic diagram of the block matching method in the Bayer domain.

[0019] Figure 4 It is a schematic diagram of controlling the galvanometer motion to monitor the target.

[0020] Figure 5 It is a schematic diagram of the galvanometer deflection angle waveform. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The present invention will be further described below in conjunction with the drawings and embodiments. The embodiments of the present invention include but are not limited to the following embodiments.

[0022] The embodiments of the present invention are as Figure 1 and Figure 2 shown, and include the following steps: S1. Adjust the initial angle of the galvanometer so that the target is in a preset position within the camera's field of view, and then set the camera exposure time. The camera continuously captures the target motion images when the target moves in a straight line at high speed.

[0023] The camera exposure time is set according to the following formula: t ex <1 / 2f where t ex represents the camera exposure time, and f represents the frequency of the galvanometer motion, which is the reciprocal of the period of the sine wave.

[0024] S2. The image processor selects the first two target motion images when the target moves to a specific area for Bayer block matching to calculate the galvanometer angular velocity.

[0025] AsFigure 3 As shown, when the target makes a one-dimensional movement relative to the camera, that is, the target only changes its position in the horizontal or vertical direction of the image, the dynamic response characteristics of the scanning galvanometer are used to control the deflection of the lens, so that the optical line of sight of the camera follows the center point of the target and captures the target, realizing the accurate information acquisition of the target. When capturing a target with continuously updated relative positions during one-dimensional movement, if the real-time performance of the system is high enough to keep the relative speed constant, that is, the relative speed remains unchanged within one exposure, then a compensation method for monitoring a linearly moving target by a camera based on galvanometer control can be proposed to extend the camera exposure time to compensate for motion blur.

[0026] By comparing the characteristic information of the target with the continuously updated information in the real-time continuously acquired images, the target speed can be obtained. Therefore, the block matching method is used to set a part of the target as the search window to achieve block matching in the target movement direction; due to the one-dimensional movement of the target, under this condition, only at least one row or one column needs to be allocated in the target part as the search window, thus effectively reducing the calculation cost. At the same time, to reduce the calculation amount and improve the execution speed, the Bayer Raw format image collected by the camera can be directly processed, and block matching is realized for two adjacent frames of images in the Bayer original domain. Based on the law that pixels are repeatedly presented as RGRGRG or GBGBGB in one-dimensional direction, the block matching of every two pixels can be realized through the following equation.

[0027] S2.1. The image processor selects the first two target motion images at the moment when the target moves to a specific position and performs Bayer block matching processing according to the following formula to obtain the minimum offset pixel amount: R SSD (x d ) = ∑ j ∑ i (I 0 (i, j) - I 1 (i + x d , j)) 2 x d min = argmin xd (R SSD (x d )) where W ω represents the window width of block matching, I 0 (i, j) represents the gray value of the pixel at the i-th row and j-th column in the previous image, I 1 (i + x d , j) represents the gray value of the pixel at the (i + x d )-th row and j-th column in the subsequent image, x d min represents the minimum offset pixel amount, x d represents the offset pixel amount, and argmin xd( ) represents the value of x when the function takes the minimum value d The value of R SSD (x d ) represents the sum of the squares of the pixel differences between the corresponding blocks of the previous image and the next image when the offset pixel amount is x d ; The previous image and the next image are relative to each other in the order of shooting based on two target motion images selected by the image processor.

[0028] Figure 4 It is a schematic diagram for controlling the galvanometer movement to monitor the target. The field of view width of the camera is s w , s w is determined by the field of view angle α of the camera and the distance L between the target and the camera; the galvanometer points in the direction of v m direction, and the relative angular velocity w r between the camera and the target within a very short time can be represented by v r , so w r can be substituted into w r . From the deflection time t m of the galvanometer to the end time t 1 , the galvanometer follows the moving target at an angular velocity w 3 and exposes the camera. From t m onwards, that is, after the image acquisition is completed, the galvanometer angle returns to the initial angle for the next shot. 4

[0029] To achieve galvanometer movement control and motion blur compensation, it is necessary to obtain the value of the relative angular velocity w r . According to x d calculate w r , we can get: s w / (2L)=tan(α / 2) (x d / △t) / (2L)=tan(w r / 2) Integrate to eliminate the unknown parameter L and solve for the relative angular velocity w r .

[0030] S2.2. Calculate and obtain the galvanometer angular velocity according to the minimum offset pixel amount using the following formula: w m =w r =2tan -1 ((x d min tan(α / 2)) / (s w △t)) where w m represents the galvanometer angular velocity, s wRepresents the field of view width of the camera, x d min Represents the minimum offset pixel amount, △t represents the time interval between two images for performing block matching, w r Represents the relative angular velocity between the camera and the target, and α represents the field of view angle of the camera.

[0031] The field of view width is the horizontal field of view width that the camera image can capture, expressed as the number of horizontal pixels, and the field of view angle is the horizontal range that the camera can capture, expressed as an angle.

[0032] Therefore, for a target in a plane, w can be calculated based on two consecutive frames of images r , and at the current time t (t 1 ≤t≤t 3 ), w m is equal to w r .

[0033] Figure 2 Is the process of controlling the galvanometer to track the target to compensate for motion blur. The relative angular velocity w between the camera and the target is calculated by the block matching method r . When the current angle of the galvanometer and the target position are within the tracking range, set the relative angular velocity w r as the galvanometer speed w m and expose the image.

[0034] S3. When the target moves to a specific area, a sine wave signal is generated according to the galvanometer angular velocity and input to the galvanometer motion controller. The galvanometer motion controller controls the galvanometer to rotate at a constant angular velocity through the galvanometer motor to achieve compensation for the target motion.

[0035] The frequency f and amplitude A of the traditional galvanometer control of the galvanometer are limited by its own weight. Therefore, the acceleration of the galvanometer is restricted by its own size. When maintaining a constant angular velocity, that is, w r and w m are kept consistent, at this time, the galvanometer rotation angle θ and the galvanometer angular velocity w m are linearly related and represented by a triangular wave: θ = w m t (t 1 ≤t≤t 3 ) However, due to the sharp instantaneous turning of the triangular wave, the galvanometer requires a very large acceleration, which is not conducive to the fast response and stability of the motor. Therefore, a sine wave with a frequency of f is used to control the galvanometer motion to approximate the triangular wave motion with the same amplitude.

[0036] The sine wave signal is set according to the following formula: θ(t) = Asin(2πft) A = w m / (4f) Among them, θ(t) represents the angle of the galvanometer at time t, and w m represents the angular velocity of the galvanometer, A represents the amplitude of the sine wave, specifically 1 / 2 of the positive and negative maximum angles of the forward and backward deflection when the galvanometer moves in a sine wave, and f represents the frequency of the galvanometer movement, which is the reciprocal of the period of the sine wave.

[0037] Figure 5 For the motion comparison between the triangular wave and the sine wave of the galvanometer, where t 1 and t 3 represent the time points of the deflection direction. In this embodiment, the camera is exposed within the time from (t 2 -t ex ) / 2 to (t 2 +t ex ) / 2. Under high-frequency response, the amplitude A is relatively small, making the difference between the sine wave and the triangular wave also significantly reduced. Therefore, the sine wave can be used to replace the triangular wave to control the galvanometer. At the same time, the sine wave has a relatively smooth acceleration and is approximately linear in the part far from the turning point, further improving the stability of the system and the accuracy of the image.

[0038] S4. The target continues to perform high-speed linear motion. When the target moves into a specific area, the target motion image captured by the camera is used as a compensation image. After the target moves out of the specific area, the galvanometer stops rotating and resets to the initial angle.

[0039] When the target moves to the specific area, specifically when the target just enters the specific area. When the target moves within the specific area, specifically when the entire target is within the specific area. When the target moves out of the specific area, specifically when the entire target leaves the specific area.

[0040] In the present invention, the imaging optical path of the camera is that the target enters the camera after being reflected by the galvanometer.

[0041] The target performs high-speed linear motion, and the specific area is the area on the preset target motion path where the camera is used to capture the compensation picture.

[0042] During the exposure time of the camera to capture the compensation image, the entire target is within the specific area.

[0043] The method adopts a galvanometer motion compensation system. The galvanometer motion compensation system includes an image processor, a camera, a galvanometer motion controller, a galvanometer motor, and a galvanometer. The galvanometer motion controller is electrically connected to the galvanometer motor, the galvanometer motor is connected to the rotating shaft of the galvanometer, the galvanometer motion controller drives the galvanometer motor to drive the galvanometer to rotate. The imaging optical path of the camera is that the target enters the camera after being reflected by the galvanometer. The camera is electrically connected to the image processor, and the image processor is used to process the target motion image captured by the camera to calculate the angular velocity of the galvanometer.

[0044] The innovation of the present invention lies in that when the camera captures a high-speed moving object, a galvanometer is used to rotate at an angular velocity matching the moving speed of the target object, thereby effectively compensating for the blurring problem caused by the movement of the target object and ensuring clear and stable imaging of the camera. At the same time, the present invention also uses a sine wave signal to replace the existing triangular wave signal to drive the galvanometer motor, improving the smoothness and stability of the galvanometer rotation, and thus further improving the imaging accuracy and reliability.

[0045] The above describes the present invention and its implementation manners. Such description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and, without departing from the gist of the present invention, design similar structural manners and embodiments to this technical solution without creative efforts, they shall fall within the protection scope of the present invention.

Claims

1. A camera monitoring linear motion target compensation method based on galvanometer control, characterized in that: The method comprises the following steps: S1, adjusting the initial angle of the galvanometer so that the target is at a preset position within the camera field of view, then setting the camera exposure time, and the camera continuously captures the target motion image; S2, the image processor selects the first two target motion images of the target moving to the specific area for block matching and then calculates the galvanometer angular velocity; S3, when the target moves to a specific area, a sine wave signal is generated according to the angular velocity of the galvanometer and input into the galvanometer motion controller, and the galvanometer motion controller controls the rotation of the galvanometer through the galvanometer motor; S4. The target continues to move in a straight line. When the target moves into a specific area, the target motion image captured by the camera is used as a compensation image. After the target moves out of the specific area, the galvanometer stops rotating and resets to the initial angle.

2. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The imaging optical path of the camera is that the target enters the camera after being reflected by the galvanometer.

3. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The camera exposure time in S1 is set according to the following formula: t ex <1 / 2f Among them, t ex represents the camera exposure time, and f represents the frequency of the galvanometer movement.

4. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The S2 is specifically: S2.

1. The image processor selects the first two target motion images when the target moves to a specific position and performs block matching according to the following formula to obtain the minimum offset pixel amount: R SSD (x d )=∑ j ∑ i (I0(i,j)-I1(i+x d ,j)) 2 x d min = argmin xd (R SSD (x d )) Among them, W ω represents the window width of the block matching, I0(i,j) represents the gray value of the pixel in the i-th row and j-th column of the previous image, and I1(i+x d ,j) represents the i+xth d Gray value of pixel in row and column j, j=~1, i=~W ω-1 , W ω-1 Represents the total number of rows in the image, x d min Indicates the minimum offset pixel amount, x d Indicates the offset pixel amount, argmin xd ( ) represents the x when the function takes the minimum value d The value of R SSD (x d ) means that the offset pixel amount is x d The sum of squares of pixel differences between the corresponding blocks in the previous image and the next image at the time t; S2.

2. Calculate the angular velocity of the galvanometer according to the following formula based on the minimum offset pixel amount: w m =2 years -1 ((x d min tan(α / 2)) / (s w △t)) Among them, w m represents the angular velocity of the galvanometer, s w Indicates the width of the camera's field of view, x d min represents the minimum offset pixel amount, α represents the field of view of the camera, and △t represents the time interval between the two images for performing block matching.

5. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The sine wave signal in S3 is set according to the following formula: θ(t)=Asin(2πft) A=w m / (4f) Among them, θ(t) represents the angle of the galvanometer at time t, w m represents the angular velocity of the galvanometer, A represents the amplitude of the sine wave, and f represents the frequency of the galvanometer movement.

6. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The target moves in a straight line, and the specific area is an area on a preset target motion path where the camera is used to take a compensation picture.

7. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: During the exposure time of the camera taking the compensation image, the entire target is within the specific area.

8. The method for compensating a linear motion target monitored by a camera based on galvanometer control according to claim 1, characterized in that: The method adopts a galvanometer tracking and compensation system, which includes an image processor, a camera, a galvanometer motion controller, a galvanometer motor and a galvanometer. The galvanometer motion controller is electrically connected to the galvanometer motor, and the galvanometer motor is connected to the rotating shaft of the galvanometer. The imaging light path of the camera is that the target enters the camera after being reflected by the galvanometer. The camera is electrically connected to the image processor, and the image processor is used to process the target motion image taken by the camera to calculate the galvanometer angular velocity.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images