A step-by-step autofocus method

Abstract

The present invention belongs to the field of visual camera focusing technology, and specifically relates to a step-by-step automatic focusing method, including the first step of focusing: determining the initial moving range of the image plane, moving the image plane to change the image distance, shooting the target image after each movement, calculating the clarity of the target image, and moving a certain distance to the left and right with the image distance of the target image with a target and high clarity as the center to form an initial single valley interval; the second step of focusing: using the Fibonacci method to cyclically reduce the initial single valley interval until the single valley interval meets the accuracy requirements; the third step of focusing: moving the image plane to use the image distance of the local target image with the highest clarity as the final image distance to complete the camera focusing. The present invention uses the image plane to make the imaging clear, and combines the Laplacian operator, the neural network and the Fibonacci method to determine the final clarity image distance, so that the target photo taken by the camera is the clearest.

Description

Technical Field

[0001] The invention belongs to the technical field of visual camera focusing, and in particular relates to a step-by-step automatic focusing method. Background Art

[0002] In view of the demand for high-precision aviation hole making, the research on high-precision robot posture measurement technology based on monocular vision is particularly important; posture measurement includes camera calibration - target design - image acquisition and processing - posture solution, and the Zhang Zhengyou calibration method is used to obtain the camera internal parameters, so as to establish the corresponding relationship model between the coordinates of the three-dimensional world point and the two-dimensional pixel point of the object, and in the subsequent steps, the designed target is photographed to solve the spatial coordinates of the target. Among them, obtaining clear images is particularly important for the posture measurement results.

[0003] At present, the commonly used methods for obtaining clear images are ranging method and focus detection method. The ranging method measures the distance between the target object and the lens, that is, the object distance, through infrared, laser, ultrasonic and other equipment, and then calculates the corresponding image distance according to the Gaussian imaging formula, and finally adjusts the image distance to complete the automatic focusing of the system. However, since the focusing process of this method requires the camera to actively transmit a signal and then receive the signal reflected by the target object, if this signal is absorbed or scattered by the target object, the signal received by the receiving end is relatively weak or even no signal is received, so the object distance cannot be accurately obtained, which will lead to focusing failure. At the same time, the auxiliary signal sending and receiving device required by this method will increase the hardware volume of the entire imaging system, increase energy consumption, and the system complexity is high, and it is easily affected by the external environment.

[0004] Focus detection methods include contrast detection and phase difference detection. In contrast detection, a sensor is placed at two locations with equal distance from the focal plane. The sensor can collect image contrast information and then control the motor to move the focal plane to obtain a clear image. Phase difference detection mainly relies on autofocus sensors that detect phase difference to make the image clear. Both are autofocus methods designed based on the camera hardware itself, and require the operator to check the camera focus screen at all times to determine whether the camera is focused on the target object, and do not achieve true "autofocus". Summary of the invention

[0005] The object of the present invention is to provide a step-by-step autofocus method, which utilizes an image plane to make the image clear, and determines the final photographing image distance by combining a Laplacian operator, a neural network and a Fibonacci method, so that the target photo taken by the camera is the clearest.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A step-by-step auto-focusing method comprises the following steps:

[0008] The first step is focusing: initialize the camera, determine the initial moving range of the image plane, set the initial moving interval, move the image plane so that the image distance moves from the minimum value of the initial moving range to the maximum value, and the distance of each movement is the initial moving interval. After each movement, the camera takes a target image and calculates the clarity of the target image. When the clarity fluctuates from low to high to low, a neural network model is used to detect the target image with high clarity. If there is no target in the target image, continue to move until the clarity fluctuates from low to high to low, and then perform target detection again until there is a target in the target image. The initial single valley interval [v 1 ,v 3 ];

[0009] The second step is to focus: use the Fibonacci method to narrow the initial single valley interval;

[0010]

[0011] In the formula, v 2 、v 4 is the reduced image distance value, v 1 、v 3 is the image distance endpoint value of the initial single valley interval, F 1 =F 2 =1, F n+2 =F n+1 +F n , F n is the Fibonacci number;

[0012] Move the image plane so that the image distance is v 2 and v 4 , take a target image, calculate the clarity of the two target images, and compare the two clarity. If g(v 2 )>g(v 4 ), then the reduced single valley interval is [v 1 , v 4 ]; if g(v 2 )<g(v 4 ), then the reduced single valley interval is [v 2 , v 3 ]; if g(v 2 )=g(v 4 ), then the reduced single valley interval is [v 2 , v 4 ]; where g(v 2 )、g(v 4 ) are respectively the image distance v 2 、v4 The clarity of the target image captured at the time;

[0013] The shrunken single valley interval is further shrunken using the Fibonacci method until the length of the single valley interval is less than the threshold ε, and it is determined as the final single valley interval;

[0014] The third step is focusing: move the image plane so that the image distance moves from the minimum value to the maximum value of the final single valley interval, and move a distance m each time, and take an image after each movement; use a neural network model to detect the target on the image, and obtain the local image of the target, calculate the clarity of the local image of the target, and obtain the image distance with the highest clarity as the final image distance to complete the camera focusing.

[0015] Furthermore, the initial moving range of the image plane is determined according to the relationship between the focal length, the object distance and the image distance during the camera imaging process.

[0016] Furthermore, the clarity is calculated using a Laplacian operator, the Laplacian operator performs convolution on the target image channel, and then calculates the variance of the output, and the larger the variance obtained, the clearer the image.

[0017] Furthermore, the neural network model adopts a YOLO v9 network or a Swin Transformer network.

[0018] The present invention firstly detects the clarity of the image using the Laplacian operator to obtain a single valley interval of "low-high-low" clarity, then uses a neural network to perform the first target detection to detect whether the camera is aimed at the target or other objects, and uses the Fibonacci method to narrow the single valley interval, with high accuracy, high efficiency and full automation, to complete automatic target alignment and obtain a clear photo of the target.

[0019] The autofocus method and target detection in the present invention can be integrated into camera software after being lightweight, so as to achieve more efficient, more accurate and more automated autofocus. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the automatic focusing flow chart of the present invention.

[0021] Figure 2 Schematic diagram of the relationship between the focal length, object distance and image distance of the camera.

[0022] Figure 3 It is a schematic diagram of the relationship between clarity and image distance fitting of the present invention.

[0023] Figure 4 Target image taken for the present invention. DETAILED DESCRIPTION

[0024] like Figure 1As shown, a step-by-step auto-focusing method provided in this embodiment includes the following steps:

[0025] The first step is focusing: initializing the camera, based on the relationship between focal length, object distance and image distance during camera imaging Determine the initial movement range of the image plane, such as Figure 2 As shown, f is the focal length, u is the object distance, and v is the image distance. In the actual imaging process, the object distance u is greater than 2f, and the image distance v is between [f, 2f].

[0026] Set the moving interval d, move the image plane, and make the image distance move from f. After the movement, take a target image. Since there is a circular directional reflective mark on the head of the target, this embodiment uses the Laplacian operator to calculate the clarity of the target image. Move another distance d to take a target image until the clarity fluctuates from "low-high-low". Perform target detection on the target image at the "high" position to detect whether there is a complete target. If there is no target, continue to move until the next "low-high-low" fluctuation occurs. Perform target detection again until a complete target image appears in the target image at the "high" position. Move a certain distance left and right with the image distance of the complete target image as the center to obtain a single valley interval [v 1 , v 3 ].

[0027] The Laplacian operator performs convolution on all target image channels and then calculates the variance of the output. The variance result retains integer bits. The larger the variance, the clearer the image. The relationship between image distance and clarity is simulated according to the clarity, as shown in Figure 3 The target detection is performed on the image with the peak clarity to detect whether there is a complete target image in the image. If there is a complete target image, the image distance corresponding to the clear image is considered to be the clarity image distance. The image distance is moved left and right according to the set threshold to obtain the initial single valley interval [v 1 , v 3 ].

[0028] The target detection is performed using a neural network model, and the neural network model uses a YOLO v9 network or a Swin Transformer network, or other neural network models capable of target detection.

[0029] The second step is to focus: use the Fibonacci method to narrow the initial single valley interval;

[0030]

[0031] In the formula, v 2 、v 4 is the reduced image distance value, v 1 、v 3is the image distance endpoint value at both ends of the initial single valley interval, F 1 =F 2 =1, F n+2 =F n+1 +F n , F n is the Fibonacci number.

[0032] The moving image plane is the image distance v 2 and v 4 , take a target image, use the Laplacian operator to calculate the clarity of the two target images, and compare the two clarity. If g(v 2 )>g(v 4 ), then the reduced single valley interval is [v 1 , v 4 ]; if g(v 2 )<g(v 4 ), then the reduced single valley interval is [v 2 , v 3 ]; if g(v 2 )=g(v 4 ), then the reduced single valley interval is [v 2 , v 4 ]; where g(v 2 )、g(v 4 ) are respectively the image distance v 2 、v 4 The clarity of the target image captured.

[0033] The shrunken single valley interval is further shrunken using the Fibonacci method until the length of the single valley interval is less than the threshold ε, and the final single valley interval is determined.

[0034] The third step is focusing: take the left endpoint of the final single valley interval as the initial value, move the image plane, each moving distance is m, take an image after each movement, until the end value of the final single valley interval is reached, use the neural network model to detect the position of the target in all images, and obtain the local image of the target, use the Laplacian operator to calculate the clarity of the local image of the target, and obtain the image distance when the clarity of the local image of the target is the highest as the final image distance to complete the camera automatic focus.

[0035] The step-by-step autofocus method adopted in this embodiment is applicable to an aviation assembly robot posture measurement system based on zoom monocular vision to achieve posture measurement of the aviation assembly robot; Figure 4 As shown, the image is taken using the method of this embodiment.

[0036] The above description is only a preferred implementation manner of the present invention, but the protection scope of the present invention is not limited thereto, and any modification and replacement based on the technical solution and inventive concept provided by the present invention should be included in the protection scope of the present invention.

Claims

1. A step-by-step autofocus method, characterized in that: It includes the following steps: The first step of focusing: Initialize the camera, determine the initial moving range of the image plane, set the initial moving interval, move the image plane so that the image distance moves from the minimum value to the maximum value of the initial moving range, and the distance of each movement is the initial moving interval. After each movement, the camera takes a target image, calculates the sharpness of the target image. When the sharpness shows a low-high-low fluctuation, perform target detection on the target image with high sharpness using a neural network model. If there is no target in the target image, continue to move until the next low-high-low fluctuation of the sharpness appears, and perform target detection again until there is a target in the target image. Take the image distance of the target image with high sharpness as the center and move a certain distance left and right to form an initial single-valley interval [v1, v3]; The second step of focusing: Use the Fibonacci method to narrow the initial single-valley interval; In the formula, v2 and v4 are the reduced image distance values, v1 and v3 are the image distance endpoint values ​​of the initial single valley interval, n≥1, F1=F2=1, F n+2 =F n+1 +F n , F n is the Fibonacci number; Move the image plane so that the image distances are v2 and v4, take target images, calculate the sharpness of the two target images, compare the two sharpness values. If g(v2) > g(v4), the narrowed single-valley interval is [v1, v4]; if g(v2) < g(v4), the narrowed single-valley interval is [v2, v3]; if g(v2) = g(v4), the narrowed single-valley interval is [v2, v4]; where g(v2) and g(v4) are the sharpness values of the target images taken when the image distances are v2 and v4 respectively; Use the Fibonacci method to narrow the narrowed single-valley interval again until the length of the single-valley interval is less than the threshold ε to obtain the final single-valley interval; The third step of focusing: Move the image plane so that the image distance moves from the minimum value to the maximum value of the final single-valley interval, and the distance of each movement is m. After each movement, take an image; perform target detection on the image using a neural network model and obtain the local target image, calculate the sharpness of the local target image, and obtain the image distance when the sharpness is the highest as the final image distance to complete the camera focusing.

2. A step-by-step autofocus method according to claim 1, characterized in that: Determine the initial moving range of the image plane according to the relationship between the focal length, object distance and image distance in the camera imaging process.

3. The step-by-step auto-focusing method according to claim 1, wherein: The sharpness is calculated using the Laplacian operator. The Laplacian operator convolves the target image channels, and then calculates the variance of the output. The larger the obtained variance, the clearer the image.

4. The step-by-step auto-focusing method according to claim 1, characterized in that: The neural network model uses the YOLO v9 network or the Swin Transformer network.

5. Apply the step-by-step automatic focusing method described in any one of claims 1 to 4 to the attitude measurement system of an aviation assembly robot based on zoom monocular vision.

Images