Automatic focusing method based on out-of-focus estimation of single-frame image

Abstract

The invention belongs to the technical field of automatic focusing, and particularly relates to an automatic focusing method based on out-of-focus estimation of a single-frame image. Comprising the following steps: S1, carrying out data preprocessing on an out-of-focus image data set; s2, an improved MobileNetV2 network model is obtained; s3, constructing an out-of-focus feature vector data set by using the out-of-focus feature vector of each out-of-focus image; s4, training a gradient boosting regression tree by using the out-of-focus feature vector data set to obtain a trained gradient boosting regression tree; s5, carrying out the out-of-focus regression prediction through an improved MobileNetV2 network model and the trained gradient lifting regression tree; and S6, a defocusing amount code value is calculated according to the defocusing amount normalization output value, the optical system adjusts the position of a focusing motor of the optical system according to the defocusing amount code value, and automatic focusing of the optical system is completed. According to the invention, the automatic focusing precision and efficiency of the optical measurement imaging equipment under the dynamic target can be improved.

Description

Technical Field

[0001] The invention belongs to the technical field of automatic focusing, and in particular relates to an automatic focusing method based on single-frame image defocus estimation. Background Art

[0002] Clear imaging is the basis of optical imaging equipment, and precise focusing is the key to achieving this goal. Faced with targets with variable positions and different motion states, the optical system must have real-time and rapid automatic focusing capabilities, and maintain the clarity and accuracy of the image by adjusting the lens position in real time. At present, the more popular automatic focusing methods for optical imaging equipment include ranging methods and methods based on digital images. The ranging method uses infrared, laser or ultrasonic ranging equipment to measure the distance between the target and the lens (object distance), and then calculates the corresponding image distance according to the Gaussian imaging formula to drive the lens to achieve fast and accurate focusing, but it is greatly affected by the environment (environmental factors such as temperature and air pressure), and requires additional equipment and high costs. The method based on digital images only relies on the acquired image information, uses digital image processing technology to analyze its defocus state, control the focusing mechanism, and effectively search for the quasi-focus position. It mainly includes two types: defocus depth method and focus depth method. The defocus depth method estimates the defocus amount by analyzing a small amount of defocused images, thereby adjusting the image plane position to achieve focusing. However, its accuracy is limited by the parameters of the imaging system and the accuracy of the model. In addition, since the model uses an approximate mathematical model, it cannot be accurately determined, resulting in low focusing accuracy. The focus depth method uses a clarity evaluation function to determine the defocus degree and defocus direction of the image based on images with different defocus degrees, and uses a focus search strategy to drive the lens to move toward the peak point of the evaluation curve until it reaches the optimal focus position. However, the repeated focusing process and the problem of being easily trapped in local extreme values ​​limit its use in the field of optical measurement. Therefore, how to improve the automatic focusing accuracy and efficiency of optical measurement imaging equipment under dynamic targets is a difficult task. Summary of the invention

[0003] In view of this, the present invention aims to provide an automatic focusing method based on single-frame image defocus estimation to solve the problem that the prior art cannot improve the automatic focusing accuracy and efficiency of photometric imaging equipment under dynamic targets. The present invention can improve the automatic focusing accuracy and efficiency of photometric imaging equipment under dynamic targets.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows: An automatic focusing method based on single-frame image defocus estimation specifically comprises the following steps: S1: construct a defocused image dataset and perform data preprocessing on the defocused image dataset; S2: construct an improved MobileNetV2 network, use the preprocessed defocused image dataset to train the improved MobileNetV2 network, and obtain an improved MobileNetV2 network model; S3: constructing a defocus feature vector dataset using the defocus feature vectors corresponding to each defocus image contained in the defocus image dataset output by the improved MobileNetV2 network model; S4: training the gradient boosting regression tree using the defocused feature vector dataset to obtain a trained gradient boosting regression tree; S5: obtaining the image to be predicted collected by the optical system, preprocessing the image to be predicted, performing defocus regression prediction on the preprocessed image to be predicted through an improved MobileNetV2 network model and a trained gradient boosting regression tree, and obtaining a normalized output value of the defocus amount corresponding to the image to be predicted; S6: Calculate the defocus code value according to the defocus normalized output value, and the optical system adjusts the position of the focusing motor of the optical system according to the defocus code value to complete the automatic focusing of the optical system.

[0005] Furthermore, step S1 specifically includes the following steps: S11: Align the optical system with the observation scene, adjust the position of the focus motor according to the preset step length, so that the optical system changes the focus from the minimum focus value to the maximum focus value at the current observation position, and record the image corresponding to each focus motor position in turn to obtain a continuous defocus image stack I; S12: Processing the continuous out-of-focus image stack using a comprehensive clarity evaluation method to obtain an in-focus image; S13: Calculate the defocus code value corresponding to each defocus image included in the continuous defocus image stack I based on the quasi-focus code value of the quasi-focus image, and normalize each defocus code value to obtain a normalized defocus amount normalization label: ; in, is the defocus amount corresponding to the i-th defocus image, is the focus code value of the i-th out-of-focus image, is the in-focus code value of the in-focus image; ; in, is the normalized label of the defocus amount of the i-th defocus image, As a constant, take F max -F min ; S14: repeating steps S11-S13 to obtain normalized defocused image data sets under different observation scenes; S15: expanding the defocused image data set obtained in step S14, and performing a filtering operation on the expanded defocused image data set using a fast guided filter; S16: normalizing the defocused images contained in the defocused image data set after the filtering operation, so that the pixel value of each channel of each defocused image is scaled to the range of [0, 1], thereby completing the data preprocessing of the defocused image data set.

[0006] Furthermore, in step S12, the calculation formula of the comprehensive clarity evaluation method is: S= + ; Among them, S is the comprehensive clarity evaluation value, is the score value of the energy gradient function, is the score value of the Tenengrad function, It is the score value of the improved grayscale difference absolute value sum function.

[0007] Furthermore, the calculation formula of the improved grayscale difference absolute value sum function is: ; Among them, the resolution of the defocused image I is , x is the row pixel coordinate position of the defocused image I, y is the column pixel coordinate position of the defocused image I, is the length of the defocused image I, is the width of the defocused image I.

[0008] Furthermore, in step S2, the improved MobileNetV2 network model takes the MobileNetV2 network model as the basic framework, embeds a coordinate attention mechanism module after the depth-separable convolution of each bottleneck layer of the MobileNetV2 network model, and embeds a cascaded coordinate attention mechanism module and a global average pooling layer after the last convolution layer of the MobileNetV2 network model.

[0009] Furthermore, the coordinate attention mechanism module includes an X-direction average pooling layer, a Y-direction average pooling layer, a Concat module, a first 2D convolution layer, a block normalization module, a nonlinear layer, a second 2D convolution layer, a third 2D convolution layer, a first Sigmoid module, and a second Sigmoid module, wherein the feature map A input to the coordinate attention mechanism module is respectively input to the X-direction average pooling layer and the Y-direction average pooling layer for processing, and feature maps A1 and A2 are obtained correspondingly, and feature maps A1 and A2 are input to the Concat module for processing to obtain feature map A3, and feature map A3 is input to the first 2D The convolution layer is used to obtain feature map A4. Feature map A4 is processed by the block normalization module and the nonlinear layer to obtain feature map A5. Feature map A5 is processed by the split function to obtain feature map A6. Feature map A6 is input to the second 2D convolution layer and the third 2D convolution layer for processing respectively to obtain feature map A7 and feature map A8 respectively. Feature map A7 is input to the first Sigmoid module to obtain feature map A9. Feature map A8 is input to the second Sigmoid module to obtain feature map A10. Feature map A9, feature map A10 and feature map A are multiplied to obtain the output features of the coordinate attention mechanism module.

[0010] Further, in step S3, the defocus feature vector dataset ,in, is the first defocused image, is the defocus feature vector corresponding to the first defocus image, is the normalized label of the defocus amount corresponding to the first defocus image, is the second defocused image, is the defocus feature vector corresponding to the second defocus image, is the normalized label of the defocus amount corresponding to the second defocus image, is the nth out-of-focus image, is the defocus feature vector corresponding to the nth defocused image, is the normalized label of the defocus amount corresponding to the nth defocused image.

[0011] Furthermore, in step S4, the prediction accuracy of the prediction code value output by the trained gradient boosting regression tree is calculated: ; ; in, is the accuracy, N is the total number of test samples, The error of the predicted code value is less than The number of samples, is a constant, is the prediction error, is the predicted code value, is the normalized label for the defocus amount, is a constant.

[0012] Furthermore, the defocus code value The calculation formula is: .

[0013] Furthermore, in step S5, the specific process of preprocessing the image to be predicted is: using a fast guided filter to filter the image to be predicted, and normalizing the image to be predicted after the filtering operation, so that the pixel value of each channel of the image to be predicted after the filtering operation is scaled to the range of [0, 1].

[0014] Compared with the prior art, the invention can achieve the following beneficial effects: (1) The present invention creates an automatic focusing method based on single-frame image defocus estimation, regards the automatic focusing process as a regression problem, designs an improved MobileNetV2 network model to extract features of salient target areas, constructs a strong regressor model of the gradient boosted regression tree (XGBoost), and learns the mapping relationship between features and defocus amounts, thereby achieving accurate estimation of the defocus amount of a single-frame image and realizing automatic focusing of the optical system.

[0015] (2) The present invention creates the automatic focusing method based on single-frame image defocus estimation, proposes an improved lightweight MobileNetV2 network model, and embeds a coordinate attention mechanism (CA) after the depth-separable convolution of the bottleneck layer of the MobileNetV2 network model and after the last convolution layer of the MobileNetV2 network model, respectively, thereby enhancing the ability to extract deep features of salient target areas and improving the quality of feature representation. A global average pooling layer is added at the end of the MobileNetV2 network model, thereby obtaining a feature vector of fixed length, which is convenient for subsequent defocus regression prediction.

[0016] (3) The present invention creates the automatic focusing method based on single-frame image defocus estimation, creates a defocus image dataset with defocus amount annotation, and uses a comprehensive clarity evaluation method (energy gradient function, Tenengrad function, improved SMD function) to determine the position of the in-focus image; the defocus image dataset is the basis for training the improved MobileNetV2 network model, ensuring that the improved MobileNetV2 network model can learn the relationship between defocus amount and image features. The present invention also uses a fast guided filter (FGIF) to pre-process the defocus image, effectively reducing noise while avoiding destroying important structural features such as image edges and details, so as to improve the feature extraction quality of subsequent defocus images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of the structure of the focusing device according to an embodiment of the present invention; Figure 2 A schematic flow chart of an automatic focusing method based on single-frame image defocus estimation according to an embodiment of the present invention; Figure 3 A schematic diagram of a processing flow of an automatic focusing method based on single-frame image defocus estimation according to an embodiment of the present invention; Figure 4 A schematic diagram of the network structure of the improved MobileNetV2 network model described in the embodiment of the present invention; Figure 5 A schematic diagram of the network structure of the bottleneck layer of the improved MobileNetV2 network model described in the embodiment of the present invention.

[0018] Description of reference numerals: 1. Optical system; 2. Focusing mechanism; 3. Focusing controller; 4. Image processing software; 41. Image acquisition module; 42. Image display module; 43. Image storage module; 44. Image tracking module; 45. Defocus prediction module; 46. Communication module. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0020] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0021] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0022] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0023] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0024] like Figure 1As shown, the focusing device includes an optical system 1, a focusing mechanism 2, a focusing controller 3 and an image processing software 4. The optical system 1 includes an optical lens and an image detector. The optical system 1 is used to collect optical signals and focus them on the image detector to form an image; the focusing mechanism 2 is used to adjust the relative position between the lens and the image detector to ensure that the image is clear, and belongs to the execution component; the focusing controller 3 mainly controls the work of the focusing mechanism 2 and is a control component; the image processing software 4 mainly includes an image acquisition module 41, an image display module 42, an image storage module 43, an image tracking module 44, a defocus prediction module 45 and a communication module 46; the image acquisition module 41 is used for image acquisition; FIG. The image display module 42 is used to display the collected image; the image storage module 43 is used for synchronous storage of image data and corresponding focus feedback code values, and the image storage module 43 is mainly used in the establishment of the defocus data set; the image tracking module 44 is used to extract the target tracking wave gate and provide the focus interest area for the defocus prediction module 45; the defocus amount prediction module is used to predict the defocus amount of the current image using the trained model; the communication module 46 encodes the current focus amount according to the defocus amount and sends a focus command to the focus controller 3; the focus controller 3 sends a drive signal to the focus mechanism 2 according to the focus command of the communication module 46 to drive the focus motor to adjust the position and achieve focus. At the same time, the focus mechanism 2 feeds back the position of the focus motor to the image processing software 4 in real time during the focusing process.

[0025] like Figure 2-Figure 3 As shown, the present invention provides an automatic focusing method based on single-frame image defocus estimation, which specifically includes the following steps: S1: constructing a defocus image data set and performing data preprocessing on the defocus image data set; S2: constructing an improved MobileNetV2 network, using the preprocessed defocus image data set to train the improved MobileNetV2 network to obtain an improved MobileNetV2 network model; S3: constructing a defocus feature vector using the defocus feature vectors corresponding to each defocus image contained in the defocus image data set output by the improved MobileNetV2 network model Data set; S4: Use the defocus feature vector data set to train the gradient boosting regression tree to obtain a trained gradient boosting regression tree; S5: Obtain the image to be predicted collected by the optical system, preprocess the image to be predicted, and perform defocus regression prediction on the preprocessed image to be predicted through the improved MobileNetV2 network model and the trained gradient boosting regression tree to obtain a normalized output value of the defocus amount corresponding to the image to be predicted; S6: Calculate the defocus amount code value according to the normalized output value of the defocus amount, and the optical system adjusts the position of the focusing motor of the optical system according to the defocus amount code value to complete the automatic focusing of the optical system.

[0026] The present invention transforms the autofocus problem into a regression problem. First, a defocus image dataset with defocus amount annotation is established, and a comprehensive clarity evaluation method is used to determine the in-focus image. Next, an improved MobileNetV2 network model is proposed. By introducing the coordinate attention mechanism (CA), a more concentrated deep feature extraction of the salient target area is achieved, thereby improving the quality of feature representation. Next, the feature vector extracted by the improved lightweight MobileNetV2 network model is trained using the gradient boosting regression tree (XGBoost), the mapping relationship between the feature and the defocus amount is learned, and the step size and direction required for the focusing process of the optical system 1 are obtained, thereby reducing the unnecessary reciprocating movement of the optical system 1 during the focusing process, thereby achieving accurate estimation of the defocus amount of a single frame image, and realizing automatic focusing of the optical system 1. The present invention not only ensures the accuracy of image focusing, but also significantly improves the efficiency of focusing.

[0027] In some embodiments, step S1 specifically includes the following steps: S11: aligning the optical system 1 with the observation scene, adjusting the position of the focus motor according to a preset step length, so that the optical system 1 changes the focus from the minimum focus value to the maximum focus value at the current observation position, and recording the image corresponding to each focus motor position in turn to obtain a continuous defocus image stack I; S12: processing the continuous defocus image stack using a comprehensive clarity evaluation method to obtain an in-focus image; S13: calculating the defocus code value corresponding to each defocus image contained in the continuous defocus image stack I based on the in-focus code value of the in-focus image, and performing an image analysis on each defocus code. The values ​​are normalized to obtain normalized defocus value normalization labels; S14: repeating steps S11-S13 to obtain normalized defocus image data sets under different observation scenes; S15: expanding the defocus image data set obtained in step S14, and filtering the expanded defocus image data set using a fast guided filter; S16: normalizing the defocus images contained in the defocus image data set after the filtering operation, so that the pixel values ​​of each channel of each defocus image are scaled to the range of [0,1], and completing the data preprocessing of the defocus image data set.

[0028] It should be noted that in step S11, a manual focusing method is usually used to adjust the position of the focus motor according to a preset step length, so that the optical system 1 changes from the minimum focus value F to the minimum focus value F at the current observation position. min To the maximum focus value F max The focus changes during this process will result in a continuous out-of-focus image stack from blurry-sharp-blurry.

[0029] Further, in step S13, the focus code value is F in-focus , then the defocus code value corresponding to each defocus image contained in the continuous defocus image stack I is i for: ; in, is the defocus amount corresponding to the i-th defocus image, is the focus code value of the i-th out-of-focus image, is the in-focus code value of the in-focus image; Normalize the defocus code value of each defocus image to create a defocus normalization label: ; in, is the normalized label of the defocus amount of the i-th defocus image, As a constant, take F max -F min In addition, in step S14, in order to ensure the generalization ability of the defocus regression prediction algorithm based on the gradient boosting regression tree (XGBoost), the more observation scenes are selected, the better. In step S15, the expansion method for the defocus image data set is as follows: in order to prevent background interference, the focus area of ​​interest is generally set to the center area of ​​the field of view. The position of the center area of ​​the field of view depends on the specific usage. i The central region of interest I i M×N Perform random cropping to obtain k images I i m×n , to increase sample diversity. The fast guided filter can effectively reduce noise while avoiding destroying important structural features such as image edges and details. Represents the filtered image, and the guide image uses the input image I i m×n itself.

[0030] In some embodiments, in step S12, the calculation formula of the comprehensive clarity evaluation method is: S= + ; Among them, S is the comprehensive clarity evaluation value, is the score value of the energy gradient function, is the score value of the Tenengrad function, It is the score value of the improved grayscale difference absolute value sum function.

[0031] It should be noted that, in order to accurately screen out the clearest in-focus image and ensure the high stability and sensitivity of the autofocus method, a comprehensive clarity evaluation method is used to process the out-of-focus image stack I.

[0032] In some embodiments, the calculation formula of the improved grayscale difference absolute value sum function is: ; Among them, the resolution of the defocused image I is , x is the row pixel coordinate position of the defocused image I, y is the column pixel coordinate position of the defocused image I, is the length of the defocused image I, is the width of the defocused image I.

[0033] It should be noted that the improved grayscale difference absolute value sum function considers the differences between the target pixel and adjacent pixels from multiple directions.

[0034] In some embodiments, the improved MobileNetV2 network model takes the MobileNetV2 network model as the basic framework, embeds a coordinate attention mechanism module after the depth-separable convolution of each bottleneck layer of the MobileNetV2 network model, and embeds a cascaded coordinate attention mechanism module and a global average pooling layer after the last convolution layer of the MobileNetV2 network model.

[0035] It should be noted that, considering that most of the task scenarios are embedded applications, there are great restrictions on computing resources and storage space. Therefore, the mainstream lightweight MobileNetV2 network model is used as the basic network architecture for feature extraction, excluding the top fully connected layer. And improve MobileNetV2. Furthermore, after the depth-separable convolution of each bottleneck layer of the MobileNetV2 network model, a coordinate attention mechanism module (referred to as CA module) is embedded, which compresses the feature map in the spatial dimension to capture the position information, and then generates attention weights through a series of convolutional layers. The original feature map is re-weighted using the generated attention weights to emphasize the features of important positions and suppress the features of unimportant positions, thereby improving the quality of feature representation. Furthermore, after the last convolutional layer of the MobileNetV2 network model, the CA module is embedded again, which can further refine the feature map, so that the improved MobileNetV2 network model can focus on the key features more accurately before making the final decision. Help the improved MobileNetV2 network model better adapt to different degrees of blur, which is convenient for subsequent regression tasks. In addition, a global average pooling layer is added after the output of the last convolutional layer of the improved MobileNetV2 network model to reduce the spatial dimension of the features and obtain a defocus feature vector of fixed length.

[0036] like Figure 4 As shown, the improved MobileNetV2 network model ( Figure 3The improved MobileNetV2 network model (abbreviated as MobileNetV2-CA) includes a 2d convolutional layer, seven bottleneck layers embedded in CA modules, a 2d convolutional layer, a CA module and a global average pooling layer connected in sequence. The features input to the improved MobileNetV2 network model are output through the 2d convolutional layer and the seven new bottleneck layers embedded in CA modules to obtain the output features. After the features F are processed by the 2d convolutional layer, the CA module and the global average pooling layer, the output features are obtained.

[0037] like Figure 5 As shown in the figure, the bottleneck layer of the improved MobileNetV2 network model is a 1×1 convolution layer, a 3×3 deep convolution layer, a coordinate attention mechanism module (CA module), and a 1×1 convolution layer connected in sequence. After the input features are processed by the 1×1 convolution layer, the 3×3 deep convolution layer, the coordinate attention mechanism module, and the 1×1 convolution layer, the intermediate features are obtained, and the intermediate features are added to the input features to obtain the output features of the bottleneck layer of the improved MobileNetV2 network model.

[0038] In some examples, the coordinate attention mechanism module includes an X-direction average pooling layer, a Y-direction average pooling layer, a Concat module, a first 2D convolution layer, a block normalization module, a nonlinear layer, a second 2D convolution layer, a third 2D convolution layer, a first Sigmoid module, and a second Sigmoid module, wherein the feature map A input to the coordinate attention mechanism module is respectively input to the X-direction average pooling layer and the Y-direction average pooling layer for processing, and feature maps A1 and A2 are obtained correspondingly, and feature maps A1 and A2 are input to the Concat module for processing to obtain feature map A3, and feature map A3 is input to the first 2D convolution layer. The feature map A4 is processed by the block normalization module and the nonlinear layer to obtain the feature map A5. The feature map A5 is processed by the split function to obtain the feature map A6. The feature map A6 is respectively input to the second 2D convolutional layer and the third 2D convolutional layer for processing to obtain the feature map A7 and the feature map A8 respectively. The feature map A7 is input to the first Sigmoid module to obtain the feature map A9. The feature map A8 is input to the second Sigmoid module to obtain the feature map A10. The feature map A9, the feature map A10 and the feature map A are multiplied to obtain the output features of the coordinate attention mechanism module.

[0039] In some examples, in step S3, the defocus feature vector dataset ,in, is the first defocused image, is the defocus feature vector corresponding to the first defocus image, is the normalized label of the defocus amount corresponding to the first defocus image, is the second defocused image, is the defocus feature vector corresponding to the second defocus image, is the normalized label of the defocus amount corresponding to the second defocus image, is the nth out-of-focus image, is the defocus feature vector corresponding to the nth defocused image, is the normalized label of the defocus amount corresponding to the nth defocused image.

[0040] It should be noted that in step S4, the gradient boosting regression tree (XGBoost) is an ensemble learning method that builds a powerful prediction model by integrating multiple weak classifiers. By continuously adding tree models, a new tree is added each time to fit the residual of the previous prediction, and finally the prediction results of all trees are added together to obtain the final prediction code value.

[0041] The training process of the gradient boosting regression tree is: The defocus feature vectors corresponding to each defocus image contained in the defocus image dataset output by the improved MobileNetV2 network model are divided into training set, validation set and test set in proportion, with the division ratios of 7:2:1 respectively.

[0042] Set the number of decision trees to 200, the learning rate to 0.1, the maximum depth of the decision tree to 16, and the loss function to minimize the squared error loss.

[0043] Setting up the initial model for the decision tree : ; in, is the average value of the normalized defocus labels in the training set, A is the total number of samples in the training set, is the i-th normalized defocus label; Calculate the residual of each defocus eigenvector under the current model: ; in, is the residual, is the current regression tree model, is the input defocus feature vector; for the square loss function, the residual is the difference between the target value and the current prediction code value.

[0044] Fitting a Regression Tree: Using Residuals As the target variable, a regression tree T is trained to fit the data, which divides the input space into multiple regions.

[0045] Calculate leaf node values: For each leaf node of the regression tree T, find the optimal output value , which is achieved by minimizing the loss function: Calculate each leaf node of the decision tree, minimize the square loss function, and obtain: ; in, is the optimal output value for each leaf node, is the set of defocus feature vectors in the jth leaf node, L() is the loss function, is a constant.

[0046] Update the model: Add the newly trained regression tree to the model and update the model's prediction code value: ; in, is the learning rate, is the predicted code value of x by the zth tree, are the parameters of the regression tree T.

[0047] Repeat the above steps, and stop the iteration when the predetermined number of iterations Z is reached, or when adding new trees no longer significantly reduces the loss function, to obtain the final strong regression tree model:

[0048] This process generates an additive model consisting of multiple trees, each of which is trained based on the residuals of the previous tree, gradually building up a strong regression model.

[0049] The performance of the gradient boosting regression tree is evaluated using the validation set, and the performance indicators used are the mean square error (MSE) and the coefficient of determination (r2_score).

[0050] After completing the training of the improved MobileNetV2 network model and the gradient boosting regression tree, the defocus prediction evaluation is performed: the accuracy of the defocus prediction is evaluated using the test set. Considering that the focusing system of the optical measurement equipment is driven by a high-precision motor, the lens can be focused within a larger range of travel, and the focusing value has a higher accuracy. The error in the defocus prediction is generally acceptable if it is within the image clarity range that is indistinguishable to the human eye. The impact of this error on the actual observation task is minimal. In this case, even if there is a certain prediction error, the image is still considered to be clear enough to meet most observation needs. Therefore, the accuracy formula for the defocus prediction is as follows: ; in, is the accuracy, N is the total number of test samples, The error of the predicted code value is less than The number of samples, is a constant, is the prediction error, is the predicted code value, is the normalized label for the defocus amount, is a constant, depending on the situation. Represents the prediction error, and the expression is: ; in, is the predicted code value of the defocus amount, is the normalized label for the defocus amount, is a constant. After testing, It can reach more than 95%, and it is considered that the performance of the present invention is good. After the training is completed, the model (the improved MobileNetV2 network model and the trained gradient boosting regression tree) is saved for subsequent use or deployment.

[0051] The trained model is deployed to the focus control software. If the tracking gate of the image tracking module 44 is received, the tracking target area is set as the focus area of ​​interest. If the tracking gate of the image tracking module 44 is not received, the image center area is set as the focus area of ​​interest. The focus area of ​​interest image of the current frame image is input into the defocus prediction module 45 for defocus amount prediction. The prediction steps are as follows: Figure 4 First, the image of the region of interest is preprocessed, and then the defocus feature is extracted using the improved MobileNetV2 network model to obtain the feature vector, which is then output to the XGBoost regression model to obtain the normalized output value of the defocus amount at the current image position. (positive and negative values ​​with direction), and by the formula Convert to defocus code value. Coder <0, control the focus motor to F max Direction adjustment, when Coder >0, control the focus motor to F min Direction adjustment, the adjusted code values ​​are The absolute value of can complete the automatic focusing of the system.

[0052] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0053] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic focusing method based on single-frame image defocus estimation, characterized in that: The specific steps include: S1: construct a defocused image dataset and perform data preprocessing on the defocused image dataset; S2: construct an improved MobileNetV2 network, use the preprocessed defocused image dataset to train the improved MobileNetV2 network, and obtain an improved MobileNetV2 network model; S3: constructing a defocus feature vector dataset using the defocus feature vectors corresponding to each defocus image contained in the defocus image dataset output by the improved MobileNetV2 network model; S4: training the gradient boosting regression tree using the defocused feature vector dataset to obtain a trained gradient boosting regression tree; S5: obtaining the image to be predicted collected by the optical system, preprocessing the image to be predicted, performing defocus regression prediction on the preprocessed image to be predicted through an improved MobileNetV2 network model and a trained gradient boosting regression tree, and obtaining a normalized output value of the defocus amount corresponding to the image to be predicted; S6: calculating a defocus code value according to the defocus normalized output value, and the optical system adjusts the position of a focus motor of the optical system according to the defocus code value to complete automatic focusing of the optical system.

2. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Align the optical system with the observation scene, adjust the position of the focus motor according to a preset step length, so that the focus of the optical system changes from a minimum focus value to a maximum focus value at the current observation position, and record the image corresponding to each focus motor position in sequence to obtain a continuous defocus image stack I; S12: Processing the continuous out-of-focus image stack using a comprehensive clarity evaluation method to obtain an in-focus image; S13: Calculate the defocus code value corresponding to each defocus image included in the continuous defocus image stack I based on the quasi-focus code value of the quasi-focus image, and normalize each defocus code value to obtain a normalized defocus amount normalization label: ； in, is the defocus amount corresponding to the i-th defocus image, is the focus code value of the i-th out-of-focus image, is the in-focus code value of the in-focus image; ； in, is the normalized label of the defocus amount of the i-th defocus image, As a constant, take F max -F min ; S14: repeating steps S11-S13 to obtain normalized defocused image data sets under different observation scenes; S15: expanding the defocused image data set obtained in step S14, and performing a filtering operation on the expanded defocused image data set using a fast guided filter; S16: normalizing the defocused images contained in the defocused image data set after the filtering operation, so that the pixel value of each channel of each defocused image is scaled to the range of [0, 1], thereby completing the data preprocessing of the defocused image data set.

3. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: In step S12, the calculation formula of the comprehensive clarity evaluation method is: S= + ； Among them, S is the comprehensive clarity evaluation value, is the score value of the energy gradient function, is the score value of the Tenengrad function, It is the score value of the improved grayscale difference absolute value sum function.

4. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: The calculation formula of the improved grayscale difference absolute value sum function is: ； Among them, the resolution of the defocused image I is , x is the row pixel coordinate position of the defocused image I, y is the column pixel coordinate position of the defocused image I, is the length of the defocused image I, is the width of the defocused image I.

5. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: In step S2, the improved MobileNetV2 network model takes the MobileNetV2 network model as the basic framework, embeds a coordinate attention mechanism module after the depth-separable convolution of each bottleneck layer of the MobileNetV2 network model, and embeds a cascaded coordinate attention mechanism module and a global average pooling layer after the last convolution layer of the MobileNetV2 network model.

6. The automatic focusing method based on single-frame image defocus estimation according to claim 5, characterized in that: The coordinate attention mechanism module includes an X-direction average pooling layer, a Y-direction average pooling layer, a Concat module, a first 2D convolution layer, a block normalization module, a nonlinear layer, a second 2D convolution layer, a third 2D convolution layer, a first Sigmoid module, and a second Sigmoid module, wherein the feature map A input to the coordinate attention mechanism module is respectively input to the X-direction average pooling layer and the Y-direction average pooling layer for processing, and feature maps A1 and A2 are obtained correspondingly, and feature maps A1 and A2 are input to the Concat module for processing to obtain feature map A3, and feature map A3 is input to the first 2D convolution layer. The feature map A4 is processed by the block normalization module and the nonlinear layer to obtain the feature map A5. The feature map A5 is processed by the split function to obtain the feature map A6. The feature map A6 is respectively input to the second 2D convolutional layer and the third 2D convolutional layer for processing to obtain the feature map A7 and the feature map A8 respectively. The feature map A7 is input to the first Sigmoid module to obtain the feature map A9. The feature map A8 is input to the second Sigmoid module to obtain the feature map A10. The feature map A9, the feature map A10 and the feature map A are multiplied to obtain the output features of the coordinate attention mechanism module.

7. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: In step S3, the defocus feature vector dataset ,in, is the first defocused image, is the defocus feature vector corresponding to the first defocus image, is the normalized label of the defocus amount corresponding to the first defocus image, is the second defocused image, is the defocus feature vector corresponding to the second defocus image, is the normalized label of the defocus amount corresponding to the second defocus image, is the nth out-of-focus image, is the defocus feature vector corresponding to the nth defocused image, is the normalized label of the defocus amount corresponding to the nth defocused image.

8. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: In step S4, the prediction accuracy of the prediction code value output by the trained gradient boosting regression tree is calculated: ； ； in, is the accuracy, N is the total number of test samples, The error of the predicted code value is less than The number of samples, is a constant, is the prediction error, is the predicted code value, is the normalized label for the defocus amount, is a constant.

9. The automatic focusing method based on single-frame image defocus estimation according to claim 8, characterized in that: Defocus code value The calculation formula is: 。 10. The automatic focusing method based on single-frame image defocus estimation according to claim 1, characterized in that: In step S5, the specific process of preprocessing the image to be predicted is: using a fast guided filter to filter the image to be predicted, normalizing the image to be predicted after the filtering operation, so that the pixel value of each channel of the image to be predicted after the filtering operation is scaled to the range of [0, 1].

Images