A machine vision automatic precision focusing method and system based on dual-mode evaluation

Abstract

The present application belongs to the field of machine vision and precise optical imaging technology, and particularly relates to a kind of automatic precision focusing method and system based on double mode evaluation of machine vision.The present application realizes the adaptive switching of focusing mode through texture evaluation, for the scene with rich texture features and high contrast, the fast focusing mechanism can carry out efficient focusing search based on image gradient, quickly locates the coarse focus position, significantly improves the focusing efficiency, and for the challenging working condition with weak surface texture and low contrast, it can automatically switch to fine focusing mechanism, determines the optimal focus position by constructing image stack and extracting the aberration coefficient of pupil area, and further ensures the reliability and accuracy of the optimal focus position by confidence verification of the weighted combination of multi-dimensional indicators such as aberration coefficient, astigmatism coefficient and pupil area roundness, effectively avoids the focusing failure caused by accidental error or interference.

Description

Technical Field

[0001] This invention belongs to the field of machine vision and precision optical imaging technology, specifically relating to a machine vision automatic precision focusing method and system based on dual-mode evaluation. Background Technology

[0002] In industrial applications of machine vision, especially in automated lighting platforms and inspection systems that require high precision under multiple operating conditions, accurate focusing is a key prerequisite for ensuring image quality and inspection accuracy. Currently, the mainstream technical approach is to use passive autofocus technology. Its core is to build a feedback system based on a single image sharpness evaluation function (such as image gradient function, frequency domain function, or information entropy). This type of system adjusts the object distance by driving a motor and calculates the image sharpness evaluation value in real time. It uses search algorithms (such as hill climbing method or Fibonacci method) to find the peak point of the evaluation function, thereby determining the optimal focus position.

[0003] Existing technical solutions have revealed several inherent defects in practice. First, the single evaluation function has poor environmental adaptability. For example, when processing common industrial workpieces with weak surface texture features and low image contrast (such as smooth metal parts, transparent glass, or uniform plastic parts), the gradient-based evaluation function has an extremely weak response, and its evaluation curve is too flat or subject to severe noise interference, causing the search algorithm to fail, fail to converge, or take an extremely long time. Second, under complex multi-angle lighting conditions, interference such as non-uniform illumination and high light reflection can easily cause the evaluation function to generate multiple local extrema, causing the focusing system to get stuck in a local optimum and unable to locate the true physical optimal focal plane, which seriously affects the reliability and repeatability of detection. In addition, the traditional search strategy does not have determinism in the search direction near the peak and is prone to oscillation, which not only reduces efficiency, but its mechanical repetitive movement also poses a potential threat to long-term accuracy stability.

[0004] Although existing technologies have attempted to integrate multiple evaluation functions or introduce predictive mechanisms to improve performance, most of these improvements have not addressed the core issue: under specific challenging conditions, all evaluation functions based on image grayscale or gradient statistics may fail in principle, and the system lacks a mechanism that can fundamentally assess the defocus state and select only the best countermeasure. Therefore, there is an urgent need for a more robust, intelligent new autofocus technology based on the principles of imaging optics to meet the increasingly stringent requirements of high-end industrial inspection. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision automatic precision focusing method and system based on dual-mode evaluation, which can make the focusing process more intelligent and robust through automatic switching of the dual-mode focusing mechanism.

[0006] The specific technical solution adopted by this invention is as follows:

[0007] A machine vision-based automatic precision focusing method based on dual-mode evaluation includes:

[0008] The industrial camera is moved above the area to be inspected and the target in the area to be inspected is captured to obtain initial image data. Then, the initial image data is preprocessed to output the image to be identified.

[0009] The texture of the image to be recognized is evaluated to obtain an image sharpness score, and the corresponding focusing mode is matched according to the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism.

[0010] Under the fast focusing mechanism, focus search based on image gradient is performed to locate the coarse focus position and determine whether it meets the fast convergence condition. If it does not meet the condition, it switches to the fine focusing mechanism.

[0011] Under the fine focusing mechanism, fine scanning is performed with the coarse focus position or the initial position as the center to build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient.

[0012] The confidence level of the optimal focus position is verified, and the focusing operation is completed after the confidence level verification is passed at the optimal focus position.

[0013] In a preferred embodiment, the step of preprocessing the initial image data to output the image to be recognized includes:

[0014] The initial image data is denoised by using an adaptive median filter to eliminate random noise during the imaging process, resulting in a denoised image.

[0015] Histogram equalization is used to enhance the contrast of the denoised image and improve the texture discernibility in low-light areas, resulting in an image to be identified for focusing.

[0016] In a preferred embodiment, the step of evaluating the texture of the image to be recognized to obtain an image sharpness score includes:

[0017] The image to be recognized is processed into a grid to obtain multiple sub-image blocks, and the standard deviation of the pixel values ​​within each sub-image block is calculated;

[0018] Calculate the arithmetic mean of the standard deviations of all sub-image patches and the overall standard deviation, and record them as the first texture index and the second texture index, respectively;

[0019] The first texture index and the second texture index are input into the image sharpness evaluation function, and the output of the sharpness evaluation function is recorded as the image sharpness score.

[0020] In a preferred embodiment, the step of matching the corresponding focusing mode based on the image sharpness score includes:

[0021] Obtain the mode switching threshold and compare the image sharpness score with the mode switching threshold;

[0022] When the image sharpness score is greater than or equal to the mode switching threshold, it indicates that the sharpness of the image to be identified meets the conditions for fast focusing, and it directly enters the fast focusing mechanism.

[0023] When the image sharpness score is less than the mode switching threshold, it indicates that the image to be identified is blurry and cannot meet the requirements of fast focusing, and a fine focusing mechanism is required.

[0024] The mode switching threshold is adaptively set according to the imaging scene, and the specific process is as follows:

[0025] Multiple reference images are obtained by scanning a fixed area at the initial position using an industrial camera;

[0026] Calculate the image sharpness score for all reference images, and then take a weighted average of the maximum and minimum image sharpness scores to obtain the dynamic mode switching threshold.

[0027] In a preferred embodiment, the fast focusing mechanism includes the following steps: performing a focus search based on image gradients to locate the coarse focus position, determining whether it meets the fast convergence condition, and switching to the fine focusing mechanism if it does not.

[0028] The industrial camera is moved by a preset initial step size, and real-time images are acquired at the stopping position of the industrial camera.

[0029] Calculate the mean gradient of each real-time image frame and determine whether the gradient change rate between two adjacent frames is less than the allowable convergence threshold.

[0030] If the gradient change rate is less than or equal to the allowable convergence threshold, and the corresponding image gradient mean reaches a local maximum, then the current stopping position is determined to be the coarse adjustment focus.

[0031] If the gradient rate of change is greater than the allowable convergence threshold, the position of the industrial camera will continue to be adjusted with decreasing step size until the convergence condition is met and then stop.

[0032] In the process of adjusting the position of the industrial camera with decreasing step size, if the gradient change rate shows a monotonically increasing trend, then the current search direction and search step size are maintained and the search continues.

[0033] When the gradient rate of change shows a monotonically decreasing trend, the search direction is reversed and the step size is reduced;

[0034] If the gradient change rate does not show a monotonically changing trend, it is judged as a search anomaly, and the fast focusing process is immediately terminated, and a mode switching signal is triggered to start the fine focusing mechanism.

[0035] If the industrial camera fails to meet the convergence condition even after adjusting its position to reach the maximum number of searches, it indicates that the current imaging environment does not meet the conditions for fast focusing, and it directly switches to the fine focusing mechanism.

[0036] In a preferred embodiment, the step of performing fine scanning centered on the coarsely adjusted focal position or initial position, constructing an image stack, and extracting the pupil region of each frame in the image stack includes:

[0037] Centered on the coarse focus position or initial position, perform bidirectional scanning near the focal plane at a preset fine step size to acquire multiple images at different focal length positions and form an image stack;

[0038] Binarization segmentation is performed on each frame of the image stack to separate the foreground from the background and extract the region contour of the pupil area.

[0039] Perform morphological closing operations on the binarized image to fill holes in the region contour and smooth the edges;

[0040] Connected regions are extracted from the smoothed binarized image, and the geometric features of each connected region are calculated. Based on the geometric features, a single region that meets the preset pupil features is selected from all connected regions and recorded as the pupil region.

[0041] In a preferred embodiment, the step of calculating the aberration coefficients of each pupil region and then determining the optimal focal position based on the aberration coefficients includes:

[0042] Calculate the Zernike polynomial coefficients within each pupil region and extract the defocus term coefficients that directly correspond to the defocus aberration as aberration coefficients;

[0043] Within the entire focal length range of the fine scan, find the image frame with the smallest absolute value of the aberration coefficient, and determine the focal length position corresponding to the image frame as the optimal focus position;

[0044] If multiple image frames have the same absolute value of aberration coefficients and all are the minimum value, then the image frame with the middle focal length is selected as the optimal focal position.

[0045] In a preferred embodiment, the step of verifying the confidence level of the optimal focal position includes:

[0046] The astigmatic coefficients are extracted from the Zernike polynomial coefficients, and the absolute values ​​of the astigmatic coefficients are processed and recorded as the first verification index.

[0047] The aberration coefficients are processed by absolute value and recorded as the second verification index;

[0048] The roundness of the pupil area was calculated and recorded as the third verification indicator;

[0049] The first, second, and third verification indicators are weighted and combined to form a comprehensive confidence score.

[0050] Compare the overall confidence score with the confidence threshold;

[0051] If the overall confidence score is not lower than the confidence threshold, the optimal focus position is determined to be valid, and the optimal focus position is output as the final focusing result.

[0052] If the overall confidence score is lower than the confidence threshold, the optimal focus position is deemed invalid, and a small step trial movement is performed. Based on the current optimal focus position, the movement is performed in both positive and negative directions by a preset step size. Two frames of images are acquired and the aberration coefficient calculation and optimal focus determination process are repeated until the confidence threshold is met.

[0053] If the number of small-step trial moves reaches the upper limit of the number of trials but still does not meet the confidence threshold, the focusing is determined to have failed and a focusing failure log is recorded.

[0054] The present invention also provides a machine vision automatic precision focusing system based on dual-mode evaluation, which uses the above-described machine vision automatic precision focusing method based on dual-mode evaluation, including:

[0055] The initialization module is used to move the industrial camera above the area to be inspected and to acquire images of the target in the area to be inspected, thereby obtaining initial image data. Then, by preprocessing the initial image data, the image to be identified is output.

[0056] The pattern matching module is used to evaluate the texture of the image to be recognized, obtain an image sharpness score, and match the corresponding focusing mode based on the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism.

[0057] The fast focus module is used to perform focus search based on image gradient under the fast focus mechanism, locate the coarse focus position, determine whether it meets the fast convergence condition, and switch to the fine focus mechanism if it does not meet the condition.

[0058] The fine focus module is used to perform fine scanning with the coarse focus position or the initial position as the center under the fine focus mechanism, build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient.

[0059] The focus evaluation module is used to verify the confidence level of the optimal focus position, and after the confidence level verification is passed at the optimal focus position, the focusing operation is completed.

[0060] And, an electronic device, the electronic device comprising:

[0061] At least one processor;

[0062] and a memory communicatively connected to the at least one processor;

[0063] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described machine vision automatic precision focusing method based on dual-mode evaluation.

[0064] The technical effects achieved by this invention are as follows:

[0065] This invention achieves intelligent and robust focusing processes by constructing a dual-mode evaluation mechanism. On one hand, it enables adaptive switching of focusing modes through texture evaluation. For scenes with rich texture features and high contrast, the fast focusing mechanism can efficiently search for focus based on image gradients, quickly locating the coarse focus position and significantly improving focusing efficiency. On the other hand, for challenging conditions with weak surface texture and low contrast, it can automatically switch to a fine focusing mechanism, determining the optimal focus position by constructing an image stack and extracting the aberration coefficients of the pupil region. This addresses the issue of susceptibility to interference from a single evaluation function at the level of imaging optics principles. While it is prone to getting stuck in local optima, this system further ensures the reliability and accuracy of the optimal focal position by performing confidence verification on a weighted combination of multiple indicators such as aberration coefficient, astigmatism coefficient, and pupil area roundness. This effectively avoids focusing failures caused by accidental errors or interference. In addition, the setting of the dynamic mode switching threshold and the judgment of search anomalies and convergence conditions in the fast focusing mechanism enhance the system's adaptability to complex and ever-changing imaging environments, reduce repetitive mechanical movements, and help ensure long-term accuracy stability. This provides strong support for the requirements of imaging quality and detection accuracy in industrial inspection. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0067] Figure 2 This is a schematic diagram of the system modules of the present invention;

[0068] Figure 3 This is a schematic diagram of the electronic device structure of the present invention. Detailed Implementation

[0069] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0070] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0071] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0072] Please see Figure 1 As shown, this invention provides a machine vision automatic precision focusing method based on dual-mode evaluation, comprising:

[0073] S1. Move the industrial camera above the area to be inspected and acquire images of the target in the area to be inspected to obtain initial image data. Then, preprocess the initial image data to output the image to be identified.

[0074] In step S1, in industrial applications of machine vision, especially in automated lighting platforms and inspection systems involving multiple working conditions and high precision requirements, accurate focusing is a key prerequisite for ensuring image quality and inspection accuracy. In this embodiment, the industrial camera is first moved directly above the area to be inspected to ensure that the optical axis is perpendicular to the target surface. Simultaneously, the target in the area to be inspected is image-captured to obtain initial image data. Then, by preprocessing the initial image data to eliminate noise, correct distortion, and enhance contrast, the image to be identified for subsequent recognition can be output. The step of preprocessing the initial image data to output the image to be identified includes:

[0075] The initial image data is denoised by using an adaptive median filter to eliminate random noise during the imaging process, resulting in a denoised image.

[0076] Histogram equalization is used to enhance the contrast of the denoised image and improve the texture discernibility in low-light areas, resulting in an image to be identified for focusing.

[0077] Specifically, during the preprocessing of the initial image data, an adaptive median filtering algorithm is first used to denoise the initial image data. This effectively eliminates random noise such as salt-and-pepper noise and Gaussian noise generated during the imaging process, while preserving the edge and detail information of the image to the greatest extent. This avoids the image blurring problem caused by traditional fixed window filtering, resulting in a denoised image. After denoising, histogram equalization is used to adjust the gray-level distribution of the denoised image to address potential uneven lighting or low contrast. By stretching the gray-level histogram of the original image, the gray-level distribution of the image becomes more uniform, enhancing the overall contrast of the image, especially improving the texture discernibility in low-light areas. This provides a clearer and easier-to-analyze image for subsequent texture evaluation and focusing operations.

[0078] S2. Evaluate the texture of the image to be recognized to obtain an image sharpness score, and match the corresponding focusing mode according to the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism.

[0079] In step S2, after the image to be recognized is generated, a corresponding texture evaluation is performed, and the result of the texture evaluation is recorded as an image sharpness score. Based on this score, a corresponding focusing mode is matched. The focusing modes are mainly classified into two categories: fast focusing mechanism and fine focusing mechanism. The fast focusing mechanism is mainly suitable for scenarios with high image sharpness scores. At this time, the system judges that the image already has basic sharpness, and only minor adjustments are needed to reach the optimal focal plane. The fine focusing mechanism is used for scenarios with low image sharpness scores. It accurately locates the globally optimal focal plane through multi-level step scanning and local gradient analysis. The step of evaluating the texture of the image to be recognized to obtain the image sharpness score includes:

[0080] The image to be recognized is processed into a grid to obtain multiple sub-image blocks, and the standard deviation of the pixel values ​​within each sub-image block is calculated;

[0081] Calculate the arithmetic mean of the standard deviations of all sub-image patches and the overall standard deviation, and record them as the first texture index and the second texture index, respectively;

[0082] The first texture index and the second texture index are input into the image sharpness evaluation function, and the output of the sharpness evaluation function is recorded as the image sharpness score.

[0083] Specifically, when evaluating the texture of the image to be recognized, it is first divided into several uniform grids to obtain multiple uniform sub-image blocks. This ensures that each sub-image block covers a local area of ​​the image and preserves spatial structure information. Then, the standard deviation of the pixel grayscale values ​​of each sub-image block is calculated to characterize the intensity of grayscale changes within the sub-image block. A larger standard deviation indicates richer texture in the region, and vice versa. The arithmetic mean of the standard deviations of all sub-image blocks is then taken as the first texture index, reflecting the overall texture activity of the image. Next, the overall standard deviation of all standard deviations is calculated as the second texture index, reflecting the unevenness of texture distribution. Finally, an image sharpness evaluation function is introduced, and the first and second texture indices are used as input parameters to calculate the image sharpness score of the image to be recognized. The expression of the image sharpness evaluation function is as follows: In the formula, Indicates the image sharpness score. Indicates the first texture index, Indicates the second texture index. This represents the adjustment coefficient, which takes the value of a positive real number and is used to control the relative importance of the second texture index in the sharpness score.

[0084] Secondly, the steps for matching the corresponding focus mode based on the image sharpness score include:

[0085] Obtain the mode switching threshold and compare the image sharpness score with the mode switching threshold;

[0086] When the image sharpness score is greater than or equal to the mode switching threshold, it indicates that the sharpness of the image to be identified meets the conditions for fast focusing, and it directly enters the fast focusing mechanism.

[0087] When the image sharpness score is less than the mode switching threshold, it indicates that the image to be identified is blurry and cannot meet the requirements of fast focusing, and a fine focusing mechanism is required.

[0088] The mode switching threshold is adaptively set according to the imaging scene, and the specific process is as follows:

[0089] Multiple reference images are obtained by scanning a fixed area at the initial position using an industrial camera;

[0090] Calculate the image sharpness score of all reference images, and take a weighted average of the maximum and minimum values ​​of the image sharpness scores to obtain the dynamic mode switching threshold;

[0091] After the image sharpness score is output, the threshold comparison logic is immediately triggered. The image sharpness score of the target to be identified is compared with the mode switching threshold to determine the specific focusing mechanism to be executed. Specifically, the mode switching threshold is not a fixed constant, but is adaptively set according to the imaging scene. First, a fixed range scan is performed at the initial position using an industrial camera. For example, several reference images are acquired within a focal depth range of ±5mm with a step size of 0.5mm to cover possible out-of-focus states. After calculating the image sharpness scores of all reference images, a weighted average of the maximum and minimum values ​​is used to obtain the dynamic mode switching threshold. When the image sharpness score is higher than or equal to the mode switching threshold, it indicates that the image to be identified has sufficient texture activity and distribution balance. At this time, a fast focusing mechanism is matched to perform focusing operation to shorten the response time. When the image sharpness score is lower than the mode switching threshold, it is determined that the image texture activity is insufficient or the distribution is unbalanced. At this time, it will automatically switch to a fine focusing mechanism to perform step-by-step fine adjustment of the image to be identified.

[0092] S3. Under the fast focusing mechanism, focus search based on image gradient is performed to locate the coarse focus position and determine whether it meets the fast convergence condition. If it does not meet the condition, it switches to the fine focusing mechanism.

[0093] In step S3, when the fast focusing mechanism is executed, a coarse-grained focus search based on the image gradient magnitude is performed, advancing along the optical axis with a large step size to quickly locate the coarse focus position. During this process, the gradient magnitude change rate is monitored in real time to see if it meets the preset fast convergence condition. If it does not meet the condition, the mechanism automatically switches to the fine focusing mechanism, initiating multi-scale local gradient extremum tracking and Zernike polynomial fitting correction to avoid falling into local extremum traps. The steps of performing a focus search based on the image gradient, locating the coarse focus position, determining whether it meets the fast convergence condition, and switching to the fine focusing mechanism if it does not meet the condition include:

[0094] The industrial camera is moved by a preset initial step size, and real-time images are acquired at the stopping position of the industrial camera.

[0095] Calculate the mean gradient of each real-time image frame and determine whether the gradient change rate between two adjacent frames is less than the allowable convergence threshold.

[0096] If the gradient change rate is less than or equal to the allowable convergence threshold, and the corresponding image gradient mean reaches a local maximum, then the current stopping position is determined to be the coarse adjustment focus.

[0097] If the gradient rate of change is greater than the allowable convergence threshold, the position of the industrial camera will continue to be adjusted with decreasing step size until the convergence condition is met and then stop.

[0098] In the process of adjusting the position of the industrial camera with decreasing step size, if the gradient change rate shows a monotonically increasing trend, then the current search direction and search step size are maintained and the search continues.

[0099] When the gradient rate of change shows a monotonically decreasing trend, the search direction is reversed and the step size is reduced;

[0100] If the gradient change rate does not show a monotonically changing trend, it is judged as a search anomaly, and the fast focusing process is immediately terminated, and a mode switching signal is triggered to start the fine focusing mechanism.

[0101] If the industrial camera fails to meet the convergence condition even after adjusting its position to reach the maximum number of searches, it indicates that the current imaging environment does not meet the conditions for fast focusing, and it directly switches to the fine focusing mechanism.

[0102] Specifically, in the fast focusing mechanism, the industrial camera is first driven to move along the optical axis with a preset large initial step size. Real-time images are acquired at each stop position, and the average gradient of the real-time image in each frame is calculated. Typically, the Sobel or Prewitt operator is used to extract the horizontal and vertical gradients of the image. The average gradient magnitude is then used to characterize the overall image sharpness; the larger the average gradient, the sharper the image. Next, it is determined whether the gradient change rate between two adjacent real-time images is less than a preset acceptable convergence threshold. The gradient change rate is obtained by dividing the difference between the average gradient of the current frame and the average gradient of the previous frame by the average gradient of the previous frame. If the gradient change rate is less than or equal to the acceptable convergence threshold, and the corresponding average gradient of the image at this time reaches a local maximum compared to the adjacent positions, then the current stop position is determined to be the coarse focus position. If the gradient change rate is greater than the acceptable convergence threshold, the position of the industrial camera continues to be adjusted, using a decreasing step size strategy, for example, from... The focus is gradually reduced from 2mm to 1mm, 0.5mm, etc., to achieve a gradual approach to the focus point. During the adjustment of the industrial camera position with decreasing step size, the trend of the gradient change rate is continuously monitored. If the gradient change rate shows a monotonically increasing trend, it indicates that the current search direction is correct, and the current search direction and current search step size should be maintained to continue searching. If the gradient change rate shows a monotonically decreasing trend, it indicates that the focus may have been exceeded, and the search direction needs to be reversed and the step size further reduced. If the gradient change rate does not show a monotonically increasing or decreasing regular change, such as frequent fluctuations or jumps, it is judged as a search anomaly, which may be trapped in a local extremum or be affected by strong noise interference. At this time, the fast focusing process will be terminated immediately, and a mode switching signal will be triggered to start the fine focusing mechanism. In addition, in order to avoid infinite loop search, when the industrial camera has reached the preset maximum number of search attempts by adjusting its position but still has not met the convergence condition, it is directly determined that the current imaging environment does not meet the fast focusing condition, and then it will switch to the fine focusing mechanism.

[0103] S4. Under the fine focusing mechanism, fine scanning is performed with the coarse focus position or the initial position as the center to build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient.

[0104] In step S4, after the fine focusing mechanism is activated, a fine scan is performed in its neighborhood, using the coarsely adjusted position as a reference or the initial position as a reference. The scanned frames are then stacked sequentially to form an image stack. The pupil region is automatically segmented for each frame in the image stack to extract the grayscale distribution features of the pupil region. By combining Zernike polynomial fitting with the wavefront aberrations, the aberration coefficients of each order can be determined. Finally, the optimal focus position is determined based on the aberration coefficients. The steps of performing a fine scan centered on the coarsely adjusted focus position or the initial position, constructing the image stack, and extracting the pupil region of each frame in the image stack include:

[0105] Centered on the coarse focus position or initial position, perform bidirectional scanning near the focal plane at a preset fine step size to acquire multiple images at different focal length positions and form an image stack;

[0106] Binarization segmentation is performed on each frame of the image stack to separate the foreground from the background and extract the region contour of the pupil area.

[0107] Perform morphological closing operations on the binarized image to fill holes in the region contour and smooth the edges;

[0108] The connected regions are extracted from the smoothed binarized image, and the geometric features of each connected region are calculated. Based on the geometric features, a single region that meets the preset pupil features is selected from all connected regions and recorded as the pupil region.

[0109] Specifically, when extracting the pupil region from each frame in the image stack, a bidirectional scan is first performed around the coarse focus position or initial position, with a preset fine step size, covering a small focal length range (circular or near-circular elliptical). This acquires multiple images at different focal length positions, which are then arranged sequentially to form an image stack for fine focus analysis. For each frame in the image stack, binarization segmentation is first performed. Using a preset grayscale threshold, the brighter foreground region (i.e., the potential pupil region) is separated from the darker background region, resulting in a preliminary binarized image. Due to potential noise interference or incomplete target edges during imaging, a morphological closing operation is performed on the binarized image, using a pre-set size structuring element to... The image is eroded and then dilated to fill in any tiny holes that may exist inside the region's outline and smooth out any burrs and irregular details at the region's edges, making the pupil region's outline more complete and continuous. After morphological processing, all connected regions are extracted from the smoothed binarized image. Each connected region represents a possible target region. By calculating the geometric features of each connected region (including but not limited to area, perimeter, aspect ratio of the circumscribed rectangle, circularity, and centroid coordinates), a single region that meets the preset pupil features is finally selected from all connected regions based on the geometric features. For example, connected regions with an area within a preset range, a circularity higher than a set threshold, and an aspect ratio of the circumscribed rectangle close to 1 are selected and identified as the pupil region of that frame image, providing the corresponding regional basis for subsequent aberration coefficient calculations.

[0110] Secondly, the steps of calculating the aberration coefficients of each pupil region and then determining the optimal focal position based on the aberration coefficients include:

[0111] Calculate the Zernike polynomial coefficients within each pupil region and extract the defocus term coefficients that directly correspond to the defocus aberration as aberration coefficients;

[0112] Within the entire focal length range of the fine scan, find the image frame with the smallest absolute value of the aberration coefficient, and determine the focal length position corresponding to the image frame as the optimal focus position;

[0113] If multiple image frames have the same absolute value of aberration coefficient and all of them are the minimum value, then the image frame with the middle focal length position is selected as the optimal focal position.

[0114] Specifically, when determining the optimal focus position, a Zernike polynomial is first fitted to each pupil region. The coefficients of each term correspond to different types of aberrations. In this mechanism, the coefficients of the defocus term, which are directly related to defocus aberrations, are focused on and extracted as aberration coefficients. The absolute value of the aberration coefficient directly reflects the degree to which the current focal length position deviates from the ideal focal plane. The smaller the absolute value, the smaller the defocus amount and the closer the image is to a clear state. Within the entire focal length range acquired by fine scanning, the aberration coefficients corresponding to each frame in the image stack are traversed. By comparison, the image frame with the smallest absolute value of the aberration coefficient is found. The focal length position corresponding to this image frame is the theoretically optimal focus position. If multiple image frames within the scanning range have the same absolute value of the aberration coefficient and are all the minimum value, to ensure the stability and centrality of the focusing result, the frame with the central focal length position is selected as the final optimal focus position, thus completing the fine focusing process.

[0115] S5. Perform confidence verification on the optimal focus position, and complete the focusing operation after passing the confidence verification at the optimal focus position;

[0116] In step S5, after the optimal focus position is determined, a further confidence verification process is performed to evaluate the reliability of the optimal focus position. The focusing action is only formally executed after the verification is passed. The confidence verification step for the optimal focus position includes:

[0117] The astigmatic coefficients are extracted from the Zernike polynomial coefficients, and the absolute values ​​of the astigmatic coefficients are processed and recorded as the first verification index.

[0118] The aberration coefficients are processed by absolute value and recorded as the second verification index;

[0119] The roundness of the pupil area was calculated and recorded as the third verification indicator;

[0120] The first, second, and third verification indicators are weighted and combined to form a comprehensive confidence score.

[0121] Compare the overall confidence score with the confidence threshold;

[0122] If the overall confidence score is not lower than the confidence threshold, the optimal focus position is determined to be valid, and the optimal focus position is output as the final focusing result.

[0123] If the overall confidence score is lower than the confidence threshold, the optimal focus position is deemed invalid, and a small step trial movement is performed. Based on the current optimal focus position, the movement is performed in both positive and negative directions by a preset step size. Two frames of images are acquired and the aberration coefficient calculation and optimal focus determination process are repeated until the confidence threshold is met.

[0124] If the number of small-step trial moves reaches the upper limit of the number of trials and still does not meet the confidence threshold, the focusing is determined to be a failure and a focusing failure log is recorded.

[0125] Specifically, when verifying the confidence level of the optimal focus position, the astigmatism coefficient is first extracted from the Zernike polynomial coefficients. Astigmatism is another common type of aberration, and the absolute value of the astigmatism coefficient reflects the difference in image sharpness in different directions. In this implementation, it is recorded as the first verification indicator. Simultaneously, the absolute value of the previously calculated aberration coefficient (i.e., the defocus term coefficient) is used as the second verification indicator. Additionally, the roundness of the pupil region is calculated. The closer the roundness is to 1, the more regular the shape of the pupil region and the more stable the image quality. This is recorded as the third verification indicator. Then, the first, second, and third verification indicators are weighted and combined. For example, based on the degree of influence of each indicator on focusing accuracy in practical applications, the aberration coefficient (second verification indicator) is given the highest weight, followed by the astigmatism coefficient (first verification indicator), and then the roundness (third verification indicator) is given an appropriate weight. A comprehensive confidence score is obtained through weighted summation. This comprehensive confidence score is then compared with a preset confidence threshold. If the comprehensive confidence score is not lower than the preset threshold, the confidence level is determined. If the confidence score is below the confidence threshold, the currently determined optimal focus position is considered highly reliable and valid, and the optimal focus position is directly output as the final focusing result. If the overall confidence score is below the confidence threshold, the optimal focus position is considered to have errors or be affected by interference. In this case, a small-step trial movement mechanism is initiated. Using the current optimal focus position as a reference, the system moves a preset small step along the positive and negative directions of the optical axis, acquiring one frame at each new position. Then, the pupil region extraction, aberration coefficient calculation, and optimal focus determination process are re-executed on the two new frames to obtain new candidate optimal focus positions, and the confidence is verified again. This process is repeated until the overall confidence score meets the confidence threshold, at which point the optimal focus position is output. If the overall confidence score fails to reach the confidence threshold within the preset limit of small-step trial movements, the focusing is considered a failure, and relevant log information about the focusing failure is automatically recorded, including the focus position at the time of failure, the values ​​of each verification index, and the overall confidence score, for subsequent analysis and system optimization.

[0126] Please see Figure 2 A machine vision automatic precision focusing system based on dual-mode evaluation, using the aforementioned machine vision automatic precision focusing method based on dual-mode evaluation, includes:

[0127] The initialization module is used to move the industrial camera above the area to be inspected and to acquire images of the target in the area to be inspected, thereby obtaining initial image data. Then, by preprocessing the initial image data, the image to be identified is output.

[0128] The pattern matching module is used to evaluate the texture of the image to be recognized, obtain an image sharpness score, and match the corresponding focusing mode based on the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism.

[0129] The fast focus module is used to perform focus search based on image gradient under the fast focus mechanism, locate the coarse focus position, determine whether it meets the fast convergence condition, and switch to the fine focus mechanism if it does not meet the condition.

[0130] The fine focus module is used to perform fine scanning with the coarse focus position or the initial position as the center under the fine focus mechanism, build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient.

[0131] The focus evaluation module is used to verify the confidence level of the optimal focus position, and after the confidence level verification is passed at the optimal focus position, the focusing operation is completed.

[0132] The execution process of the above focusing system corresponds exactly to the process of the aforementioned method, and will not be repeated here.

[0133] Please see Figure 3 An electronic device, characterized in that the electronic device comprises:

[0134] At least one processor;

[0135] and a memory communicatively connected to the at least one processor;

[0136] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described machine vision automatic precision focusing method based on dual-mode evaluation.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0138] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A machine vision automatic precision focusing method based on dual-mode evaluation, characterized in that, include: The industrial camera is moved above the area to be inspected and the target in the area to be inspected is captured to obtain initial image data. Then, the initial image data is preprocessed to output the image to be identified. The texture of the image to be recognized is evaluated to obtain an image sharpness score, and the corresponding focusing mode is matched according to the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism. Under the fast focusing mechanism, focus search based on image gradient is performed to locate the coarse focus position and determine whether it meets the fast convergence condition. If it does not meet the condition, it switches to the fine focusing mechanism. Under the fine focusing mechanism, fine scanning is performed with the coarse focus position or the initial position as the center to build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient. The confidence level of the optimal focus position is verified, and the focusing operation is completed after the confidence level verification is passed at the optimal focus position; The step of evaluating the texture of the image to be recognized to obtain an image sharpness score includes: The image to be recognized is processed into a grid to obtain multiple sub-image blocks, and the standard deviation of the pixel values ​​within each sub-image block is calculated. Calculate the arithmetic mean of the standard deviations of all sub-image patches and the overall standard deviation, and record them as the first texture index and the second texture index, respectively. The first texture index and the second texture index are input into the image sharpness evaluation function, and the output of the sharpness evaluation function is recorded as the image sharpness score. The expression of the image sharpness evaluation function is as follows: In the formula, Indicates the image sharpness score. Indicates the first texture index, Indicates the second texture index. This represents the adjustment coefficient, which takes the value of a positive real number.

2. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, The step of preprocessing the initial image data and outputting the image to be recognized includes: The initial image data is denoised by using an adaptive median filter to eliminate random noise during the imaging process, resulting in a denoised image. Histogram equalization is used to enhance the contrast of the denoised image and improve the texture discernibility in low-light areas, resulting in an image to be identified for focusing.

3. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, The step of matching the corresponding focusing mode based on the image sharpness score includes: Obtain the mode switching threshold and compare the image sharpness score with the mode switching threshold; When the image sharpness score is greater than or equal to the mode switching threshold, it indicates that the sharpness of the image to be identified meets the conditions for fast focusing, and it directly enters the fast focusing mechanism. When the image sharpness score is less than the mode switching threshold, it indicates that the image to be identified is blurry and cannot meet the requirements of fast focusing, and a fine focusing mechanism is required. The mode switching threshold is adaptively set according to the imaging scene, and the specific process is as follows: Multiple reference images are obtained by scanning a fixed area at the initial position using an industrial camera; Calculate the image sharpness score for all reference images, and then take a weighted average of the maximum and minimum image sharpness scores to obtain the dynamic mode switching threshold.

4. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, Under the fast focusing mechanism, the steps of performing focus search based on image gradients to locate the coarse focus position, determining whether it meets the fast convergence condition, and switching to the fine focusing mechanism if it does not, include: The industrial camera is moved by a preset initial step size, and real-time images are acquired at the stopping position of the industrial camera. Calculate the mean gradient of each real-time image frame and determine whether the gradient change rate between two adjacent frames is less than the allowable convergence threshold. If the gradient change rate is less than or equal to the allowable convergence threshold, and the corresponding image gradient mean reaches a local maximum, then the current stopping position is determined to be the coarse adjustment focus. If the gradient rate of change is greater than the allowable convergence threshold, the position of the industrial camera will continue to be adjusted with decreasing step size until the convergence condition is met and then stop. In the process of adjusting the position of the industrial camera with decreasing step size, if the gradient change rate shows a monotonically increasing trend, then the current search direction and search step size are maintained and the search continues. When the gradient rate of change shows a monotonically decreasing trend, the search direction is reversed and the step size is reduced; If the gradient change rate does not show a monotonically changing trend, it is judged as a search anomaly, and the fast focusing process is immediately terminated, and a mode switching signal is triggered to start the fine focusing mechanism. If the industrial camera fails to meet the convergence condition even after adjusting its position to reach the maximum number of searches, it indicates that the current imaging environment does not meet the conditions for fast focusing, and it directly switches to the fine focusing mechanism.

5. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, The steps of performing fine scanning centered on the coarsely adjusted focal position or initial position, constructing an image stack, and extracting the pupil region of each frame in the image stack include: Centered on the coarse focus position or initial position, perform bidirectional scanning near the focal plane at a preset fine step size to acquire multiple images at different focal length positions and form an image stack; Binarization segmentation is performed on each frame of the image stack to separate the foreground from the background and extract the region contour of the pupil area. Perform morphological closing operations on the binarized image to fill holes in the region contour and smooth the edges; Connected regions are extracted from the smoothed binarized image, and the geometric features of each connected region are calculated. Based on the geometric features, a single region that meets the preset pupil features is selected from all connected regions and recorded as the pupil region.

6. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, The step of calculating the aberration coefficients of each pupil region and then determining the optimal focal position based on the aberration coefficients includes: Calculate the Zernike polynomial coefficients within each pupil region and extract the defocus term coefficients that directly correspond to the defocus aberration as aberration coefficients; Within the entire focal length range of the fine scan, find the image frame with the smallest absolute value of the aberration coefficient, and determine the focal length position corresponding to the image frame as the optimal focus position; If multiple image frames have the same absolute value of aberration coefficients and all are the minimum value, then the image frame with the middle focal length is selected as the optimal focal position.

7. The machine vision automatic precision focusing method based on dual-mode evaluation according to claim 1, characterized in that, The step of performing confidence verification on the optimal focal position includes: The astigmatic coefficients are extracted from the Zernike polynomial coefficients, and the absolute values ​​of the astigmatic coefficients are processed and recorded as the first verification index. The aberration coefficients are processed by absolute value and recorded as the second verification index; The roundness of the pupil area was calculated and recorded as the third verification indicator; The first, second, and third verification indicators are weighted and combined to form a comprehensive confidence score. Compare the overall confidence score with the confidence threshold; If the overall confidence score is not lower than the confidence threshold, the optimal focus position is determined to be valid, and the optimal focus position is output as the final focusing result. If the overall confidence score is lower than the confidence threshold, the optimal focus position is deemed invalid, and a small step trial movement is performed. Based on the current optimal focus position, the movement is performed in both positive and negative directions by a preset step size. Two frames of images are acquired and the aberration coefficient calculation and optimal focus determination process are repeated until the confidence threshold is met. If the number of small-step trial moves reaches the upper limit of the number of trials but still does not meet the confidence threshold, the focusing is determined to have failed and a focusing failure log is recorded.

8. A machine vision automatic precision focusing system based on dual-mode evaluation, characterized in that, The machine vision automatic precision focusing method based on dual-mode evaluation according to any one of claims 1 to 7 includes: The initialization module is used to move the industrial camera above the area to be inspected and to acquire images of the target in the area to be inspected, thereby obtaining initial image data. Then, by preprocessing the initial image data, the image to be identified is output. The pattern matching module is used to evaluate the texture of the image to be recognized, obtain an image sharpness score, and match the corresponding focusing mode based on the image sharpness score. The focusing mode includes a fast focusing mechanism and a fine focusing mechanism. The fast focus module is used to perform focus search based on image gradient under the fast focus mechanism, locate the coarse focus position, determine whether it meets the fast convergence condition, and switch to the fine focus mechanism if it does not meet the condition. The fine focus module is used to perform fine scanning with the coarse focus position or the initial position as the center under the fine focus mechanism, build an image stack, extract the pupil region of each frame in the image stack, calculate the aberration coefficient of each pupil region, and then determine the optimal focus position based on the aberration coefficient. The focus evaluation module is used to verify the confidence level of the optimal focus position, and after the confidence level verification is passed at the optimal focus position, the focusing operation is completed.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the machine vision automatic precision focusing method based on dual-mode evaluation as described in any one of claims 1 to 7.

Images