Multi-focal-plane focusing window adaptive selection method and device, storage medium and program product

Abstract

The invention discloses a multi-focal plane focusing window adaptive selection method and device, a storage medium and a program product, and the method specifically comprises the steps: firstly obtaining a plurality of focusing sequence images, screening out a plurality of candidate focal plane points through the full-image definition detection, then segmenting the images into a plurality of small regions, carrying out the definition detection of each small region, and obtaining a plurality of focusing sequence images; judging whether candidate focal plane points are supported or not; the focal plane which is closest to the current position of the camera and has the largest supporting area is selected as a final focusing window by counting the supporting condition of the focusing plane of each area; according to the invention, by combining the bilateral filter denoising and high-frequency information extraction technologies, the precision of definition detection can be improved, a multi-focal-plane scene can be automatically adapted, the focusing precision and robustness in a complex environment are ensured, and a focusing window selection function is realized.

Description

Technical Field

[0001] The present invention relates to the field of automatic focusing technology, and in particular to a method, device, storage medium and program product for adaptively selecting focus windows on multiple focal planes. Background Art

[0002] Existing autofocus technology is widely used in photography, security and other fields. Figure 2 As shown in Figure 1, focusing in these fields is usually based on the central area of ​​the image, using central area method, multi-area method or non-uniform sampling method to determine the focus point. These methods assume that the subject is mostly in the central area of ​​the image, so when dealing with industrial measurement where the object distribution is not central or there are multiple focal planes, it is easy to cause focusing failure.

[0003] In industrial measurement scenarios, samples are irregular in shape and may have multiple focal planes at the same time. If we rely on traditional focusing algorithms, it is often difficult to accurately select the best focal plane. Therefore, there is an urgent need for an autofocus technology that can adaptively select the focus window and effectively deal with multiple focal planes to improve measurement accuracy and focusing effect. Summary of the invention

[0004] Technical purpose: In view of the shortcomings of existing autofocus technology in industrial measurement scenarios, the present invention discloses a multi-focal plane focus window adaptive selection method, device, storage medium and program product, which can automatically select the best focus area of ​​multiple focal planes, and is particularly suitable for application scenarios with complex focal plane distribution.

[0005] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0006] A method for adaptively selecting a focus window for multiple focal planes, comprising the following steps:

[0007] Acquire multiple focus sequence images, with the index being k, k=1, 2, 3…, n, and each image being a frame image;

[0008] Set the focus area of ​​each frame image to the full image size, recorded as ROI t , and perform clarity detection on each frame of image, and calculate the full image clarity value of each frame of image based on high-frequency information, recorded as V(ROI t ,k);

[0009] The focus area is divided into small areas of r×c, recorded as ROI (r,c) , and perform clarity detection on each small area, calculate the clarity value of each small area, and record it as V(ROI (r,c) ,k);

[0010] For each frame of the image, the full image clarity value V(ROI t,k) to perform statistics and find the image index k corresponding to all maximum points t , and add these maximum value points to the focus point set E1 to be selected, and the images corresponding to these maximum value points are recorded as candidate focus images;

[0011] For each small area definition value V(ROI (r,c) ,k) to perform statistics and find the image index k corresponding to the maximum value point of each small area (r,c) , and compare it with the image index k corresponding to all the maximum points in the selected focal point set E1 t In comparison, if k (r,c) Equal to k t , then determine the small area as the support area of ​​the candidate focus plane image;

[0012] Count the support regions of each candidate focus plane image, and combine these regions to form a candidate focus window;

[0013] Get the current position of the camera, determine the distance between the camera position and each candidate focus window, select the candidate focus plane image closest to the camera position as the final focus window, and perform autofocus based on this focus plane.

[0014] Preferably, performing definition detection on each frame of image and calculating the full-image definition value of each frame of image based on high-frequency information specifically includes the following steps:

[0015] Use bilateral filter to denoise each frame of image to eliminate the interference of random noise in imaging process on definition calculation;

[0016] The high-frequency information of each frame image is extracted through the first-order derivative and the second-order derivative, and the high-frequency information points are screened, and the screened high-frequency information points are added to the set E;

[0017] Calculate the clarity value of the high-frequency information points in set E.

[0018] Preferably, the formula of the bilateral filter is:

[0019]

[0020] Among them, g represents the filtering result, (i, j) represents the coordinates of other pixels in the template window, (m, l) represents the coordinates of the center point in the template window, f represents the pixel gray value, and w represents the weighted sum of the spatial domain parameter and the value range parameter.

[0021] Preferably, the calculation formula of the spatial domain kernel of the bilateral filter is:

[0022]

[0023] The calculation formula of the range kernel of the bilateral filter is:

[0024]

[0025] Among them, w d represents the spatial domain parameter, w r represents the range parameter, σ d With σ r Represent the standard deviation parameters of the spatial domain and range respectively.

[0026] Preferably, extracting high-frequency information of each frame of image by first-order derivatives and second-order derivatives and screening high-frequency information points, and adding the screened high-frequency information points to set E specifically comprises the following steps:

[0027] Use the convolution kernel to perform convolution operation on each frame of the image to obtain the first-order derivatives in the X and Y directions, where the convolution kernel in the X direction is The convolution kernel in the Y direction is The convolution operation results corresponding to the X and Y directions are G1(x,y) and G2(x,y) respectively. The first-order derivative value of the results in the two directions is unified to be

[0028] Use the Laplace convolution kernel to perform convolution operation on each frame of the image to obtain the second-order derivative value L(x,y), where the Laplace convolution kernel is

[0029] The image is traversed. If the first-order derivative G(x, y) of a pixel point is greater than half of the maximum value of the first-order derivative on the image or the second-order derivative L(x, y) of the pixel point is greater than half of the maximum value of the second-order derivative on the image, the point is considered to be a high-frequency information point and is added to the set E.

[0030] Preferably, the calculation formula for the clarity value of the high-frequency information points in the set E is as follows:

[0031]

[0032] Among them, C represents the clarity value, (x, y) represents the coordinates of the high-frequency information point, N represents the number of high-frequency information points in the E set, and G(x, y) represents the first-order derivative value.

[0033] Preferably, the focus area is divided into small areas of 12×12.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for adaptively selecting a focus window on a multi-focal plane as described above is implemented.

[0035] The present invention also provides a storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the multi-focal plane focus window adaptive selection method as described above.

[0036] The present invention also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned multi-focal plane focus window adaptive selection method.

[0037] Beneficial effects: The method, device, storage medium and program product for adaptively selecting a focus window on a multi-focal plane provided by the present invention have the following beneficial effects:

[0038] 1. The present invention combines the method of full-image clarity detection with the method of local clarity detection, and can automatically identify and select the most suitable focus plane in complex scenes with multiple focal planes, thereby avoiding the focus failure problem of the traditional single focal plane algorithm; by comparing the maximum clarity points of the full image and the local area, the present invention can dynamically select a focus window that adapts to the current scene, and combine the camera position and the size of the focal plane support area to give priority to automatically focusing on the nearest focal plane with the largest support area, thereby improving the flexibility and accuracy of focusing.

[0039] 2. The present invention adopts bilateral filter denoising and extracts high-frequency information through first-order derivatives and second-order derivatives, which effectively eliminates the interference of noise, improves the accuracy and robustness of clarity calculation, and ensures that accurate focusing results can be obtained even in high-noise environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0041] Figure 1 It is an overall flow chart of the method for adaptively selecting focus windows on multiple focal planes of the present invention;

[0042] Figure 2 A schematic diagram of a multi-focal window selection method in the prior art;

[0043] Figures 3 to 6 A schematic diagram of the process of selecting a focus window under different focal planes of the present invention;

[0044] Figure 7 This is a full-image clarity curve diagram of the present invention when performing full-image clarity detection. DETAILED DESCRIPTION

[0045] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0046] like Figure 1 As shown, the present invention provides a method for adaptively selecting a focus window on multiple focal planes, which specifically includes the following steps:

[0047] S1. Acquire multiple focus sequence images, with the index being k, k=1, 2, 3, ..., n, and each image being a frame image.

[0048] S2. Set the focus area of ​​each frame image to the size of the full image, recorded as ROI t , and perform clarity detection on each frame of image, and calculate the full image clarity value of each frame of image based on high-frequency information, recorded as V(ROI t ,k).

[0049] In one embodiment, performing definition detection on each frame of image and calculating the definition value of the entire image of each frame of image based on high-frequency information specifically includes the following steps:

[0050] S21. Use a bilateral filter to perform denoising on each frame of the image to eliminate the interference of random noise in the imaging process on the clarity calculation.

[0051] The imaging process of the camera will be affected by random noise, which is reflected in the image as the noise component superimposed on the grayscale value of the pixel. When calculating the clarity, the interference of the noise will cause the clarity curve to not satisfy the unimodal property. Therefore, the image needs to be filtered before the clarity calculation.

[0052] The bilateral filter used in the present invention can better protect high-frequency details while removing noise. The calculation formula of the spatial domain kernel of the bilateral filter is:

[0053]

[0054] The calculation formula of the range kernel of the bilateral filter is:

[0055]

[0056] Among them, w d represents the spatial domain parameter, w r represents the range parameter, σ d With σ r Represent the standard deviation parameters of the spatial domain and range respectively.

[0057] The multiplication of the spatial domain kernel and the range kernel is:

[0058]

[0059] From this, the formula for the bilateral filter can be obtained as follows:

[0060]

[0061] Among them, g represents the filtering result, (i, j) represents the coordinates of other pixels in the template window, (m, l) represents the coordinates of the center point in the template window, f represents the pixel gray value, and w represents the weighted sum of the spatial domain parameter and the value range parameter.

[0062] S22. Based on the denoised image, high-frequency information of each frame of the image is extracted through first-order derivatives and second-order derivatives, and high-frequency information points are screened, and the screened high-frequency information points are added to the set E.

[0063] There are high-frequency areas with richer information and low-frequency areas with less information in the focus area. When the low-frequency areas are involved in the clarity calculation, they will interfere with the clarity value. Therefore, it is necessary to extract the high-frequency information and perform clarity calculation on it separately. The present invention extracts high-frequency information from the image based on the first-order and second-order derivatives, and specifically includes the following steps:

[0064] Use the convolution kernel to perform convolution operation on each frame of the image to obtain the first-order derivatives in the X and Y directions, where the convolution kernel in the X direction is The convolution kernel in the Y direction is The convolution operation results corresponding to the X and Y directions are G1(x,y) and G2(x,y) respectively. The first-order derivative value of the results in the two directions is unified to be

[0065] Use the Laplace convolution kernel to perform convolution operation on each frame of the image to obtain the second-order derivative value L(x,y), where the Laplace convolution kernel is

[0066] The image is traversed. If the first-order derivative G(x,y) of a pixel point is greater than half of the maximum value of the first-order derivative on the image or the second-order derivative L(x,y) of the pixel point is greater than half of the maximum value of the second-order derivative on the image, then the point is considered to be a high-frequency information point and added to the set E, that is, the first 50% of the pixels of the first-order derivative or the second-order derivative are selected. If the first-order derivative G(x1,y1) of the pixel point (x1,y1)>max(G(x,y)) / 2 or the second-order derivative L(x1,y1)>max(L(x,y)) / 2, then the point is considered to be a high-frequency pixel point and added to the set E.

[0067] S23. Calculate the clarity value of the high-frequency information points in set E.

[0068] In one embodiment, the formula for calculating the clarity value of high-frequency information points in set E is as follows:

[0069]

[0070] Among them, C represents the clarity value, (x, y) represents the coordinates of the high-frequency information point, N represents the number of high-frequency information points in the E set, and G(x, y) represents the first-order derivative value.

[0071] S3: Divide the focus area into r×c small areas, recorded as ROI (r,c) , and perform clarity detection on each small area, calculate the clarity value of each small area, and record it as V(ROI (r,c) ,k).

[0072] In one embodiment, the focus area may be divided into 12×12 small areas, each of which is a small rectangle.

[0073] S4, the full image clarity value V(ROI t ,k) to perform statistics and find the image index k corresponding to all maximum points t , that is, the clarity value of this point is greater than the clarity values ​​of the two points on its left and right, and these maximum value points are added to the focus point set E1 to be selected, and the images corresponding to these maximum value points are recorded as candidate focus images.

[0074] S5, for each small area definition value V(ROI (r,c) ,k) to perform statistics and find the image index k corresponding to the maximum value point of each small area (r,c) , and compare it with the image index k corresponding to all the maximum points in the selected focal point set E1 t In comparison, if k (r,c) Equal to k t , then the small area is determined to be the support area of ​​the candidate focus plane image.

[0075] like Figure 7 As shown, the horizontal axis represents the image index and the vertical axis represents the clarity value. When there are multiple focal planes in the field of view, when the captured image is subjected to full-image clarity detection, the focus curve will have multiple peaks, and each peak corresponds to a focal plane. When the peak value of a small area coincides with one of the focal planes, the small area is the support area of ​​the focal plane, and the superposition of the support areas of each focal plane constitutes a focus area that supports the focal plane.

[0076] After all regions are compared, the maximum points that do not have a supporting region will be deleted as noise points, and the maximum points that exist in a supporting region will be defined as positive focal points, and the set of small regions that support them will be the focus window of the positive focal point.

[0077] S6. Count the support areas of each candidate focus plane image, and combine these areas to form a candidate focus window. The autofocus system performs a final focus operation based on this window.

[0078] S7, obtaining the current position of the camera, determining the distance between the camera position and each candidate focus window, selecting the candidate focus plane image closest to the camera position as the final focus window, and performing autofocus based on this focus plane.

[0079] like Figures 3 to 6 As shown in the figure, the selection process and selection results of the focus window in the two cases of focus plane 1 and focus plane 2 are included, wherein Figure 3 and Figure 4 They are respectively the focus window selection diagram for focal plane 1 and the focus window selection diagram for focal plane 2. Figure 5 and Figure 6 They are respectively a schematic diagram of the focusing result of focal plane 1 and a schematic diagram of the focusing result of focal plane 2.

[0080] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned methods for adaptively selecting a focus window in a multi-focal plane is implemented. The memory may be any type of memory, such as a random access memory, a read-only memory, a flash memory, etc. The processor may be any type of processor, such as a central processing unit, a microprocessor, a digital signal processor, or an image processor, etc.

[0081] The present invention also provides a storage medium storing computer executable instructions for executing any of the above-mentioned methods for adaptively selecting a focus window on a multi-focal plane. The storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0082] The present invention also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the multi-focal plane focus window adaptive selection method as described above, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0083] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for adaptively selecting a focus window for multiple focal planes, characterized in that: The specific steps include: Acquire multiple focus sequence images, with the index being k, k=1, 2, 3…, n, and each image being a frame image; Set the focus area of ​​each frame image to the full image size, recorded as ROI t , and perform clarity detection on each frame of image, and calculate the full image clarity value of each frame of image based on high-frequency information, recorded as V(ROI t ,k); The focus area is divided into small areas of r×c, recorded as ROI (r,c) , and perform clarity detection on each small area, calculate the clarity value of each small area, and record it as V(ROI (r,c) ,k); For each frame of the image, the full image clarity value V(ROI t ,k) to perform statistics and find the image index k corresponding to all maximum points t , and add these maximum value points to the focus point set E1 to be selected, and the images corresponding to these maximum value points are recorded as candidate focus images; For each small area definition value V(ROI (r,c) ,k) to perform statistics and find the image index k corresponding to the maximum value point of each small area (r,c) , and compare it with the image index k corresponding to all the maximum points in the selected focal point set E1 t In comparison, if k (r,c) Equal to k t , then determine the small area as the support area of ​​the candidate focus plane image; Count the support regions of each candidate focus plane image, and combine these regions to form a candidate focus window; Get the current position of the camera, determine the distance between the camera position and each candidate focus window, select the candidate focus plane image closest to the camera position as the final focus window, and perform autofocus based on this focus plane.

2. The method for adaptively selecting a focus window for multiple focal planes according to claim 1, characterized in that: Performing clarity detection on each frame of image and calculating the full-image clarity value of each frame of image based on high-frequency information specifically includes the following steps: Use bilateral filter to denoise each frame of image to eliminate the interference of random noise in imaging process on definition calculation; The high-frequency information of each frame image is extracted through the first-order derivative and the second-order derivative, and the high-frequency information points are screened, and the screened high-frequency information points are added to the set E; Calculate the clarity value of the high-frequency information points in set E.

3. The method for adaptively selecting a focus window for multiple focal planes according to claim 2, characterized in that: The formula for the bilateral filter is: Among them, g represents the filtering result, (i, j) represents the coordinates of other pixels in the template window, (m, l) represents the coordinates of the center point in the template window, f represents the pixel gray value, and w represents the weighted sum of the spatial domain parameter and the value range parameter.

4. The method for adaptively selecting a focus window for multiple focal planes according to claim 3, characterized in that: The calculation formula of the spatial domain kernel of the bilateral filter is: The calculation formula of the range kernel of the bilateral filter is: Among them, w d represents the spatial domain parameter, w r represents the range parameter, σ d With σ r Represent the standard deviation parameters of the spatial domain and range respectively.

5. The method for adaptively selecting a focus window for multiple focal planes according to claim 2, characterized in that: Extracting high-frequency information of each frame of image through first-order derivatives and second-order derivatives and filtering high-frequency information points, and adding the filtered high-frequency information points to set E specifically includes the following steps: Use the convolution kernel to perform convolution operation on each frame of the image to obtain the first-order derivatives in the X and Y directions, where the convolution kernel in the X direction is The convolution kernel in the Y direction is The convolution operation results corresponding to the X and Y directions are G1(x,y) and G2(x,y) respectively. The first-order derivative value of the results in the two directions is unified to be Use the Laplace convolution kernel to perform convolution operation on each frame of the image to obtain the second-order derivative value L(x,y), where the Laplace convolution kernel is The image is traversed. If the first-order derivative G(x, y) of a pixel point is greater than half of the maximum value of the first-order derivative on the image or the second-order derivative L(x, y) of the pixel point is greater than half of the maximum value of the second-order derivative on the image, the point is considered to be a high-frequency information point and is added to the set E.

6. The method for adaptively selecting a focus window for multiple focal planes according to claim 2, characterized in that: The formula for calculating the clarity value of high-frequency information points in set E is as follows: Among them, C represents the clarity value, (x, y) represents the coordinates of the high-frequency information point, N represents the number of high-frequency information points in the E set, and G(x, y) represents the first-order derivative value.

7. The method for adaptively selecting a focus window for multiple focal planes according to claim 1, characterized in that: The focus area is divided into small areas of 12×12.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a multi-focal plane focus window adaptive selection method as described in any one of claims 1 to 7 is implemented.

9. A storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the multi-focal plane focus window adaptive selection method as described in any one of claims 1-7.

10. A computer program product comprising instructions, characterized in that When the instruction is executed on a computer, the computer is caused to execute the method for adaptively selecting focus windows on multiple focal planes as described in any one of claims 1 to 7.

Images