Zoom lens automatic focusing region-of-interest selection method

Abstract

The invention provides an automatic focusing region-of-interest selection method for a zoom lens, which meets the requirement that a scene focus can fall in an ideal region when the depth of field is small, and ensures that the region is clearest. The method comprises the following steps: 1, segmenting an acquired image, calculating an FV value of each region, and outputting and storing the FV value as an original FV value matrix; 2, respectively carrying out convolution calculation on the vertical kernel weight matrix and the horizontal kernel weight matrix and the original FV value matrix; 3, when the focusing motor is pulled once, a vertical kernel weight FV value matrix and a horizontal kernel weight FV value matrix of the current frame image are respectively calculated and obtained; 4, comparing the area vertical kernel weight FV value and the area horizontal kernel weight FV value of the continuous front and back adjacent frames in the same area, and recording the maximum value and the position of the focusing motor corresponding to the maximum value; 5, taking an area corresponding to the minimum value in the positions of the focusing motor, and setting the area as a focusing interested area; and 6, focusing the focusing region of interest.

Description

Technical Field

[0001] The present invention relates to a method for selecting an area of interest for automatic focusing of a zoom lens, which is applied in the field of cameras. Background Art

[0002] In recent years, with the rapid development of video education, remote conferencing, and live streaming e-commerce, it has greatly promoted the development of educational cameras, conference cameras, and live cameras, and put forward higher requirements for the focusing effect of cameras.

[0003] In the online education scenario, the main method is in-class recording in the classroom. Generally, high-power zoom cameras are used in classrooms. The cameras are installed on the back wall of the classroom. When the teacher is teaching, the camera will be zoomed in at a high magnification to give a close-up of the lecturer. At this time, the lecturer accounts for 1 / 3 - 2 / 3 in the picture. When the zoom lens is pulled in, the lens magnification becomes larger and the depth of field becomes smaller, and the clarity will be significantly different.

[0004] During video conferencing, when someone is giving a report or speech, the zoom lens also needs to give a close-up of the main person. Similarly, after the picture becomes larger, the depth of field will become very small, and there will be a very obvious difference in the front and back clarity. In addition, at the live streaming e-commerce site that is very popular now, when the anchor is introducing the products he is selling, he needs to show some specific food production ingredients, production time, and some more delicate parts of clothes and packaging to the audience. He needs to enlarge the picture and move the product to a region closer to the camera, so as to better show the current products for sale to the audience.

[0005] Whether in the scenarios of online video education, online video conferencing, or live streaming e-commerce, when using a high-power zoom lens, when pulling the zoom to give a close-up picture, the depth of field of the lens will become very small, and the front and back clarity will become particularly obvious. Under the most conforming to human senses and scene requirements, it is hoped that the scene focus will fall on the ideal area, and it is necessary to ensure that this area must be the clearest. Summary of the Invention

[0006] The purpose of the present invention is to overcome the above-mentioned deficiencies existing in the prior art, and provide a method for selecting an area of interest for automatic focusing of a zoom lens, which can satisfy that the scene focus can also fall on the ideal area when the depth of field is small, and ensure that this area is the clearest.

[0007] The technical solution adopted by the present invention to solve the above problems is: a method for selecting an area of interest for automatic focusing of a zoom lens, which is characterized in that it includes the following steps:

[0008] Step 1: Segment the collected image to form n×n regions, calculate the FV value of each region, and then output and store it as the original FV value matrix;

[0009] Step 2: Corresponding to the original FV value matrix in Step 1, a vertical kernel weight matrix and a horizontal kernel weight matrix are adopted.

[0010] Step 3: Respectively perform convolution calculations on the vertical kernel weight matrix and the horizontal kernel weight matrix with the original FV value matrix to obtain a vertical kernel weight FV value matrix and a horizontal kernel weight FV value matrix.

[0011] Step 4: Respectively save the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image.

[0012] Step 5: Drive and control the movement of the focusing motor. Each time the focusing motor is pulled, according to the above steps, calculate and obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image respectively.

[0013] Step 6: During the continuous pulling process in Step 5, compare the regional vertical kernel weight FV values and regional horizontal kernel weight FV values at the same region (i, j) of two adjacent frames before and after; where the regional vertical kernel weight FV value is the clarity value at the region (i, j) in the vertical kernel weight FV value matrix, and the regional horizontal kernel weight FV value is the clarity value at the region (i, j) in the horizontal kernel weight FV value matrix.

[0014] Record the maximum value maxFV of the regional vertical kernel weight FV values corresponding to each region 1-(i,j) , and the maximum value maxFV of the regional horizontal kernel weight FV values 2-(i,j) ;

[0015] Record the positions maxFVPos of the focusing motor corresponding to the maximum values maxFV 1-(i,j) and maxFV 2-(i,j) respectively 1-(i,j) and maxFVPos 2-(i,j) ;

[0016] Step 7: Compare maxFVPos 1-(i,j) and maxFVPos 2-(i,j) in turn, and take the region corresponding to the minimum value of maxFVPos 1-(i,j) and maxFVPos 2-(i,j) as the region of interest for focusing.

[0017] Step 8: Perform focusing on the region of interest for focusing to complete the entire autofocus process.

[0018] The n×n regions described in the present invention are 16×16 regions.

[0019] The vertical kernel weight matrix and the horizontal kernel weight matrix described in the present invention are as follows:

[0020]

[0021] is the vertical kernel weight matrix, is the horizontal kernel weight matrix.

[0022] Step (1) of the present invention specifically includes the following steps:

[0023] (11) Align the upper left corner of the vertical kernel weight matrix with the upper left corner of the original FV value matrix, multiply the vertical kernel weight matrix with the corresponding positions of the local matrix in the upper left corner of the original FV value matrix respectively, then calculate the weighted average of the multiplied and accumulated values, and use this weighted average as the weighted average FV value matrix of this area;

[0024] (12) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (11), slide the vertical kernel weight matrix to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current area, and finally obtain the vertical kernel weight FV value matrix.

[0025] Step (2) of the present invention specifically includes the following steps:

[0026] (21) Align the upper left corner of the horizontal kernel weight matrix with the upper left corner of the original FV value matrix, multiply the horizontal kernel weight matrix with the corresponding positions of the local matrix in the upper left corner of the original FV value matrix respectively, then calculate the weighted average of the multiplied and accumulated values, and use this weighted average as the weighted average FV value of this area;

[0027] (22) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (21), slide the horizontal kernel weight matrix to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current area, and finally obtain the horizontal kernel weight FV value matrix.

[0028] In step five of the present invention, the initial movement direction of the focusing motor is defined as the proximal end.

[0029] In step six of the present invention, it is necessary to judge the reverse condition of the focusing lens. After meeting the condition, it is necessary to reverse the focusing motor for processing.

[0030] The reverse condition of the focusing lens described in the present invention is that the FV values of all areas continuously decrease, or the FV values of most areas are lower than 85% of the maximum FV value corresponding to this area.

[0031] In step eight of the present invention, the focusing region of interest obtained in step seven is focused by using the hill climbing algorithm.

[0032] The specific method of the climbing algorithm of the present invention is as follows: Set the current operating speed of the focusing motor according to the image clarity of the focused region of interest; When the focusing motor is moving in the current direction and the FV values of multiple frames show an upward trend, the operating speed of the focusing motor needs to be appropriately increased; If a downward trend appears later and the FV values of several consecutive frames show an obvious downward trend, it can be judged that the highest point of the current scene has been searched along this direction, so the direction of the focusing motor is reversed to continue the search; If after several consecutive frames with continuously increasing FV values, several consecutive frames show an obvious downward trend in the FV values, it can be determined that the clearest point of the scene has been searched, and the focusing motor goes to the position with the highest scene clarity, and the current climbing search algorithm ends, that is, the position value of the focusing motor corresponding to the clearest point of the current scene image. Compared with the prior art, the present invention has the following advantages and effects: It can quickly lock the focusing area, be compatible with foreground objects in multiple orientations, and is suitable for video education, remote conferencing, and live streaming e-commerce scenarios, meeting the requirement that the scene focus can also fall in the ideal area near the center front when the depth of field is small.

[0033] Description of the drawings

[0034] Figure 1 Schematic diagram of 16×16 regions in Embodiment 1 of the present invention.

[0035] Figure 2 Vertical kernel weight matrix in Embodiment of the present invention And fv i,j Schematic diagram of convolution.

[0036] Figure 3 Horizontal kernel weight matrix in Embodiment of the present invention And fv i,j Schematic diagram of convolution.

[0037] Figure 4 Vertical kernel weight matrix in Embodiment of the present invention And fv i,j FV after convolution 1-i,j Schematic diagram of the region and the central region.

[0038] Figure 5 Horizontal kernel weight matrix in Embodiment of the present invention And fv i,j FV after convolution 2-i,j Schematic diagram of the region and the central region.

[0039] Figure 6 Schematic diagram of the climbing algorithm in the automatic calibration curve in Embodiment of the present invention. Detailed implementation manners

[0040] The present invention will be further described in detail below with reference to the drawings and through embodiments. The following embodiments are explanations of the present invention, and the present invention is not limited to the following embodiments.

[0041] A method for selecting an autofocus region of interest for a zoom lens according to an embodiment of the present invention includes the following steps:

[0042] Step 1: Taking the Xingchen Technology chip (SSD268G) as an example, with the help of hardware devices, including an integrated circuit composed of a varifocal lens, a sensor, a chip, and various electronic devices, the chip captures a frame of image, and then evenly divides the captured image to form 16×16 square regions, as Figure 1 shown. And through the internal processing mechanism of the chip, after processing the image data of each region by filtering, the FV value of each region is calculated. The FV value is the image sharpness value FocusValue. There are a total of 16×16 region matrix data, and the data size is 256 block data, and then output and stored as the original FV value matrix fv i,j :

[0043] fv i,j , i∈[0,16], j∈[0,16].

[0044] Step 2: Corresponding to the original FV value matrix fv i,j in Step 1, different kernel weight matrices are selected. The present invention adopts two different kernel weight matrices, namely a vertical kernel weight matrix and a horizontal kernel weight matrix. The size of the vertical kernel weight matrix is 8×4, and the size of the horizontal kernel weight matrix is 4×8, and the weight values gradually decrease from the center to the edge. The data of the vertical kernel weight matrix and the horizontal kernel weight matrix are as follows:

[0045]

[0046] is the vertical kernel weight matrix, is the horizontal kernel weight matrix.

[0047] Step 3: Obtain the convolutional FV value matrix FV i,j after convolving the kernel weight matrix and the original FV value matrix fv i,j , the steps are as follows:

[0048] (1) Convolve the vertical kernel weight matrix with the original FV value matrix fv i,j to obtain the vertical kernel weight FV value matrix FV 1-(i,j) , i∈[0,9], j∈[0,13], i is the horizontal row number, and j is the vertical column number. The FV value matrix changes from the original fv i,j to FV 1-i,j , as Figure 4 shown, and the original image sharpness matrix fv i,j is dimension-reduced.

[0049] Specifically, it includes the following steps:

[0050] (11) Align the upper left corner of the vertical kernel weight matrix with the upper left corner of the original FV value matrix fv i,j as shown. Multiply the corresponding positions of the vertical kernel weight matrix with the 8×4 local matrix at the upper left corner of the original FV value matrix fv Figure 2 respectively. Then calculate the weighted average of the products after accumulation, and take this weighted average as the weighted average FV value matrix FV″ i,j of this area. The calculation method is: 1-(i,j) where fv′

[0051]

[0052] is the local sharpness value matrix corresponding to the m×n area in the original FV value matrix fv m,n . i,j

[0053] (12) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (11), slide the vertical kernel weight matrix to the right and down by a step size of 1 and calculate the weighted average FV value matrix of the current area. Finally, obtain the vertical kernel weight FV value matrix FV 1-(i,j) .

[0054] (2) Perform convolution calculation on the horizontal kernel weight matrix and the original FV value matrix fv i,j to obtain the horizontal kernel weight FV value matrix FV 2-(i,j) , where i∈[0,13], j∈[0,9], i is the horizontal row number, and j is the vertical column number. The FV value matrix changes from the original fv i,j to FV 1-(i,j) , as shown in Figure 5 . Similarly, dimensionality reduction processing is performed on the original FV value matrix fv i,j .

[0055] Specifically, it includes the following steps:

[0056] (21) Align the upper left corner of the horizontal kernel weight matrix with the upper left corner of the original FV value matrix fv i,j as shown. Multiply the corresponding positions of the horizontal kernel weight matrix with the 4×8 local matrix at the upper left corner of the original FV value matrix fv Figure 3 respectively. Then calculate the weighted average of the products after accumulation, and take this weighted average as the weighted average FV value FV″ i,j of this area. The calculation method is: 2-(i,j)

[0057]

[0058] Among them, fv ′ m,n is the local sharpness value matrix corresponding to the m×n area in the original FV value matrix fv i,j in the middle.

[0059] (22) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (21), the horizontal kernel weight matrix is slid to the right and down with a step size of 1, and the weighted average FV value matrix of the current area is calculated. Finally, the horizontal kernel weight FV value matrix FV 2-(i,j) is obtained.

[0060] Step 4: Save the vertical kernel weight FV value matrix FV 1-(i,j) and the horizontal kernel weight FV value matrix FV 2-(i,j) of the current frame image respectively.

[0061] Step 5: Drive and control the movement of the focusing motor. The initial movement direction is defined as the proximal end. Each time the focusing motor is pulled, according to the above steps, the vertical kernel weight FV value matrix FV 1-(i,j) and the horizontal kernel weight FV value matrix FV 2-(i,j) of the current frame image are calculated and obtained respectively.

[0062] Step 6: During the continuous pulling process in Step 5, compare the vertical kernel weight FV value FV′ 1-(i,j) and the horizontal kernel weight FV value FV′ 2-(i,j) of the adjacent frames before and after in the same area (i, j). Among them, the vertical kernel weight FV value FV′ 1-(i,j) is the sharpness value at the area (i, j) of the vertical kernel weight FV value matrix FV 1-(i,j) , and the horizontal kernel weight FV value FV′ 2-(i,j) is the sharpness value at the area (i, j) of the horizontal kernel weight FV value matrix FV 2-(i,j) ;

[0063] Record the maximum value maxFV ′ 1-(i,j) in the vertical kernel weight FV value FV 1-(i,j) corresponding to each area, as well as the maximum value maxFV 2-(i,j) of the horizontal kernel weight FV value FV′ 2-(i,j) ;

[0064] Record the position maxFVPos 1-(i,j) where the focusing motor is located corresponding to maxFV 2-(i,j) and maxFV 1-(i,j)and maxFVPos 2-(i , j) 。

[0065] In this step, it is necessary to judge the reverse condition of the focusing lens. After the condition is met, the focusing motor needs to be reversed, that is, processed in the left rotation direction. The reverse condition of the focusing lens is that the FV values in all regions continuously decrease, or the FV values in most regions are lower than 85% of the maximum FV value corresponding to that region.

[0066] Step Seven: According to the principle of the scene focusing center from the center towards the edge direction, that is, it is considered that the sense is mainly focused on the image center, select the focus comparison region. The way is that the region taken in the embodiment of the present invention is at 2 / 3 of the center block region of the focus center, as Figure 4 、 Figure 5 shown, that is, the edge 1 / 3 region is not considered in the comparison region; compare maxFVPos 1-(i,j) and maxFVPos 2-(i,j) , take the region corresponding to the minimum value of maxFVPos 1-(i,j) and maxFVPos 2-(i,j) , and define this region as the region of interest ROI i,j of focus, as the main focus region.

[0067] Step Eight: According to the region of interest ROI i,j of focus obtained in Step Seven, as Figure 6 shown, use the ramp algorithm to perform autofocus on the region of interest ROI i,j . When the focus of the region of interest ROI i,j is completed, the entire autofocus process is completed.

[0068] The specific method of the ramp algorithm is: Set the current operating speed of the focusing motor Focus according to the image sharpness of the region of interest ROI i,j . Among them, the operating speed of the focusing motor Focus is positively correlated with the change trend of the image sharpness of the region of interest ROI i,j . When the focusing motor Focus is along the current direction and the FV values of multiple frames are in an upward trend, the operating speed of the focusing motor Focus needs to be appropriately increased. If a downward trend appears later and the FV values of several consecutive frames show an obvious downward trend, it can be judged that the highest point of the current scene has been searched along this direction, so the direction of the focusing motor Focus is reversed to continue the search. If after the FV values continuously rise, several consecutive frames of FV values show an obvious downward trend, the clearest point of the scene can be determined. The focusing motor Focus goes to the position with the highest scene sharpness, and the current hill-climbing search algorithm ends, that is, the position value of the focusing motor Focus corresponding to the clearest point of the current scene image.

[0069] In addition, it should be noted that for the specific embodiments described in this specification, the shapes and names of the components can be different. The above content described in this specification is only an illustrative example of the structure of the present invention. Any equivalent changes or simple changes made to the structure, features, and principles according to the concept of the present invention are included in the protection scope of the present invention. Those skilled in the technical field to which the present invention pertains can make various modifications, supplements, or use similar methods for substitution to the specific embodiments described, as long as they do not deviate from the structure of the present invention or exceed the scope defined by this claims, they should fall within the protection scope of the present invention.

Claims

1. A method for selecting an autofocus region of interest for a zoom lens, characterized in that: It includes the following steps: Step 1: Segment the collected image to form n×n regions, calculate the FV value of each region, and then output and store it as the original FV value matrix; Step 2: Corresponding to the original FV value matrix in Step 1, use the vertical kernel weight matrix and the horizontal kernel weight matrix; Step 3: Respectively perform convolution calculations on the vertical kernel weight matrix and the horizontal kernel weight matrix with the original FV value matrix to obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix; Step 4: Respectively save the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image; Step 5: Drive and control the movement of the focusing motor. Each time the focusing motor is pulled, calculate and obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image according to the above steps; Step 6: During the continuous pulling process in Step 5, compare the regional vertical kernel weight FV value and the regional horizontal kernel weight FV value at the same region (i, j) of two adjacent consecutive frames; where the regional vertical kernel weight FV value is the clarity value at the region (i, j) in the vertical kernel weight FV value matrix, and the regional horizontal kernel weight FV value is the clarity value at the region (i, j) in the horizontal kernel weight FV value matrix; Record the maximum value maxFV of the regional vertical kernel weight FV value corresponding to each region 1-(i,j) , and the maximum value of the regional level kernel weight FV maxFV 2-(i,j) ; Record the maximum value maxFV 1-(i,j) and maxFV 2-(i,j) The positions maxFVPos of the focus motors corresponding respectively 1-(i,j) and maxFVPos 2-(i,j) ; Step 7. Compare maxFVPos 1-(i,j) and maxFVPos 2-(i,j) in sequence, and take the region corresponding to the minimum value of maxFVPos 1-(i,j) and maxFVPos 2-(i,j) as the focused region of interest; Step 8: Focus on the region of interest for focusing to complete the entire autofocus process.

2. The method for selecting an automatically focused region of interest of a zoom lens according to claim 1, wherein: The n×n regions are 16×16 regions.

3. The method for selecting an automatically focused region of interest of a zoom lens according to claim 2, wherein: The vertical kernel weight matrix and the horizontal kernel weight matrix are as follows: is the vertical kernel weight matrix, is the horizontal kernel weight matrix.

4. The method for selecting the region of interest for autofocus of a zoom lens according to claim 1, wherein: The specific steps of Step 3(1) are as follows: (11) Align the upper left corner of the vertical kernel weight matrix with the upper left corner of the original FV value matrix, multiply the vertical kernel weight matrix with the corresponding positions of the local matrix in the upper left corner of the original FV value matrix respectively, then calculate the weighted average of the multiplied and accumulated values, and use this weighted average as the weighted average FV value matrix of this region; (12) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (11), slide the vertical kernel weight matrix to the right and down by a step size of 1 and calculate the weighted average FV value matrix of the current region, and finally obtain the vertical kernel weight FV value matrix.

5. The method for selecting an autofocus region of interest of a zoom lens according to claim 1, characterized in that: The specific steps of Step 3(2) are as follows: (21) Align the upper left corner of the horizontal kernel weight matrix with the upper left corner of the original FV value matrix, multiply the horizontal kernel weight matrix with the corresponding positions of the local matrix in the upper left corner of the original FV value matrix respectively, then calculate the weighted average of the multiplied and accumulated values, and use this weighted average as the weighted average FV value of this region; (22) Define the convolution step size as 1, that is, the movement amount in the horizontal and vertical directions each time is 1. Then, similarly to the above step (21), slide the horizontal kernel weight matrix to the right and down by a step size of 1 and calculate the weighted average FV value matrix of the current region, and finally obtain the horizontal kernel weight FV value matrix.

6. The method for selecting an automatically focused region of interest of a zoom lens according to claim 1, characterized in that: In Step 5, the initial movement direction of the focusing motor is defined as the proximal end.

7. The method for selecting an autofocus region of interest of a zoom lens according to claim 1, wherein: In Step 6, it is necessary to judge the reverse condition of the focusing lens. After meeting the condition, it is necessary to reverse the focusing motor for processing.

8. The method for selecting an automatically focused region of interest of a zoom lens according to claim 7, characterized in that: The reverse condition of the focusing lens is that the FV values in all regions continuously decrease, or the FV values in most regions are lower than 85% of the maximum FV value corresponding to that region.

9. The method for selecting an automatically focused region of interest of a zoom lens according to claim 1, characterized in that: In step eight, the focusing region of interest obtained in step seven is focused using the hill climbing algorithm.

10. The method for selecting an autofocus region of interest of a zoom lens according to claim 9, wherein: The specific method of the hill climbing algorithm is as follows: Set the current operating speed of the focusing motor according to the image sharpness of the focusing region of interest; when the FV values of multiple frames show an upward trend as the focusing motor moves in the current direction, the operating speed of the focusing motor needs to be appropriately increased; if a downward trend appears afterwards and the FV values of several consecutive frames show an obvious downward trend, it can be judged that the highest point of the current scene has been searched along this direction, so the direction of the focusing motor is reversed to continue the search; if after the FV values show a continuous upward trend, the FV values of several consecutive frames show an obvious downward trend, it can be determined that the clearest point of the scene has been searched, and the focusing motor goes to the position with the highest scene sharpness. The current hill climbing search algorithm ends, that is, the position value of the focusing motor corresponding to the clearest point of the current scene image.

Images