Zoom lens auto focus region of interest selection method

Abstract

The present application provides a zoom lens automatic focusing region selection method, which can meet the requirement that the scene focus can fall in the ideal region when the depth of field is small, and ensure that the region is the clearest. The present application comprises the following steps: 1, segmenting the collected image, calculating the FV value of each region, and then outputting and storing the original FV value matrix; 2, respectively performing convolution calculation on the vertical kernel weight matrix and the horizontal kernel weight matrix and the original FV value matrix; 3, calculating the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image respectively each time the focusing motor is pulled; 4, comparing the region vertical kernel weight FV value and the region horizontal kernel weight FV value of the same region of the continuous adjacent frames, and recording the maximum value and the position of the focusing motor corresponding to the maximum value; 5, taking the region corresponding to the minimum value of the position of the focusing motor as the focusing region of interest; 6, focusing on the focusing region of interest.

Description

Technical Field

[0001] This invention relates to a method for selecting the region of interest in autofocus of a zoom lens, which is applied in the field of cameras. Background Technology

[0002] In recent years, the rapid development of video education, remote conferencing, and live-streaming e-commerce has greatly promoted the development of educational cameras, conference cameras, and live-streaming cameras, and has also placed higher demands on the focusing effect of cameras.

[0003] In online education scenarios, the main method is classroom-based in-class recording and broadcasting. Classrooms typically use high-magnification zoom cameras, which are mounted on the back wall of the classroom. When the teacher is lecturing, the camera is zoomed in at high magnification to give the lecturer a close-up shot. At this time, the lecturer occupies 1 / 3 to 2 / 3 of the frame. When the zoom lens zooms in, the magnification increases and the depth of field decreases, resulting in a noticeable difference in clarity.

[0004] During video conferences, when someone is giving a report or speaking, zoom lenses should also focus on close-ups of key personnel. Similarly, as the image zooms in, the depth of field becomes very shallow, resulting in a noticeable difference in sharpness between foreground and background. Furthermore, in the currently popular live-streaming e-commerce, when the host is introducing products and needs to show viewers specific food ingredients, production time, and even finer details like clothing and packaging, it's necessary to zoom in and bring the product closer to the camera. This allows for a more effective demonstration of the product to the audience.

[0005] Whether in online video education, online video conferencing, or live-streaming e-commerce scenarios, when using a high-magnification zoom lens, zooming in to provide a close-up shot results in a very shallow depth of field and a significant difference in sharpness between the foreground and background. To best meet human sensory needs and scenario requirements, it is desirable for the focus of the scene to fall on an ideal area, and for that area to be as sharp as possible. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned shortcomings in the prior art and provide a method for selecting the region of interest in autofocus of a zoom lens, which satisfies the requirement that the focus of the scene can fall on the ideal area even when the depth of field is small, and ensures that the area is the clearest.

[0007] The technical solution adopted by the present invention to solve the above problems is: a method for selecting the region of interest in autofocus of a zoom lens, characterized by including the following steps:

[0008] Step 1: Divide the acquired image into n×n regions, calculate the FV value of each region, and then output and store it as the original FV value matrix;

[0009] Step 2: Using the vertical kernel weight matrix and the horizontal kernel weight matrix corresponding to the original FV value matrix in Step 1;

[0010] Step 3: Perform convolution calculations between the vertical kernel weight matrix and the horizontal kernel weight matrix and the original FV value matrix respectively to obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix;

[0011] Step 4: Save the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image respectively;

[0012] Step 5: Drive and control the focusing motor to move. Each time the focusing motor is pulled, calculate and obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image according to the above steps.

[0013] Step 6: During the continuous dragging process in Step 5, compare the region vertical kernel weight FV value and the region horizontal kernel weight FV value in the same region (i,j) of consecutive adjacent frames; where the region vertical kernel weight FV value is the sharpness value in the vertical kernel weight FV value matrix at region (i,j), and the region horizontal kernel weight FV value is the sharpness value in the horizontal kernel weight FV value matrix at region (i,j);

[0014] Record the maximum value (maxFV) of the vertical kernel weight FV for each region. 1-(i,j) And the maximum value of the regional level kernel weight FV, maxFV. 2-(i,j) ;

[0015] Record the maximum value maxFV 1-(i,j) and maxFV 2-(i,j) The corresponding positions of the focusing motors are maxFVPos 1-(i,j) and maxFVPos 2-(i,j) ;

[0016] Step 7: Compare maxFVPos sequentially. 1-(i,j) and maxFVPos 2-(i,j) Take maxFVPos 1-(i,j) and maxFVPos 2-(i,j) The region corresponding to the minimum value is defined as the region of interest.

[0017] Step 8: Focus on the area of ​​interest to complete the entire autofocus process.

[0018] The n×n regions described in this invention are 16×16 regions.

[0019] The vertical kernel weight matrix and the horizontal kernel weight matrix described in this invention are as follows:

[0020]

[0021] This is the vertical kernel weight matrix. This is the horizontal kernel weight matrix.

[0022] Step three (1) of the present invention specifically includes the following steps:

[0023] (11) Align the top left corner of the vertical kernel weight matrix with the top left corner of the original FV value matrix, multiply the vertical kernel weight matrix with the corresponding positions of the local matrix at the top left corner of the original FV value matrix, calculate the weighted average after multiplication and summation, and use the weighted average as the weighted average FV value matrix of the region.

[0024] (12) Define the convolution step size as 1, that is, the horizontal and vertical movement amount is 1 each time. Then, similarly as in step (11) above, slide the vertical kernel weight matrix to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current region, and finally obtain the vertical kernel weight FV value matrix.

[0025] Step three (2) of the present invention specifically includes the following steps:

[0026] (21) Align the top left corner of the horizontal kernel weight matrix with the top left corner of the original FV value matrix, multiply the horizontal kernel weight matrix with the corresponding positions of the local matrix at the top left corner of the original FV value matrix, calculate the weighted average after multiplication and summation, and use the weighted average as the weighted average FV value of the region.

[0027] (22) Define the convolution step size as 1, that is, the amount of horizontal and vertical movement is 1 each time. Then, similarly as in step (21) above, slide the horizontal kernel weight matrix to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current region, and finally obtain the horizontal kernel weight FV value matrix.

[0028] In step five of this invention, the initial direction of motion of the focusing motor is defined as the near end.

[0029] In step six of this invention, it is necessary to determine the reversal condition of the focusing lens. If the condition is met, the focusing motor needs to be reversed.

[0030] The reverse condition for the focusing lens described in this invention is: the FV value of all regions decreases continuously, or the FV value of most regions is lower than 85% of the maximum FV value corresponding to that region.

[0031] In step eight of this invention, the region of interest obtained in step seven is focused using a hill-climbing algorithm.

[0032] The specific method of the climbing algorithm of this invention is as follows: The current speed of the focusing motor is set according to the image sharpness of the focused region of interest; when the FV value of the focusing motor is increasing in multiple frames along the current direction, the speed of the focusing motor needs to be appropriately increased; if a downward trend subsequently appears, and the FV value shows a significant downward trend for several consecutive frames, it can be determined that the highest point of the current scene has been found along this direction, and the direction of the focusing motor is then reversed to continue searching; if the FV value shows a continuous increase followed by a significant downward trend for several consecutive frames, it can be determined that the sharpest point of the scene has been found, and the focusing motor moves to the position with the highest scene sharpness, thus ending the current climbing search algorithm. That is, the position of the sharpest point in the current scene image corresponds to the position value of the focusing motor. Compared with existing technologies, this invention has the following advantages and effects: it can quickly lock the focus area, is compatible with multiple foreground objects facing different directions, and is applicable to video education, remote conferencing, and live-streaming e-commerce scenarios, ensuring that the scene focus falls on an ideal area near the center even when the depth of field is small.

[0033] Instruction manual illustrations

[0034] Figure 1 This is a schematic diagram of 16×16 regions in an embodiment of the present invention.

[0035] Figure 2 The vertical kernel weight matrix in this embodiment of the invention With fv i,j A schematic diagram of convolution.

[0036] Figure 3 The horizontal kernel weight matrix of this invention embodiment With fv i,j A schematic diagram of convolution.

[0037] Figure 4 The vertical kernel weight matrix in this embodiment of the invention With fv i,j FV after convolution 1-i,j A schematic diagram of the region and the central region.

[0038] Figure 5 The horizontal kernel weight matrix of this invention embodiment With fv i,j FV after convolution 2-i,j A schematic diagram of the regional and central areas.

[0039] Figure 6 This is a schematic diagram of the hill-climbing algorithm in the automatic calibration curve of this invention. Detailed Implementation

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0041] An embodiment of the present invention provides a method for selecting a region of interest in autofocus using a zoom lens, comprising the following steps:

[0042] Step 1: Taking the StarChip SSD268G chip as an example, using hardware devices including a zoom lens, sensor, chip, and various electronic components forming an integrated circuit, the chip acquires a frame of image, and then evenly divides the acquired image into 16×16 square regions, such as... Figure 1 As shown, the chip's internal processing mechanism filters the image data of each region to calculate the FV value of each region. The FV value is the image sharpness value (FocusValue). There are a total of 16×16 region matrix data, with a data size of 256 blocks. The data is then output and stored as the original FV value matrix fv. i,j :

[0043] fv i,j , i∈[0,16], j∈[0,16].

[0044] Step 2: The original FV value matrix fv from Step 1. i,j Different kernel weight matrices are used in this invention: a vertical kernel weight matrix and a horizontal kernel weight matrix. The vertical kernel weight matrix is ​​8×4 in size, and the horizontal kernel weight matrix is ​​4×8 in size. The weight values ​​in both matrices gradually decrease from the center to the edge. The data for the vertical and horizontal kernel weight matrices are as follows:

[0045]

[0046] This is the vertical kernel weight matrix. This is the horizontal kernel weight matrix.

[0047] Step 3: Obtain the kernel weight matrix and the original FV value matrix fv i,j The convolutional FV value matrix FV after convolution i,j The steps are as follows:

[0048] (1) Vertical kernel weight matrix Compared with the original FV value matrix fv i,j Perform convolution calculations to obtain the vertical kernel weight matrix FV. 1-(i,j) , i∈[0,9], j∈[0,13], where i is the horizontal row number and j is the vertical column number. The FV value matrix is ​​changed from the original fv i,j It became FV 1-i,j ,like Figure 4 As shown, the original image sharpness matrix fv i,j Dimensionality reduction was performed.

[0049] Specifically, the steps include the following:

[0050] (11) Vertical kernel weight matrix The top left corner is the original FV value matrix fv i,j Align with top left corner, as shown Figure 2 As shown, the vertical kernel weight matrix is ​​compared with the original FV value matrix fv. i,j Multiply the corresponding positions of the 8×4 local matrix in the upper left corner, then calculate the weighted average of the multiplied and accumulated matrices, and use this weighted average as the weighted average FV value matrix FV″ for that region. 1-(i,j) The calculation method is as follows:

[0051]

[0052] Among them, fv′ m,n The original FV value matrix fv i,j The local sharpness value matrix corresponding to the m×n region.

[0053] (12) Define the convolution stride as 1, that is, the horizontal and vertical movement is 1 each time. Then, similarly to step (11) above, adjust the vertical kernel weight matrix. Slide to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current region, finally obtaining the vertical kernel weight FV value matrix. 1-(i,j) .

[0054] (2) The horizontal kernel weight matrix Compared with the original FV value matrix fv i,j Perform convolution calculations to obtain the horizontal kernel weight matrix FV. 2-(i,j) , i∈[0,13], j∈[0,9], where i is the horizontal row number and j is the vertical column number. The FV value matrix is ​​changed from the original fv i,j It became FV 1-(i,j) ,like Figure 5 As shown, the same applies to the original FV value matrix fv i,j Dimensionality reduction was performed.

[0055] Specifically, the steps include the following:

[0056] (21) The horizontal kernel weight matrix The top left corner is the original FV value matrix fv i,j Align with top left corner, as shown Figure 3 As shown, the horizontal kernel weight matrix is ​​compared with the original FV value matrix fv. i,j Multiply the corresponding positions of the 4×8 local matrix in the top left corner, then calculate the weighted average of the multiplied and accumulated matrices, and use this weighted average as the weighted average FV value FV″ for that region. 2-(i,j) The calculation method is as follows:

[0057]

[0058] Among them, fv ′ m,n The original FV value matrix fv i,j The local sharpness value matrix corresponding to the m×n region.

[0059] (22) Define the convolution stride as 1, that is, the horizontal and vertical movement is 1 each time. Then, similarly to step (21) above, adjust the horizontal kernel weight matrix. Slide right and down with a step size of 1 and calculate the weighted average FV value matrix for the current region, ultimately obtaining the horizontal kernel weight FV value matrix. 2-(i,j) .

[0060] Step 4: Save the vertical kernel weight FV value matrix FV of the current frame image. 1-(i,j) And the horizontal kernel weight FV value matrix FV 2-(i,j) .

[0061] Step 5: Drive and control the focusing motor to move. The initial direction of movement is defined as near end. Each time the focusing motor is pulled, calculate and obtain the vertical kernel weight FV value matrix of the current frame image according to the above steps. 1-(i,j) And the horizontal kernel weight FV value matrix FV 2-(i,j) .

[0062] Step Six: During the continuous dragging process in Step Five, compare the regional vertical kernel weight FV value FV′ of adjacent frames in the same region (i,j). 1-(i,j) and regional level kernel weight FV value FV′ 2-(i,j) The region vertical kernel weight FV value FV′ 1-(i,j) The vertical kernel weight FV value matrix FV 1-(i,j) The sharpness value at region (i,j), and the region-level kernel weight FV value FV′. 2-(i,j) The horizontal kernel weight FV value matrix FV 2-(i,j) The sharpness value at region (i,j);

[0063] Record the region vertical kernel weight FV value for each region. ′ 1-(i,j) The maximum value of maxFV in 1-(i,j) And the regional level kernel weight FV value FV′ 2-(i,j) The maximum value of maxFV 2-(i,j) ;

[0064] Record the maximum value maxFV 1-(i,j) and maxFV 2-(i,j) The corresponding position of the focusing motor is maxFVPos 1-(i,j)and maxFVPos 2-(i , j) .

[0065] This step requires determining the reversal condition of the focusing lens. Once the condition is met, the focusing motor needs to be reversed, i.e., turned to the left. The reversal condition of the focusing lens is: the FV value of all areas continuously decreases, or the FV value of most areas is lower than 85% of the maximum FV value corresponding to that area.

[0066] Step 7: Following the principle of scene focus center, from the center outwards, and assuming that the sensory focus is primarily on the image center, select a focus comparison area. In this embodiment, the selected area is located at 2 / 3 of the focal block region. Figure 4 , Figure 5 As shown, the comparison area does not consider the region at the edge 1 / 3; maxFVPos is compared sequentially. 1-(i,j) and maxFVPos 2-(i,j) Take maxFVPos 1-(i,j) and maxFVPos 2-(i,j) The region corresponding to the minimum value is defined as the region of interest (ROI). i,j , as the main focus area.

[0067] Step 8: Based on the Region of Interest (ROI) obtained in Step 7 i,j ,like Figure 6 As shown, the hill-climbing algorithm is used to focus on the region of interest (ROI). i,j Perform autofocus and focus on the region of interest (ROI). i,j Once focus is achieved, the entire autofocus process is complete.

[0068] The specific method of the hill-climbing algorithm is as follows: based on the region of interest (ROI) i,j Image sharpness is set by the current focus motor operating speed, where the focus motor operating speed is related to the region of interest (ROI) being focused. i,j The trend of image sharpness change is positively correlated. When the focusing motor focuses along the current direction, if the FV value of multiple frames shows an upward trend, the focusing motor's running speed needs to be appropriately increased. If a downward trend appears afterward, and the FV value shows a significant downward trend for several consecutive frames, it can be determined that the highest point of the current scene has been found along this direction. Therefore, the focusing motor's focusing direction is reversed to continue searching. If the FV value shows a continuous upward trend, and then a significant downward trend appears for several consecutive frames, it can be determined that the sharpest point of the scene has been found. The focusing motor then moves to the position with the highest scene sharpness, and the current hill-climbing search algorithm ends. That is, the position of the sharpest point of the current scene image corresponds to the focusing motor's focusing point value.

[0069] Furthermore, it should be noted that the specific embodiments described in this specification may differ in the shape and name of their components, etc. The above description is merely illustrative of the structure of the present invention. All equivalent or simple variations made based on the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art can make various modifications or additions to the described specific embodiments or use similar methods to substitute them, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, all of which should fall within the protection scope of this invention.

Claims

1. A method for selecting a region of interest in autofocus using a zoom lens, characterized in that: Includes the following steps: Step 1: Divide the acquired image into n×n regions, calculate the FV value of each region, and then output and store it as the original FV value matrix; Step 2: Using the vertical kernel weight matrix and the horizontal kernel weight matrix corresponding to the original FV value matrix in Step 1; Step 3: Perform convolution calculations between the vertical kernel weight matrix and the horizontal kernel weight matrix and the original FV value matrix respectively to obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix; Step 3 includes the following steps: (11) Align the upper left corner of the vertical kernel weight matrix with the upper left corner of the original FV value matrix, multiply the vertical kernel weight matrix with the corresponding positions of the local matrix at the upper left corner of the original FV value matrix, calculate the weighted average after multiplication and summation, and use the weighted average as the weighted average FV value matrix of the region. (12) Define the convolution stride as 1, that is, the amount of horizontal and vertical movement is 1 each time. Then, in the same way as the above steps (11), slide the vertical kernel weight matrix to the right and down with a stride of 1 and calculate the weighted average FV value matrix of the current region, and finally obtain the vertical kernel weight FV value matrix. (21) Align the top left corner of the horizontal kernel weight matrix with the top left corner of the original FV value matrix, multiply the horizontal kernel weight matrix with the corresponding positions of the local matrix at the top left corner of the original FV value matrix, calculate the weighted average after multiplication and summation, and use the weighted average as the weighted average FV value of the region. (22) Define the convolution step size as 1, that is, the horizontal and vertical movement amount is 1 each time. Then, similarly as in step (21) above, slide the horizontal kernel weight matrix to the right and down with a step size of 1 and calculate the weighted average FV value matrix of the current region to finally obtain the horizontal kernel weight FV value matrix. Step 4: Save the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image respectively; Step 5: Drive and control the focusing motor to move. Each time the focusing motor is pulled, calculate and obtain the vertical kernel weight FV value matrix and the horizontal kernel weight FV value matrix of the current frame image according to the above steps. Step 6: During the continuous dragging process in Step 5, compare the regional vertical kernel weight FV value and the regional horizontal kernel weight FV value in the same region (i,j) of consecutive adjacent frames; where the regional vertical kernel weight FV value is the sharpness value in the vertical kernel weight FV value matrix at region (i,j), and the regional horizontal kernel weight FV value is the sharpness value in the horizontal kernel weight FV value matrix at region (i,j); Record the maximum value (maxFV) of the vertical kernel weight FV for each region. 1-(i,j) And the maximum value of the regional level kernel weight FV, maxFV. 2-(i,j) ; Record the maximum value maxFV 1-(i,j) and maxFV 2-(i,j) The corresponding positions of the focusing motors are maxFVPos 1-(i,j) and maxFVPos 2-(i,j) ; Step 7: Compare maxFVPos sequentially. 1-(i,j) and maxFVPos 2-(i,j) Take maxFVPos 1-(i,j) and maxFVPos 2-(i,j) The region corresponding to the minimum value is defined as the region of interest. Step 8: Focus on the area of ​​interest to complete the entire autofocus process.

2. The method for selecting the region of interest in autofocus of a zoom lens according to claim 1, characterized in that: The n×n regions refer to 16×16 regions.

3. The method for selecting the region of interest in autofocus of a zoom lens according to claim 2, characterized in that: The vertical kernel weight matrix and the horizontal kernel weight matrix are as follows: ， This is the vertical kernel weight matrix. This is the horizontal kernel weight matrix.

4. The method for selecting the region of interest in autofocus of a zoom lens according to claim 1, characterized in that: In step five, the initial direction of motion of the focusing motor is defined as the near end.

5. The method for selecting the region of interest in autofocus of a zoom lens according to claim 1, characterized in that: In step six, it is necessary to determine the reversal condition of the focusing lens. Once the condition is met, the focusing motor needs to be reversed.

6. The method for selecting the region of interest in autofocus of a zoom lens according to claim 5, characterized in that: The reversal condition for the focusing lens is: the FV value of all regions decreases continuously, or the FV value of most regions is lower than 85% of the maximum FV value corresponding to that region.

7. The method for selecting the region of interest in autofocus of a zoom lens according to claim 1, characterized in that: In step eight, the climbing algorithm is used to focus the region of interest obtained in step seven.

8. The method for selecting the region of interest in autofocus of a zoom lens according to claim 7, characterized in that: The specific method of the hill-climbing algorithm is as follows: the current speed of the focusing motor is set according to the image sharpness of the region of interest; when the FV value of the focusing motor is increasing in multiple frames along the current direction, the speed of the focusing motor needs to be increased appropriately; if a downward trend appears afterward, and the FV value shows a significant downward trend in several consecutive frames, it can be determined that the highest point of the current scene has been found along this direction, so the direction of the focusing motor is reversed to continue searching; if the FV value increases continuously and then shows a significant downward trend in several consecutive frames, it can be determined that the sharpest point of the scene has been found, and the focusing motor moves to the position with the highest scene sharpness. The current hill-climbing search algorithm ends, that is, the position of the sharpest point of the current scene image corresponds to the position value of the focusing motor.

Images