A method for calculating FV values ​​based on images, and a DSP device.

Abstract

This invention discloses an image-based FV value calculation method and DSP device, comprising: acquiring a first image; scaling the first image according to Y-luminance information; and denoising the scaled first image to acquire a second image; wherein the Y-luminance information is the Y component in YUV data; obtaining first edge data of the second image by performing edge detection and content filtering on the second image; and obtaining an FV value based on the first edge data. This invention reduces the computational load of FV value calculation and the impact of noise on the calculation by scaling and denoising the image according to the Y component. Simultaneously, by performing edge detection and content filtering on different image contents, it achieves FV value calculation based on the image content to complete focusing, avoids blurring of zoomed images due to scene anomalies, and improves focusing effect.

Description

Technical Field

[0001] This invention relates to the field of zoom control technology, and in particular to an image-based FV value calculation method and DSP device. Background Technology

[0002] Contrast-based autofocus (also known as contrast-detection autofocus) calculates the FV (Focus Value, the amount of contrast in the statistical area) of the image as the lens is gradually moved from bottom to top, and continuously records and compares the FV values ​​to form an FV curve. During the scanning process, the peak (maximum value) of the FV curve is considered to be the focus position.

[0003] Currently, in traditional contrast-detection autofocus systems, the FV value is directly obtained from the hardware statistics of the ISP (Image Signal Processing) subsystem. Different ISP solutions produce different FV values; for example, in brightly lit scenes, obvious false peaks may appear when there is severe defocusing. Because IIR and FIR filters have limited configuration parameters, they cannot calculate the FV value based on the scene content, leading to poor focusing performance in some scenarios. Summary of the Invention

[0004] This invention provides an image-based FV value calculation method and DSP device to solve the technical problem in the prior art that the FV value cannot be calculated according to different scenes, resulting in poor focusing effect.

[0005] To address the aforementioned technical problems, in a first aspect, embodiments of the present invention provide an image-based FV value calculation method, comprising:

[0006] A first image is acquired, scaled according to the Y brightness information, and the scaled first image is denoised to acquire a second image; the Y brightness information is the Y component in the YUV data.

[0007] By performing edge detection and content filtering on the second image, the first edge data of the second image is obtained;

[0008] The FV value is obtained based on the first edge data.

[0009] This invention reduces the computational load of FV value calculation and the impact of noise on the calculation by scaling and denoising the image based on the Y component. At the same time, it calculates the FV value based on the image content by performing edge detection and content filtering on different image contents to achieve focusing, avoids blurring of zoomed images caused by scene abnormalities, and improves the focusing effect.

[0010] Furthermore, the step of obtaining the first edge data of the second image by performing edge detection and content filtering on the second image specifically involves:

[0011] Edge detection is performed on the second image to obtain second edge data, and content filtering is performed on the second image to obtain mask data;

[0012] By combining the second edge data and the mask data, the first edge data of the second image is obtained.

[0013] This invention obtains edge data of different images by performing edge detection on different images, using formic acid FV values, and simultaneously filters content for abnormal lighting scenes to improve the focusing effect of abnormal scenes.

[0014] Furthermore, the step of performing edge detection on the second image to obtain second edge data specifically involves:

[0015] The second edge data of the second image is calculated using a DSP edge detection operator. The second edge data includes a set of points where the gray values ​​of the image are discontinuous.

[0016] Furthermore, the step of filtering the second image to obtain mask data specifically involves:

[0017] Binarize the second image and detect the brightness areas of the lights;

[0018] The edges of the light brightness area are filtered through an erosion operation, and mask data is generated.

[0019] This invention uses binarization to detect areas of high light intensity and then filters out the edges of bright areas through etching. This creates Mask data, i.e., a binarized weight table, to reduce the impact of light intensity on focusing performance.

[0020] Furthermore, the step of combining the second edge data and the mask data to obtain the first edge data of the second image specifically involves:

[0021] The second edge data is converted into an edge matrix, and the Mask data is converted into a Mask matrix;

[0022] Multiply the edge matrix and the mask matrix to obtain the first edge data, which is weighted edge data.

[0023] This invention reduces the influence of lighting by assigning different weights to the values ​​of the lighting components, i.e., Mask data. Finally, by multiplying the edge matrix and the Mask matrix at corresponding positions, weighted edge data can be obtained, thus reducing the impact of lighting on the calculation of FV values.

[0024] Further, the step of scaling the first image based on the Y luminance information and then denoising the scaled first image to obtain the second image specifically involves:

[0025] The resolution is determined based on the performance of the DSP, and the first image is scaled according to the resolution and the Y component of the first image.

[0026] The first image, after being scaled up, is filtered and denoised using a Gaussian blur operator to obtain the second image.

[0027] This invention selects different resolutions for scaling based on DSP performance, improving the adaptability of image processing. It also employs Gaussian blur to filter out noise in the image, reducing the impact of noise on image recognition.

[0028] Further, obtaining the FV value based on the first edge data specifically involves:

[0029] The standard deviation of the first edge data is calculated via the DSP interface, and the standard deviation is the FV value.

[0030] This invention uses standard deviation to represent FV value, which can increase the peak value of FV value and reduce the number when the focus is blurred, thereby reducing the requirement of hill climbing algorithm on the monotonicity of FV value.

[0031] Secondly, embodiments of the present invention provide a DSP device, including: a scaling and noise reduction module, a detection and filtering module, and an FV value calculation module;

[0032] The scaling and noise reduction module is used to acquire a first image, scale the first image according to the Y brightness information, and reduce the noise of the scaled first image to acquire a second image; the Y brightness information is the Y component in YUV data.

[0033] The detection and filtering module is used to obtain the first edge data of the second image by performing edge detection and content filtering on the second image;

[0034] The FV value calculation module is used to obtain the FV value based on the first edge data.

[0035] Thirdly, embodiments of the present invention provide a contrast-based focusing method, including:

[0036] Acquire an image, and calculate the image's FV value using the image-based FV value calculation method described above;

[0037] Based on the focusing algorithm, zoom is achieved by controlling the lens motor through the FV value.

[0038] Fourthly, embodiments of the present invention provide a contrast focusing device, including an FV value calculation module and a focusing module;

[0039] The FV value calculation module is used to acquire an image and use the DSP device to calculate the FV value of the image.

[0040] The focusing module is used to control the lens motor to achieve zoom by using the FV value according to the focusing algorithm. Attached Figure Description

[0041] Figure 1 A flowchart illustrating an image-based FV value calculation method provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of an edge detection step 102 provided in an embodiment of the present invention;

[0043] Figure 3 This is another schematic diagram of edge detection in step 102 provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of a content filtering step 102 provided in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of a content filtering step 102 provided in an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of a content filtering process for step 102 provided in an embodiment of the present invention;

[0047] Figure 7 A comparison diagram of a focusing process in step 102 provided in an embodiment of the present invention;

[0048] Figure 8 A schematic diagram of the structure of a DSP device provided in an embodiment of the present invention;

[0049] Figure 9 This is a schematic flowchart of a contrast focusing method provided in an embodiment of the present invention;

[0050] Figure 10 This is a schematic diagram of a contrast focusing device provided in an embodiment of the present invention;

[0051] Figure 11 This is another flowchart illustrating the image-based FV value calculation method provided in an embodiment of the present invention.

[0052] Figure 12 This is a flowchart illustrating another contrast-based focusing method provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1

[0055] Please refer to Figure 1 , Figure 1 A flowchart illustrating an image-based FV value calculation method provided in an embodiment of the present invention is shown, which mainly includes steps 101 to 102, as detailed below:

[0056] Step 101: Obtain a first image, scale the first image according to the Y brightness information, and reduce the noise of the scaled first image to obtain a second image; the Y brightness information is the Y component in the YUV data;

[0057] In this embodiment, the step of scaling the first image based on the Y luminance information and then denoising the scaled first image to obtain the second image specifically involves:

[0058] The resolution is determined based on the performance of the DSP, and the first image is scaled according to the resolution and the Y component of the first image.

[0059] The first image, after being scaled up, is filtered and denoised using a Gaussian blur operator to obtain the second image.

[0060] In this embodiment, the acquired data source is processed. Since the data source may have various resolutions, different resolutions can be processed according to the DSP performance. Because YUV data is relatively large, only the luminance Y component is processed, which reduces the amount of computation. Taking the commonly used NV12 format as an example, taking only the Y component can reduce the amount of data by 1 / 3.

[0061] In this embodiment, a DSP operator, namely a 5*5 Gaussian blur operator, is used to denoise the image. After the Y brightness image is blurred by Gaussian, the noise in the image can be filtered out, effectively reducing the impact of noise.

[0062] Step 102: Obtain the first edge data of the second image by performing edge detection and content filtering on the second image;

[0063] In this embodiment, obtaining the first edge data of the second image by performing edge detection and content filtering on the second image specifically involves:

[0064] Edge detection is performed on the second image to obtain second edge data, and content filtering is performed on the second image to obtain mask data;

[0065] By combining the second edge data and the mask data, the first edge data of the second image is obtained.

[0066] In this embodiment, the step of performing edge detection on the second image to obtain second edge data specifically involves:

[0067] The second edge data of the second image is calculated using a DSP edge detection operator. The second edge data includes a set of points where the gray values ​​of the image are discontinuous.

[0068] As a specific example of an embodiment of the present invention, please refer to Figure 2 , Figure 2 This is a schematic diagram of an edge detection step 102 provided in an embodiment of the present invention;

[0069] In this embodiment, the DSP edge detection operator, SOBEL operator, or Laplacian is called through the basic operator development library. The SOBEL operator calculates image edges that are relatively clear and have different intensities; SOBEL can be used for calculation in most scenarios. Figure 2 As shown, in a focusing process, the focus moves from the far point to the point of sharpness and then to the near point.

[0070] Please refer to Figure 3 , Figure 3 This is another schematic diagram of edge detection in step 102 provided in an embodiment of the present invention.

[0071] In this embodiment, for a scene with lights, if only the SOBEL operator is used, the edges of the lights will be magnified when the image is out of focus and blurred, forming stronger and more obvious edges. At this time, the edge detection value will be relatively large. When the hill climbing algorithm is processed, because the larger the FV value, the more difficult it is to focus, even though the image is blurry.

[0072] In this embodiment, the step of filtering the second image to obtain mask data specifically involves:

[0073] Binarize the second image and detect the brightness areas of the lights;

[0074] The edges of the light brightness area are filtered through an erosion operation, and mask data is generated.

[0075] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a content filtering step 102 provided in an embodiment of the present invention.

[0076] In this embodiment, the image is already blurry under abnormal lighting conditions, but the edges of the lights are still very noticeable due to the out-of-focus state. Content filtering can effectively solve the problem of blurry images under abnormal lighting conditions. To address the issue of inability to focus in abnormal scenes, it is necessary to filter out abnormal lights. This is done by directly detecting areas with high light brightness; erosion is used to include the edges of bright areas as well. The resulting mask will filter out the edges of the lights, and this mask is equivalent to a binarized weight table.

[0077] In this embodiment, the step of combining the second edge data and the mask data to obtain the first edge data of the second image specifically involves:

[0078] The second edge data is converted into an edge matrix, and the Mask data is converted into a Mask matrix;

[0079] Multiply the edge matrix and the mask matrix to obtain the first edge data, which is weighted edge data.

[0080] In this embodiment, the Mask matrix is:

[0081]

[0082] In this embodiment, the edge matrix obtained by edge detection is:

[0083]

[0084] In this embodiment, Mask is a binary image. The black parts are defined as those that do not require weight adjustment, so their values ​​are configured to 1. The values ​​for the light parts can be set to different weights, such as 0.5, to reduce the influence of the light. Finally, multiplying the two matrices at corresponding positions yields the weighted compiled detection image Result, specifically:

[0085]

[0086] In this embodiment, the Result is weighted edge data. This Result is the result of multiplying the edge detection and Mask weights. Using this result to calculate the FV value can filter out the influence of the light.

[0087] As a specific example of an embodiment of the present invention, please refer to Figure 4 , 5 and Figure 6 , Figure 4 This is a schematic diagram of image inflation for step 102 provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a content filtering step 102 provided in an embodiment of the present invention; Figure 6This is a schematic diagram of a content filtering process for step 102 provided in an embodiment of the present invention; the image after binarization and dilation can make the light tube larger and cover the edge of the light. In this way, the Mask data obtained by erosion and the image data of edge detection can be multiplied to obtain an image that can calculate and analyze the FV value, that is, to filter abnormal light edges.

[0088] Please refer to Figure 7 , Figure 7 A comparison diagram of a focusing process in step 102 provided in an embodiment of the present invention.

[0089] In this embodiment, `sobel_std` (the upper curve) is the FV value calculated by standard edge detection. The horizontal direction represents the focus position. Scanning from far to near focus, the image goes through a process of blurring to sharpness and then back to blurring. The upper curve shows the FV value change during this process. It can be observed that in scenes with lighting, the FV value obtained directly using the Sobel algorithm cannot accurately represent image sharpness. This causes the image to drift towards blurriness during the hill-climbing focusing process. The FV value curve obtained by the `sobel+mask` method (the lower curve) is better; it is not affected by lighting edges when blurry, making it more conducive to focusing.

[0090] In this embodiment, content filtering can achieve different focusing requirements based on different algorithms. For example, incorporating a face recognition algorithm can accurately obtain the FV value of the face region, while traditional FV values ​​may include the background region behind the face because traditional values ​​are rectangular and lack precision. The content filtering module can accurately calculate the focusing FV value as long as the algorithm provides a mask for the region. The content filtering module can use a human body or other objects, depending on the algorithm type.

[0091] Step 103: Obtain the FV value based on the first edge data.

[0092] In this embodiment, obtaining the FV value based on the first edge data specifically involves:

[0093] The standard deviation of the first edge data is calculated via the DSP interface, and the standard deviation is the FV value.

[0094] In this embodiment, the image after edge detection can be obtained by summing the values ​​directly. This value can basically represent the image clarity. However, the verification test found that using the standard deviation is more effective. The standard deviation can increase the peak value and decrease the number when the image is blurry, which is more conducive to the hill climbing algorithm's requirement for the monotonicity of the FV value.

[0095] In this embodiment, the standard deviation is calculated using meanStdDev in OpenCV and fcvImageIntensityStats interface in DSP.

[0096] In this embodiment, by scaling and denoising the image based on the Y component, the computational load of FV value calculation is reduced and the impact of noise on the calculation is reduced. At the same time, by performing edge detection and content filtering on different image contents, the FV value is calculated based on the image content to achieve focusing, avoid blurring of zoomed images caused by scene abnormalities, and improve the focusing effect.

[0097] Please refer to Figure 8 , Figure 8 A schematic diagram of a DSP device provided in an embodiment of the present invention, which mainly includes: a scaling and noise reduction module 801, a detection and filtering module 802, and an FV value calculation module 803;

[0098] The scaling and noise reduction module 801 is used to acquire a first image, scale the first image according to the Y brightness information, and reduce the noise of the scaled first image to acquire a second image; the Y brightness information is the Y component in YUV data.

[0099] The detection and filtering module 802 is used to obtain the first edge data of the second image by performing edge detection and content filtering on the second image;

[0100] The FV value calculation module 803 is used to obtain the FV value based on the first edge data.

[0101] Please refer to Figure 9 , Figure 9 This is a schematic flowchart of a contrast focusing method provided in an embodiment of the present invention, which mainly includes steps 901 to 902, as detailed below:

[0102] Step 901: Acquire the image and calculate the FV value of the image using the image-based FV value calculation method described above;

[0103] Step 902: Based on the focusing algorithm, zoom is achieved by controlling the lens motor through the FV value.

[0104] Please refer to Figure 10 , Figure 10 This is a schematic diagram of a contrast focusing device provided in an embodiment of the present invention, which mainly includes: an FV value calculation module 1001 and a focusing module 1002;

[0105] The FV value calculation module 1001 is used to acquire an image and calculate the FV value of the image using the DSP device.

[0106] The focusing module 1002 is used to control the lens motor to achieve zoom by means of the FV value according to the focusing algorithm.

[0107] Example 2

[0108] Please refer to Figure 11 , Figure 11 This is another schematic diagram of the image-based FV value calculation method provided in an embodiment of the present invention.

[0109] In this embodiment, it mainly includes a YUV scaling module, a Gaussian filtering module, an edge detection module, a content filtering module, and a standard deviation calculation module.

[0110] The YUV and scaling module primarily handles data source processing. Since the data source may have various resolutions, processing can be performed at different resolutions based on the DSP's performance. Because YUV data is quite large, only the luminance (Y) component is processed, thus reducing computational load. Taking the commonly used NV12 format as an example, processing only the Y component can reduce the data volume by one-third.

[0111] The Gaussian filtering module uses DSP operators to reduce noise in the Y-brightness image. After the Y-brightness image is blurred by Gaussian, the noise in the image can be filtered out, effectively reducing the impact of noise.

[0112] The edge detection module directly uses DSP edge detection operators, such as the SOBEL operator or the Laplacian operator. The SOBEL operator produces image edges that are more obvious and have different intensities. SOBEL can be used for calculation in most scenarios.

[0113] The content filtering module directly detects areas with high light brightness. Erosion is used to include the edges of the bright edges, so the resulting mask will filter out the edges of the light.

[0114] The standard deviation calculation module can directly sum the values ​​of the edge-detected image to obtain a numerical value, which can basically represent the image sharpness. However, verification tests have shown that using the standard deviation is more effective. The standard deviation can increase the peak value and decrease the number when the image is blurry, which is more conducive to the hill-climbing algorithm's requirement for the monotonicity of the FV value. We use the standard deviation value to represent the FV value.

[0115] Please refer to Figure 12 , Figure 12 This is a flowchart illustrating another contrast-based focusing method provided in an embodiment of the present invention.

[0116] In this embodiment, RAW data source is acquired through ISP and converted into YUV data, FV value is calculated through DSP device, and lens motor is controlled to complete focusing according to focusing algorithm.

[0117] In this embodiment, traditional ISP statistics can only analyze 16*16 areas to obtain statistical information for 32*32 areas. However, this embodiment can perform pixel-level analysis and processing, without being limited by this hardware. It can solve the problem of focusing based on the content of the image, rather than focusing on a fixed position, thus improving the focusing effect.

[0118] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. An image-based FV value calculation method, characterized by, The method comprises the following steps: obtaining a first image, scaling the first image according to Y luminance information, and performing noise reduction on the scaled first image to obtain a second image; the Y luminance information is a Y component in YUV data; performing edge detection on the second image to obtain second edge data, and performing content filtering on the second image to obtain Mask data; combining the second edge data and the Mask data to obtain first edge data of the second image; the step of performing edge detection on the second image to obtain second edge data specifically comprises: using a DSP edge detection operator to calculate second edge data of the second image, wherein the second edge data comprises a set of points with discontinuous image grayscale values; the step of performing content filtering on the second image to obtain Mask data specifically comprises: performing binaryzation on the second image to detect a light brightness area; and filtering edges of the light brightness area through an erosion operation to generate Mask data; the step of combining the second edge data and the Mask data to obtain first edge data of the second image specifically comprises: converting the second edge data into an edge matrix and converting the Mask data into a Mask matrix; and multiplying the edge matrix and the Mask matrix to obtain the first edge data, wherein the first edge data is weighted edge data; obtaining an FV value according to the first edge data.

2. The image-based FV value calculation method of claim 1, wherein, the step of scaling the first image according to Y luminance information and performing noise reduction on the scaled first image to obtain a second image specifically comprises: determining a resolution according to the performance of a DSP, and scaling the first image according to the resolution and a Y component of the first image; performing filter noise reduction on the scaled first image using a Gaussian blur operator to obtain the second image.

3. The image-based FV value calculation method of claim 1, wherein, the step of obtaining an FV value according to the first edge data specifically comprises: calculating a standard deviation of the first edge data through a DSP interface, wherein the standard deviation is the FV value.

4. A DSP device, characterized by The method comprises the following steps: a scaling and noise reduction module, a detection and filtering module, and an FV value calculation module; the scaling and noise reduction module is configured to obtain a first image, scale the first image according to Y luminance information, and perform noise reduction on the scaled first image to obtain a second image; the Y luminance information is a Y component in YUV data; the detection and filtering module is configured to perform edge detection on the second image to obtain second edge data, and perform content filtering on the second image to obtain Mask data; and combine the second edge data and the Mask data to obtain first edge data of the second image; the step of performing edge detection on the second image to obtain second edge data specifically comprises: using a DSP edge detection operator to calculate second edge data of the second image, wherein the second edge data comprises a set of points with discontinuous image grayscale values; the step of performing content filtering on the second image to obtain Mask data specifically comprises: performing binaryzation on the second image to detect a light brightness area; and filtering edges of the light brightness area through an erosion operation to generate Mask data; The second edge data and the Mask data are combined to obtain first edge data of the second image, specifically, the second edge data is converted into an edge matrix, the Mask data is converted into a Mask matrix, the edge matrix and the Mask matrix are multiplied to obtain the first edge data, and the first edge data is weighted edge data. The FV value calculation module is configured to obtain an FV value according to the first edge data.

5. A contrast focusing method, characterized by, The method comprises: An image is obtained, and an FV value calculation method based on an image according to claims 1 to 3 is applied to calculate the FV value of the image. According to a focusing algorithm, the FV value is used to control a lens motor to realize zooming.

6. A contrast focusing device, characterized by The method comprises an FV value calculation module and a focusing module. The FV value calculation module is configured to obtain an image, and a DSP device according to claim 4 is applied to calculate the FV value of the image. The focusing module is configured to use the FV value to control a lens motor according to a focusing algorithm to realize zooming.

Images