Camera module optimization method and device, electronic equipment and storage medium

Abstract

The invention provides a camera module optimization method and device, electronic equipment and a storage medium, and relates to the technical field of quality classification of camera modules. The method comprises the following steps: acquiring an image shot by a camera module, and performing definition verification on the image to obtain actually measured definition values of a central region and four corner regions of the image; determining the category of the camera module based on preset definition reference thresholds of the central area and the four-corner area and the actually measured definition values of the central area and the four-corner area of the image; if the category of the camera module is an edge product, performing region enhancement processing on the image, reducing the definition difference between the center and the edge of the image, and improving the visual impression; the optimization processing of the camera module is realized, the edge lens can be improved into a good product, the production yield is improved, and the cost is saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of quality classification of camera modules, and particularly relates to a camera module optimization method and device, an electronic device and a storage medium. BACKGROUND

[0002] As one of the core indicators for measuring the imaging quality of a camera module, the sharpness directly affects the final imaging effect. In the actual production process, due to the limitations of the manufacturing process of optical lenses and the accumulation of assembly tolerances, the lens often has field curvature problems to varying degrees. This phenomenon is specifically manifested in that there is a significant difference in sharpness between the center area and the four corners of the imaging plane: the center area is sharp, while the edge area may have problems such as resolution decline and detail loss. In extreme cases, some local areas may even have obvious imaging blur or distortion, which not only reduces the overall picture quality of the photo, but also seriously affects the user's experience in shooting long-range, document scanning and other high-precision scenarios. Therefore, how to optimize the camera module has become a technical problem to be solved. SUMMARY

[0003] In view of the above problems, the present application is proposed to provide a camera module optimization method and device, an electronic device and a storage medium which overcome the above problems or at least partially solve the above problems, can upgrade the edge product lens to good product, improve the production yield and save cost. The technical solution is as follows:

[0004] In the first aspect, a camera module optimization method is provided, and the method comprises:

[0005] An image photographed by a camera module is acquired, and sharpness verification is performed on the image to obtain measured sharpness values of a center area and four corner areas of the image;

[0006] Based on preset center area and four corner area sharpness reference thresholds and the measured sharpness values of the center area and the four corner areas of the image, the category of the camera module is determined;

[0007] If the category of the camera module is an edge product, a region enhancement process is performed on the image to realize optimization of the camera module.

[0008] In the second aspect, a camera module optimization device is provided, and the device comprises:

[0009] A verification unit is configured to acquire an image photographed by a camera module, and perform sharpness verification on the image to obtain measured sharpness values of a center area and four corner areas of the image;

[0010] determining unit configured to determine the category of the camera module based on a pre-set center region and corner region definition threshold and measured definition values of the center region and the corner region of the image;

[0011] a region enhancement unit configured to perform a region enhancement process on the image to achieve the optimization of the camera module if the category of the camera module is an edge product.

[0012] In a third aspect, an electronic device is provided, which includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the optimization method of the camera module according to any one of the preceding aspects.

[0013] In a fourth aspect, a storage medium is provided, which stores a computer program, wherein the computer program is configured to perform the optimization method of the camera module according to any one of the preceding aspects when running.

[0014] By means of the above technical solutions, the optimization method and device of the camera module, the electronic device and the storage medium provided by the embodiments of the present application, the optimization method of the camera module is based on a pre-set center region and corner region definition threshold and measured definition values of the center region and the corner region of the image to determine the category of the camera module. If the category of the camera module is an edge product, a region enhancement process is performed on the image to reduce the definition difference between the center and the edge of the image and improve the visual perception. The optimization of the camera module is achieved, which can improve the edge product lens to a good product, improve the production yield and save costs. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced.

[0016] Figure 1 a flowchart of the optimization method of the camera module provided by the embodiments of the present application is shown;

[0017] Figure 2 a schematic diagram of the measured definition values of the center region and the corner region of the image provided by the embodiments of the present application is shown;

[0018] Figure 3 a schematic diagram of the calculation of the pixel-level enhancement intensity coefficient matrix provided by the embodiments of the present application is shown;

[0019] Figure 4 a schematic diagram of the pixel-level enhancement intensity coefficient matrix represented by polar coordinates provided by the embodiments of the present application is shown;

[0020] Figure 5A schematic diagram of calculating the enhanced intensity of any pixel point P in an image is shown.

[0021] Figure 6 An effect schematic diagram of the pixel-level enhanced intensity coefficient matrix after visualization is shown.

[0022] Figure 7 A structure diagram of an optimization device of a camera module is shown.

[0023] Figure 8 A structure diagram of an electronic device is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thoroughly and completely understood, and will fully convey the scope of the application to those skilled in the art.

[0025] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that such use is interchangeable under appropriate circumstances so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the term "comprising" and variations thereof are to be construed as meaning "including but not limited to", an open-ended term.

[0026] To solve the above technical problems, the present application provides an optimization method of a camera module, as shown in Figure 1 The optimization method of the camera module can include the following steps S101-S103:

[0027] Step S101, an image captured by a camera module is obtained, and a sharpness check is performed on the image to obtain measured sharpness values of the center region and the four corner regions of the image.

[0028] In this step, the center region and the four corner regions of the image can be selected, and a sharpness check calculation is performed on the center region and the four corner regions of the image using a commonly used sharpness statistical function or operator for contrast focusing. The corresponding sharpness check calculation method includes but is not limited to: gray difference method, gradient function (such as Roberts operator, Sobel operator, Tenengrad function, Laplacian operator, etc.), and frequency domain energy analysis (based on Fourier transform, wavelet transform or IIR filter) and the like.

[0029] AsFigure 2 The figure shows the measured sharpness value of the image center region and the four corner regions, 0 is the image center region; the four corner regions include the upper left corner region, the upper right corner region, the lower left corner region, and the lower right corner region, which are 1, 2, 3, and 4, respectively.

[0030] The current camera module sharpness verification method is to verify the horizontal V and vertical H sharpness of the image center region and the four corner regions by the camera module shooting line. The higher the verification value (also known as the measured sharpness value or measurement value) is, the clearer the image picture is.

[0031] In the Figure 2 , the typical measurement values of the sharpness of the image center and the four corners are known. The typical measurement value here is a representative of the current region sharpness measurement value, and does not require consistency in all image positions, which can well meet the practicality principle.

[0032] Specifically, the horizontal V and vertical H sharpness typical measurement values of the image center are 243820 and 233371, respectively, which are the representative of the sharpness measurement value of the image center region 0; the horizontal V and vertical H sharpness typical measurement values of the upper left corner are 253534 and 233150, respectively, which are the representative of the sharpness measurement value of the upper left corner region 1; the horizontal V and vertical H sharpness typical measurement values of the lower left corner are 233452 and 246132, respectively, which are the representative of the sharpness measurement value of the lower left corner region 3; the horizontal V and vertical H sharpness typical measurement values of the upper right corner are 113822 and 156684, respectively, which are the representative of the sharpness measurement value of the upper right corner region 2; the horizontal V and vertical H sharpness typical measurement values of the lower right corner are 112471 and 190011, respectively, which are the representative of the sharpness measurement value of the lower right corner region 4; the sharpness of the center region, the upper left corner region, and the lower left corner region in the current image picture is good, while the upper right corner region and the lower right corner region are relatively blurred and need to be optimized.

[0033] Step S102, based on the pre-set center region and corner region sharpness reference threshold, and the measured sharpness value of the image center region and the corner region, determine the category of the camera module.

[0034] In an optional embodiment, based on the pre-set center region and corner region definition threshold and the measured definition values of the image center region and the corner region, if the measured definition values of the image center region and the corner region are all more than the first multiple of the respective definition threshold, the camera module is determined as a good product; if the measured definition values of the image center region and the corner region are less than the second multiple of the respective definition threshold, the camera module is determined as a bad product; wherein the second multiple is less than the first multiple; if the measured definition values of the image center region and the corner region are all more than the second multiple of the respective definition threshold, but not all more than the first multiple of the respective definition threshold, the camera module is determined as an edge product.

[0035] For example, based on the pre-set center region and corner region definition threshold and the measured definition values of the image center region and the corner region, if the measured definition values of the image center region and the corner region are all more than 1.2 times of the respective definition threshold, the camera module is determined as a good product; if the measured definition values of the image center region and the corner region are less than 0.8 times of the respective definition threshold, the camera module is determined as a bad product; if the measured definition values of the image center region and the corner region are all more than 0.8 times of the respective definition threshold, but not all more than 1.2 times of the respective definition threshold, the camera module is determined as an edge product. It should be noted that the above example is only illustrative and does not limit the present embodiment.

[0036] In step S103, if the camera module is an edge product, the image is subjected to a region enhancement processing to realize the optimization processing of the camera module.

[0037] The present embodiment optimizes the edge product in software aspect, improves the definition thereof, and makes it a good product, thereby improving the yield and saving the cost.

[0038] In the present embodiment, a possible implementation is provided. In step S103, if the camera module is an edge product, the image is subjected to a region enhancement processing to realize the optimization processing of the camera module, which can include the following steps S103-1 to S103-3:

[0039] In step S103-1, if the camera module is an edge product, the definition expected difference of the image center region and the corner region is calculated according to the measured definition values of the image center region and the corner region.

[0040] In step S103-2, the pixel-level enhancement intensity coefficient matrix is calculated according to the definition expected difference of the image center region and the corner region, wherein the element in the pixel-level enhancement intensity coefficient matrix describes the enhancement intensity corresponding to the definition expected difference of the pixel point.

[0041] Step S103-3, performing a region-adaptive enhancement processing on the image by the pixel-level enhancement intensity coefficient matrix.

[0042] The embodiment can adaptively perform a region enhancement on the image according to the pixel-level enhancement intensity coefficient, reduce the difference in definition between the center and the edge of the image, and improve the visual perception; the optimization processing on the camera module is realized, the edge lens can be improved to a good product, the production yield is improved, and the cost is saved.

[0043] In the embodiment of the application, the expected difference in definition of the center region and the corner region mentioned in step S103-1 refers to the difference between the definition of the center region and the corner region and the definition expected to be reached. There are two calculation schemes for the difference:

[0044] (1) Taking the measured definition value of the center region as a reference, the four-corner definition detection result of the enhanced image is made to approach the center region;

[0045] (2) Taking the preset definition threshold of the center region and the corner region as a reference, the definition of the center region and the corner region of the enhanced image is made to approach the respective threshold.

[0046] For scheme (1), the following steps A1 and A2 can be specifically included:

[0047] Step A1, for each corner region in the corner region, taking the measured definition value of the center region of the image as a reference, the horizontal measured definition value of the corner region is subtracted from the horizontal measured definition value of the center region of the image, and divided by the horizontal measured definition value of the center region of the image to obtain a first value of the corner region; and the vertical measured definition value of the corner region is subtracted from the vertical measured definition value of the center region of the image, and divided by the vertical measured definition value of the center region of the image to obtain a second value of the corner region;

[0048] Step A2, selecting the maximum value of the first value and the second value of the corner region as the expected difference in definition of the corner region.

[0049] For example, taking Figure 2 as an example, the horizontal V and vertical H measured definition values of the center region of the image are denoted as and The horizontal V and vertical H measured definition values of the four corner regions 1, 2, 3 and 4 are denoted as

[0050] Using steps A1 and A2, the expected differences in definition of the four corner regions 1, 2, 3 and 4 are value1, value2, value3 and value4, respectively, that is:

[0051]

[0052] The embodiment takes the measured sharpness value of the center region of the image as a reference to calculate the expected difference in sharpness of the four corner regions, and further calculates a pixel-level enhancement intensity coefficient matrix according to the expected difference in sharpness of the center region and the four corner regions, so that the four corner regions of the image after enhancement are closer to the center region.

[0053] For scheme (2), the following steps B1 to B4 can be specifically included:

[0054] Step B1, for the center region of the image, taking the preset sharpness reference threshold of the center region of the image as a reference, the horizontal measured sharpness value of the center region of the image is subtracted from the preset horizontal sharpness reference threshold of the center region of the image, and divided by the preset horizontal sharpness reference threshold of the center region of the image to obtain the first value of the center region of the image; and the vertical measured sharpness value of the center region of the image is subtracted from the preset vertical sharpness reference threshold of the center region of the image, and divided by the preset vertical sharpness reference threshold of the center region of the image to obtain the second value of the center region of the image;

[0055] Step B2, selecting the maximum value of the first value and the second value of the center region of the image as the expected difference in sharpness of the center region of the image;

[0056] Step B3, for each corner region in the four corner regions, taking the respective preset sharpness reference threshold of each corner region as a reference, the horizontal measured sharpness value of the corner region is subtracted from the preset horizontal sharpness reference threshold of the corner region, and divided by the preset horizontal sharpness reference threshold of the corner region to obtain the third value of the corner region; and the vertical measured sharpness value of the corner region is subtracted from the preset vertical sharpness reference threshold of the corner region, and divided by the preset vertical sharpness reference threshold of the corner region to obtain the fourth value of the corner region;

[0057] Step B4, selecting the maximum value of the third value and the fourth value of the corner region as the expected difference in sharpness of the corner region.

[0058] In the image captured by the camera module, the measured sharpness value of the image has a trend of highest center region sharpness and gradually decreasing to the four corner regions. The embodiment takes the respective preset sharpness reference threshold of the center region and the four corner regions as a reference, so that the center region and the four corner regions of the image after enhancement are closer to the respective threshold, thereby being able to improve the sharpness according to the respective threshold of the relevant region.

[0059] The pixel-level enhancement intensity coefficient matrix mentioned in step S103-2 in the embodiments of the present application includes elements describing the enhancement intensity corresponding to the expected difference in definition of the pixel points, and the pixel-level enhancement intensity coefficient matrix is generally consistent with the size of the image to be enhanced.

[0060] The design of the pixel-level enhancement intensity coefficient matrix should meet the following two design principles 1) and 2):

[0061] 1) Intensity adaptation: The pixel-level enhancement intensity coefficient matrix can be used to determine the enhancement intensity according to the expected difference in definition in different regions during image enhancement, so as to achieve adaptive enhancement. Specifically, for the regions with poor definition, the corresponding enhancement intensity in the pixel-level enhancement intensity coefficient matrix is large, and the definition is significantly improved after image enhancement; for the regions with good definition, the corresponding enhancement intensity in the pixel-level enhancement intensity coefficient matrix is small or zero, and the definition is slightly improved or not enhanced, thereby avoiding excessive and unreasonable enhancement of the regions with good definition due to global enhancement of the image, and further expanding the definition difference of different regions of the entire image or making the regions with good definition unnatural.

[0062] 2) Continuity: The pixel-level enhancement intensity coefficient matrix should have continuity in position and smooth transition, that is, there should be no sudden change. In the specific implementation process, interpolation can be used to avoid visible image defects caused by enhancement.

[0063] The following takes the expected difference in definition calculation scheme (1) as an embodiment (not limited thereto, and any strategy meeting the requirements of intensity differentiation in different positions and continuity in position can be used), and the steps of the pixel-level enhancement intensity coefficient matrix include steps C1 and C2:

[0064] Step C1, calculating the pixel-level enhancement intensity coefficient matrix of image enhancement.

[0065] Taking the center of the image as the center of a circle, denoting the distance of a pixel point to the center of the image as R, denoting the distance of the upper left corner of the image to the center of the circle as D, setting R0 as a threshold value of the proportion of the radius of the central region circle in D (which can be flexibly adjusted), and setting R1 as a threshold value of the proportion of the radius of the four-corner region circle in D (which can be flexibly adjusted); denoting the enhancement intensity value of a single pixel point as α, and the calculation method is as follows:

[0066] When R is less than R0, the value of α is 0, and no image enhancement is performed.

[0067] When R is greater than or equal to R0, α is calculated according to the expected difference in definition of the four-corner region, and the pixel-level enhancement intensity coefficient matrix is obtained.

[0068] For example, Figure 3As shown, the original image has a width W and a height H, the distance from the top left corner of the image to the center of the circle is D, R0 is set as a threshold value of the proportion of the radius of the central region circle in D (flexible adjustment), and R1 is set as a threshold value of the proportion of the radius of the four corner region circle in D (flexible adjustment); the original image is divided into four regions from the center of the four corners, and the expected difference in sharpness of the top left corner region, the top right corner region, the bottom left corner region, and the bottom right corner region is value1, value2, value3, and value4, respectively, representing the maximum value of the sharpness difference of the corresponding region and the central region, which is referred to as the maximum value of the sharpness difference. The enhancement intensity of each pixel point is related to the maximum value of the sharpness difference of the four corners, but considering that the difference value of the regions with similar distances can better reflect the difference degree of the sharpness of the position, the maximum value of the sharpness difference of the nearest two regions is used to represent the sharpness difference between the position and the central region. Specifically, the expected difference in sharpness of the nearest two corner regions in the four corner regions is used to calculate a, that is:

[0069] a = a * value x + (1-a) * value y (x, y = 1, 2, 3, 4)

[0070] wherein: x and y are the numbers of the two adjacent corner regions, a is the proportion of the expected difference in sharpness of the corner region indicated by x in determining a, and * is a multiplication sign.

[0071] In order to quantify the proportion of the maximum value of the sharpness difference of the adjacent regions in determining a, the ratio of the arc formed by the line connecting the pixel point and the image center to the right half axis to the arc formed by the line connecting the two adjacent corner regions and the center is taken as a, which is as follows:

[0072] Taking the right half axis of the image center as 0 radian, the counterclockwise direction as the positive direction, recording the arc angle formed by the line connecting the pixel point and the image center to the right half axis as θ, recording the angle between the line connecting the top right corner of the image and the center to the right half axis as angle1, recording the angle between the line connecting the top left corner of the image and the center to the right half axis as angle2, recording the angle between the line connecting the bottom left corner of the image and the center to the right half axis as angle3, and recording the angle between the line connecting the bottom right corner of the image and the center to the right half axis as angle4.

[0073] As shown in Figure 3 , the total circumference is 2π, according to the four angles angle1, angle2, angle3, and angle4 provided above, the image is divided into five regions ①, ②, ③, ④, and ⑤, and the sharpness difference of each pixel point and the center region is calculated in turn (the θ region is determined by the nearest two value values), and the specific steps are as follows:

[0074] When θ≤angle1, the value of a is determined by value2 and value4, and the proportion of value4 is: a=(angle1-θ) / (angle1*2);

[0075] The value of a is: a=a*value4+(1-a)*value2;

[0076] When angle1<θ≤angle2, the value of a is determined by value1 and value2, and the proportion of value1 is: a=(θ-angle1) / (angle2-angle1);

[0077] The value of a is: a=a*value1+(1-a)*value2;

[0078] When angle2<θ≤angle3, the value of a is determined by value1 and value3, and the proportion of value3 is: a=(θ-angle2) / (angle3-angle2);

[0079] The value of a is: a=a*value3+(1-a)*value1;

[0080] When angle3<θ≤angle4, the value of a is determined by value3 and value4, and the proportion of value4 is: a=(θ-angle3) / (angle4-angle3);

[0081] The value of a is: a=a*value4+(1-a)*value3;

[0082] When θ>angle4, the value of a is determined by value4 and value2, and the proportion of value2 is: a=(θ-angle4) / (angle1*2);

[0083] The value of a is: a=a*value2+(1-a)*value4.

[0084] Further, in order to ensure the smooth transition of the image enhancement effect, that is, gradually increasing from the small intensity of the center area to the large intensity of the corner area, but not suddenly changing, a transition function is introduced to achieve this function. The transition function can achieve the transition effect of starting and ending slowly, and the implementation is as follows:

[0085] When R≥R1, the value of a can be used as the enhancement intensity value of the pixel point;

[0086] When R0≤R<R1, the difference between R1 and R0 is ΔR, specifically:

[0087] When a < 0.5, a = a*((R-R0) / AR) 4 *8;

[0088] When a > 0.5, a = a*(1.0-(1.0-(R-R0) / AR) 4 *8).

[0089] In an optional embodiment, in order to improve processing speed or reduce memory occupation, if the long-focus image resolution required for image enhancement is large, and the pixel-level enhancement intensity coefficient matrix and the original image have the same specifications, the down-sampled (i.e. reduced) pixel-level enhancement intensity coefficient matrix can be calculated first, and then interpolated to the pixel-level enhancement intensity coefficient matrix corresponding to the original image. For example, if the original image has a resolution of 3840x2160, the pixel-level enhancement intensity coefficient matrix can be calculated first by down-sampling to 1280x720. The effect of down-sampling is to significantly reduce the calculation amount and improve the speed under the premise of basically not affecting the enhancement effect by appropriately reducing the image resolution.

[0090] It should be noted that the above area division and calculation method is only one embodiment, and in order to more generally describe the enhancement intensity of the pixel points in the image, the polar coordinate method can be used.

[0091] As shown in Figure 4 , the point P in the figure represents the image enhancement intensity of the corresponding position, and any pixel point P in the image is represented as (r, β) in the Cartesian coordinate system; according to the expected difference in sharpness of the center area and the corner area of the image, the image enhancement intensities of the center area and the corner area of the image are determined as Then the problem is converted into calculating the mapping relationship between the image intensity S of an arbitrary pixel point and the known sharpness image enhancement intensities of the center and the four corners, i.e.

[0092] S = f(L)

[0093] Where S represents the enhancement intensity corresponding to any point in the figure, and f(L) represents the mapping relationship between the image enhancement intensity of any point in the image and the sharpness image enhancement intensities of the five positions (Location).

[0094] The calculation of the image enhancement intensity of any point in the image is divided into three steps (bilinear interpolation method):

[0095] 1) Determine the interpolation reference point: as Figure 5 shown, the entire image frame is divided into four blocks by the diagonal lines, and for any point P, the left and right reference points are determined tangentially, i.e. taking the radius r of the P point as the reference, finding the intersection points of the left and right diagonal lines closest to P, denoted as P A and P BAs an interpolation reference point, any pixel point in the image can find the corresponding left and right two interpolation reference points, and so on.

[0096] 2) Radial P A , P B The corresponding image enhancement intensity value S A , S B : Here, the radial refers to the extension direction of the diagonal line where the interpolation reference point is located, and the image enhancement intensity of the corresponding interpolation reference point is obtained by selecting the nearest two image enhancement intensity interpolation of the definition typical measurement value area on the diagonal line. As shown in Figure 5 P A The nearest two definition measurement value areas are Record the distance from P A to r A ,

[0097]

[0098] P B The nearest two definition measurement value areas are Record the distance from P B to r B ,

[0099]

[0100] 3) S A , S B Based on the tangential P, the image enhancement intensity value S corresponding to P is obtained:

[0101] Record the angle between P and P A on the diagonal line as α A , record the angle between P and P B on the diagonal line as α B , and the image enhancement intensity S of any pixel point P is:

[0102]

[0103] Traverse the whole image to obtain the pixel-level enhancement intensity coefficient matrix.

[0104] Step C2, calculate the pixel-level enhancement intensity coefficient matrix of the original image.

[0105] Interpolation is performed on the pixel-level enhanced intensity coefficient matrix of the image obtained in step C1 above, i.e., a pixel-level enhanced intensity coefficient matrix of the same size as the original image is obtained. Specifically, the interpolation methods include nearest-neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. Preferably, the bilinear interpolation method is used in this embodiment (the specific interpolation formula is known to those skilled in the art and will not be described here), and the final effect is visualized as shown in FIG. 8. It should be noted that the numerical values of the elements in the pixel-level enhanced intensity coefficient matrix of the original image can be floating-point numbers or fixed-point numbers (0 to 255), depending on the sharpening parameters or numerical conversion logic. For example, if the numerical values of the elements in the pixel-level enhanced intensity coefficient matrix are floating-point numbers, they are only converted to the range of [0, 255] for visualization. Figure 6

[0106] It should be further noted that the pixel-level enhanced intensity coefficient matrix corresponding to the same camera is actually unchanged during use. In order to improve the calculation efficiency and save computing resources, preferably, the measured sharpness value of the module can be obtained during factory calibration, and based on the pre-set center region and corner region sharpness reference threshold and the measured sharpness values of the image center region and corner region, the category of the camera module is determined as an edge product, i.e., the corresponding pixel-level enhanced intensity coefficient matrix is calculated and written into the embedded platform. When image region enhancement is needed, the stored pixel-level enhanced intensity coefficient matrix is directly read without the need for further calculation. Alternatively, the measured sharpness value of the module is read when the camera system is started, and based on the pre-set center region and corner region sharpness reference threshold and the measured sharpness values of the image center region and corner region, the category of the camera module is determined as an edge product, and the pixel-level enhanced intensity coefficient matrix is calculated asynchronously in a separate thread (without affecting the system startup time) and stored in the memory (such as DDR) or disk (such as Flash, SDCard, etc.), and directly read when image region enhancement is performed without the need for repeated calculation. Here, DDR stands for Double Data Rate, double data rate memory; Flash is flash memory; and SDCard stands for Secure Digital Card, secure digital storage card.

[0107] ​A possible implementation manner is provided in the embodiment of the present application, and the image enhancement manner adopted in the embodiment is usm sharpening (only a preferred manner). The usm sharpening is a classical image sharpening algorithm, which makes the image look clearer by enhancing the edge contrast. The effect of the method has three influence amounts, i.e., amount, radius and threshold. The amount can be understood as the intensity of sharpening. When global sharpening is performed on the image, the value is a constant value. If the value is replaced by a pixel-level enhancement intensity coefficient matrix, different enhancement intensities can be adaptively adopted for different regions in the sharpening process, so that the enhanced image result is obtained, and the optimization processing of the camera module is realized.

[0108] It should be noted that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In actual application, all the above possible implementation manners can be combined in any combination to form possible embodiments of the present application, which will not be repeated here.

[0109] Based on the camera module optimization method provided in each of the above embodiments, based on the same inventive concept, the embodiment of the present application further provides a camera module optimization device.

[0110] Figure 7 is a structural diagram of the camera module optimization device provided by the embodiment of the present application. As shown in Figure 7 The camera module optimization device can specifically include a verification unit 710, a determination unit 720 and a region enhancement unit 730.

[0111] The verification unit 710 is configured to obtain an image captured by a camera module, and perform sharpness verification on the image to obtain measured sharpness values of a center region and four corner regions of the image.

[0112] The determination unit 720 is configured to determine a category of the camera module based on a pre-set center region and four corner region sharpness reference threshold and the measured sharpness values of the center region and the four corner regions of the image.

[0113] The region enhancement unit 730 is configured to perform region enhancement processing on the image to realize optimization processing of the camera module if the category of the camera module is an edge product.

[0114] A possible implementation manner is provided in the embodiment of the present application. The determination unit 720 is further configured to:

[0115] determining the camera module to be a good product if the measured sharpness values of the center region and the corner regions of the image are all greater than a first multiple of the respective sharpness reference threshold value;

[0116] determining the camera module to be a bad product if the measured sharpness values of the center region and the corner regions of the image are all less than a second multiple of the respective sharpness reference threshold value, wherein the second multiple is less than the first multiple;

[0117] determining the camera module to be an edge product if the measured sharpness values of the center region and the corner regions of the image are all greater than a second multiple of the respective sharpness reference threshold value, but not all greater than the first multiple of the respective sharpness reference threshold value.

[0118] In an embodiment of the present application, the region enhancement unit 730 is further configured to:

[0119] if the camera module is determined to be an edge product, calculating a sharpness expectation difference of the center region and the corner regions of the image according to the measured sharpness values of the center region and the corner regions of the image;

[0120] calculating a pixel-level enhancement intensity coefficient matrix according to the sharpness expectation difference of the center region and the corner regions of the image, wherein an element in the pixel-level enhancement intensity coefficient matrix describes an enhancement intensity corresponding to the sharpness expectation difference of a pixel point;

[0121] performing a region-adaptive enhancement processing on the image by using the pixel-level enhancement intensity coefficient matrix.

[0122] In an embodiment of the present application, the region enhancement unit 730 is further configured to:

[0123] for each corner region in the corner regions, taking the measured sharpness value of the center region of the image as a reference, calculating a first value of the corner region by subtracting a horizontal measured sharpness value of the corner region from a horizontal measured sharpness value of the center region of the image and dividing the result by the horizontal measured sharpness value of the center region of the image, and calculating a second value of the corner region by subtracting a vertical measured sharpness value of the corner region from a vertical measured sharpness value of the center region of the image and dividing the result by the vertical measured sharpness value of the center region of the image;

[0124] taking a maximum value between the first value and the second value of the corner region as a sharpness expectation difference of the corner region.

[0125] In an embodiment of the present application, the region enhancement unit 730 is further configured to:

[0126] For the image center region, a difference between the transverse measured sharpness value of the image center region and the preset transverse sharpness reference threshold of the image center region is obtained based on the preset transverse sharpness reference threshold of the image center region, and the difference is divided by the preset transverse sharpness reference threshold of the image center region to obtain a first value of the image center region; and a difference between the longitudinal measured sharpness value of the image center region and the preset longitudinal sharpness reference threshold of the image center region is obtained, and the difference is divided by the preset longitudinal sharpness reference threshold of the image center region to obtain a second value of the image center region;

[0127] The maximum value of the first value and the second value of the image center region is selected as the sharpness expected difference of the image center region.

[0128] For each corner region in the four-corner region, a difference between the transverse measured sharpness value of the corner region and the preset transverse sharpness reference threshold of the corner region is obtained based on the preset transverse sharpness reference threshold of the corner region, and the difference is divided by the preset transverse sharpness reference threshold of the corner region to obtain a third value of the corner region; and a difference between the longitudinal measured sharpness value of the corner region and the preset longitudinal sharpness reference threshold of the corner region is obtained, and the difference is divided by the preset longitudinal sharpness reference threshold of the corner region to obtain a fourth value of the corner region.

[0129] The maximum value of the third value and the fourth value of the corner region is selected as the sharpness expected difference of the corner region.

[0130] In an embodiment of the present application, a possible implementation manner is provided, and the region enhancement unit 730 is further configured to:

[0131] Taking the image center as the center of a circle, denoting the distance from a pixel point to the image center as R, denoting the distance from the top-left corner of the image to the center of the circle as D, setting R0 as a threshold of the proportion of the radius of the center region circle in D, and setting R1 as a threshold of the proportion of the radius of the four-corner region circle in D; denoting the enhancement intensity value of a single pixel point as α, and the calculation manner is as follows:

[0132] When R is less than R0, the value of α is 0, and no image enhancement is performed.

[0133] When R is greater than or equal to R0, α is calculated according to the sharpness expected difference of the four-corner region to obtain a pixel-level enhancement intensity coefficient matrix.

[0134] In an embodiment of the present application, a possible implementation manner is provided, and the region enhancement unit 730 is further configured to:

[0135] Denote the sharpness expected differences of the top-left corner region, the top-right corner region, the bottom-left corner region and the bottom-right corner region as value1, value2, value3 and value4 respectively.

[0136] The definition of alpha is the difference of the definition of the two adjacent corner regions, i.e.:

[0137] alpha = a*value x +(1-a)*value y (x,y = 1,2,3,4)

[0138] where x and y are the numbers of the two adjacent corner regions, a is the proportion of the definition of the corner region x in the definition of alpha, and * is the multiplication sign.

[0139] In an embodiment of the present application, the region enhancement unit 730 is further configured to:

[0140] The proportion of the angle between the line connecting the pixel point and the image center and the right half axis and the angle between the line connecting the two adjacent corner regions and the center and the right half axis is taken as a, and the specific process is as follows:

[0141] The right half axis of the image center is taken as 0 radian, the counterclockwise direction is taken as the positive direction, the angle between the line connecting the pixel point and the image center and the right half axis is taken as theta, the angle between the line connecting the right upper corner of the image and the center and the right half axis is taken as angle1, the angle between the line connecting the left upper corner of the image and the center and the right half axis is taken as angle2, the angle between the line connecting the left lower corner of the image and the center and the right half axis is taken as angle3, and the angle between the line connecting the right lower corner of the image and the center and the right half axis is taken as angle4.

[0142] When theta is less than or equal to angle1, the value of alpha is determined by value2 and value4, and the proportion of value4 is a = (angle1-theta) / (angle1*2).

[0143] The value of alpha is a*value4+(1-a)*value2.

[0144] When angle1 is less than theta and theta is less than or equal to angle2, the value of alpha is determined by value1 and value2, and the proportion of value1 is a = (theta-angle1) / (angle2-angle1).

[0145] The value of alpha is a*value1+(1-a)*value2.

[0146] When angle2 is less than theta and theta is less than or equal to angle3, the value of alpha is determined by value1 and value3, and the proportion of value3 is a = (theta-angle2) / (angle3-angle2).

[0147] The value of alpha is a*value3+(1-a)*value1.

[0148] When angle3<θ≤angle4, the value of α is determined by value3 and value4, and the proportion of value4 is: a=(θ-angle3) / (angle4-angle3);

[0149] Then the value of α is: α=a*value4+(1-a)*value3;

[0150] When θ>angle4, the value of α is determined by value4 and value2, and the proportion of value2 is: a=(θ-angle4) / (angle1*2);

[0151] Then the value of α is: α = a*value2+(1-a)*value4.

[0152] An embodiment of the present application provides a possible implementation method, in which the region enhancement unit 730 is further configured to:

[0153] When R≥R1, the α value can be used as the enhancement intensity value of the pixel;

[0154] When R0≤R<R1, the difference between R1 and R0 is ΔR, specifically:

[0155] When α<0.5, α=α*((R-R0) / ΔR) 4 *8;

[0156] When α≥0.5, α=α*(1.0-(1.0-(R-R0) / ΔR) 4 *8).

[0157] An embodiment of the present application provides a possible implementation method, in which the region enhancement unit 730 is further configured to:

[0158] According to the expected difference in clarity between the center area and the four corner areas of the image, the image enhancement strengths of the center area and the four corner areas are determined as follows:

[0159] Assume that any pixel point P in the Cartesian coordinate system is represented by (r, β). Taking the radius r of point P as the reference, find the intersection point of the left and right diagonals of the image closest to P, which is recorded as P. A and P B As an interpolation reference point, by analogy, any pixel in the image can find the corresponding left and right interpolation reference points;

[0160] Radial P A 、P B The corresponding image enhancement strength value S A 、S B ;

[0161] P A The two closest clarity measurement values ​​are remember to P A The distance is r A ,but

[0162]

[0163] P B The two closest clarity measurement values ​​are remember to P B The distance is r B ,but

[0164]

[0165] Remember P and P A The angle between the diagonals is α A , remember P and P B The angle between the diagonals is α B , the image enhancement intensity S of any pixel point P is:

[0166]

[0167] Traverse the entire image and obtain the pixel-level enhancement intensity coefficient matrix.

[0168] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the camera module optimization method of any one of the above embodiments.

[0169] In an exemplary embodiment, an electronic device is provided, such as Figure 8 As shown, Figure 8 The electronic device 800 shown includes a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, for example, via a bus 802. Optionally, the electronic device 800 may further include a transceiver 804. It should be noted that in actual applications, the number of transceivers 804 is not limited to one, and the structure of the electronic device 800 does not constitute a limitation on the embodiments of the present application.

[0170] The processor 801 can be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array) or other programmable logic device, transistor logic device, hardware component or any combination thereof. It can implement or execute the various exemplary logical blocks, modules and circuits described in connection with the disclosure. The processor 801 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0171] The bus 802 can include a path that transmits information between the above-mentioned components. The bus 802 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 802 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 8 In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0172] The memory 803 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but not limited to this.

[0173] The memory 803 is configured to store computer program codes for implementing the solutions of the present application, and the processor 801 is configured to execute the computer program codes stored in the memory 803.

[0174] The electronic device includes, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (for example, a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 8 The electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0175] Based on the same inventive concept, the embodiments of the present application further provide a storage medium, which stores a computer program, and the computer program is configured to execute the optimization method of the camera module of any one of the above-mentioned embodiments when running.

[0176] Those skilled in the art can clearly understand the specific working process of the above-mentioned system, device and module, and can refer to the corresponding process in the above-mentioned method embodiments. For the sake of brevity, no further description is given here.

[0177] Those skilled in the art can understand that the technical solutions of the present application can be embodied in the form of software product in essence or all or part of the technical solutions, and the computer software product is stored in a storage medium, which includes a plurality of program instructions for making an electronic device (such as a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application when running the program instructions. The storage medium mentioned above includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0178] Alternatively, all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware (such as an electronic device of a personal computer, a server, or a network device, etc.), and the program instructions can be stored in a computer readable storage medium, and when the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the method described in the embodiments of the present application.

[0179] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application is described in detail with reference to the above embodiments, those skilled in the art should understand that, within the spirit and principle of the present application, the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the protection scope of the present application.

Claims

1. A camera module optimization method, characterized in that: The method comprises: Obtain the image captured by the camera module and perform clarity verification on the image to obtain the measured clarity values ​​of the center area and the four corner areas of the image; Determining the type of the camera module based on pre-set clarity benchmark thresholds for the center area and the four corner areas, and the measured clarity values ​​for the center area and the four corner areas of the image; If the category of the camera module is a marginal product, regional enhancement processing is performed on the image to achieve optimization processing of the camera module.

2. The method according to claim 1, characterized in that Determining the category of the camera module based on pre-set clarity benchmark thresholds for the center area and the four corner areas, and the measured clarity values ​​for the center area and the four corner areas of the image, includes: Based on the preset clarity reference thresholds for the center area and the four corner areas, and the actually measured clarity values ​​for the center area and the four corner areas of the image, if the actually measured clarity values ​​for the center area and the four corner areas of the image all exceed a first multiple of their respective clarity reference thresholds, then the camera module is determined to be a good product. If the measured clarity values ​​of the image center area and the four corner areas are lower than the second multiple of the respective clarity reference thresholds, the camera module is determined to be defective; wherein the second multiple is smaller than the first multiple; If the measured clarity values ​​of the image center area and the four corner areas all exceed the second multiple of their respective clarity reference thresholds, but not all exceed the first multiple of their respective clarity reference thresholds, the camera module is determined to be a marginal product.

3. The method according to claim 1 or 2, characterized in that If the camera module is of marginal quality, the image is subjected to regional enhancement processing, including: If the camera module is of marginal quality, the expected difference in clarity between the center and four corners of the image is calculated based on the measured clarity values ​​of the center and four corners of the image. Calculate a pixel-level enhancement intensity coefficient matrix based on the expected difference in clarity of the central area and the four corner areas of the image, wherein the elements in the pixel-level enhancement intensity coefficient matrix describe the enhancement intensity corresponding to the expected difference in clarity of the pixel point; The image is enhanced in a region-adaptive manner using the pixel-level enhancement intensity coefficient matrix.

4. The method according to claim 3, characterized in that Based on the measured sharpness values ​​of the image center and the four corners, calculate the expected sharpness difference between the image center and the four corners, including: For each of the four corner regions, taking the measured clarity value of the image center region as a reference, calculating the difference between the horizontal measured clarity value of the corner region and the horizontal measured clarity value of the image center region, and dividing the difference by the horizontal measured clarity value of the image center region, to obtain a first value for the corner region; and calculating the difference between the vertical measured clarity value of the corner region and the vertical measured clarity value of the image center region, and dividing the difference by the vertical measured clarity value of the image center region, to obtain a second value for the corner region; The maximum value between the first value and the second value of the angular area is selected as the expected definition difference of the angular area.

5. The method according to claim 3, characterized in that Based on the measured sharpness values ​​of the image center and the four corners, calculate the expected sharpness difference between the image center and the four corners, including: For the central area of ​​the image, taking a preset clarity reference threshold for the central area of ​​the image as a reference, calculating the difference between the horizontal clarity value actually measured in the central area of ​​the image and the preset horizontal clarity reference threshold for the central area of ​​the image, and dividing the difference by the preset horizontal clarity reference threshold for the central area of ​​the image, to obtain a first value for the central area of ​​the image; and calculating the difference between the vertical clarity value actually measured in the central area of ​​the image and the preset vertical clarity reference threshold for the central area of ​​the image, and dividing the difference by the preset vertical clarity reference threshold for the central area of ​​the image, to obtain a second value for the central area of ​​the image; Selecting the maximum value of the first value and the second value of the central area of ​​the image as the expected clarity difference of the central area of ​​the image; For each of the four corner areas, taking the preset reference threshold value of the respective corner areas as a reference, calculating the difference between the horizontally measured clarity value of the corner area and the preset reference threshold value of the horizontal clarity for the corner area, and dividing the difference by the preset reference threshold value of the horizontal clarity for the corner area, to obtain a third value for the corner area; and calculating the difference between the vertically measured clarity value of the corner area and the preset reference threshold value of the vertical clarity for the corner area, and dividing the difference by the preset reference threshold value of the vertical clarity for the corner area, to obtain a fourth value for the corner area; The maximum value between the third value and the fourth value of the angular area is selected as the expected definition difference of the angular area.

6. The method according to claim 3, characterized in that According to the expected difference in clarity between the center and four corners of the image, the pixel-level enhancement coefficient matrix is ​​calculated, including: With the center of the image as the center of the circle, let the distance from the pixel to the center of the image be R, and the distance from the upper left corner of the image to the center of the circle be D. Set R0 as the threshold of the proportion of the radius of the circle dividing the central area on D, and R1 as the threshold of the proportion of the radius of the circle dividing the four corner areas on D. Let the enhancement strength value of a single pixel be α, and the calculation method is: When R is less than R0, the α value is 0 and no image enhancement is performed; When R is greater than or equal to R0, α is calculated based on the expected difference in clarity of the four corner areas to obtain the pixel-level enhancement intensity coefficient matrix.

7. The method according to claim 6, characterized in that Calculate α based on the expected difference in sharpness in the four corners, including: Assume that the expected differences in sharpness of the upper left corner, upper right corner, lower left corner, and lower right corner are value1, value2, value3, and value4 respectively; The expected difference in sharpness between the two nearest corners in the four corners is used to calculate α, namely: α=a*value x +(1-a)*value y (x,y=1,2,3,4) Where: x and y are the numbers of two adjacent angular regions, a is the proportion of the expected difference in sharpness of the angular region pointed to by x in determining α, and * is the multiplication sign.

8. The method according to claim 7, characterized in that The method further comprises: The ratio of the arc formed by the line connecting the pixel point and the center of the image and the right semi-axis to the arc formed by the line connecting the edges of the two adjacent corner areas and the center is taken as a, as follows: Take the right semi-axis of the image center as 0 radians, counterclockwise as the positive direction, and record the radian angle formed by the line connecting the pixel point, the image center, and the right semi-axis as θ. The angle formed by the line connecting the upper right corner of the image and the center and the right semi-axis is recorded as angle1; the angle formed by the line connecting the upper left corner of the image and the center and the right semi-axis is recorded as angle2; the angle formed by the line connecting the lower left corner of the image and the center and the right semi-axis is recorded as angle3; the angle formed by the line connecting the lower right corner of the image and the center and the right semi-axis is recorded as angle4; When θ≤angle1, the value of α is determined by value2 and value4, and the proportion of value4 is: a=(angle1-θ) / (angle1*2); Then the value of α is: α=a*value4+(1-a)*value2; When angle1<θ≤angle2, the value of α is determined by value1 and value2, and the proportion of value1 is: a=(θ-angle1) / (angle2-angle1); Then the value of α is: α=a*value1+(1-a)*value2; When angle2<θ≤angle3, the value of α is determined by value1 and value3, and the proportion of value3 is: a=(θ-angle2) / (angle3-angle2); Then the value of α is: α=a*value3+(1-a)*value1; When angle3<θ≤angle4, the value of α is determined by value3 and value4, and the proportion of value4 is: a=(θ-angle3) / (angle4-angle3); Then the value of α is: α=a*value4+(1-a)*value3; When θ>angle4, the value of α is determined by value4 and value2, and the proportion of value2 is: a=(θ-angle4) / (angle1*2); Then the value of α is: α = a*value2+(1-a)*value4.

9. The method according to claim 8, characterized in that The method further comprises: When R≥R1, the α value can be used as the enhancement intensity value of the pixel; When R0≤R<R1, the difference between R1 and R0 is ΔR, specifically: When α<0.5, α=α*((R-R0) / ΔR) 4 *8; When α≥0.5, α=α*(1.0-(1.0-(R-R0) / ΔR) 4 *8).

10. The method according to claim 3, characterized in that According to the expected difference in clarity between the center and four corners of the image, the pixel-level enhancement coefficient matrix is ​​calculated, including: According to the expected difference in clarity between the center area and the four corner areas of the image, the image enhancement strengths of the center area and the four corner areas are determined as follows: Assume that any pixel point P in the Cartesian coordinate system is represented by (r, β). Taking the radius r of point P as the reference, find the intersection point of the left and right diagonals of the image closest to P, which is recorded as P. A and P B As an interpolation reference point, by analogy, any pixel in the image can find the corresponding left and right interpolation reference points; Radial P A 、P B The corresponding image enhancement strength value S A 、S B ; P A The two closest clarity measurement values ​​are remember to P A The distance is r A ,but P B The two closest clarity measurement values ​​are remember to P B The distance is r B ,but Remember P and P A The angle between the diagonals is α A , remember P and P B The angle between the diagonals is α B , the image enhancement intensity S of any pixel point P is: Traverse the entire image and obtain the pixel-level enhancement intensity coefficient matrix.

11. A camera module optimization device, characterized in that: The device comprises: The calibration unit is used to obtain the image captured by the camera module and perform clarity calibration on the image to obtain the measured clarity values ​​of the center area and the four corner areas of the image; a determination unit, configured to determine the type of the camera module based on preset clarity reference thresholds of the central area and the four corner areas, and measured clarity values ​​of the central area and the four corner areas of the image; The regional enhancement unit is used to perform regional enhancement processing on the image if the category of the camera module is a marginal product, so as to achieve optimization processing of the camera module.

12. An electronic device, characterized in that: The device comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the camera module optimization method according to any one of claims 1 to 10.

13. A storage medium, characterized in that The storage medium stores a computer program, wherein the computer program is configured to execute the camera module optimization method according to any one of claims 1 to 10 when running.

Images