Camera lens shadow compensation method, device, equipment and camera equipment

Abstract

The present application relates to a camera lens shadow compensation method, device, equipment and camera equipment, the method comprising: pre-processing the original image captured by the camera to obtain a compressed image; calculating the field of view of each pixel in the compressed image, and dividing the compressed image into a central area and a corner area around the central area according to the field of view of each pixel; respectively calculating the shadow compensation value of each field of view in the central area and the corner area; obtaining the correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field of view in the central area and the corner area; performing shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image. Compensating pixels in different areas of the original image separately according to the correction compensation values ​​of pixels in different areas, accurate image compensation can also be achieved for ultra-wide-angle cameras, and the accuracy of image compensation is improved.

Description

Technical Field

[0001] The present application relates to the field of photographic technology, and in particular to a camera lens shadow compensation method, device, equipment and camera equipment. Background Art

[0002] Currently, ultra-wide-angle cameras are being used more and more widely, from security and vehicle-mounted cameras to mobile phone rear cameras. The magnificent imaging effect brought by the ultra-wide viewing angle of mobile phones is increasingly favored by mobile phone terminal users, and the efficiency of traditional security and vehicle-mounted cameras has also been greatly improved due to the application of ultra-wide angle cameras.

[0003] The vignetting caused by the uneven illumination of ultra-wide-angle cameras (Lens Shading) has a great impact on the image quality. Traditional lens shading compensation methods include concentric circle compensation method and grid compensation method. However, for the ultra-wide-angle imaging characteristics, these two methods cannot perfectly compensate the image and have the disadvantage of low image compensation accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a camera lens shading compensation method, device, equipment and camera equipment that can improve the accuracy of image compensation for ultra-wide-angle cameras in order to address the problem of low image compensation accuracy when using traditional lens shading compensation methods to perform shadow compensation for ultra-wide-angle cameras.

[0005] A camera lens shadow compensation method comprises: performing compression preprocessing on an original image captured by a camera to obtain a compressed image; calculating the field of view of each pixel in the compressed image, and dividing the compressed image into a central area and a corner area around the central area according to the field of view of each pixel; respectively calculating the shadow compensation value of each field of view in the central area and the corner area; obtaining a correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field of view in the central area and the corner area; performing shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image.

[0006] The above-mentioned camera lens shadow compensation method processes the original image taken by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view for the central area and the corner area respectively, and the correction compensation value of the pixels in each area, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0007] In one of the embodiments, the original image captured by the camera is compressed and preprocessed to obtain a compressed image, including: performing image preprocessing on the original image captured by the camera to obtain a preprocessed image; and compressing the preprocessed image to obtain a compressed image.

[0008] The original image is preprocessed and compressed to obtain a compressed image, which is convenient for subsequent processing and improves the convenience of image processing.

[0009] In one embodiment, the image preprocessing includes at least one of automatic white balance, bad pixel correction and black level correction. The corresponding method can be selected for image preprocessing according to actual needs, and the operation is convenient and reliable.

[0010] In one of the embodiments, compressing the preprocessed image to obtain a compressed image includes: determining whether the preprocessed image can be evenly divided; if the preprocessed image can be evenly divided, performing ROI (Region Of Interest) segmentation on the preprocessed image to obtain a segmented image, and if the preprocessed image cannot be evenly divided, performing boundary expansion on the preprocessed image, performing ROI segmentation on the image after boundary expansion to obtain a segmented image; and obtaining a compressed image based on the grayscale average value of each segmented image.

[0011] According to the actual situation of the preprocessed image, the corresponding image segmentation method is adopted, and the compressed image is generated according to the grayscale average value of the segmented image, which can ensure that images of different sizes can be compressed and improve the reliability of image compression processing.

[0012] In one of the embodiments, the method of calculating the field of view of each pixel in the compressed image and dividing the compressed image into a central area and corner areas around the central area according to the field of view of each pixel includes: calculating the optical center of the compressed image and calculating the field of view of each pixel according to the distance from each pixel to the optical center; dividing the compressed image into a central area and corner areas around the central area according to the field of view of each pixel and a preset field of view threshold.

[0013] The field of view of each pixel is determined according to the distance from each pixel of the compressed image to the optical center, and the compressed image is divided into regions based on the field of view of the pixel, so that compensation values ​​can be calculated separately for different regions to compensate the original image.

[0014] In one of the embodiments, respectively calculating the shadow compensation values ​​of each field of view in the central area and the corner area includes: calculating the grayscale average value of each field of view of the compressed image; calculating the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area according to the grayscale average value of each field of view of the compressed image and the grayscale average value of the set field of view; extracting the shadow compensation value of each field of view located in the central area of ​​the compressed image as the shadow compensation value of each field of view in the central area; performing alpha bending processing on the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area to obtain the shadow compensation value of each field of view in the corner area.

[0015] Combined with the grayscale average value of each field of view in the compressed image, the shadow compensation value of each field of view and the initial shadow compensation value of each field of view in the corner area are determined respectively. Alpha bending processing is performed on the initial shadow compensation value of each field of view in the corner area and the shadow compensation value of each field of view in the compressed image to determine the shadow compensation value of each field of view in the corner area, so as to facilitate the subsequent calculation of the correction compensation value of the corresponding area by combining the shadow compensation values ​​of each field of view in the central area and the corner area.

[0016] In one of the embodiments, the correction compensation value of each pixel in the central area and the corner area is obtained according to the shadow compensation value of each field of view in the central area and the corner area, including: calculating the initial correction compensation value of each pixel in the central area according to the shadow compensation value of each field of view in the central area and the matrix data of the field of view where each pixel in the central area is located; obtaining the correction compensation value of each pixel in the central area according to the initial correction compensation value of each pixel in the central area and the center correction coefficient; wherein the center correction coefficient is obtained according to the field of view where the pixel in the central area is located; calculating the initial correction compensation value of each pixel in the corner area according to the shadow compensation value of each field of view in the corner area and the matrix data of the field of view where each pixel in the corner area is located; obtaining the correction compensation value of each pixel in the corner area according to the initial correction compensation value of each pixel in the corner area and the corner correction coefficient; wherein the corner correction coefficient is obtained according to the field of view where the pixel in the corner area is located.

[0017] Combined with the shadow compensation values ​​of each field of view in the center area and the corner area, and the matrix data of the field of view of each pixel in the corresponding area, the initial correction compensation values ​​of each pixel in the center area and the corner area are calculated respectively. Then, the correction compensation values ​​of each pixel in the center area and the corner area are calculated according to the initial correction compensation values ​​of each pixel in the center area and the corner area and the corresponding correction coefficients. Correcting the initial correction compensation values ​​with the correction coefficients can control the noise introduced after the shadow compensation, and improve the accuracy of the shadow compensation of the camera lens.

[0018] A camera lens shadow compensation device comprises: an image processing module, used for compressing and preprocessing an original image captured by a camera to obtain a compressed image; an area division module, used for calculating the field of view of each pixel in the compressed image, and dividing the compressed image into a central area and a corner area around the central area according to the field of view of each pixel; a first calculation module, used for respectively calculating the shadow compensation value of each field of view in the central area and the corner area; a second calculation module, used for obtaining the correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field of view in the central area and the corner area; an image compensation module, used for performing shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image.

[0019] The above-mentioned camera lens shadow compensation device processes the original image captured by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view and the correction compensation value of the pixels in each area for the central area and the corner area respectively, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0020] A camera lens shading compensation device comprises an image receiving interface and a processor, wherein the processor is connected to a camera via the image receiving interface, the image receiving interface receives an original image captured by the camera and sends the original image to the processor, and the processor performs lens shading compensation on the camera according to the above method.

[0021] The above-mentioned camera lens shadow compensation device processes the original image taken by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view for the central area and the corner area respectively, and the correction compensation value of the pixels in each area, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0022] A camera device comprises a camera and the camera lens shadow compensation device.

[0023] The above-mentioned camera device processes the original image taken by the camera to obtain a compressed image, and then divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, and calculates the shadow compensation value of each field of view and the correction compensation value of the pixels in each area for the central area and the corner area respectively, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a camera lens shadow compensation method in one embodiment;

[0025] Figure 2 is a flow chart of compressing a preprocessed image to obtain a compressed image in one embodiment;

[0026] Figure 3 A flowchart of calculating the field of view of each pixel in a compressed image and dividing the compressed image into a central area and corner areas around the central area according to the field of view of each pixel in an embodiment;

[0027] Figure 4 A flowchart for calculating shadow compensation values ​​of each field of view in a central area and a corner area respectively in an embodiment;

[0028] Figure 5 It is a flow chart of obtaining the correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field in the central area and the corner area in one embodiment;

[0029] Figure 6 A schematic diagram of extending the border of a portrait in one embodiment;

[0030] Figure 7 This is an example diagram of image boundary mirroring extension in one embodiment;

[0031] Figure 8An example diagram of performing ROI segmentation on an image in an embodiment;

[0032] Fig. 9 is an example diagram of grayscale values ​​of a compressed image in one embodiment;

[0033] Fig.10 An example diagram of calculating the field of view according to the distance from each pixel to the optical center in one embodiment;

[0034] Fig.11 is an example diagram of the grayscale average value of each field of view of a compressed image in one embodiment;

[0035] Fig.12 is an example diagram of shadow compensation values ​​of each field of view of a compressed image in an embodiment;

[0036] Fig.13 An example diagram of dividing a compressed image into regions in an embodiment;

[0037] Fig.14 An example diagram of calculating shadow compensation values ​​of each field of view in a corner area in an embodiment;

[0038] Fig.15 An example diagram of performing shadow correction compensation on an original image in one embodiment;

[0039] Fig.16 is a structural block diagram of a camera lens shadow compensation device in one embodiment;

[0040] Fig.17 4 is a structural block diagram of a camera lens shadow compensation device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, a camera lens shadow compensation method is provided, and the camera may be a telephoto camera, a thermal imaging camera, or an ultra-wide-angle camera. Figure 1 As shown, the method includes:

[0043] Step S110: performing compression preprocessing on the original image captured by the camera to obtain a compressed image.

[0044] The original image captured by the camera can be processed and compressed by the processor to facilitate subsequent area division and calculation of shadow compensation values. There is no unique way of image processing, which can include automatic white balance, bad pixel correction, etc.

[0045] In addition, before step S110, the method may further include a step of receiving an original image captured by a camera. The processor may be connected to the camera through an image receiving interface to receive the original image captured by the camera. Specifically, a uniform light panel imaging of a D50 light source (matching the color temperature of daylight 5000k) may be used to capture a portrait under a D50 light source environment with a camera, and then the processor receives the original image output from the sensor of the camera and processes the original image to compress the image.

[0046] Step S120: Calculate the field of view of each pixel in the compressed image, and divide the compressed image into a central area and corner areas around the central area according to the field of view of each pixel.

[0047] Specifically, the processor calculates the field of view of all pixels in the compressed image based on the data extracted from the compressed image, and then divides the compressed image into a central area and a corner area based on the field of view. The field of view of the pixel can be obtained by dividing the distance from the pixel to the OC (optical center) of the camera sensor by the half-diagonal distance of the image. The way of dividing the area is not unique. In this embodiment, the processor divides the area with a field of view less than or equal to 80 into the central area Center, and divides the area with a field of view greater than 80 into the corner area, and obtains the corner area Corner located at the four corners of the compressed image.

[0048] Step S130: Calculate the shadow compensation value of each field of view in the central area and the corner area respectively.

[0049] After dividing the area, the processor combines the grayscale data of the center area Center and the corner area Corner to calculate the shadow compensation value of each field of view in the corresponding area. Specifically, the grayscale average value of the same field of view in the calculation area can be combined with the grayscale average value of each field of view to calculate the shadow compensation value ShadingCenter of each field of view in the center area, and the shadow compensation value ShadingCorner of each field of view in the corner area.

[0050] Step S140: obtaining correction compensation values ​​of each pixel in the central area and the corner area according to the shadow compensation values ​​of each field of view in the central area and the corner area.

[0051] Specifically, the processor performs interpolation calculations based on the shadow compensation values ​​of each field of view in the center area Center and the corner area Corner, and separately calculates the correction compensation value Correct for each pixel in the center area Center and the corner area Corner. Among them, the correction compensation value CorrectionCenter of each pixel in the center area is calculated based on the shadow compensation value ShadingCenter of each field of view in the center area and the matrix data of the field of view where each pixel in the center area is located. The correction compensation value CorrectionCorner of each pixel in the corner area is calculated based on the shadow compensation value ShadingCorner of each field of view in the corner area and the matrix data of the field of view where each pixel in the corner area is located.

[0052] Step S150: performing shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image.

[0053] After calculating the correction compensation value CorrectionCenter of each pixel in the center area and the correction compensation value CorrectionCorner of each pixel in the corner area, the processor can use the correction compensation value Correct as the compensation value of each pixel, and then interpolate according to the size of the original image to obtain the compensated image. The processor can also set a coefficient for the correction compensation value Correct in combination with the field of view of the pixel, and then determine the final compensation value of each pixel as Correction*Rs^2*ACOS (CorrectionFactor) by combining the coefficient and the correction compensation value Correct, and then interpolate according to the size of the original image to obtain the compensated image.

[0054] The coefficient of the compensation value Correct is Rs^2.*ACOS (CorrectionFactor), Rs is the field of view of each pixel, and CorrectionFactor is a value ranging from 0 ～ The compensation factor is 1.

[0055] The above-mentioned camera lens shadow compensation method processes the original image taken by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view for the central area and the corner area respectively, and the correction compensation value of the pixels in each area, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0056] In one embodiment, step S110 specifically includes: performing image preprocessing on the original image captured by the camera to obtain a preprocessed image; and compressing the preprocessed image to obtain a compressed image.

[0057] The processor first performs image preprocessing on the original image. The specific method of image preprocessing is not unique. In one embodiment, the image preprocessing includes at least one of automatic white balance, bad pixel correction and black level correction. The corresponding method can be selected for image preprocessing according to actual needs, and the operation is convenient and reliable. Taking the image preprocessing including automatic white balance, bad pixel correction and black level correction as an example, the processor obtains the preprocessed image after performing automatic white balance, bad pixel correction and black level correction on the original image in sequence. Then the image is compressed to obtain a compressed image.

[0058] In this embodiment, the original image is preprocessed and compressed to obtain a compressed image, which facilitates subsequent processing and improves the convenience of image processing.

[0059] Further, in one embodiment, if Figure 2 As shown, compressing the preprocessed image to obtain a compressed image includes steps S114 to S118.

[0060] Step S114: Determine whether the pre-processed image can be evenly divided.

[0061] Specifically, the processor can determine whether the preprocessed image can be evenly divided according to the pre-saved segmentation size. For example, if the segmentation size can completely divide the preprocessed image without any remaining image part, it can be considered that the preprocessed image can be evenly divided, otherwise it cannot be.

[0062] Step S116: If the preprocessed image can be evenly segmented, ROI segmentation is performed on the preprocessed image to obtain a segmented image. If the image cannot be evenly segmented, the processor performs boundary expansion on the image and performs ROI segmentation on the boundary-expanded image to obtain a segmented image.

[0063] The processor detects whether the preprocessed image can be evenly segmented according to the segmentation size. If it can, the image is segmented by ROI to obtain a segmented image. The size of the image segmentation can be determined according to the resolution of the camera sensor and the expected amount of calculation. If the preprocessed image cannot be evenly segmented, the preprocessed image is extended at the boundary, and the image after boundary extension is segmented by ROI to obtain a segmented image. In machine vision and image processing, the area to be processed is outlined from the processed image in the form of a box, circle, ellipse, irregular polygon, etc., which is called the region of interest (ROI). Various operators and functions are often used in machine vision software to obtain the region of interest ROI and perform the next step of image processing. The region of interest ROI is an image area selected from the image. This area is the focus of image analysis, and the area is circled for further processing. Using ROI to circle the target area can reduce processing time and increase accuracy.

[0064] Step S118: Obtaining a compressed image based on the grayscale average value of each segmented image. After segmenting the image, the processor obtains the grayscale average value of each segmented image to obtain a compressed image for the next step of calculation.

[0065] In this embodiment, a corresponding image segmentation method is adopted according to the actual situation of the preprocessed image, and a compressed image is generated according to the grayscale average value of the segmented image, which can ensure that images of different sizes can be compressed, thereby improving the reliability of image compression processing.

[0066] In one embodiment, Figure 3 As shown, step S120 includes step S122 and step S124.

[0067] Step S122: Calculate the optical center of the compressed image, and calculate the field of view of each pixel based on the distance from each pixel to the optical center. The processor can calculate the optical center (Optical Center, OC) of the compressed image by using a method of finding the centroid of a binary image, and then calculate the field of view of each pixel based on the distance from each pixel to the optical center.

[0068] Step S124: Divide the compressed image into a central area and corner areas around the central area according to the field of view of each pixel and the preset field of view threshold. The specific value of the field of view threshold is not unique. In this embodiment, the field of view threshold is 80. The processor defines the area in the compressed image where the field of view of the pixels is less than or equal to 80 as the central area, and defines the four corners where the field of view of the pixels is greater than 80 as the corner areas. It can be understood that the setting of the field of view threshold of 80 can be appropriately modified according to the specific situation.

[0069] In this embodiment, the field of view is determined according to the distance from each pixel of the compressed image to the optical center, and the compressed image is divided into regions based on the field of view where the pixel is located, so that compensation values ​​can be calculated separately for different regions to compensate the original image.

[0070] In one embodiment, Figure 4 As shown, step S130 includes steps S132 to S138.

[0071] Step S132: Calculate the grayscale average value of each field of view of the compressed image. Specifically, the processor averages the grayscale values ​​of the same field of view in the compressed image to obtain the grayscale average value of the 0-100 field of view. For example, if the total number of pixels in the field of view M is N, then the grayscale average value of the field of view M is:

[0072]

[0073] Among them, AVG (M) is the grayscale average value of the field of view M, and Grayscale represents the grayscale value of each pixel in the field of view M.

[0074] Step S134: Calculate the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area according to the grayscale average value of each field of view of the compressed image and the grayscale average value of the set field of view.

[0075] The specific selection of setting the field of view is not unique. In this embodiment, field of view 1 is selected as the setting field of view. Then, the processor divides the grayscale average value of field of view 1 by the grayscale average value of each field of view to obtain the shadow compensation value of each field of view of the compressed image.

[0076] ShadingAll(M)=AVG(1) / AVG(M)

[0077] Wherein, ShadingAll(M) is the shading compensation value of field of view M, AVG(1) is the grayscale average value of field of view 1, and AVG(M) is the grayscale average value of field of view M.

[0078] In addition, the processor also calculates the initial shadow compensation value shadingCorner' of each field of view in the four corner areas separately according to the above method. That is, take the pixels with a field of view greater than 80 in the four corner areas respectively, calculate the shadow compensation value of 80 to 100 according to the above method, and obtain the initial shadow compensation value ShadingCorner'.

[0079] Step S136: extracting the shadow compensation value of each field of view located in the center area of ​​the compressed image as the shadow compensation value of each field of view in the center area. The processor extracts the shadow compensation value of the field of view located in the center area from the shadow compensation value calculated in step S134, and obtains the shadow compensation value of each field of view in the center area, that is, the shadow compensation value of each field of view in the center area ShadingCenter=ShadingAll(0:80).

[0080] Step S138: performing alpha bending processing on the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area to obtain the shadow compensation value of each field of view in the corner area.

[0081] Among them, alpha bending is an image processing technology that mixes source pixels and target pixels according to the value of the "Alpha" mixing vector. After obtaining the shadow compensation value ShadingAll of each field of view of the compressed image and the initial shadow compensation value ShadingCorner' of each field of view in the corner area, the processor takes the 80-90 field of view as the transition area, performs alpha bending on the shadow compensation value ShadingAll of each field of view of the compressed image and the initial shadow compensation value shadingCorner' of each field of view in the corner area, and obtains the shadow compensation value ShadingCorner of each field of view in the corner area.

[0082] Specifically, in this embodiment, the alpha bending processing method is performed as follows:

[0083] WeightA=[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]

[0084] WeightB=[0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1]

[0085] ShadingCorner=[shadingAll(81:89)*WeightB+shadingCorner'(81:89).*WeightA,shadingCorner'(90:100)].

[0086] In this embodiment, the shadow compensation value of each field of view and the initial shadow compensation value of each field of view in the corner area are determined in combination with the grayscale average value of each field of view of the compressed image. Alpha bending processing is performed in combination with the initial shadow compensation value of each field of view in the corner area and the shadow compensation value of each field of view in the compressed image to determine the shadow compensation value of each field of view in the corner area, so as to facilitate the subsequent calculation of the correction compensation value of the corresponding area in combination with the shadow compensation values ​​of each field of view in the central area and the corner area.

[0087] In one embodiment, Figure 5 As shown, step S140 includes steps S142 to S148.

[0088] Step S142: according to the shadow compensation value of each field of view in the central area and the matrix data of the field of view where each pixel in the central area is located, the initial correction compensation value of each pixel in the central area is calculated respectively.

[0089] Specifically, the processor calculates the initial correction compensation value of each pixel in the central area including:

[0090] CorrectionCenter=interp1(0:80,ShadingCenter,RsCenter)

[0091] Among them, CorrectionCenter represents the initial correction compensation value of each pixel in the central area, ShadingCenter represents the shadow compensation value of each field in the central area, and RsCenter is the matrix data of the field of view of each pixel in the central area (field of view 0~80).

[0092] Step S144: Obtain the correction compensation value of each pixel in the central area according to the initial correction compensation value of each pixel in the central area and the central correction coefficient. The central correction coefficient is obtained according to the field of view of the pixel in the central area. The correction compensation value of each pixel in the central area is specifically:

[0093] CorrectionCenter*Rs^2*ACOS(CorrectionFactor)

[0094] Among them, CorrectionCenter is the initial correction compensation value of each pixel in the central area, Rs^2.*ACOS (CorrectionFactor) is the center correction coefficient, and Rs is the field of view of each pixel (the percentage of the distance from the center of the sensor to the semi-diagonal). CorrectionFactor is a value ranging from 0 ～ A compensation coefficient of 1 can be used to control the noise introduced after shadow compensation by adjusting the compensation strength appropriately according to the distance of each pixel from the center.

[0095] Step S146: according to the shadow compensation value of each field of view in the corner area and the matrix data of the field of view where each pixel in the corner area is located, the initial correction compensation value of each pixel in the corner area is calculated respectively.

[0096] Specifically, the processor calculates the initial correction compensation value of each pixel in the corner area including:

[0097] CorrectionCorner=interp1(80:100,ShadingCorner,RsCorner)

[0098] Among them, CorrectionCorner represents the initial correction compensation value of each pixel in the corner area, ShadingCorner represents the shadow compensation value of each field in the corner area, and RsCorner is the matrix data of the field of view where each pixel in the corner area (field of view 80~100) is located.

[0099] Step S148: Obtain the correction compensation value of each pixel in the corner area according to the initial correction compensation value of each pixel in the corner area and the corner correction coefficient. The corner correction coefficient is obtained according to the field of view of the pixel in the corner area. The correction compensation value of each pixel in the corner area is specifically:

[0100] CorrectionCorner*Rs^2*ACOS(CorrectionFactor)

[0101] Among them, CorrectionCorner is the initial correction compensation value of each pixel in the corner area, Rs^2.*ACOS (CorrectionFactor) is the corner correction coefficient, Rs is the field of view of the sensor where each pixel is located (the percentage of the distance from the center of the sensor to the semi-diagonal line). CorrectionFactor is a compensation coefficient with a value range of 0 to 1.

[0102] In this embodiment, the shadow compensation values ​​of each field of view in the central area and the corner area are combined with the matrix data of the field of view of each pixel in the corresponding area to calculate the initial correction compensation value of each pixel in the central area and the corner area. Then, the correction compensation value of each pixel in the central area and the corner area is calculated according to the initial correction compensation value of each pixel in the central area and the corner area and the corresponding correction coefficient. By correcting the initial correction compensation value with the correction coefficient, the noise introduced after the shadow compensation can be controlled, and the accuracy of the shadow compensation of the camera lens can be improved.

[0103] In order to facilitate a better understanding of the above-mentioned camera lens shadow compensation method, it is explained in detail below in conjunction with a specific embodiment.

[0104] For the ultra-wide-angle imaging characteristics, the traditional lens shading compensation method cannot perfectly compensate the image. Based on this, this application provides a new lens shading compensation algorithm to more accurately compensate for the lens shading of the ultra-wide-angle camera. For ease of understanding, the following names are simplified.

[0105] The above camera lens shading compensation method uses a uniform light plate of a D50 light source to form an image, and then calculates the LSC (Lens Shading Correction) coefficient of the imaging module from the raw image output by the camera sensor. Then proceed as follows:

[0106] 1. Image preprocessing (RGGB (Red Green Green Blue) module includes AWB (automatic white balance) / DPC (dead pixel correction) / Black level correction). 2. Data compression of the original image. 3. Calculate the OC of the compressed image. 3. Calculate the distance from each pixel to the OC (unit: pixel), and then divide it by the length of the half diagonal of the image to get the field of view of the pixel. 4. Define the four corners as the area outside the 0.8 field of view. 5. For 0 ～ 0.8 center area and 0.8 ～ 1 Calculate the shadow compensation values ​​for the four corners and add 0.8 ～ 0.9 area compensation value for alphablending.

[0107] Step 1: Compression preprocessing of the original image.

[0108] Preprocess the original image, perform ROI segmentation on the preprocessed image (the size is determined by the resolution of the module sensor and the expected amount of calculation). If the preprocessed image cannot be evenly segmented, the image boundary needs to be expanded. Then take the grayscale average of each segmented image to obtain the compressed image for the next step of calculation. Figure 6 This is an example of compressing a 4224×3024 pixel original image. The image is filled by border mirroring and then each 7×7 ROI is averaged and compressed to obtain a compressed image (raw: 704×504). Figure 7 This is an example of border mirroring extension. The solid-line frame is the original image, and the dotted-line frame is the extended image. Figure 8 This is an example of selecting the data of the upper left corner of the image for ROI (7×7).

[0109] Step 2: Calculate the field of view and pixel grayscale.

[0110] Calculate the optical center of the compressed image, and then calculate the field of view based on the distance from each pixel to the optical center. The optical center can be calculated by the method of finding the centroid of the binary image:

[0111] imbw=im>single(0.5*(max(im(:))+min(im(:)))); (im is the image data compressed in the previous step)

[0112] [X,Y]=meshgrid(1:size(im,2),1:size(im,1));

[0113] Xcs=mean([Xs(imbws),Ys(imbws)],1);)

[0114] Fig. 9 An example of grayscale values ​​of a compressed image (raw: 704×504). Fig.10 This is an example of calculating the field of view based on the distance from each pixel to the optical center (raw: 704×504). The field of view value can be rounded to reduce the amount of calculation.

[0115] Step 3: Calculate the shading compensation value ShadingAll for the field of view 0 to 100.

[0116] The grayscale value of the same field of view is averaged to obtain the grayscale average value of the field of view 0-100. For example, if the total number of pixels in the field of view M is N, then the grayscale average value of the field of view M is

[0117]

[0118] Fig.11 This is an example of the average grayscale value for each field of view. The horizontal axis represents the field of view, and the vertical axis represents the average grayscale value.

[0119] The shading compensation value ShadingAll is obtained by dividing the average gray value of field 1 by the gray value of each field.

[0120] ShadingAll(M)=AVG(1) / AVG(M)

[0121] Fig.12 Examples of shadow compensation values ​​for each field of view. The horizontal axis represents the field of view, and the vertical axis represents the shadow compensation value.

[0122] Step 4: Define the center area Center and four corner areas Corner in the image.

[0123] like Fig.13 In the image shown, the four corners with a field of view > 80 are defined as four corner areas Corner (the white part in the figure below). The setting of the field of view 80 can be appropriately modified according to the specific situation.

[0124] Step 5: Calculate the shadow compensation value of the corner area Corner.

[0125] Follow the method of calculating shadingAll in step 3 to calculate the four corner areas Corner (field of view 80 ～ 100) of ShadingCorner'. That is, take the pixels in the four corners with a field of view greater than 80, and calculate 80 according to step 3. ～ A shadow compensation value of 100, ie ShadingCorner'.

[0126] like Fig.14 As shown, the field of view is 80 ～ 90 is the transition area, shadingCorner is obtained by performing alpha bending on shadingAll and shadingCorner'. At the same time, ShadingCenter = ShadingAll (0:80) is defined.

[0127] The Alpha bending method is performed as follows:

[0128] WeightA=[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]

[0129] WeightB=[0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1]

[0130] ShadingCorner=[shadingAll(81:89)*WeightB+shadingCorner'(81:89).*WeightA,shadingCorner'(90:100)]

[0131] Step 6: Correction execution

[0132] The Correction value of each pixel is calculated based on the field of view of each pixel of the compressed image (704×504) according to the interpolation of shadingAll and ShadingCorner (that is, the Center and Corner are calculated separately).

[0133] CorrectionCenter=interp1(0:80,shadingCenter,RsCenter)

[0134] CorrectionCorner=interp1(80:100,shadingCorner,RsCorner);

[0135] Among them, Rscenter is Center(0 ～ 80) The matrix data of the field of view where each pixel is located, RsCorner is Corner (80 ～ 100) Matrix data of the field of view where each pixel is located.

[0136] Here you can set a value range of 0 ～ The compensation coefficient CorrectionFactor is 1, and the compensation strength is appropriately adjusted according to the distance of each pixel from the center, so that the noise introduced after shadow compensation can be controlled.

[0137] The final correction coefficient is defined as Rs^2.*ACOS (correctionFactor). Rs is the field of view of the sensor where each pixel is located (the percentage of the distance from the center of the sensor to the semi-diagonal). The compensation value of each pixel of the final original image (4224×3024) is:

[0138] Correction*Rs^2*ACOS(correctionFactor)

[0139] Then the compensated image is obtained by interpolation according to the size of the original image, such as Fig.15 shown.

[0140] In one embodiment, a camera lens shadow compensation device is also provided, and the camera may be a telephoto camera, a thermal imaging camera, or an ultra-wide-angle camera. Fig.16 As shown, the device includes an image processing module 110 , a region dividing module 120 , a first calculation module 130 , a second calculation module 140 and an image compensation module 150 .

[0141] The image processing module 110 is used to perform compression preprocessing on the original image captured by the camera to obtain a compressed image.

[0142] The region division module 120 is used to calculate the field of view of each pixel in the compressed image, and divide the compressed image into a central region and corner regions around the central region according to the field of view of each pixel.

[0143] The first calculation module 130 is used to calculate the shadow compensation value of each field of view in the central area and the corner area respectively.

[0144] The second calculation module 140 is used to obtain the correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field of view in the central area and the corner area.

[0145] The image compensation module 150 is used to perform shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image.

[0146] In one embodiment, the image processing module 110 performs image preprocessing on the original image captured by the camera to obtain a preprocessed image, and compresses the preprocessed image to obtain a compressed image. In one embodiment, the image preprocessing includes at least one of automatic white balance, bad pixel correction, and black level correction.

[0147] Furthermore, in one embodiment, the image processing module 110 determines whether the preprocessed image can be evenly segmented; if the preprocessed image can be evenly segmented, the image processing module 110 performs ROI segmentation on the preprocessed image to obtain a segmented image. If the preprocessed image cannot be evenly segmented, the image processing module 110 performs boundary extension on the preprocessed image, performs ROI segmentation on the image after boundary extension, and obtains a segmented image. The image processing module 110 also obtains a compressed image based on the grayscale average value of each segmented image.

[0148] In one embodiment, the region division module 120 calculates the optical center of the compressed image, and calculates the field of view of each pixel according to the distance from each pixel to the optical center. According to the field of view of each pixel and a preset field of view threshold, the compressed image is divided into a central region and corner regions around the central region.

[0149] In one embodiment, the first calculation module 130 calculates the grayscale average of each field of view of the compressed image. According to the grayscale average of each field of view of the compressed image and the grayscale average of the set field of view, the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area are calculated. The shadow compensation value of each field of view located in the center area of ​​the compressed image is extracted as the shadow compensation value of each field of view in the center area. The shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area are subjected to alpha bending processing to obtain the shadow compensation value of each field of view in the corner area.

[0150] In one embodiment, the second calculation module 140 calculates the initial correction compensation value of each pixel in the central area according to the shadow compensation value of each field of view in the central area and the matrix data of the field of view where each pixel in the central area is located. The correction compensation value of each pixel in the central area is obtained according to the initial correction compensation value of each pixel in the central area and the center correction coefficient. The initial correction compensation value of each pixel in the corner area is calculated according to the shadow compensation value of each field of view in the corner area and the matrix data of the field of view where each pixel in the corner area is located. The correction compensation value of each pixel in the corner area is obtained according to the initial correction compensation value of each pixel in the corner area and the corner correction coefficient.

[0151] For the specific definition of the camera lens shading compensation device, please refer to the definition of the camera lens shading compensation method above, which will not be repeated here. Each module in the above-mentioned camera lens shading compensation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0152] The above-mentioned camera lens shadow compensation device processes the original image captured by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view and the correction compensation value of the pixels in each area for the central area and the corner area respectively, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0153] In one embodiment, a camera lens shading compensation device is also provided. Fig.17 As shown, it includes an image receiving interface 210 and a processor 220. The processor 220 is connected to the camera through the image receiving interface 210. The image receiving interface 210 receives the original image taken by the camera and sends it to the processor 220. The processor 220 performs lens shading compensation on the camera according to the above method.

[0154] The above-mentioned camera lens shadow compensation device processes the original image taken by the camera to obtain a compressed image, divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, calculates the shadow compensation value of each field of view for the central area and the corner area respectively, and the correction compensation value of the pixels in each area, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0155] In one embodiment, a camera device is also provided, including a camera and the above-mentioned camera lens shading compensation device. The camera can be a telephoto camera, a thermal imaging camera, or an ultra-wide-angle camera.

[0156] The above-mentioned camera device processes the original image taken by the camera to obtain a compressed image, and then divides the image into a central area and a corner area according to the field of view of each pixel of the compressed image, and calculates the shadow compensation value of each field of view and the correction compensation value of the pixels in each area for the central area and the corner area respectively, and compensates the pixels in different areas of the original image separately according to the correction compensation values ​​of the pixels in different areas. Accurate portrait compensation can also be achieved for ultra-wide-angle cameras, thereby improving the accuracy of portrait compensation.

[0157] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A camera lens shadow compensation method, characterized in that: include: Performing compression preprocessing on the original image captured by the camera to obtain a compressed image; Calculating the field of view of each pixel in the compressed image, and dividing the compressed image into a central area and corner areas around the central area according to the field of view of each pixel; Calculating shadow compensation values ​​of each field of view in the central area and the corner area respectively; Obtaining correction compensation values ​​for each pixel in the central area and the corner area according to the shadow compensation values ​​of each field of view in the central area and the corner area; Performing shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image; The step of calculating the field of view of each pixel in the compressed image and dividing the compressed image into a central area and a corner area around the central area according to the field of view of each pixel includes: Calculate the optical center of the compressed image, and calculate the field of view of each pixel according to the distance from each pixel to the optical center; divide the compressed image into a central area and corner areas around the central area according to the field of view of each pixel and a preset field of view threshold; The calculating the shadow compensation value of each field of view in the central area and the corner area respectively includes: The grayscale average value of each field of view of the compressed image is calculated; according to the grayscale average value of each field of view of the compressed image and the grayscale average value of the set field of view, the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area are calculated; the shadow compensation value of each field of view located in the central area of ​​the compressed image is extracted as the shadow compensation value of each field of view in the central area; the shadow compensation value of each field of view of the compressed image and the initial shadow compensation value of each field of view in the corner area are subjected to alpha bending processing to obtain the shadow compensation value of each field of view in the corner area.

2. The camera lens shadow compensation method according to claim 1, characterized in that: The method of performing compression preprocessing on the original image captured by the camera to obtain a compressed image includes: Performing image preprocessing on the original image captured by the camera to obtain a preprocessed image; The preprocessed image is compressed to obtain a compressed image.

3. The camera lens shadow compensation method according to claim 2, characterized in that: The image preprocessing includes at least one of automatic white balance, bad pixel correction and black level correction.

4. The camera lens shadow compensation method according to claim 2, characterized in that: The step of compressing the preprocessed image to obtain a compressed image comprises: Determine whether the preprocessed image can be evenly segmented; If the preprocessed image can be evenly segmented, ROI segmentation is performed on the preprocessed image to obtain a segmented image. If the preprocessed image cannot be evenly segmented, the preprocessed image is boundary extended, and ROI segmentation is performed on the boundary extended image to obtain a segmented image. The compressed image is obtained based on the grayscale average value of each segmented image.

5. The camera lens shadow compensation method according to claim 1, characterized in that: Obtaining correction compensation values ​​of each pixel in the central area and the corner area according to the shadow compensation values ​​of each field of view in the central area and the corner area, including: According to the shadow compensation value of each field of view in the central area and the matrix data of the field of view where each pixel in the central area is located, respectively calculating the initial correction compensation value of each pixel in the central area; Obtaining a correction compensation value for each pixel in the central area according to an initial correction compensation value for each pixel in the central area and a central correction coefficient; wherein the central correction coefficient is obtained according to a field of view in which the pixel in the central area is located; According to the shadow compensation value of each field of view in the corner area and the matrix data of the field of view where each pixel in the corner area is located, respectively calculating the initial correction compensation value of each pixel in the corner area; The correction compensation value of each pixel in the corner area is obtained according to the initial correction compensation value of each pixel in the corner area and the corner correction coefficient; wherein the corner correction coefficient is obtained according to the field of view where the pixel in the corner area is located.

6. A camera lens shadow compensation device, characterized in that: include: The image processing module is used to compress and preprocess the original image captured by the camera to obtain a compressed image; A region division module, used for calculating the field of view of each pixel in the compressed image, and dividing the compressed image into a central region and corner regions around the central region according to the field of view of each pixel; A first calculation module, used to calculate the shadow compensation value of each field of view in the central area and the corner area respectively; A second calculation module is used to obtain a correction compensation value of each pixel in the central area and the corner area according to the shadow compensation value of each field of view in the central area and the corner area; The image compensation module is used to perform shadow correction compensation on the original image according to the correction compensation value of each pixel in the central area and the corner area to obtain a compensated image.

7. A camera lens shadow compensation device, characterized in that: It includes an image receiving interface and a processor, the processor is connected to a camera via the image receiving interface, the image receiving interface receives the original image taken by the camera and sends it to the processor, and the processor performs lens shading compensation on the camera according to the method described in any one of claims 1-5.

8. A camera device, characterized in that: It comprises a camera and a camera lens shading compensation device as claimed in claim 7.

Images