Image shadow compensation methods, apparatus, devices and storage media

Abstract

This application discloses an image shadow compensation method applied to an image shadow compensation device. The image shadow compensation device is equipped with a first camera and a second camera. When switching from the first camera to the second camera to capture a target object, a grid position mapping relationship is determined between a first image and a second image of the target object. The first image is captured by the first camera, and the second image is captured by the second camera. Based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined. Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined. The second image is then subjected to shadow compensation processing based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image shadow compensation method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous advancement and development of terminal technology, terminal devices can be equipped with multiple cameras, such as wide-angle cameras, main cameras, and telephoto cameras, and can switch between different zoom levels using these multiple cameras. The field of view (FOV) varies between these different cameras.

[0003] In multi-camera switching shooting scenarios, the current Lens Shading Correction (LSC) solution cannot effectively respond to the FOV differences between different cameras, resulting in brightness jumps during zoom switching and reducing image quality. Summary of the Invention

[0004] This application provides an image shadow compensation method, apparatus, device, and storage medium, which can improve the accurate recognition of dynamic photos, expand the application scenarios of dynamic photos, and enhance the intelligence of electronic devices.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide an image shadow compensation method, applied to an image shadow compensation device, the image shadow compensation device being configured with a first camera and a second camera, the method comprising:

[0007] When switching from the first camera to the second camera to capture the target object, determine the grid position mapping relationship between the first image and the second image of the target object; wherein the first image is captured by the first camera and the second image is captured by the second camera.

[0008] Based on the grid position mapping relationship and the preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image;

[0009] Based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera, determine the lens shadow compensation parameters corresponding to the second image.

[0010] The second image is processed by performing shadow compensation based on the lens shadow compensation parameters corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0011] Secondly, embodiments of this application provide an image shadow compensation device, which includes:

[0012] The determining unit is used to determine the grid position mapping relationship between a first image and a second image of the target object when switching from a first camera to a second camera to capture the target object; wherein the first image is acquired by the first camera and the second image is acquired by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, the unit determines the compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, the unit determines the lens shadow compensation parameter corresponding to the second image.

[0013] The compensation unit is used to perform shadow compensation processing on the second image based on the lens shadow compensation parameters corresponding to the second image, so as to obtain the processed image of the target object corresponding to the second camera.

[0014] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory storing processor-executable instructions, wherein when the instructions are executed by the processor, the method as described in the first aspect is implemented.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0016] This application provides an image shadow compensation method, apparatus, device, and storage medium. When switching from a first camera to a second camera to capture a target object, a grid position mapping relationship is determined between a first image and a second image of the target object. The first image is captured by the first camera, and the second image is captured by the second camera. Based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined. Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined. The second image is then subjected to shadow compensation processing based on the lens shadow compensation parameter to obtain the processed image of the target object corresponding to the second camera. In other words, in this application, during zoom switching, the relative positional relationship between different images captured by different cameras can be determined first, i.e., the grid position mapping relationship can be determined. Then, based on this grid position mapping relationship, the corresponding compensation intensity adjustment parameter is determined, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter to finally obtain the corresponding lens shadow compensation parameter for lens shadow compensation of the image. This achieves synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. As can be seen, this application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve image quality. Attached Figure Description

[0017] Figure 1 A schematic diagram illustrating the differences in field of view between different cameras;

[0018] Figure 2 This is a schematic diagram illustrating the implementation process of the image shadow compensation method proposed in the embodiments of this application;

[0019] Figure 3 This is a schematic diagram illustrating the FOV difference between the first image and the second image proposed in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram illustrating the grid position mapping relationship between the first image and the second image as proposed in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the mapped grid position proposed in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram illustrating the determination of brightness ratio parameters according to an embodiment of this application;

[0023] Figure 7 This is a schematic diagram illustrating the determination of brightness ratio parameters according to an embodiment of this application;

[0024] Figure 8 This is a schematic diagram illustrating the implementation process of the image shadow compensation method proposed in the embodiments of this application;

[0025] Figure 9 This is a schematic diagram of the composition structure of the image shadow compensation device proposed in the embodiments of this application;

[0026] Figure 10 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the differences between the applications and are not intended to limit the application. Furthermore, it should be noted that, for ease of description, only the parts that differ from the relevant applications are shown in the accompanying drawings.

[0028] In the field of imaging products, due to the light-focusing effect of convex lenses, the light intensity in the center of the camera module is greater than that at the edges, resulting in vignetting. Ultimately, this will result in an image that is bright in the center and dark around the edges.

[0029] Lens shading compensation (LSC), also known as vignetting correction or shadow compensation, is a post-processing technique used by camera lenses through built-in software algorithms to eliminate or reduce the darkening of image edges caused by the optical characteristics of the lens.

[0030] With shadow compensation enabled, the camera can intelligently identify and adjust the brightness of image edges, significantly improving edge darkening, resulting in a more uniform overall brightness distribution and clearer details. This helps enhance the overall visual appeal and information delivery efficiency of the image, thereby improving image quality.

[0031] With the continuous advancement and development of terminal technology, terminal devices can be equipped with multiple cameras, such as wide-angle cameras, main cameras, and telephoto cameras, and can switch between different zoom levels using these multiple cameras. The field of view (FOV) varies between different cameras.

[0032] Figure 1 This is a diagram illustrating the differences in field of view between different cameras, such as... Figure 1 As shown, there are significant differences in the field of view (FOV) among wide-angle cameras, main cameras, and telephoto cameras. The FOV of the wide-angle camera is much larger than that of the telephoto camera.

[0033] Currently, there are three main modules in the Image Signal Processor (ISP) pipeline that affect the brightness consistency of multiple cameras: the Auto Exposure (AE) module, the Tone Mapping module, and the LSC lens shading correction module. Current industry solutions for multi-camera brightness consistency primarily focus on two dimensions: multi-camera AE synchronization algorithms and multi-camera brightness synchronization algorithms (tone synchronization).

[0034] However, due to the differences in FOV between different cameras, the LSC compensation between different cameras also varies accordingly. This leads to differences in brightness at the same object in the multi-camera images when switching between multiple cameras, and brightness jumps when zooming, which reduces image quality.

[0035] To address the aforementioned issues, this application provides an image shadow compensation method, apparatus, device, and storage medium. When switching from a first camera to a second camera to capture a target object, a grid position mapping relationship is determined between a first image and a second image of the target object. The first image is captured by the first camera, and the second image is captured by the second camera. Based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined. Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined. The second image is then subjected to shadow compensation processing based on the lens shadow compensation parameter, resulting in a processed image of the target object corresponding to the second camera. In other words, in this application, during zoom switching, the relative positional relationship between different images captured by different cameras can be determined first, i.e., the grid position mapping relationship can be determined. Then, based on this grid position mapping relationship, the corresponding compensation intensity adjustment parameter is determined, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter, ultimately obtaining the corresponding lens shadow compensation parameter to perform lens shadow compensation on the image. This achieves synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. As can be seen, this application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve image quality.

[0036] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0037] One embodiment of this application provides an image shadow compensation method, which can be applied to an image shadow compensation device or electronic device, and can also be applied to any terminal that includes an image shadow compensation device or electronic device.

[0038] It is understood that the image shadow compensation method proposed in the embodiments of this application mainly includes a method for lens shading compensation (LSC) of images.

[0039] Furthermore, in the embodiments of this application, the image shadow compensation method proposed in the embodiments of this application can be applied to scenarios where multiple cameras switch shooting. That is, the image shadow compensation method of this application can be used to perform LSC synchronous compensation for different cameras when multiple cameras switch zoom, which can ensure the brightness consistency of the same FOV area of ​​the images of different cameras during zoom switching.

[0040] The image shadow compensation method proposed in this application will be described below using an image shadow compensation device as an example.

[0041] Furthermore, in the embodiments of this application, Figure 2 This is a schematic diagram illustrating the implementation process of the image shadow compensation method proposed in the embodiments of this application, as follows: Figure 2 As shown, the image shadow compensation method may include the following steps:

[0042] Step 101: When switching from the first camera to the second camera to capture the target object, determine the grid position mapping relationship between the first image and the second image of the target object; wherein the first image is captured by the first camera and the second image is captured by the second camera.

[0043] In the embodiments of this application, when switching from the first camera to the second camera to capture the target object, the image shadow compensation device can first determine the grid position mapping relationship between the first image of the target object and the second image of the target object; wherein, the first image is captured by the first camera and the second image is captured by the second camera.

[0044] In the embodiments of this application, the image shadow compensation device can be configured with multiple different cameras, wherein the number of cameras configured in the image shadow compensation device can be greater than or equal to two. That is to say, in this application, the image shadow compensation device can be a multi-camera device.

[0045] It is understood that, in the embodiments of this application, the various cameras in the image shadow compensation device are typically designed with different functions and characteristics to meet the shooting needs in different scenarios. This application does not specifically limit the performance and type of the different cameras in the image shadow compensation device.

[0046] For example, in some embodiments, the multiple cameras configured in the image shadow compensation device may include, but are not limited to, at least two of the following: a main camera, a wide-angle camera, a telephoto camera, a macro camera, a depth-sensing camera, etc.

[0047] The main camera is the most basic camera, typically boasting high resolution and good image quality. It handles most everyday shooting tasks, such as landscapes and portraits. The main camera has relatively high sensor size, aperture, and pixel count to ensure clear and detailed photos in various lighting conditions.

[0048] Wide-angle cameras offer a wider field of view than main cameras, capturing more of the scene and making them ideal for shooting landscapes, architecture, and other similar subjects. They deliver a stronger visual impact, showcasing a broader perspective. Furthermore, wide-angle cameras excel at shooting in confined spaces, giving photos a greater sense of space and depth.

[0049] Telephoto cameras are typically used for optical zoom, allowing you to bring distant objects closer for shooting, making them suitable for landscapes, portrait close-ups, and more. Telephoto cameras have a shallow depth of field, which helps to highlight the subject and blur the background, creating a more professional shooting effect. Some high-end phones also support high-magnification optical zoom in their telephoto cameras, enabling them to capture images from even greater distances while maintaining sharpness.

[0050] Macro cameras support close-up focusing, allowing you to capture amazing details of the microscopic world, such as flowers and insects. Macro cameras typically have high magnification and good focusing performance, enabling them to present a delicate and clear microscopic world.

[0051] Depth-field cameras are primarily used to enhance the bokeh effect in photos, making the subject stand out more and the background more blurred. Through algorithmic processing, depth-field cameras can achieve more natural and softer bokeh effects, enhancing the artistic appeal of photos.

[0052] Of course, in addition to the common camera types mentioned above, the image shadow compensation device can also be equipped with some special cameras, such as Time-of-Flight (ToF) lenses and cinema lenses. ToF lenses are mainly used for 3D perception and depth measurement, and can be used for augmented reality, facial recognition, and other functions. Cinema lenses typically have high pixel counts and excellent color reproduction capabilities, making them suitable for shooting high-quality video. Furthermore, in the embodiments of this application, the image shadow compensation device can be configured with a first camera and a second camera, wherein the first camera and the second camera are different. For example, the image shadow compensation device is configured with a main camera, a wide-angle camera, and a telephoto camera, with the first camera being the main camera, and the second camera being a wide-angle camera and / or a telephoto camera.

[0053] In the embodiments of this application, the image shadow compensation device can acquire images of the target object through a first camera and a second camera, respectively obtaining a first image and a second image under the target object.

[0054] In other words, in the embodiments of this application, the image shadow compensation device can acquire a first image corresponding to the target object through a first camera, and can also acquire a second image corresponding to the target object through a second camera.

[0055] It is understood that, in the embodiments of this application, the image shadow compensation device can switch zoom between different cameras. Specifically, during the process of taking a picture of the target object, the zoom switching time from the first camera to the second camera is negligible; that is, the first image and the second image can be considered to be acquired synchronously through the first and second cameras.

[0056] Furthermore, in the embodiments of this application, after acquiring the first image and the second image corresponding to the target object through the first camera and the second camera respectively, the grid position mapping relationship between the first image and the second image can be further determined.

[0057] The grid position mapping relationship between the first image and the second image can be used to determine the correspondence between the grid positions of pixels in the first image and the second image. Specifically, the grid position mapping relationship can be the correspondence between the grid positions of pixels in the grid image of the first image and the grid positions of pixels in the grid image of the second image.

[0058] It is understood that, in the embodiments of this application, the first image captured by the first camera can be divided according to a preset grid radius parameter to obtain a grid image of the first image. Similarly, the second image captured by the second camera can also be divided according to the preset grid radius parameter to obtain a grid image of the second image.

[0059] In embodiments of this application, preset mesh parameters can be used to determine the size of the mesh. These preset mesh parameters may include the horizontal radius parameter and the vertical radius parameter of the mesh, or they can be understood as the radius parameter of the mesh length and the radius parameter of the mesh width.

[0060] For example, in some embodiments, it is assumed that the preset grid radius parameters are the horizontal radius parameter M and the vertical radius parameter N, that is, the preset grid parameters can be represented as M×N, and the size of each grid obtained after dividing according to the preset grid radius parameters is M×N. Wherein, both M and N are greater than 0, and M and N can be the same or different. This application does not specifically limit the values ​​of M and N.

[0061] For example, in some embodiments, it is assumed that the preset grid radius parameters are a horizontal radius parameter of 64 and a vertical radius parameter of 48, and the size of each grid obtained after dividing according to the preset grid radius parameters is 64×48.

[0062] For example, in some embodiments, it is assumed that the preset grid radius parameters are a horizontal radius parameter 32 and a vertical radius parameter 32, and the size of each grid obtained after dividing according to the preset grid radius parameters is 32×32.

[0063] Furthermore, in the embodiments of this application, after dividing the first image into grid images of the first image according to the preset grid radius parameter, for any one of the grids, the average value of all pixels in the grid can be determined as the pixel value of the grid; at the same time, for the grid images of the first image, the grid position of any pixel can be determined based on the center coordinates of the grid where the pixel is located.

[0064] Furthermore, in the embodiments of this application, after the second image is divided into grid images of the second image according to the preset grid radius parameter, for any one of the grids, the average value of all pixels in the grid can be determined as the pixel value of the grid; at the same time, for the grid image of the second image, the grid position of any pixel can be determined based on the center coordinates of the grid where the pixel is located.

[0065] Furthermore, in the embodiments of this application, when determining the grid position mapping relationship between the first image and the second image of the target object, the zoom ratio parameter can be determined first based on the field of view parameter of the first image and the field of view parameter of the second image; at the same time, the offset parameter can be determined based on the positional relationship between the first camera and the second camera; then the grid position mapping relationship between the first image and the second image can be determined based on the zoom ratio parameter and the offset parameter.

[0066] In the embodiments of this application, the zoom ratio parameter can be understood as zoomRatio. The zoom ratio parameter can be used to determine the range of focal length that the camera can adjust, and is usually expressed in the form of a ratio.

[0067] For example, in some embodiments, for a first image captured by a first camera and a second image captured by a second camera, the field of view parameters of the first image and the second image can be determined first, and then the ratio of the field of view parameters of the two can be calculated to finally determine the corresponding zoom ratio parameter zoomRatio.

[0068] In the embodiments of this application, the field of view parameter can be used to determine the specific size of the field of view (FOV).

[0069] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the field of view parameter of the first image acquired by the first camera is 1000 and the field of view parameter of the second image acquired by the second camera is 100, then the ratio between the field of view parameter of the first image and the field of view parameter of the second image can be determined as the corresponding zoom ratio parameter zoomRatio, that is, zoomRatio = 10 can be calculated.

[0070] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a wide-angle camera, the field of view parameter of the first image acquired by the first camera is 1000 and the field of view parameter of the second image acquired by the second camera is 10000, then the ratio between the field of view parameter of the first image and the field of view parameter of the second image can be determined as the corresponding zoom ratio parameter zoomRatio, that is, zoomRatio = 0.1 can be calculated.

[0071] In the embodiments of this application, there may be positional differences between the first camera and the second camera. Therefore, for the first image and the second image of the same target object acquired by the first camera and the second camera, the pixel positions corresponding to the same content may be offset. Therefore, it is necessary to determine the offset parameter. The offset parameter can be understood as the offset amount.

[0072] Exemplary, in some embodiments, Figure 3 This is a schematic diagram illustrating the FOV difference between the first and second images presented in an embodiment of this application, as shown below. Figure 3As shown, assuming the first image is captured by the main camera and the second image is captured by the telephoto camera, the large black box can represent the FOV range of the first image, and the small black box can represent the FOV range of the second image.

[0073] Furthermore, in the embodiments of this application, after determining the zoom ratio parameter and the offset parameter respectively, the grid position mapping relationship between the first image and the second image can be further determined based on the zoom ratio parameter and the offset parameter, that is, the mapping relationship between the grid position of the pixel in the grid image after the first image is divided into grids and the grid position of the pixel in the grid image after the second image is divided into grids is determined.

[0074] It is understood that in the embodiments of this application, since the first image and the second image are acquired by different cameras targeting the same object, the FOV of the first image and the FOV of the second image are different. Consequently, for the same pixel, the position of the pixel in the first image is different from the position in the second image. Therefore, it is necessary to determine the grid position mapping relationship between the first image and the second image, so that the position of the pixel in the second image can be accurately mapped to the first image based on the grid position mapping relationship.

[0075] Therefore, in the embodiments of this application, the grid position mapping relationship between the first image and the second image can be used to match and align the pixels of the first image and the pixels of the second image.

[0076] It is understood that, in the embodiments of this application, it is assumed that the grid position of the pixels in the grid image of the second image is (stats'). x stats′ y Therefore, after mapping the second image onto the first image, the grid positions of the pixels in the grid image of the first image are (stats). x stats x ), where, (stats′ x stats′ y ) and (stats x stats x The correspondence between the first and second images is the grid position mapping relationship between the first and second images.

[0077] For example, in some embodiments, the grid position mapping relationship between the first image and the second image can be expressed as the following formula:

[0078] stats x =stats' x *zoomRatio+offset (1)

[0079] stats y =stats' y *zoomRatio+offset (2)

[0080] Where zoomRatio is the zoom ratio parameter and offset is the offset parameter.

[0081] Exemplary, in some embodiments, Figure 4 This is a schematic diagram illustrating the grid position mapping relationship between the first image and the second image proposed in an embodiment of this application, as shown below. Figure 4 As shown, if the first image is chosen as the reference for synchronous shadow compensation processing during zoom switching, then the correspondence between the grid positions stats of the pixels in the first image and the grid positions stats′ of the pixels in the second image can be determined by using the FOV of the first image as the reference and combining it with the FOV of the second image. For example, (stats′ x stats′ y ) corresponds to (stats x stats x ).

[0082] Step 102: Based on the grid position mapping relationship and the preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image.

[0083] In the embodiments of this application, after determining the grid position mapping relationship between the first image of the target object and the second image of the target object, the compensation intensity adjustment parameter corresponding to the second image can be further determined based on the grid position mapping relationship and the preset grid radius parameter.

[0084] In the embodiments of this application, the compensation intensity adjustment parameter corresponding to the second image can be used to adaptively adjust the pre-acquired initial shadow compensation gain parameter corresponding to the second camera during the lens shadow compensation process of the second image. Since the compensation intensity adjustment parameter corresponding to the second image is determined based on the grid position mapping relationship between the first and second images, using the compensation intensity adjustment parameter corresponding to the second image to control the lens shadow compensation process of the second image can ensure synchronous lens shadow compensation for both the first and second images.

[0085] Furthermore, in the embodiments of this application, when determining the compensation intensity adjustment parameters corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter, for the grid position of any pixel in the grid image of the second image, the mapped grid position corresponding to the grid image of the first image is determined based on the grid position mapping relationship; the first distance parameter corresponding to the grid image of the first image is determined; and the compensation intensity adjustment parameters corresponding to the second image are determined based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image.

[0086] It is understood that, in the embodiments of this application, the grid position of any pixel in the grid image of the second image can be determined based on the center coordinates of the grid where the pixel is located.

[0087] In the embodiments of this application, the grid position of a pixel in the grid image of the second image corresponds to the grid position after mapping the grid image of the first image. This can be understood as the grid position corresponding to the pixel in the grid image of the second image after mapping to the grid image of the first image.

[0088] Exemplary, in some embodiments, Figure 5 This is a schematic diagram of the mapped grid position proposed in an embodiment of this application, as shown below. Figure 5 As shown, assuming the first camera is the main camera and the second camera is the telephoto camera, the first image is acquired by the main camera and the second image is acquired by the telephoto camera. Then, by using the grid position mapping relationship, we can determine the mapped grid position of the pixels in the grid image of the second image to the grid image of the first image.

[0089] Furthermore, in the embodiments of this application, after determining the mapped grid position, a first distance parameter corresponding to the grid image of the first image can be determined based on the mapped grid position and the center position of the grid image of the first image.

[0090] It is understood that, in the embodiments of this application, the first distance parameter corresponding to the mapped grid position of the first image's grid image can be used to determine the distance between the mapped grid position and the center position of the first image's grid image. The first distance parameter may include the horizontal and vertical distances between the mapped grid position and the center position of the first image's grid image.

[0091] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the first image is acquired by the main camera and the second image is acquired by the telephoto camera, the first distance parameter of the grid image corresponding to the grid position of the first image after mapping can be represented as dist_w, including the horizontal distance dist_w x and vertical distance dist_w y .

[0092] It is understood that, in the embodiments of this application, by traversing the grid position of each pixel in the grid image of the second image according to the above method, the mapped grid position of each pixel to the grid image of the first image can be determined based on the grid position mapping relationship, and then the first distance parameter of the grid image of the first image corresponding to the mapped grid position of each pixel can be determined.

[0093] Furthermore, in the embodiments of this application, when determining the compensation intensity adjustment parameter corresponding to the second image based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the first adjustment parameter and the second adjustment parameter can be determined first based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image; then the compensation intensity adjustment parameter corresponding to the second image can be determined based on the first adjustment parameter and / or the second adjustment parameter.

[0094] Furthermore, in embodiments of this application, the brightness ratio parameter corresponding to the first image can be determined by the brightness parameters at different grid positions in the grid image of the first image. For example, the first brightness parameter corresponding to the grid edge position in the grid image of the first image and the second brightness parameter corresponding to the grid center position in the grid image of the first image can be determined respectively; then, the brightness ratio parameter corresponding to the first image is determined based on the first brightness parameter and the second brightness parameter.

[0095] It is understood that, in the embodiments of this application, the brightness parameter corresponding to the grid position is the brightness value of the pixel corresponding to that grid position.

[0096] Of course, in the embodiments of this application, when determining the brightness ratio parameter corresponding to the first image, brightness parameters corresponding to other grid positions can also be used, and are not limited to the grid edge position and / or grid center position. This application does not make specific limitations.

[0097] Exemplary, in some embodiments, Figure 6 This is a schematic diagram illustrating the determination of brightness ratio parameters proposed in an embodiment of this application, as shown below. Figure 6As shown, the ratio of the brightness value of pixel A at the grid edge position to the brightness value of pixel B at the grid center position in the grid image of the first image is used as the brightness ratio parameter.

[0098] Exemplary, in some embodiments, Figure 7 This is a schematic diagram illustrating the determination of brightness ratio parameters proposed in an embodiment of this application, as shown below. Figure 7 As shown, the ratio of the brightness value of pixel C at any grid position in the grid image of the first image to the brightness value of pixel B at the center of the grid is used as the brightness ratio parameter.

[0099] In other words, in the embodiments of this application, for any pixel in the grid image of the second image, after determining the corresponding first distance parameter, the corresponding first adjustment parameter can be further determined by combining the preset grid radius parameter and the brightness ratio parameter corresponding to the first image, and / or the corresponding second adjustment parameter can be determined.

[0100] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameter corresponding to the second image can be understood as the ratio map corresponding to the second image.

[0101] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the first image is acquired by the main camera and the second image is acquired by the telephoto camera, the first distance parameter of the grid image whose mapped grid position corresponds to the first image may include the horizontal distance dist_w x and vertical distance dist_w y The preset grid radius parameter 'radius' includes the horizontal radius parameter 'radius'. x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is ratio. For the pixels in the grid image of the second image, the corresponding first adjustment parameter ratioMap_w1 can be determined by referring to the following formula:

[0102]

[0103] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the first image is acquired by the main camera and the second image is acquired by the telephoto camera, the first distance parameter of the grid image whose mapped grid position corresponds to the first image may include the horizontal distance dist_w x and vertical distance dist_w y The preset grid radius parameter 'radius' includes the horizontal radius parameter 'radius'.x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is ratio. For the pixels in the grid image of the second image, the corresponding second adjustment parameter ratioMap_w2 can be determined by referring to the following formula:

[0104]

[0105] In other words, in the embodiments of this application, the first adjustment parameter and the second adjustment parameter can be calculated based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, using different mathematical operations. The calculation method for the adjustment parameter is not limited to the aforementioned square root calculation, square calculation, etc.

[0106] Furthermore, in the embodiments of this application, when determining the compensation intensity adjustment parameter corresponding to the second image based on the first adjustment parameter and / or the second adjustment parameter, it is possible to directly determine the first adjustment parameter as the compensation intensity adjustment parameter corresponding to the second image; or, it is possible to directly determine the second adjustment parameter as the compensation intensity adjustment parameter corresponding to the second image; or, it is possible to determine the compensation intensity adjustment parameter corresponding to the second image based on the preset weight, the first adjustment parameter, and the second adjustment parameter.

[0107] It is understood that, in the embodiments of this application, the calculated first adjustment parameter or the second adjustment parameter can be directly determined as the final compensation intensity adjustment parameter for adaptively adjusting the lens shadow compensation intensity, or the calculated first adjustment parameter and the second adjustment parameter can be weighted to obtain the final compensation intensity adjustment parameter.

[0108] For example, in some embodiments, assuming the preset weight is w1, the first adjustment parameter is ratioMap_w1, and the second adjustment parameter is ratioMap_w2, the following formula can be used to perform a weighted operation on the first adjustment parameter and the second adjustment parameter using the preset weight to determine the final compensation intensity adjustment parameter ratioMap_w.

[0109] ratioMap_w=w1*ratioMap_w1+(1.0-w1)*ratioMap_w2 (5)

[0110] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameters are calculated based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. This results in a pixel in the second image exhibiting a gradual attenuation trend from the center outwards, similar to the brightness change trend of the lens shadow image. Furthermore, preset weights can be introduced to adjust the proportions of the first and second adjustment parameters, flexibly adjusting the rate of ratio attenuation from the center outwards, thus better adapting to the changing needs of different scenarios.

[0111] Step 103: Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, determine the lens shadow compensation parameter corresponding to the second image.

[0112] In the embodiments of this application, after determining the compensation intensity adjustment parameters corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter, the lens shadow compensation parameters corresponding to the second image can be further determined based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera.

[0113] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameter corresponding to the second image can be used to adjust the shadow compensation intensity during the lens shadow compensation process of the second image. Therefore, the initial shadow compensation gain parameter corresponding to the second camera can be adjusted according to the compensation intensity adjustment parameter corresponding to the second image, thereby obtaining the adjusted lens shadow compensation parameter corresponding to the second image. The lens shadow compensation parameter corresponding to the second image can be used to perform lens shadow compensation on the second image.

[0114] In the embodiments of this application, the initial shadow compensation gain parameter corresponding to the second camera may be pre-generated by the image shadow compensation device, or it may be generated by other devices or apparatus and sent to the image shadow compensation device. This application does not impose specific limitations.

[0115] It is understood that, in the embodiments of this application, the initial shadow compensation gain parameter corresponding to the second camera can be understood as the LSC luminance full complement gain map corresponding to the second camera.

[0116] For example, in some embodiments, lens shading compensation calibration can be performed on each camera in advance to calculate the LSC brightness full-fill gain map. For instance, based on white field images captured by a wide-angle camera, a main camera, and a telephoto camera under uniform lighting, the images can be divided into grids according to a preset grid radius parameter (M×N). The average pixel value within each grid is statistically analyzed to obtain the average statistics (stats). The stats then exhibit a distribution trend of bright center and dark periphery. Using the center value of stats as the brightness compensation benchmark, the stats gain map is calculated when the brightness of each value in stats is fully filled until the brightness value at the center of stats is the same.

[0117] For example, in some embodiments, for pixel (i,j), the initial shadow compensation gain parameter gain corresponding to the second camera is... (i,j) You can refer to the following formula:

[0118]

[0119] Among them, stats (icenter,jcenter) For central statistics, stats (i,j) These are the statistical values ​​corresponding to pixel (i,j).

[0120] Furthermore, in the embodiments of this application, the compensation intensity adjustment parameter and the corresponding initial shadow compensation gain parameter can be multiplied based on the correspondence of grid positions, and then interpolation can be performed based on the product result to finally obtain the lens shadow compensation parameter corresponding to the second image.

[0121] For example, in some embodiments, the compensation intensity adjustment parameter corresponding to the second image can be understood as the ratio map corresponding to the second image, and the initial shadow compensation gain parameter corresponding to the second camera can be understood as the full brightness fill gain gain map corresponding to the second camera. The ratio map and the full brightness fill gain gain map are multiplied based on the grid position to obtain the LSC adjustment compensation gain correction map, and the correction map is interpolated and enlarged to the full image size to obtain the lens shadow compensation parameter that satisfies the size of the second image.

[0122] Step 104: Perform shadow compensation processing on the second image based on the lens shadow compensation parameters corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0123] In the embodiments of this application, after determining the lens shadow compensation parameters corresponding to the second image based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera, the second image can be further processed for shadow compensation based on the lens shadow compensation parameters corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0124] It is understood that, in the embodiments of this application, the lens shading compensation parameter corresponding to the second image is obtained by adjusting the initial shading compensation gain parameter corresponding to the second camera based on the compensation intensity adjustment parameter corresponding to the second image. The compensation intensity adjustment parameter corresponding to the second image is determined based on the grid position mapping relationship between the first and second images. Therefore, applying the lens shading compensation parameter corresponding to the second image to perform LSC lens shading correction ensures synchronous lens shading compensation for both the first and second images, thus ensuring brightness consistency in the same FOV area across multiple cameras.

[0125] Furthermore, in the embodiments of this application, Figure 8 This is a schematic diagram illustrating the implementation process of the image shadow compensation method proposed in the embodiments of this application, as follows: Figure 8 As shown, the image shadow compensation method may include the following steps:

[0126] Step 105: For any pixel in the grid image of the first image, determine the second distance parameter corresponding to the grid position of the first image.

[0127] In the embodiments of this application, after dividing the first image into grid images of the first image according to a preset grid radius parameter, for any pixel in the grid image of the first image, a second distance parameter corresponding to the grid position of the first image is determined.

[0128] It is understood that, in the embodiments of this application, the grid position of any pixel in the grid image of the first image can be determined based on the center coordinates of the grid where the pixel is located.

[0129] Furthermore, in the embodiments of this application, after determining the grid position of any pixel, a second distance parameter corresponding to the grid position of the first image can be determined based on the grid position and the center position of the grid image of the first image.

[0130] It is understood that, in the embodiments of this application, the grid position corresponds to a second distance parameter of the grid image of the first image, which can be used to determine the distance between the grid position and the center position of the grid image of the first image. The second distance parameter may include the horizontal and vertical distances between the grid position and the center positions of the grid images of the second and third images.

[0131] For example, in some embodiments, assuming the first camera is a main camera and the first image is acquired by the main camera, the grid position of any pixel in the grid image of the first image corresponds to a second distance parameter of the grid image of the first image, which can be represented as dist, including the horizontal distance dist. x and vertical distance dist y .

[0132] It is understood that, in the embodiments of this application, by traversing the grid position of each pixel in the grid image of the first image according to the above method, it is possible to determine the second distance parameter of the grid image of the first image corresponding to the grid position of each pixel.

[0133] Step 106: Based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, determine the compensation intensity adjustment parameter corresponding to the first image.

[0134] In the embodiments of this application, for any pixel in the grid image of the first image, after determining the second distance parameter of the grid image corresponding to the grid position of the first image, the compensation intensity adjustment parameter corresponding to the first image can be further determined based on the second distance parameter, the preset grid radius parameter and the brightness ratio parameter corresponding to the first image.

[0135] Furthermore, in the embodiments of this application, when determining the compensation intensity adjustment parameter corresponding to the first image based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the third adjustment parameter and the fourth adjustment parameter can be determined first based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image; then the compensation intensity adjustment parameter corresponding to the first image can be determined based on the third adjustment parameter and / or the fourth adjustment parameter.

[0136] Furthermore, in embodiments of this application, the brightness ratio parameter corresponding to the first image can be determined by the brightness parameters at different grid positions in the grid image of the first image. For example, the first brightness parameter corresponding to the grid edge position in the grid image of the first image and the second brightness parameter corresponding to the grid center position in the grid image of the first image can be determined respectively; then, the brightness ratio parameter corresponding to the first image is determined based on the first brightness parameter and the second brightness parameter.

[0137] In other words, in the embodiments of this application, for any pixel in the grid image of the first image, after determining the corresponding first distance parameter, the corresponding third adjustment parameter can be further determined by combining the preset grid radius parameter and the brightness ratio parameter corresponding to the first image, and / or the corresponding fourth adjustment parameter can be determined.

[0138] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameter corresponding to the first image can be understood as the ratio map corresponding to the first image.

[0139] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the first image is acquired by the main camera, and the second image is acquired by the telephoto camera, the second distance parameter of the grid image corresponding to the grid position of the first image may include the horizontal distance dist. x and vertical distance dist y The preset grid radius parameter 'radius' includes the horizontal radius parameter 'radius'. x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is ratio. For each pixel in the grid image of the first image, the corresponding third adjustment parameter ratioMap1 can be determined by referring to the following formula:

[0140]

[0141] For example, in some embodiments, assuming the first camera is a main camera and the second camera is a telephoto camera, the first image is acquired by the main camera, and the second image is acquired by the telephoto camera, the second distance parameter of the grid image whose mapped grid position corresponds to the first image may include the horizontal distance dist. x and vertical distance dist y The preset grid radius parameter 'radius' includes the horizontal radius parameter 'radius'. x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is ratio. For each pixel in the grid image of the first image, the corresponding fourth adjustment parameter ratioMap2 can be determined by referring to the following formula:

[0142]

[0143] In other words, in the embodiments of this application, the third and fourth adjustment parameters can be calculated based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, using different mathematical operations. The calculation methods for the adjustment parameters are not limited to the aforementioned square root calculation, square calculation, etc.

[0144] Furthermore, in the embodiments of this application, when determining the compensation intensity adjustment parameter corresponding to the first image based on the third adjustment parameter and / or the fourth adjustment parameter, it is possible to directly determine the third adjustment parameter as the compensation intensity adjustment parameter corresponding to the first image; or, it is possible to directly determine the fourth adjustment parameter as the compensation intensity adjustment parameter corresponding to the first image; or, it is possible to determine the compensation intensity adjustment parameter corresponding to the first image based on the preset weight, the third adjustment parameter, and the fourth adjustment parameter.

[0145] It is understood that, in the embodiments of this application, the calculated third adjustment parameter or the fourth adjustment parameter can be directly determined as the final compensation intensity adjustment parameter used for adaptively adjusting the lens shadow compensation intensity, or the calculated third adjustment parameter and the fourth adjustment parameter can be weighted to obtain the final compensation intensity adjustment parameter.

[0146] For example, in some embodiments, assuming the preset weight is w1, the third adjustment parameter is ratioMap1, and the fourth adjustment parameter is ratioMap2, the final compensation intensity adjustment parameter ratioMap can be determined by weighting the third and fourth adjustment parameters using the preset weights, as shown in the following formula:

[0147] ratioMap=w1*ratioMap1+(1.0-w1)*ratioMap2 (9)

[0148] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameters are calculated based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. This results in a pixel in the first image exhibiting a gradual attenuation trend from the center outwards, similar to the brightness change trend of the lens shadow image. Furthermore, preset weights can be introduced to adjust the proportions of the third and fourth adjustment parameters, flexibly adjusting the rate of ratio attenuation from the center outwards, thus better adapting to the changing needs of different scenarios.

[0149] Step 107: Determine the lens shadow compensation parameters corresponding to the first image based on the compensation intensity adjustment parameters corresponding to the first image and the initial shadow compensation gain parameters corresponding to the first camera.

[0150] In the embodiments of this application, after determining the compensation intensity adjustment parameters corresponding to the first image, the lens shadow compensation parameters corresponding to the first image can be further determined based on the compensation intensity adjustment parameters corresponding to the first image and the initial shadow compensation gain parameters corresponding to the first camera.

[0151] It is understood that, in the embodiments of this application, the compensation intensity adjustment parameter corresponding to the first image can be used to adjust the shadow compensation intensity during the lens shadow compensation process of the first image. Therefore, the initial shadow compensation gain parameter corresponding to the first camera can be adjusted according to the compensation intensity adjustment parameter corresponding to the first image, thereby obtaining the adjusted lens shadow compensation parameter corresponding to the first image. The lens shadow compensation parameter corresponding to the first image can be used to perform lens shadow compensation on the first image.

[0152] In the embodiments of this application, the initial shadow compensation gain parameter corresponding to the first camera may be pre-generated by the image shadow compensation device, or it may be generated by other devices or apparatus and sent to the image shadow compensation device. This application does not impose specific limitations.

[0153] It is understood that, in the embodiments of this application, the initial shadow compensation gain parameter corresponding to the first camera can be understood as the LSC luminance full complement gain map corresponding to the first camera.

[0154] For example, in some embodiments, lens shading compensation calibration can be performed on each camera in advance to calculate the LSC brightness full-fill gain map. For instance, based on white field images captured by a wide-angle camera, a main camera, and a telephoto camera under uniform lighting, the images can be divided into grids according to a preset grid radius parameter (M×N). The average pixel value within each grid is statistically analyzed to obtain the average statistics (stats). The stats then exhibit a distribution trend of bright center and dark periphery. Using the center value of stats as the brightness compensation benchmark, the stats gain map is calculated when the brightness of each value in stats is fully filled until the brightness value at the center of stats is the same.

[0155] For example, in some embodiments, for pixel (i,j), the initial shadow compensation gain parameter gain corresponding to the first camera is... (i,j) You can refer to formula (6).

[0156] Furthermore, in the embodiments of this application, the compensation intensity adjustment parameter and the corresponding initial shadow compensation gain parameter can be multiplied based on the correspondence of grid positions, and then interpolation can be performed based on the product result to finally obtain the lens shadow compensation parameter corresponding to the first image.

[0157] For example, in some embodiments, the compensation intensity adjustment parameter corresponding to the first image can be understood as the ratio map corresponding to the first image, and the initial shadow compensation gain parameter corresponding to the first camera can be understood as the full brightness fill gain gain map corresponding to the first camera. The ratio map and the full brightness fill gain gain map are multiplied based on the grid position to obtain the LSC adjustment compensation gain correction map, and the correction map is interpolated and enlarged to the full image size to obtain the lens shadow compensation parameter that satisfies the size of the first image.

[0158] Step 108: Perform shadow compensation processing on the first image based on the lens shadow compensation parameters corresponding to the first image to obtain the processed image of the target object corresponding to the first camera.

[0159] In the embodiments of this application, after determining the lens shadow compensation parameters corresponding to the first image based on the compensation intensity adjustment parameters corresponding to the first image and the initial shadow compensation gain parameters corresponding to the first camera, the first image can be further processed for shadow compensation based on the lens shadow compensation parameters corresponding to the first image to obtain the processed image of the target object corresponding to the first camera.

[0160] It is understood that, in the embodiments of this application, the lens shadow compensation parameter corresponding to the first image is obtained by adjusting the initial shadow compensation gain parameter corresponding to the first camera based on the compensation intensity adjustment parameter corresponding to the first image. Applying the lens shadow compensation parameter corresponding to the first image to the first image for LSC lens shadow correction can achieve accurate lens shadow compensation.

[0161] Therefore, in the embodiments of this application, for images captured by different cameras in the image shadow compensation device, corresponding lens shadow compensation parameters can be determined to adaptively adjust the initial shadow compensation gain parameters corresponding to each camera. During zoom switching, the lens shadow compensation parameters for each camera can be determined based on the grid position mapping relationship between different images from different cameras. This ensures the consistency of LSC compensation among multiple cameras during lens shadow compensation of the corresponding images based on the lens shadow compensation parameters, resulting in smooth, flicker-free image brightness during multi-camera zoom switching.

[0162] It is understood that in the embodiments of this application, for the reference image, such as the first image captured by the main camera, the lens shadow compensation parameter corresponding to the first image does not need to be based on the grid position mapping relationship between different images, that is, the determination of the grid position mapping relationship is skipped. Instead, the lens shadow compensation parameter corresponding to the first image is directly determined, and the lens shadow compensation parameter is used to perform lens shadow compensation on the first image.

[0163] In summary, the image shadow compensation method proposed in this application is an LSC compensation intensity ratio map adjustment model based on radial information, which can flexibly adjust the LSC compensation intensity according to changes in the scene environment and improve image quality.

[0164] Furthermore, the image shadow compensation method proposed in this application, based on the above radial compensation intensity adjustment model, proposes an adaptive adjustment scheme for the synchronous alignment of multi-camera LSC compensation intensity under different FOV conditions, which controls the synchronization effect of LSC compensation when switching between multiple cameras, thereby improving the user experience of multi-camera zoom.

[0165] Therefore, the image shadow compensation method proposed in this application restores the FOV position relationship between multiple cameras based on image registration technology. It proposes a synchronous compensation scheme for multi-camera LSC lens shadow correction based on radial position. Under difficult conditions such as large differences in optical lens structure, large differences in zoom FOV, and large dynamic range, it can ensure the consistency of LSC compensation between multiple cameras in real time, so that the image brightness is smooth and flicker-free when switching between multiple cameras, which greatly improves the image effect.

[0166] This application proposes an image shadow compensation method. When switching from a first camera to a second camera to capture a target object, a grid position mapping relationship is determined between a first image and a second image of the target object. The first image is captured by the first camera, and the second image is captured by the second camera. Based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined. Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined. The second image is then subjected to shadow compensation processing based on the lens shadow compensation parameter to obtain the processed image of the target object corresponding to the second camera. In other words, in this application, during zoom switching, the relative positional relationship between different images captured by different cameras can be determined first, i.e., the grid position mapping relationship can be determined. Then, based on this grid position mapping relationship, the corresponding compensation intensity adjustment parameter is determined, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter to finally obtain the corresponding lens shadow compensation parameter for lens shadow compensation of the image. This achieves synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. As can be seen, this application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve image quality.

[0167] Based on the above embodiments, another embodiment of this application proposes an image shadow compensation method, which includes a multi-camera LSC lens shadow synchronization compensation method. Specifically, based on the LSC calibration brightness full-complementation gain map (initial shadow compensation gain parameters), a lens shadow compensation gain adjustment model based on radial position is designed. Through image registration, the FOV positional relationship between different cameras such as wide-angle, main camera, and telephoto is restored. A lens shadow compensation gain adjustment ratio map (compensation intensity adjustment parameters) for each camera in the same scene is calculated. After combining this with the brightness full-complementation gain map, the compensation is applied to the images of each of the multiple cameras, achieving a consistent brightness compensation alignment effect in the same area.

[0168] The initial shadow compensation gain parameters for each camera can be pre-generated or pre-received from other devices or apparatuses; this application does not impose any specific limitations on them.

[0169] For example, in some embodiments, when setting the initial shadow compensation gain parameters (brightness full-fill gain map), lens shadow compensation calibration can be performed on each camera to calculate the LSC brightness full-fill gain map. Wide-angle, main, and telephoto lenses capture white field images under uniform lighting. The images are divided into grids according to a preset grid radius parameter (M×N, e.g., 64×48). The average pixel value within each grid is calculated to obtain the average statistical values ​​(stats). The stats show a distribution trend of bright center and dark periphery. Using the center value of stats as the brightness compensation benchmark, the brightness of each value in stats is fully filled until the center brightness value of stats is the same. The resulting stats gain map shows the initial shadow compensation gain parameters (gain) for each camera at pixel (i,j). (i,j) You can refer to formula (6).

[0170] For example, in some embodiments, for the reference image, such as the first image captured by the main camera, the lens shadow compensation parameters corresponding to the first image can be designed based on the corresponding full brightness fill gain map (initial shadow compensation gain parameters) and the distance relationship between the stats grid position and the image center. The adjustment model of lens shadow brightness compensation intensity ratio map is designed, with the ratio (brightness ratio parameter) of the brightness of the grid position point at the edge of stats to the brightness of the position point at the center of stats as the adjustment starting point. For pixel (i, j), the adjustment mode formula, that is, the third adjustment parameter ratioMap1(i, j) and the fourth adjustment parameter ratioMap2(i, j) corresponding to the first image refer to the following formulas respectively:

[0171]

[0172] Where, dist(i, j) x Let dist(i,j) be the horizontal distance of the grid image of the first image corresponding to the grid position of pixel (i,j). y The grid position of pixel (i, j) corresponds to the vertical distance of the grid image of the first image. The preset grid radius parameter radius includes the horizontal radius parameter radius. x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is ratio.

[0173] Furthermore, in the embodiments of this application, the third adjustment parameter can be directly determined as the compensation intensity adjustment parameter corresponding to the first image; or, the fourth adjustment parameter can be directly determined as the compensation intensity adjustment parameter corresponding to the first image; or, the compensation intensity adjustment parameter corresponding to the first image can be determined based on the preset weight, the third adjustment parameter, and the fourth adjustment parameter.

[0174] For example, in some embodiments, assuming the preset weight is w1, the third adjustment parameter ratioMap1(i,j) and the fourth adjustment parameter ratioMap2(i,j) can be weighted using the preset weight to determine the final compensation intensity adjustment parameter ratioMap(i,j) by referring to the following formula:

[0175] ratioMap(i,j)=w1*ratioMap1(i,j)+(1.0-w1)*ratioMap2(i,j) (12)

[0176] As can be seen, the center point value of the ratio map is 1.0, and the edge point value is ratio. This model exhibits a gradual decay trend from the center outwards, which is similar to the brightness change trend of the lens shadow image. This adjustment mode can also adjust the weight between the square root ratioMap1(i,j) and the square ratioMap2(i,j) through the w1 parameter, flexibly adjusting the speed at which the ratio decays from the center outwards, and better adapting to the changing needs of different scenarios.

[0177] For example, in some embodiments, for non-reference images, such as those captured by the second camera after switching from the first camera to the second camera, assuming the first camera is the main camera and the second camera is a wide-angle camera, the ratio map adjustment mode can be adapted to the wide-angle camera based on the FOV size and position relationship, with the main camera as the reference.

[0178] In other words, the relationship between the FOVs of multiple cameras, assuming the main camera coordinate system is used as the reference, can be determined by using the image registration method between multiple cameras. Through the correspondence of feature points, the coordinates of the stats grid (grid image) of the wide-angle camera (or telephoto camera, etc.) can be transformed to the main camera FOV coordinate system, that is, the grid position mapping relationship between the first image and the second image can be determined. The transformation process is as shown in formulas (1) and (2). Where zoomRatio is the zoom ratio parameter and offset is the offset parameter. zoomRatio and offset are the coordinate transformation parameters of the wide-angle camera (or telephoto camera, etc.) obtained by image registration and transformed to the main camera coordinate system.

[0179] For example, in some embodiments, for non-reference images, based on the determined grid position mapping relationship between different images captured by different cameras, the ratio map of LSC sync compensation alignment between multiple cameras is further calculated, that is, the corresponding compensation intensity adjustment parameter is determined based on the grid position mapping relationship.

[0180] For example, in some embodiments, it is assumed that the first camera is a main camera and the second camera is a telephoto camera. The first image is acquired by the main camera and the second image is acquired by the telephoto camera. The horizontal distance between the grid position of pixel (i, j) in the second image and the grid image of the first image is dist_w(i, j). x and vertical distance dist_w(i, j) y .

[0181] For example, in some embodiments, it is assumed that the first camera is a main camera and the second camera is a telephoto camera. The first image is acquired by the main camera and the second image is acquired by the telephoto camera. For pixel (i, j), the horizontal distance of the mapped grid position to the grid image of the first image is dist_w(i, j). x and vertical distance dist_w(i, j) y The preset grid radius parameter 'radius' includes the horizontal radius parameter 'radius'. x and the radius parameter in the vertical direction. y The brightness ratio parameter corresponding to the first image is 'ratio'. The adjustment mode formulas, namely the first adjustment parameter 'ratioMap_w1(i,j)' and the second adjustment parameter 'ratioMap_w2(i,j)' corresponding to the second image, are respectively referred to as the following formulas:

[0182]

[0183] Furthermore, in the embodiments of this application, the first adjustment parameter can be directly determined as the compensation intensity adjustment parameter corresponding to the second image; or, the second adjustment parameter can be directly determined as the compensation intensity adjustment parameter corresponding to the second image; or, the compensation intensity adjustment parameter corresponding to the second image can be determined based on the preset weight, the first adjustment parameter, and the second adjustment parameter.

[0184] For example, in some embodiments, assuming the preset weight is w1, the first adjustment parameter ratioMap_w1(i,j) and the second adjustment parameter ratioMap_w2(i,j) can be weighted using the preset weight to determine the final compensation intensity adjustment parameter ratioMap_w(i,j) by referring to the following formula:

[0185] ratioMap_w(i,j)=w1*ratioMap_w1(i,j)+(1.0-w1)*ratioMap_w2(i,j)(15)

[0186] Furthermore, the ratio map of each of the multiple cameras is multiplied by the full brightness compensation gain map to obtain the LSC adjustment compensation gain correction map. The correction map is then interpolated and enlarged to the full image size and finally applied to the multi-camera image. This process not only completes the LSC lens shadow correction for each camera, but also maintains brightness consistency in the same FOV area among the multiple cameras.

[0187] Furthermore, in the embodiments of this application, the color transformation matrix of the image color between multiple cameras can be calibrated according to the differences in the absorption characteristics of each camera module for different wavelength spectra. This allows the LSC lens shadow compensation to maintain color consistency within the same FOV area when switching zoom between multiple cameras, further improving the user's viewing experience.

[0188] In summary, this application proposes an LSC compensation intensity ratio map adjustment model based on radial information, which can flexibly adjust the LSC compensation intensity according to changes in the scene environment, thereby improving image quality.

[0189] Furthermore, the image shadow compensation method proposed in this application, based on the above radial compensation intensity adjustment model, proposes an adaptive adjustment scheme for the synchronous alignment of multi-camera LSC compensation intensity under different FOV conditions, which controls the synchronization effect of LSC compensation when switching between multiple cameras, thereby improving the user experience of multi-camera zoom.

[0190] Therefore, the image shadow compensation method proposed in this application restores the FOV position relationship between multiple cameras based on image registration technology. It proposes a synchronous compensation scheme for multi-camera LSC lens shadow correction based on radial position. Under difficult conditions such as large differences in optical lens structure, large differences in zoom FOV, and large dynamic range, it can ensure the consistency of LSC compensation between multiple cameras in real time, so that the image brightness is smooth and flicker-free when switching between multiple cameras, which greatly improves the image effect.

[0191] This application proposes an image shadow compensation method. When switching from a first camera to a second camera to capture a target object, a grid position mapping relationship is determined between a first image and a second image of the target object. The first image is captured by the first camera, and the second image is captured by the second camera. Based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined. Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined. The second image is then subjected to shadow compensation processing based on the lens shadow compensation parameter to obtain the processed image of the target object corresponding to the second camera. In other words, in this application, during zoom switching, the relative positional relationship between different images captured by different cameras can be determined first, i.e., the grid position mapping relationship can be determined. Then, based on this grid position mapping relationship, the corresponding compensation intensity adjustment parameter is determined, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter to finally obtain the corresponding lens shadow compensation parameter for lens shadow compensation of the image. This achieves synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. As can be seen, this application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve image quality.

[0192] Based on the above embodiments, in another embodiment of this application... Figure 9 This is a schematic diagram of the composition structure of the image shadow compensation device proposed in the embodiments of this application, as shown below. Figure 9 As shown, the image shadow compensation device 110 proposed in this application embodiment may include:

[0193] The determining unit 1101 is used to determine the grid position mapping relationship between a first image and a second image of the target object when switching from a first camera to a second camera to capture the target object; wherein the first image is acquired by the first camera and the second image is acquired by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, the compensation intensity adjustment parameter corresponding to the second image is determined; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, the lens shadow compensation parameter corresponding to the second image is determined.

[0194] The compensation unit 1102 is used to perform shadow compensation processing on the second image based on the lens shadow compensation parameters corresponding to the second image, so as to obtain the processed image of the target object corresponding to the second camera.

[0195] In the embodiments of this application, further, Figure 10 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application, such as... Figure 10 As shown, the electronic device 120 proposed in this application embodiment may include a processor 1201, a memory 1202, a communication interface 1203, and a bus 1204 for connecting the processor 1201, the memory 1202 and the communication interface 1203.

[0196] In the embodiments of this application, the processor 1201 can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other types, and this application embodiment does not specifically limit this. The electronic device 120 may also include a memory 1202, which can be connected to the processor 1201. The memory 1202 is used to store executable program code, which includes computer operation instructions. The memory 1202 may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk drives.

[0197] In embodiments of this application, bus 1204 is used to connect communication interface 1203, processor 1201, and memory 1202, as well as the mutual communication between these devices.

[0198] In practical applications, the aforementioned memory 1202 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 1201.

[0199] Further, in an embodiment of this application, the processor 1201 is configured to, when switching from a first camera to a second camera to capture a target object, determine a grid position mapping relationship between a first image and a second image of the target object; wherein the first image is acquired by the first camera and the second image is acquired by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, determine a compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, determine a lens shadow compensation parameter corresponding to the second image; and perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

[0200] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0201] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0202] This application provides a computer-readable storage medium storing a program that, when executed by a processor, implements the image shadow compensation method described above.

[0203] Specifically, the program instructions corresponding to an image shadow compensation method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to an image shadow compensation method in the storage media are read or executed by an electronic device, the following steps are included:

[0204] When switching from the first camera to the second camera to capture the target object, determine the grid position mapping relationship between the first image and the second image of the target object; wherein the first image is captured by the first camera and the second image is captured by the second camera.

[0205] Based on the grid position mapping relationship and the preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image;

[0206] Based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera, determine the lens shadow compensation parameters corresponding to the second image.

[0207] The second image is processed by performing shadow compensation based on the lens shadow compensation parameters corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0208] This application also provides a computer program product.

[0209] In some embodiments, the computer program product may include a computer program or instructions.

[0210] In some embodiments, the computer program product can be applied to the computer device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the computer device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0211] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0212] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0213] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0214] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0215] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An image shadow compensation method, characterized in that, The method is applied to an image shadow compensation device, the image shadow compensation device being configured with a first camera and a second camera, and the method includes: When switching from the first camera to the second camera to capture a target object, a grid position mapping relationship is determined between a first image of the target object and a second image of the target object; wherein the first image is captured by the first camera and the second image is captured by the second camera; Based on the grid position mapping relationship and the preset grid radius parameter, the compensation intensity adjustment parameter corresponding to the second image is determined; Based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera, the lens shadow compensation parameters corresponding to the second image are determined. The second image is subjected to shadow compensation processing based on the lens shadow compensation parameters corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

2. The method according to claim 1, characterized in that, Determining the grid position mapping relationship between the first image and the second image of the target object includes: Based on the field of view parameters of the first image and the field of view parameters of the second image, the zoom ratio parameter is determined; The offset parameter is determined based on the positional relationship between the first camera and the second camera; Based on the zoom ratio parameter and offset parameter, the grid position mapping relationship between the first image and the second image is determined.

3. The method according to claim 1, characterized in that, The step of determining the compensation intensity adjustment parameters corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter includes: For any pixel in the grid image of the second image, the mapped grid position of the grid position corresponding to the grid image of the first image is determined based on the grid position mapping relationship. The first distance parameter is determined to correspond to the grid position of the first image after mapping; Based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the compensation intensity adjustment parameter corresponding to the second image is determined.

4. The method according to claim 3, characterized in that, The step of determining the compensation intensity adjustment parameter corresponding to the second image based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image includes: Based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, a first adjustment parameter and a second adjustment parameter are determined. The compensation intensity adjustment parameter corresponding to the second image is determined based on the first adjustment parameter and / or the second adjustment parameter.

5. The method according to claim 4, characterized in that, The step of determining the compensation intensity adjustment parameter corresponding to the second image based on the first adjustment parameter and / or the second adjustment parameter includes: The first adjustment parameter is determined as the compensation intensity adjustment parameter corresponding to the second image; or... The second adjustment parameter is determined as the compensation intensity adjustment parameter corresponding to the second image; or... Based on the preset weights, the first adjustment parameter, and the second adjustment parameter, the compensation intensity adjustment parameter corresponding to the second image is determined.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the lens shadow compensation parameters corresponding to the second image based on the compensation intensity adjustment parameters corresponding to the second image and the initial shadow compensation gain parameters corresponding to the second camera includes: For any pixel position in the grid image of the second image, the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter are multiplied to obtain the product result corresponding to the grid position; Based on the product result, the lens shading compensation parameters corresponding to the second image are determined.

7. The method according to claim 1, characterized in that, The method further includes: For any pixel in the grid image of the first image, determine the second distance parameter of the grid image corresponding to the grid position of the first image; Based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the compensation intensity adjustment parameter corresponding to the first image is determined; Based on the compensation intensity adjustment parameter corresponding to the first image and the initial shadow compensation gain parameter corresponding to the first camera, the lens shadow compensation parameter corresponding to the first image is determined. The first image is subjected to shadow compensation processing based on the lens shadow compensation parameters corresponding to the first image to obtain the processed image of the target object corresponding to the first camera.

8. The method according to any one of claims 3-5, 7, characterized in that, The method further includes: Determine the first brightness parameter corresponding to the grid edge position in the grid image of the first image; Determine the second brightness parameter corresponding to the center position of the grid in the grid image of the first image; The brightness ratio parameter corresponding to the first image is determined based on the first brightness parameter and the second brightness parameter.

9. An image shadow compensation device, characterized in that, The image shadow compensation device includes: The determining unit is configured to, when switching from a first camera to a second camera to capture a target object, determine a grid position mapping relationship between a first image and a second image of the target object; wherein the first image is captured by the first camera and the second image is captured by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, determine a compensation intensity adjustment parameter corresponding to the second image; and based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera, determine a lens shadow compensation parameter corresponding to the second image. The compensation unit is used to perform shadow compensation processing on the second image based on the lens shadow compensation parameters corresponding to the second image, so as to obtain the processed image of the target object corresponding to the second camera.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images