Image shadow compensation method and device, equipment and storage medium

Abstract

The invention discloses an image shadow compensation method, which is applied to an image shadow compensation device, the image shadow compensation device is provided with a first camera and a second camera, and the first camera is switched to the second camera to shoot a target object. Determining a grid position mapping relationship between the first image of the target object and the second image of the target object; wherein the first image is acquired through a first camera, and the second image is acquired through a second camera; determining a compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relation and a preset grid radius parameter; determining a lens shadow compensation parameter corresponding to the second image based on a compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera; and performing shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to an image shadow compensation method, apparatus, device, and storage medium. Background Art

[0002] With the continuous progress and development of terminal technologies, a terminal device can be configured with multiple cameras, such as a wide-angle camera, a main camera, a telephoto camera, etc., and perform zoom switching through the configured multiple cameras. Among them, there are differences in the field of view (FOV) between different cameras.

[0003] In the multi-camera switching shooting scenario, the current lens shading correction (LSC) scheme cannot effectively respond to the FOV differences between different cameras, resulting in brightness jumps during zoom switching and reducing the image quality. Summary of the Invention

[0004] Embodiments of the present application provide an image shadow compensation method, apparatus, device, and storage medium, which can improve the accurate recognition of dynamic photos, expand the application scenarios of dynamic photos, and enhance the intelligence of electronic devices.

[0005] The technical solution of the embodiments of the present application is implemented as follows:

[0006] In a first aspect, embodiments of the present application provide an image shadow compensation method, which is applied to an image shadow compensation apparatus. The image shadow compensation apparatus is configured with a first camera and a second camera. The method includes:

[0007] When switching from the first camera to the second camera to photograph a target object, determining a grid position mapping relationship between a first image of the target object and a second image of the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera;

[0008] Based on the grid position mapping relationship and a preset grid radius parameter, determining a compensation intensity adjustment parameter corresponding to the second image;

[0009] Based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera, determining a lens shadow compensation parameter corresponding to the second image;

[0010] Performing shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

[0011] In a second aspect, embodiments of the present application provide an image shadow compensation apparatus, which includes:

[0012] A determination unit, configured to determine a grid position mapping relationship between a first image of a target object and a second image of the target object when switching from a first camera to a second camera to capture the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, determine a compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera, determine a lens shadow compensation parameter corresponding to the second image;

[0013] A compensation unit, configured to perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image, to obtain a processed image of the target object corresponding to the second camera.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory storing processor-executable instructions. When the instructions are executed by the processor, the method according to the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the method according to the first aspect is implemented.

[0016] The embodiments of the present application provide an image shadow compensation method, device, equipment and storage medium. When switching from a first camera to a second camera to capture a target object, a grid position mapping relationship between a first image of the target object and a second image of the target object is determined; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image is determined; based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image is determined; based on the lens shadow compensation parameter corresponding to the second image, shadow compensation processing is performed on the second image to obtain a processed image of the target object corresponding to the second camera. That is to say, in the present application, during the process of zoom switching, the relative position relationship between different images captured by different cameras can be determined first, that is, the grid position mapping relationship is determined, and then the corresponding compensation intensity adjustment parameter is determined based on the grid position mapping relationship, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter, and finally the corresponding lens shadow compensation parameter is obtained to perform lens shadow compensation on the image, so as to realize synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. It can be seen that the present application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve the image quality. Description of the Drawings

[0017] Figure 1 Schematic diagram of the difference in the field of view area between different cameras;

[0018] Figure 2 Schematic diagram of the implementation process of the image shadow compensation method proposed in the embodiments of the present application;

[0019] Figure 3 Schematic diagram of the FOV difference between the first image and the second image proposed in the embodiments of the present application;

[0020] Figure 4 Schematic diagram of the grid position mapping relationship between the first image and the second image proposed in the embodiments of the present application;

[0021] Figure 5 Schematic diagram of the mapped grid position proposed in the embodiments of the present application;

[0022] Figure 6 Schematic diagram of determining the brightness ratio parameter proposed in the embodiments of the present application;

[0023] Figure 7 Schematic diagram of determining the brightness ratio parameter proposed in the embodiments of the present application;

[0024] Figure 8 Schematic diagram of the implementation process of the image shadow compensation method proposed in the embodiments of the present application;

[0025] Figure 9 Schematic diagram of the composition structure of the image shadow compensation device proposed in the embodiments of the present application;

[0026] Figure 10 Schematic diagram of the composition structure of the electronic device proposed in the embodiments of the present application. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. In addition, it should be noted that for the convenience of description, only the parts related to the present application are shown in the drawings.

[0028] In the field of imaging products, due to the light-gathering effect of the convex lens in the camera module, the light intensity at the center of the sensor is greater than that at the edge, such as the vignetting phenomenon, and finally an image with a bright center and a dark periphery will be captured.

[0029] Lens Shadow Compensation (LSC), also known as vignetting correction or shadow compensation, is a post-processing of the captured image by the camera lens through an internal software algorithm to eliminate or reduce the image edge darkening phenomenon caused by the optical characteristics of the lens.

[0030] After the shadow compensation function is turned on, the camera can intelligently identify and adjust the brightness of the image edge. The darkening phenomenon of the image edge is significantly improved, the overall brightness distribution is more uniform, and the details are clearer. This helps to improve the overall visual perception and information transmission efficiency of the image, thereby improving the image quality.

[0031] With the continuous progress and development of terminal technology, terminal devices can be equipped with multiple cameras, such as wide-angle cameras, main cameras, telephoto cameras, etc., and zoom switching can be performed through the configured multiple cameras. Among them, there are differences in the FOV between different cameras.

[0032] Figure 1 Schematic diagram of the difference in the field of view area between different cameras, as Figure 1 shown, there are significant differences in the FOV between the wide-angle camera, the main camera, and the telephoto camera. Among them, the FOV of the wide-angle camera is much larger than that of the telephoto camera.

[0033] Currently, there are mainly three modules in the Image Signal Processor pipeline (ISP pipeline) that affect the multi-camera brightness consistency, namely, the Auto Exposure (AE) module, the Tone Mapping module, and the LSC (Lens Shading Correction) module. Currently, the multi-camera brightness consistency solutions in the industry mainly fall into two dimensions: the multi-camera AE synchronization algorithm (AE sync) and the multi-camera brightness synchronization algorithm (tone sync).

[0034] However, due to the differences in the Field of View (FOV) between different cameras, there are also corresponding differences in the LSC compensation between different cameras. This leads to differences in the brightness at the same object in the multi-camera images during multi-camera switching, and there is a brightness jump during zoom switching, reducing the image quality.

[0035] To solve the above problems, the embodiments of the present application provide an image shadow compensation method, apparatus, device, and storage medium. When switching from the first camera to the second camera to capture a target object, determine the grid position mapping relationship between the first image of the target object and the second image of the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; based on the grid position mapping relationship and the preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, determine the lens shadow compensation parameter corresponding to the second image; perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain the processed image of the target object corresponding to the second camera. That is to say, in the present application, during the zoom switching process, the relative position relationship between different images captured by different cameras can be determined first, that is, the grid position mapping relationship is determined, and then the corresponding compensation intensity adjustment parameter is determined based on the grid position mapping relationship, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter, and finally the corresponding lens shadow compensation parameter is obtained to perform lens shadow compensation on the image, so as to achieve synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. It can be seen that the present application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve the image quality.

[0036] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application.

[0037] An embodiment of the present application provides an image shadow compensation method. This image shadow compensation method can be applied to an image shadow compensation device or an electronic device, and can also be applied to any terminal including an image shadow compensation device or an electronic device.

[0038] It can be understood that the image shadow compensation method proposed in the embodiment of the present application mainly includes a method for performing lens shadow compensation (LSC) on an image.

[0039] Furthermore, in the embodiment of the present application, the image shadow compensation method proposed in the embodiment of the present application can be applied to a scenario of multi-camera switching shooting. That is, the image shadow compensation method of the present application can be used for a scheme of performing LSC synchronous compensation on different cameras when multiple cameras perform zoom switching, so as to ensure the brightness consistency within the same FOV area of the images of different cameras during zoom switching.

[0040] Next, taking the image shadow compensation device as an example, an exemplary description of the image shadow compensation method proposed in the embodiment of the present application is given.

[0041] Furthermore, in the embodiment of the present application, Figure 2 is a schematic diagram of the implementation process of the image shadow compensation method proposed in the embodiment of the present application. As Figure 2 shown, the image shadow compensation method may include the following steps:

[0042] Step 101, in the case of switching from the first camera to the second camera to shoot a target object, determine the grid position mapping relationship between the first image of the target object and the second image of the target object; wherein, the first image is collected by the first camera, and the second image is collected by the second camera.

[0043] In the embodiment of the present application, in the case of switching from the first camera to the second camera to shoot a target object, the image shadow compensation device may first determine the grid position mapping relationship between the first image of the target object and the second image of the target object; wherein, the first image is collected by the first camera, and the second image is collected by the second camera.

[0044] In the embodiment of the present application, the image shadow compensation device may be configured with multiple different cameras. Among them, the number of cameras configured by the image shadow compensation device may be greater than or equal to 2. That is to say, in the present application, the image shadow compensation device may be a multi-camera device.

[0045] It can be understood that in the embodiments of the present application, each camera in the image shadow compensation device generally has different functions and characteristics in design to meet the shooting requirements in different scenarios. The present application does not specifically limit the performance and type of different cameras in the image shadow compensation device.

[0046] Exemplarily, in some embodiments, the multiple cameras configured in the image shadow compensation device may include, but are not limited to, at least two of the main camera, wide-angle camera, telephoto camera, macro camera, depth-of-field camera, and other cameras.

[0047] Among them, the main camera is the most basic camera, usually having a relatively high pixel count and good imaging quality. It is responsible for handling most of the daily shooting tasks, such as landscapes, portraits, etc. The parameters such as the sensor size, aperture size, and pixel count of the main camera are relatively high to ensure clear and delicate photos under various lighting conditions.

[0048] The wide-angle camera has a wider viewing angle than the main camera, can accommodate more scenery, and is suitable for shooting scenes such as landscapes and buildings. The wide-angle camera brings a stronger visual impact and can show a broader field of view. In addition, the wide-angle camera can also perform excellently when shooting in narrow spaces, making the photo more three-dimensional and spatial.

[0049] The telephoto camera is usually used to achieve optical zoom, can zoom in on distant scenery for shooting, and is suitable for shooting distant views, portrait close-ups, etc. The telephoto camera has a shallower depth of field, can highlight the subject, blur the background, and create a more professional shooting effect. The telephoto cameras of some high-end mobile phones also support high-power optical zoom, which can achieve a longer shooting distance while maintaining clarity.

[0050] The macro camera supports close focusing and can capture wonderful details of the microscopic world, such as flowers, insects, etc. The macro camera usually has a relatively high magnification ratio and good focusing performance, and can present a delicate and clear microscopic world.

[0051] The depth-of-field camera is mainly used to enhance the blurring effect of the photo, making the subject more prominent and the background more blurred. Through algorithm processing, the depth-of-field camera can achieve a more natural and soft blurring effect, enhancing the artistic sense of the photo.

[0052] Of course, in addition to the above common types of cameras, the image shadow compensation device can also be equipped with some special cameras, such as Time of Flight (ToF) lenses, movie lenses, etc. Among them, ToF lenses are mainly used to achieve three-dimensional perception and depth measurement, and can be used for functions such as augmented reality and face recognition. Movie lenses usually have high pixels and excellent color reproduction capabilities, and are suitable for shooting high-quality videos. Further, in the embodiments of the present application, the image shadow compensation device can be configured with a first camera and a second camera, where the first camera and the second camera are different. For example, the image shadow compensation device is configured with a main camera, a wide-angle camera, and a telephoto camera, the first camera is the main camera, and the second camera can be a wide-angle camera and / or a telephoto camera.

[0053] In the embodiments of the present application, the image shadow compensation device can collect images of a target object through the first camera and the second camera, and obtain a first image and a second image of the target object respectively.

[0054] That is to say, in the embodiments of the present application, the image shadow compensation device can collect a first image corresponding to the target object through the first camera, and can also collect a second image corresponding to the target object through the second camera.

[0055] It can be understood that, in the embodiments of the present application, the image shadow compensation device can perform zoom switching between different cameras. Among them, during the process of taking a picture of the target object, the zoom switching time from the first camera to the second camera can be ignored, that is, it can be considered that the first image and the second image are collected synchronously through the first camera and the second camera.

[0056] Further, in the embodiments of the present application, after collecting the first image and the second image corresponding to the target object through the first camera and the second camera respectively, the grid position mapping relationship between the first image and the second image can be further determined.

[0057] Among them, the grid position mapping relationship between the first image and the second image can be used to determine the corresponding relationship between the grid positions of the pixel points between the first image and the second image. Specifically, the grid position mapping relationship can be the corresponding relationship between the grid positions of the pixels in the grid image of the first image and the grid positions of the pixels in the grid image of the second image.

[0058] It can be understood that, in the embodiments of the present application, the first image collected through the first camera can be divided according to a preset grid radius parameter to obtain the grid image of the first image. At the same time, the second image collected through the second camera can also be divided according to the preset grid radius parameter to obtain the grid image of the second image.

[0059] In an embodiment of the present application, the preset grid parameters can be used to determine the size of the grid. Among them, the preset grid parameters can include the radius parameter in the horizontal direction of the grid and the radius parameter in the vertical direction of the grid. Or, it can be understood as the radius parameter of the grid length and the radius parameter of the grid width.

[0060] Exemplarily, in some embodiments, assume that the preset grid radius parameter is the radius parameter M in the horizontal direction and the radius parameter N in the vertical direction. That is, the preset grid parameters can be expressed as M×N. Finally, the size of each grid obtained after dividing according to the preset grid radius parameter is M×N. Among them, both M and N are greater than 0, and M and N can be the same or different. The present application does not specifically limit the values of M and N.

[0061] Exemplarily, in some embodiments, assume that the preset grid radius parameter is the radius parameter 64 in the horizontal direction and the radius parameter 48 in the vertical direction. The size of each grid obtained after dividing according to the preset grid radius parameter is 64×48.

[0062] Exemplarily, in some embodiments, assume that the preset grid radius parameter is the radius parameter 32 in the horizontal direction and the radius parameter 32 in the vertical direction. The size of each grid obtained after dividing according to the preset grid radius parameter is 32×32.

[0063] Further, in an embodiment of the present application, after dividing the first image into the grid image of the first image according to the preset grid radius parameter, for any one of the grids, the mean value of all pixels in the grid can be determined as the pixel value of the grid; at the same time, for the grid image of the first image, the grid position of any pixel can be determined based on the center coordinates of the grid where the pixel is located.

[0064] Further, in an embodiment of the present application, after dividing the second image into the grid image of the second image according to the preset grid radius parameter, for any one of the grids, the mean value of all pixels in the grid can be determined as the pixel value of the grid; at the same time, for the grid image of the second image, the grid position of any pixel can be determined based on the center coordinates of the grid where the pixel is located.

[0065] Further, in an embodiment of the present application, when determining the grid position mapping relationship between the first image of the target object and the second image of the target object, the zoom ratio parameter can be determined first based on the field of view angle parameter of the first image and the field of view angle parameter of the second image; at the same time, the offset parameter can be determined based on the positional relationship between the first camera and the second camera; then, the grid position mapping relationship between the first image and the second image can be determined based on the zoom ratio parameter and the offset parameter.

[0066] In an embodiment of the present application, the zoom ratio parameter can be understood as zoomRatio. The zoom ratio parameter can be used to determine the range within which the camera can adjust the focal length, and is usually expressed in the form of a ratio.

[0067] Exemplarily, in some embodiments, for a first image captured by a first camera and a second image captured by a second camera, the field of view angle parameter of the first image and the field of view angle parameter of the second image can be determined respectively first, and then a ratio operation can be performed using the field of view angle parameters of both to finally determine the corresponding zoom ratio parameter zoomRatio.

[0068] In an embodiment of the present application, the field of view angle parameter can be used to determine the specific size of the field of view FOV.

[0069] Exemplarily, in some embodiments, assuming that the first camera is the main camera and the second camera is the telephoto camera, the field of view angle parameter of the first image acquired by the first camera is 1000, and the field of view angle parameter of the second image acquired by the second camera is 100, then the ratio between the field of view angle parameter of the first image and the field of view angle parameter of the second image can be determined as the corresponding zoom ratio parameter zoomRatio, that is, zoomRatio = 10 can be calculated.

[0070] Exemplarily, in some embodiments, assuming that the first camera is the main camera and the second camera is the wide-angle camera, the field of view angle parameter of the first image acquired by the first camera is 1000, and the field of view angle parameter of the second image acquired by the second camera is 10000, then the ratio between the field of view angle parameter of the first image and the field of view angle parameter of the second image can be determined as the corresponding zoom ratio parameter zoomRatio, that is, zoomRatio = 0.1 can be calculated.

[0071] In an embodiment of the present application, there may be a positional difference between the first camera and the second camera. Therefore, for the first image and the second image of the same target object acquired by the first camera and the second camera, the pixel positions corresponding to the same content may have corresponding offsets, and thus an offset parameter needs to be determined. Among them, the offset parameter can be understood as the offset amount offset.

[0072] Exemplarily, in some embodiments, Figure 3 is a schematic diagram of the FOV difference between the first image and the second image proposed in the embodiment of the present application, as Figure 3As shown, assume that the first image is captured by the main camera, and the second image is captured by the telephoto camera. The large black frame can represent the size range of the FOV of the first image, and the small black frame can represent the size range of the FOV of the second image.

[0073] Furthermore, in the embodiments of the present application, after determining the zoom ratio parameter and the offset parameter respectively, based on the zoom ratio parameter and the offset parameter, the grid position mapping relationship between the first image and the second image can be further determined, that is, the mapping relationship between the grid positions of the pixels in the grid image after the first image is divided into grids and the grid positions of the pixels in the grid image after the second image is divided into grids is determined.

[0074] It can be understood that in the embodiments of the present application, since the first image and the second image are captured by different cameras for the same target object, the FOV of the first image and the FOV of the second image are different. Therefore, for the same pixel, its position in the first image is different from its position in the second image. Thus, it is necessary to determine the grid position mapping relationship between the first image and the second image, so that the position of the pixel in the second image can be accurately mapped to the first image based on this grid position mapping relationship.

[0075] It can be seen that in the embodiments of the present application, the grid position mapping relationship between the first image and the second image can be used to match and align the pixels in the first image and the pixels in the second image.

[0076] It can be understood that in the embodiments of the present application, assume that the grid position of the pixel in the grid image of the second image is (stats′ x , stats′ y ). Then, after mapping the second image to the first image, the grid position of the pixel in the grid image of the first image is (stats x , stats x ), where the correspondence between (stats′ x , stats′ y ) and (stats x , stats x ) is the grid position mapping relationship between the first image and the second image.

[0077] Exemplarily, in some embodiments, the grid position mapping relationship between the first image and the second image can be expressed by the following formula:

[0078] stats x = stats′ x *zoomRatio + offset (1)

[0079] stats y = stats' y *zoomRatio + offset(2)

[0080] Wherein, zoomRatio is the zoom ratio parameter and offset is the offset parameter.

[0081] Exemplarily, in some embodiments, Figure 4 is a schematic diagram of the grid position mapping relationship between the first image and the second image proposed by the embodiment of the present application. As Figure 4 shown, if it is selected to perform synchronous shadow compensation processing during the zoom switching process with the first image as the reference, then the corresponding relationship between the grid position stats of the pixels of the first image and the grid position stats' of the pixels of the second image can be determined based on the FOV of the first image and in combination with the FOV of the second image. For example, (stats' x , stats' y ) corresponds to (stats x , stats x ).

[0082] Step 102: Determine the compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter.

[0083] In the embodiments of the present application, after determining the grid position mapping relationship between the first image of the target object and the second image of the target object, the compensation intensity adjustment parameter corresponding to the second image can be further determined based on the grid position mapping relationship and the preset grid radius parameter.

[0084] In the embodiments of the present application, the compensation intensity adjustment parameter corresponding to the second image can be used to adaptively adjust the initially obtained initial shadow compensation gain parameter corresponding to the second camera during the lens shadow compensation of the second image. Wherein, since the compensation intensity adjustment parameter corresponding to the second image is determined based on the grid position mapping relationship between the first image and the second image, therefore, using the compensation intensity adjustment parameter corresponding to the second image to control the lens shadow compensation process of the second image can ensure synchronous lens shadow compensation for the first image and the second image.

[0085] Further, in the embodiments of the present application, when determining the compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter, for the grid position of any pixel point in the grid image of the second image, the mapped grid position corresponding to the grid position in the grid image of the first image is determined based on the grid position mapping relationship; the first distance parameter corresponding to the mapped grid position in the grid image of the first image is determined; and the compensation intensity adjustment parameter corresponding to the second image is determined based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image.

[0086] It can be understood that in the embodiments of the present application, for the grid position of any pixel point in the grid image of the second image, it can be determined based on the central coordinates of the grid where the pixel is located.

[0087] In the embodiments of the present application, the mapped grid position corresponding to the grid position of the pixel in the grid image of the second image in the grid image of the first image can be understood as the grid position corresponding to the pixel in the grid image of the second image after being mapped to the grid image of the first image.

[0088] Exemplarily, in some embodiments, Figure 5 is a schematic diagram of the mapped grid position proposed in the embodiments of the present application. As Figure 5 shown, assuming that the first camera is the main camera and the second camera is the telephoto camera, the first image is acquired by the main camera, and the second image is acquired by the telephoto camera. Then, through the grid position mapping relationship, the mapped grid position corresponding to the pixel in the grid image of the second image in the grid image of the first image can be determined.

[0089] Further, in the embodiments of the present application, after determining the mapped grid position, the first distance parameter corresponding to the mapped grid position in the grid image of the first image can be further determined according to the mapped grid position and the central position of the grid image of the first image.

[0090] It can be understood that in the embodiments of the present application, the first distance parameter corresponding to the mapped grid position in the grid image of the first image can be used to determine the distance between the mapped grid position and the central position of the grid image of the first image. Among them, the first distance parameter may include the horizontal distance and the vertical distance between the mapped grid position and the central position of the grid image of the first image.

[0091] Exemplarily, in some embodiments, it is assumed that the first camera is the main camera and the second camera is the telephoto camera. The first image is acquired by the main camera, and the second image is acquired by the telephoto camera. The first distance parameter corresponding to the mapped grid position in the grid image of the first image can be expressed as dist_w, including the horizontal distance dist_w x and the vertical distance dist_w y .

[0092] It can be understood that in the embodiments of the present application, by traversing the grid positions of each pixel in the grid image of the second image according to the above method, the mapped grid position corresponding to each pixel mapped to the grid image of the first image can be determined based on the grid position mapping relationship, and then the first distance parameter corresponding to the mapped grid position of each pixel in the grid image of the first image can be determined.

[0093] Furthermore, in the embodiments of the present application, when determining the compensation intensity adjustment parameter corresponding to the second image based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the first adjustment parameter and the second adjustment parameter can be determined first based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image; then the compensation intensity adjustment parameter corresponding to the second image can be determined based on the first adjustment parameter and / or the second adjustment parameter.

[0094] Furthermore, in the embodiments of the present application, the brightness ratio parameter corresponding to the first image can be determined by the brightness parameters at different grid positions in the grid image of the first image. For example, the first brightness parameter corresponding to the grid edge position in the grid image of the first image and the second brightness parameter corresponding to the grid center position in the grid image of the first image can be determined respectively; then the brightness ratio parameter corresponding to the first image can be determined based on the first brightness parameter and the second brightness parameter.

[0095] It can be understood that in the embodiments of the present application, the brightness parameter corresponding to the grid position is the brightness value of the pixel point corresponding to the grid position.

[0096] Of course, in the embodiments of the present application, when determining the brightness ratio parameter corresponding to the first image, the brightness parameters corresponding to other grid positions can also be used, not limited to the grid edge position and / or the grid center position, and the present application does not make specific limitations.

[0097] Exemplarily, in some embodiments, Figure 6 is a schematic diagram for determining the brightness ratio parameter proposed in the embodiments of the present application, as Figure 6As shown, the ratio ratio of the luminance value of the pixel point A at the grid edge position to the luminance value of the pixel point B at the grid center position in the grid image of the first image is used as the luminance ratio parameter.

[0098] Exemplarily, in some embodiments, Figure 7 is a schematic diagram for determining the luminance ratio parameter proposed in the embodiments of the present application. As Figure 7 shown, the ratio ratio of the luminance value of any grid position pixel point C to the luminance value of the pixel point B at the grid center position in the grid image of the first image is used as the luminance ratio parameter.

[0099] That is to say, in the embodiments of the present application, for any pixel point in the grid image of the second image, after determining the corresponding first distance parameter, in combination with the preset grid radius parameter and the luminance ratio parameter corresponding to the first image, the corresponding first adjustment parameter and / or the corresponding second adjustment parameter can be further determined.

[0100] It can be understood that in the embodiments of the present application, the compensation intensity adjustment parameter corresponding to the second image can be understood as the ratio map corresponding to the second image.

[0101] Exemplarily, in some embodiments, assume that the first camera is the main camera, the second camera is the telephoto camera, the first image is acquired by the main camera, the second image is acquired by the telephoto camera, and the first distance parameter corresponding to the mapped grid position in the grid image of the first image may include the horizontal distance dist_w x and the vertical distance dist_w y , the preset grid radius parameter radius includes the radius parameter radius in the horizontal direction x and the radius parameter radius in the vertical direction y , the luminance ratio parameter corresponding to the first image is ratio, and for the pixel point in the grid image of the second image, the corresponding first adjustment parameter ratioMap_w can be determined with reference to the following formula 1 :

[0102]

[0103] Exemplarily, in some embodiments, assume that the first camera is the main camera, the second camera is the telephoto camera, the first image is acquired by the main camera, the second image is acquired by the telephoto camera, and the first distance parameter corresponding to the mapped grid position in the grid image of the first image may include the horizontal distance dist_w x and the vertical distance dist_w y, the preset grid radius parameter radius includes the radius parameter radius in the horizontal direction x and the radius parameter radius in the vertical direction y , the brightness ratio parameter corresponding to the first image is ratio. For the pixel points in the grid image of the second image, the corresponding second adjustment parameter ratioMap_w can be determined with reference to the following formula 2 :

[0104]

[0105] That is to say, in the embodiments of the present application, the first adjustment parameter and the second adjustment parameter can be calculated according to different mathematical operation methods based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. Among them, the calculation methods of the adjustment parameters are not limited to the above square root calculation, square calculation, etc.

[0106] Furthermore, in the embodiments of the present application, when determining the compensation intensity adjustment parameter corresponding to the second image based on the first adjustment parameter and / or the second adjustment parameter, it is possible to directly determine the first adjustment parameter as the compensation intensity adjustment parameter corresponding to the second image; or, it is also possible to directly determine the second adjustment parameter as the compensation intensity adjustment parameter corresponding to the second image; or, it is also possible to determine the compensation intensity adjustment parameter corresponding to the second image based on the preset weight, the first adjustment parameter, and the second adjustment parameter.

[0107] It can be understood that, in the embodiments of the present application, the calculated first adjustment parameter or second adjustment parameter can be directly determined as the compensation intensity adjustment parameter finally used for adaptively adjusting the lens shadow compensation intensity, or it is also possible to perform a weighted operation on the calculated first adjustment parameter and second adjustment parameter to obtain the final compensation intensity adjustment parameter.

[0108] Exemplarily, in some embodiments, assume that the preset weight is w 1 , the first adjustment parameter is ratioMap_w 1 , the second adjustment parameter is ratioMap_w 2 , then with reference to the following formula, the preset weight can be used to perform a weighted operation on the first adjustment parameter and the second adjustment parameter to determine the final compensation intensity adjustment parameter ratioMap_w;

[0109] ratioMap_w = w 1 *ratioMap_w 1 +(1.0 - w 1 )*ratioMap_w 2 (5)

[0110] It can be understood that in the embodiments of the present application, a compensation intensity adjustment parameter is calculated based on the first distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. For the pixel points in the second image, it shows a gradually decreasing trend from the center to the periphery, which is close to the brightness change trend of the lens shadow image. Among them, a preset weight can also be introduced to adjust the proportion of the first adjustment parameter and the second adjustment parameter, and flexibly adjust the speed of the ratio decay from the center to the periphery, so as to better adapt to the change requirements of different scenarios.

[0111] Step 103: Determine the lens shadow compensation parameter corresponding to the second image based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera.

[0112] In the embodiments of the present application, after determining the compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relationship and the preset grid radius parameter, the lens shadow compensation parameter corresponding to the second image can be further determined based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera.

[0113] It can be understood that in the embodiments of the present application, the compensation intensity adjustment parameter corresponding to the second image can be used to adjust the shadow compensation intensity during the lens shadow compensation of the second image. Therefore, the initial shadow compensation gain parameter corresponding to the second camera can be adjusted according to the compensation intensity adjustment parameter corresponding to the second image, so that the adjusted lens shadow compensation parameter corresponding to the second image can be obtained. Among them, the lens shadow compensation parameter corresponding to the second image can be used to perform lens shadow compensation on the second image.

[0114] In the embodiments of the present application, the initial shadow compensation gain parameter corresponding to the second camera can be pre-generated by the image shadow compensation device, or can be generated by other devices or apparatuses and sent to the image shadow compensation device, which is not specifically limited in the present application.

[0115] It can be understood that in the embodiments of the present application, the initial shadow compensation gain parameter corresponding to the second camera can be understood as the LSC brightness full compensation gain gain map corresponding to the second camera.

[0116] Exemplarily, in some embodiments, lens shadow compensation calibration can be performed on each camera in advance to calculate the LSC brightness full compensation gain gain map. For example, based on the wide-angle camera, the main camera, and the telephoto camera, white field images are captured under a uniform light source. The images can be divided into grids according to the specifications of the preset grid radius parameter (M×N). The pixel mean value in each grid is statistically calculated to obtain the mean statistics (stats). Then, stats shows a distribution trend of being bright in the center and dark around. Taking the center value of stats as the brightness compensation benchmark, calculate the stats gain gain map when each value of stats is fully compensated to the center brightness value of stats.

[0117] Exemplarily, in some embodiments, for the pixel point (i,j), the initial shadow compensation gain parameter gain corresponding to the second camera (i,j) can refer to the following formula:

[0118]

[0119] where stats (icenter,jcenter) is the center statistical value, and stats (i,j) is the statistical value corresponding to the pixel point (i,j).

[0120] Furthermore, in the embodiments of the present application, based on the corresponding relationship of the grid positions, the compensation intensity adjustment parameter and the corresponding initial shadow compensation gain parameter can be multiplied, and then interpolation operation is performed based on the product result to finally obtain the lens shadow compensation parameter corresponding to the second image.

[0121] Exemplarily, in some embodiments, the compensation intensity adjustment parameter corresponding to the second image can be understood as the ratio map corresponding to the second image, and the initial shadow compensation gain parameter corresponding to the second camera can be understood as the full brightness compensation gain gain map corresponding to the second camera. Multiply the ratio map and the full brightness compensation gain gain map based on the grid positions to obtain the LSC adjustment compensation gain correction map, and interpolate and enlarge the correction map to the full image size to obtain the lens shadow compensation parameter that meets the size of the second image.

[0122] Step 104: Perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0123] In an embodiment of the present application, after determining the lens shadow compensation parameter corresponding to the second image based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, the second image may be further subjected to shadow compensation processing based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

[0124] It can be understood that, in an embodiment of the present application, the lens shadow compensation parameter corresponding to the second image is obtained by adjusting the initial shadow compensation gain parameter corresponding to the second camera based on the compensation intensity adjustment parameter corresponding to the second image, wherein the compensation intensity adjustment parameter corresponding to the second image is determined based on the grid position mapping relationship between the first image and the second image. Therefore, applying the lens shadow compensation parameter corresponding to the second image to the second image for LSC lens shadow correction can ensure synchronous lens shadow compensation for the first image and the second image, that is, ensure that the brightness consistency is also maintained in the same FOV area between multiple cameras.

[0125] Furthermore, in an embodiment of the present application, Figure 8 is a schematic flowchart of the implementation process of the image shadow compensation method proposed in the embodiment of the present application, as Figure 8 shown, the image shadow compensation method may include the following steps:

[0126] Step 105: For the grid position of any pixel point in the grid image of the first image, determine the second distance parameter corresponding to the grid position in the grid image of the first image.

[0127] In an embodiment of the present application, after dividing the first image into the grid image of the first image according to the preset grid radius parameter, for the grid position of any pixel point in the grid image of the first image, determine the second distance parameter corresponding to the grid position in the grid image of the first image.

[0128] It can be understood that, in an embodiment of the present application, for the grid position of any pixel point in the grid image of the first image, it can be determined based on the center coordinates of the grid where the pixel is located.

[0129] Furthermore, in an embodiment of the present application, after determining the grid position of any pixel point, the second distance parameter corresponding to the grid position in the grid image of the first image may be further determined according to the grid position and the center position of the grid image of the first image.

[0130] It can be understood that, in the embodiments of the present application, the grid position corresponds to the second distance parameter of the grid image of the first image, and can be used to determine the distance between the grid position and the central position of the grid image of the first image. Among them, the second distance parameter may include the horizontal distance and the vertical distance between the grid position and the central position of the grid image of the second image.

[0131] Exemplarily, in some embodiments, it is assumed that the first camera is the main camera, the first image is acquired by the main camera, and the grid position of any pixel point in the grid image of the first image corresponding to the second distance parameter of the grid image of the first image can be expressed as dist, including the horizontal distance dist x and the vertical distance dist y .

[0132] It can be understood that, in the embodiments of the present application, by traversing the grid positions of each pixel in the grid image of the first image according to the above method, the second distance parameter corresponding to the grid position of each pixel can be determined.

[0133] Step 106: Determine the compensation intensity adjustment parameter corresponding to the first image based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image.

[0134] In the embodiments of the present application, for the grid position of any pixel point in the grid image of the first image, after determining the second distance parameter corresponding to the grid position of the grid image of the first image, the compensation intensity adjustment parameter corresponding to the first image can be further determined based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image.

[0135] Further, in the embodiments of the present application, when determining the compensation intensity adjustment parameter corresponding to the first image based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image, the third adjustment parameter and the fourth adjustment parameter can be first determined based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image; then the compensation intensity adjustment parameter corresponding to the first image can be determined based on the third adjustment parameter and / or the fourth adjustment parameter.

[0136] Further, in the embodiments of the present application, the brightness ratio parameter corresponding to the first image can be determined by the brightness parameters at different grid positions in the grid image of the first image. For example, the first brightness parameter corresponding to the grid edge position in the grid image of the first image and the second brightness parameter corresponding to the grid center position in the grid image of the first image can be determined respectively; then the brightness ratio parameter corresponding to the first image can be determined based on the first brightness parameter and the second brightness parameter.

[0137] That is to say, in the embodiments of the present application, for any pixel point in the grid image of the first image, after determining the corresponding first distance parameter, in combination with the preset grid radius parameter and the brightness ratio parameter corresponding to the first image, the corresponding third adjustment parameter and / or the corresponding fourth adjustment parameter can be further determined.

[0138] It can be understood that in the embodiments of the present application, the compensation intensity adjustment parameter corresponding to the first image can be understood as the ratio map corresponding to the first image.

[0139] Exemplarily, in some embodiments, assume that the first camera is the main camera and the second camera is the telephoto camera. The first image is acquired by the main camera, and the second image is acquired by the telephoto camera. The second distance parameter corresponding to the grid position in the grid image of the first image may include the horizontal distance dist x and the vertical distance dist y , the preset grid radius parameter radius includes the radius parameter radius x in the horizontal direction and the radius parameter radius y in the vertical direction. The brightness ratio parameter corresponding to the first image is ratio. For the pixel points in the grid image of the first image, the corresponding third adjustment parameter ratioMap can be determined with reference to the following formula 1 :

[0140]

[0141] Exemplarily, in some embodiments, assume that the first camera is the main camera and the second camera is the telephoto camera. The first image is acquired by the main camera, and the second image is acquired by the telephoto camera. The second distance parameter corresponding to the mapped grid position in the grid image of the first image may include the horizontal distance dist x and the vertical distance dist y , the preset grid radius parameter radius includes the radius parameter radius x in the horizontal direction and the radius parameter radius y in the vertical direction. The brightness ratio parameter corresponding to the first image is ratio. For the pixel points in the grid image of the first image, the corresponding fourth adjustment parameter ratioMap2 can be determined with reference to the following formula:

[0142]

[0143] That is to say, in the embodiments of the present application, the third adjustment parameter and the fourth adjustment parameter can be calculated according to different mathematical operation methods based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. Among them, the calculation methods of the adjustment parameters are not limited to the above square root calculation, square calculation, etc.

[0144] Furthermore, in the embodiments of the present application, when determining the compensation intensity adjustment parameter corresponding to the first image based on the third adjustment parameter and / or the fourth adjustment parameter, it is possible to directly determine the third adjustment parameter as the compensation intensity adjustment parameter corresponding to the first image; or, it is also possible to directly determine the fourth adjustment parameter as the compensation intensity adjustment parameter corresponding to the first image; or, it is further possible to determine the compensation intensity adjustment parameter corresponding to the first image based on the preset weight, the third adjustment parameter, and the fourth adjustment parameter.

[0145] It can be understood that, in the embodiments of the present application, the calculated third adjustment parameter or fourth adjustment parameter can be directly determined as the compensation intensity adjustment parameter finally used for adaptively adjusting the lens shadow compensation intensity, or it is also possible to perform a weighted operation on the calculated third adjustment parameter and fourth adjustment parameter to obtain the final compensation intensity adjustment parameter.

[0146] Exemplarily, in some embodiments, assume that the preset weight is w 1 , the third adjustment parameter is ratioMap 1 , the fourth adjustment parameter is ratioMap 2 , then the following formula can be referred to, and the preset weight is used to perform a weighted operation on the third adjustment parameter and the fourth adjustment parameter to determine the final compensation intensity adjustment parameter ratioMap:

[0147] ratioMap = w 1 *ratioMap 1 +(1.0 - w 1 )*ratioMap 2 (9)

[0148] It can be understood that, in the embodiments of the present application, the compensation intensity adjustment parameter is calculated based on the second distance parameter, the preset grid radius parameter, and the brightness ratio parameter corresponding to the first image. For the pixel points in the first image, it shows a gradually decreasing trend from the center to the periphery, which is close to the brightness change trend of the lens shadow image. Among them, a preset weight can also be introduced to adjust the proportion of the third adjustment parameter and the fourth adjustment parameter, and flexibly adjust the speed of the ratio decay from the center to the periphery, so as to better adapt to the change requirements of different scenarios.

[0149] Step 107: Determine the lens shadow compensation parameter corresponding to the first image based on the compensation intensity adjustment parameter corresponding to the first image and the initial shadow compensation gain parameter corresponding to the first camera.

[0150] In an embodiment of the present application, after determining the compensation intensity adjustment parameter corresponding to the first image, the lens shadow compensation parameter corresponding to the first image can be further determined based on the compensation intensity adjustment parameter corresponding to the first image and the initial shadow compensation gain parameter corresponding to the first camera.

[0151] It can be understood that in an embodiment of the present application, the compensation intensity adjustment parameter corresponding to the first image can be used to adjust the shadow compensation intensity during the lens shadow compensation of the first image. Therefore, the initial shadow compensation gain parameter corresponding to the first camera can be adjusted according to the compensation intensity adjustment parameter corresponding to the first image, so that the adjusted lens shadow compensation parameter corresponding to the first image can be obtained. Among them, the lens shadow compensation parameter corresponding to the first image can be used to perform lens shadow compensation on the first image.

[0152] In an embodiment of the present application, the initial shadow compensation gain parameter corresponding to the first camera can be pre-generated by the image shadow compensation device, or can be generated by other devices or devices and sent to the image shadow compensation device, which is not specifically limited in this application.

[0153] It can be understood that in an embodiment of the present application, the initial shadow compensation gain parameter corresponding to the first camera can be understood as the LSC brightness full compensation gain gain map corresponding to the first camera.

[0154] Exemplarily, in some embodiments, lens shadow compensation calibration can be performed on each camera in advance to calculate the LSC brightness full compensation gain gain map. For example, based on a wide-angle camera, a main camera, and a telephoto camera, white field images are taken under a uniform light source. The images can be divided into grids according to the specifications of a preset grid radius parameter (M×N). The pixel mean value in each grid is statistically calculated to obtain the mean statistics (stats). Then, stats shows a distribution trend of bright in the center and dark around. Taking the center value of stats as the brightness compensation reference, calculate the stats gain gain map when the brightness of each value of stats is fully compensated to the center brightness value of stats.

[0155] Exemplarily, in some embodiments, for the pixel point (i,j), the initial shadow compensation gain parameter gain corresponding to the first camera (i,j) can refer to formula (6).

[0156] Further, in the embodiments of the present application, based on the corresponding relationship of grid positions, a multiplication operation can be performed on the compensation intensity adjustment parameter and the corresponding initial shadow compensation gain parameter, and then an interpolation operation can be performed based on the multiplication result to finally obtain the lens shadow compensation parameter corresponding to the first image.

[0157] Exemplarily, in some embodiments, the compensation intensity adjustment parameter corresponding to the first image can be understood as the ratio map corresponding to the first image, and the initial shadow compensation gain parameter corresponding to the first camera can be understood as the full brightness filling gain gain map corresponding to the first camera. Multiply the ratio map and the full brightness filling gain gain map based on the grid positions to obtain the LSC adjustment compensation gain correction map, and interpolate and enlarge the correction map to the full image size to obtain the lens shadow compensation parameter that meets the size of the first image.

[0158] Step 108: Perform shadow compensation processing on the first image based on the lens shadow compensation parameter corresponding to the first image to obtain the processed image of the target object corresponding to the first camera.

[0159] In the embodiments of the present application, after determining the lens shadow compensation parameter corresponding to the first image based on the compensation intensity adjustment parameter corresponding to the first image and the initial shadow compensation gain parameter corresponding to the first camera, shadow compensation processing can be further performed on the first image based on the lens shadow compensation parameter corresponding to the first image to obtain the processed image of the target object corresponding to the first camera.

[0160] It can be understood that in the embodiments of the present application, the lens shadow compensation parameter corresponding to the first image is obtained by adjusting the initial shadow compensation gain parameter corresponding to the first camera based on the compensation intensity adjustment parameter corresponding to the first image. Applying the lens shadow compensation parameter corresponding to the first image to the first image for LSC lens shadow correction can achieve accurate lens shadow compensation.

[0161] Thus, in the embodiments of the present application, for the images collected by different cameras in the image shadow compensation device, the corresponding lens shadow compensation parameters can be determined respectively to adaptively adjust the initial shadow compensation gain parameters corresponding to their respective cameras. Among them, during the zoom switching process, the lens shadow compensation parameters corresponding to each camera can be determined according to the grid position mapping relationship between different images corresponding to different cameras. Therefore, during the process of performing lens shadow compensation on the corresponding images based on the lens shadow compensation parameters, the consistency of LSC compensation between multiple cameras can be ensured, making the image brightness smooth and flicker-free during multi-camera zoom switching.

[0162] It can be understood that in the embodiments of the present application, for the image serving as the reference, for example, the first image captured by the main camera is used as the reference. Then, the lens shadow compensation parameter corresponding to the first image does not need to be based on the grid position mapping relationship between different images, that is, the determination of the grid position mapping relationship is skipped. Instead, the lens shadow compensation parameter corresponding to the first image is directly determined, and the first image is compensated for lens shadow using this lens shadow compensation parameter.

[0163] In summary, the image shadow compensation method proposed in the embodiments of the present application is an LSC compensation intensity ratio map adjustment model based on radial information, which can flexibly adjust the LSC compensation intensity according to the changes in the scene environment and improve the image quality.

[0164] Furthermore, based on the above radial compensation intensity adjustment model, the image shadow compensation method proposed in the embodiments of the present application proposes an adaptive adjustment scheme for synchronously aligning the LSC compensation intensity of multiple cameras based on image registration under different FOV conditions, which controls the synchronization effect of the LSC compensation during multi-camera switching, and thus improves the user experience of multi-camera zooming.

[0165] It can be seen that the image shadow compensation method proposed in the embodiments of the present application restores the FOV position relationship between multiple cameras based on the image registration technology, and proposes a multi-camera LSC lens shadow correction synchronous compensation scheme based on the radial position. In difficult situations such as large differences in the optical lens structure, large differences in the zoom FOV, and large dynamic range spans, it can always ensure the consistency of the LSC compensation between multiple cameras in real time, making the image brightness smooth and flicker-free during multi-camera zooming switching, and greatly improving the image effect.

[0166] An embodiment of the present application proposes an image shadow compensation method. In the case of switching from the first camera to the second camera to capture a target object, the grid position mapping relationship between the first image of the target object and the second image of the target object is determined; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, the compensation intensity adjustment parameter corresponding to the second image is determined; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, the lens shadow compensation parameter corresponding to the second image is determined; based on the lens shadow compensation parameter corresponding to the second image, shadow compensation processing is performed on the second image to obtain the processed image of the target object corresponding to the second camera. That is to say, in the present application, during the zoom switching process, the relative position relationship between different images captured by different cameras can be first determined, that is, the grid position mapping relationship is determined, and then the corresponding compensation intensity adjustment parameter is determined based on this grid position mapping relationship, and the compensation intensity adjustment parameter is used to adjust the corresponding initial shadow compensation gain parameter, and finally the corresponding lens shadow compensation parameter is obtained to perform lens shadow compensation on the image, so as to achieve synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. It can be seen that the present application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zoom switching, and greatly improve the image quality.

[0167] Based on the above embodiment, another embodiment of the present application proposes an image shadow compensation method, and this image shadow compensation method includes a multi-camera LSC lens shadow synchronous compensation method. Among them, based on the LSC calibration brightness fully filled gain gainmap (initial shadow compensation gain parameter), a lens shadow compensation gain adjustment model based on radial position is designed. Through image registration, the FOV position relationship between different cameras such as wide-angle, main camera, and telephoto is restored, and the lens shadow compensation gain adjustment ratio map (compensation intensity adjustment parameter) of each camera in the same scene is calculated. After being combined with the brightness fully filled gain gain map, it is compensated onto the images of each multi-camera to achieve the effect of consistent brightness compensation alignment in the same area.

[0168] Among them, for the initial shadow compensation gain parameter of each camera, it can be pre-generated or pre-received from other devices or apparatuses, and the present application does not make specific limitations.

[0169] Exemplarily, in some embodiments, when initial shadow compensation gain parameters (brightness full compensation gain map) are used, lens shadow compensation calibration can be performed on each camera to calculate the LSC brightness full compensation gain map. The wide-angle, main camera, and telephoto lenses respectively capture white-field images under a uniform light source. The images are divided into grids according to the specifications of a preset grid radius parameter (M×N, for example, 64×48). The pixel mean value within each grid is statistically calculated to obtain the mean value statistics stats, and stats shows a distribution trend of being bright in the center and dark around. Taking the center value of stats as the brightness compensation benchmark, when calculating the brightness full compensation of each value of stats to be the same as the center brightness value of stats, the stats gain map is obtained. For the pixel point (i, j), the initial shadow compensation gain parameter gain corresponding to each camera (i,j) can refer to formula (6).

[0170] Exemplarily, in some embodiments, for the reference image, for example, taking the first image captured by the main camera as the reference, the lens shadow compensation parameter corresponding to the first image can be based on the distance relationship between the stats grid position and the image center on the corresponding brightness full compensation gain map (initial shadow compensation gain parameter), and a lens shadow brightness compensation intensity ratio map adjustment model is designed. Taking the ratio (brightness ratio parameter) of the brightness of the grid position point at the edge of stats to the brightness of the grid position point at the center of stats as the adjustment starting point, for the pixel point (i, j), the adjustment mode formula, that is, the third adjustment parameter ratioMap corresponding to the first image 1 (i, j) and the fourth adjustment parameter ratioMap 2 (i, j) respectively refer to the following formulas:

[0171]

[0172] where, dist(i, j) x is the horizontal distance of the grid position of the pixel point (i, j) corresponding to the grid image of the first image, and dist(i, j) y is the vertical distance of the grid position of the pixel point (i, j) corresponding to the grid image of the first image. The preset grid radius parameter radius includes the radius parameter radius x in the horizontal direction and the radius parameter radius y in the vertical direction. The brightness ratio parameter corresponding to the first image is ratio.

[0173] Further, in the embodiments of the present application, the third adjustment parameter may be directly determined as the compensation intensity adjustment parameter corresponding to the first image; alternatively, the fourth adjustment parameter may be directly determined as the compensation intensity adjustment parameter corresponding to the first image; or, the compensation intensity adjustment parameter corresponding to the first image may also be determined based on a preset weight, the third adjustment parameter, and the fourth adjustment parameter.

[0174] Exemplarily, in some embodiments, assume that the preset weight is w 1 , then the following formula can be referred to, and the preset weight is used to perform a weighted operation on the third adjustment parameter ratioMap 1 (i, j) and the fourth adjustment parameter ratioMap 2 (i, j) to determine the final compensation intensity adjustment parameter ratioMap(i, j):

[0175] ratioMap(i, j) = w 1 *ratioMap 1 (i, j) + (1.0 - w 1 ) * ratioMap 2 (i, j) (12)

[0176] It can be seen that the center point value of the ratio Map is 1.0, and the edge point value is ratio. This model shows a changing trend of gradually decaying from the center to the periphery, which is close to the brightness change trend of the lens shadow image. This adjustment mode can also adjust the weight between the square root ratioMap 1 (i, j) and the square ratioMap 1 (i, j) through the w 2 (i, j) parameter, and flexibly adjust the decay speed of the ratio of the model from the center to the periphery, better adapting to the changing requirements of different scenarios.

[0177] Exemplarily, in some embodiments, for a non-reference image, such as switching from the first camera to the second camera, and the second image collected by the second camera. Assume that the first camera is the main camera and the second camera is the wide-angle camera. That is, the ratio Map adjustment mode can be adapted to the wide-angle camera based on the main camera as the reference and according to the FOV size and position relationship.

[0178] That is to say, for the corresponding relationship of the FOV between multiple cameras, assuming the main camera coordinate system as the reference, the method of image registration between multiple cameras can be used to convert the coordinates of the stats grid (grid image) of the wide-angle camera (or telephoto camera, etc.) to the main camera FOV coordinate system by means of corresponding feature points, that is, to determine the grid position mapping relationship between the first image and the second image. The conversion process is as shown in formulas (1) and (2). Among them, zoomRatio is the zoom ratio parameter, and offset is the offset parameter. zoomRatio and offset are the coordinate conversion parameters for converting the wide-angle camera (or telephoto camera, etc.) obtained by image registration to the main camera coordinate system.

[0179] Exemplarily, in some embodiments, for non-reference images, based on the determined grid position mapping relationship between different images collected by different cameras, the ratio Map for LSC sync compensation alignment between multiple cameras is further calculated, that is, the corresponding compensation intensity adjustment parameter is determined based on the grid position mapping relationship.

[0180] Exemplarily, in some embodiments, assume that the first camera is the main camera, the second camera is the telephoto camera, the first image is obtained by the main camera, and the second image is obtained by the telephoto camera. The horizontal distance of the grid position corresponding to the pixel point (i, j) in the second image after mapping to the grid image of the first image is dist_w(i, j) x and the vertical distance dist_w(i, j) y .

[0181] Exemplarily, in some embodiments, assume that the first camera is the main camera, the second camera is the telephoto camera, the first image is obtained by the main camera, and the second image is obtained by the telephoto camera. For the pixel point (i, j), the horizontal distance of the grid position corresponding to the pixel point (i, j) after mapping to the grid image of the first image is dist_w(i, j) x and the vertical distance dist_w(i, j) y , the preset grid radius parameter radius includes the radius parameter radius x in the horizontal direction and the radius parameter radius y in the vertical direction. The brightness ratio parameter corresponding to the first image is ratio. The adjustment mode formula, that is, the first adjustment parameter ratioMap_w 1 (i, j) and the second adjustment parameter ratioMap_w 2 (i, j) respectively refer to the following formulas:

[0182]

[0183] Further, in the embodiments of the present application, the first adjustment parameter may be directly determined as the compensation intensity adjustment parameter corresponding to the second image; alternatively, the second adjustment parameter may be directly determined as the compensation intensity adjustment parameter corresponding to the second image; or, the compensation intensity adjustment parameter corresponding to the second image may also be determined based on a preset weight, the first adjustment parameter, and the second adjustment parameter.

[0184] Exemplarily, in some embodiments, assume that the preset weight is w 1 , then the following formula can be referred to, and the preset weight is used to perform a weighted operation on the first adjustment parameter ratioMap_w 1 (i, j) and the second adjustment parameter ratioMap_w 2 (i, j) to determine the final compensation intensity adjustment parameter ratioMap_w(i, j):

[0185] ratioMap_w(i, j) = w 1 *ratioMap_w 1 (i, j) + (1.0 - w 1 ) * ratioMap_w 2 (i, j)(15)

[0186] Further, multiply the ratio map of each multi-camera by the full-brightness filling gain gain map to obtain the LSC adjustment compensation gain correction map, and interpolate and enlarge the correction map to the full image size, and finally apply it to the multi-camera image. While completing the LSC lens shadow correction of each camera, the brightness consistency is also maintained in the same FOV area between the multi-cameras.

[0187] Further, in the embodiments of the present application, the correction conversion matrix color transfor matrix of the image color between the multi-cameras can also be calibrated according to the differences in the absorption characteristics of each camera module for different wavelength spectra, so that when the multi-camera switches and zooms, the LSC lens shadow compensation can also maintain the color consistency in the same FOV area, further improving the user's visual experience.

[0188] In summary, the embodiments of the present application propose an LSC compensation intensity ratio map adjustment model based on radial information, which can flexibly adjust the LSC compensation intensity according to the changes in the scene environment and improve the image quality.

[0189] Furthermore, based on the above radial compensation intensity adjustment model, the image shadow compensation method proposed in the embodiments of the present application presents an adaptive adjustment scheme for synchronously aligning the multi-camera LSC compensation intensity based on image registration under different FOV conditions, which controls the synchronization effect of the LSC compensation during multi-camera switching, thereby improving the user experience of multi-camera zooming.

[0190] It can be seen that the image shadow compensation method proposed in the embodiments of the present application restores the FOV position relationship between multiple cameras based on the image registration technology, and presents a multi-camera LSC lens shadow correction synchronous compensation scheme based on the radial position. In difficult situations such as large differences in optical lens structures, large differences in zoom FOVs, and large dynamic range spans, the consistency of the LSC compensation between multiple cameras can be ensured in real time, making the image brightness smooth and flicker-free during multi-camera zooming switching, and greatly improving the image effect.

[0191] The embodiments of the present application propose an image shadow compensation method. When switching from the first camera to the second camera to capture a target object, the grid position mapping relationship between the first image of the target object and the second image of the target object is determined; wherein, the first image is collected by the first camera, and the second image is collected by the second camera; based on the grid position mapping relationship and the preset grid radius parameter, the compensation intensity adjustment parameter corresponding to the second image is determined; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, the lens shadow compensation parameter corresponding to the second image is determined; based on the lens shadow compensation parameter corresponding to the second image, shadow compensation processing is performed on the second image to obtain the processed image of the target object corresponding to the second camera. That is to say, in the present application, during the zooming switching process, the relative position relationship between different images collected by different cameras can be determined first, that is, the grid position mapping relationship is determined, and then the corresponding compensation intensity adjustment parameter is determined based on the grid position mapping relationship, and the compensation intensity adjustment parameter is used to process the corresponding initial shadow compensation gain parameter, and finally the corresponding lens shadow compensation parameter is obtained to perform lens shadow compensation on the image, so as to achieve synchronous compensation for different images corresponding to different cameras during the lens shadow compensation process. It can be seen that the present application can flexibly adjust the LSC compensation intensity of different cameras through the grid position mapping relationship between different images, solve the problem of brightness jump during zooming switching, and greatly improve the image quality.

[0192] Based on the above embodiments, in another embodiment of the present application, Figure 9 is a schematic structural diagram of the composition of the image shadow compensation device proposed in the embodiments of the present application. As Figure 9 shown, the image shadow compensation device 110 proposed in the embodiments of the present application may include:

[0193] A determination unit 1101, configured to determine a grid position mapping relationship between a first image of a target object and a second image of the target object when switching from a first camera to a second camera to capture the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; determine a compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relationship and a preset grid radius parameter; determine a lens shadow compensation parameter corresponding to the second image based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera;

[0194] A compensation unit 1102, configured to perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain a processed image of the target object corresponding to the second camera.

[0195] In an embodiment of the present application, further, Figure 10 is a schematic structural diagram of a composition of an electronic device proposed in an embodiment of the present application, as Figure 10 shown, the electronic device 120 proposed in an embodiment of the present application may include a processor 1201, a memory 1202, a communication interface 1203, and a bus 1204 for connecting the processor 1201, the memory 1202, and the communication interface 1203.

[0196] In an embodiment of the present application, the above-mentioned processor 1201 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic devices for implementing the functions of the above-mentioned processor may also be others, and the embodiments of the present application do not make specific limitations. The electronic device 120 may further include a memory 1202, and the memory 1202 may be connected to the processor 1201, wherein the memory 1202 is used to store executable program codes, and the program codes include computer operation instructions. The memory 1202 may include a high-speed RAM memory and may also include non-volatile memory, for example, at least two disk memories.

[0197] In an embodiment of the present application, the bus 1204 is used to connect the communication interface 1203, the processor 1201, and the memory 1202 and enable mutual communication between these components.

[0198] In practical applications, the above-mentioned memory 1202 can be a volatile memory, such as a Random-Access Memory (RAM); or a non-volatile memory, such as a Read-Only Memory (ROM), a flash memory, a Hard Disk Drive (HDD), or a Solid-State Drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 1201.

[0199] Further, in an embodiment of the present application, the processor 1201 is configured to determine the grid position mapping relationship between the first image of the target object and the second image of the target object when switching from the first camera to the second camera to capture the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, determine the lens shadow compensation parameter corresponding to the second image; perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0200] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional module.

[0201] When the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0202] An embodiment of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the image shadow compensation method as described above.

[0203] Specifically, the program instructions corresponding to an image shadow compensation method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to an image shadow compensation method in the storage medium are read or executed by an electronic device, the following steps are included:

[0204] In the case of switching from the first camera to the second camera to capture a target object, determine the grid position mapping relationship between the first image of the target object and the second image of the target object; wherein, the first image is captured by the first camera, and the second image is captured by the second camera;

[0205] Based on the grid position mapping relationship and the preset grid radius parameter, determine the compensation intensity adjustment parameter corresponding to the second image;

[0206] Based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, determine the lens shadow compensation parameter corresponding to the second image;

[0207] Perform shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image to obtain the processed image of the target object corresponding to the second camera.

[0208] An embodiment of the present application also provides a computer program product.

[0209] In some embodiments, the computer program product may include a computer program or instructions.

[0210] In some embodiments, the computer program product can be applied to the computer device in the embodiments of the present application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the computer device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be described in detail here.

[0211] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) that contain computer-usable program code.

[0212] The present application is described with reference to the schematic flow diagrams and / or block diagrams of the implementation processes of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the schematic flow diagrams and / or block diagrams can be implemented by computer program instructions, and the combination of the processes and / or blocks in the schematic flow diagrams and / or block diagrams can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in one or more of the Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0213] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the specified functions in one or more of the Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0214] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer-implemented processing. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one or more of the Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0215] As described above, only the preferred embodiments of the present application are given, and they are not used to limit the protection scope of the present application.

Claims

1. An image shadow compensation method, characterized in that: The method is applied to an image shading compensation device, the image shading compensation device is configured with a first camera and a second camera, and the method comprises: In the case of switching from the first camera to the second camera to shoot a target object, determining a grid position mapping relationship between a first image of the target object and a second image of the target object; wherein the first image is captured by the first camera, and the second image is captured by the second camera; Determining a compensation intensity adjustment parameter corresponding to the second image based on the grid position mapping relationship and a preset grid radius parameter; Determining a lens shading compensation parameter corresponding to the second image based on a compensation intensity adjustment parameter corresponding to the second image and an initial shading compensation gain parameter corresponding to the second camera; Performing shadow compensation processing on the second image based on the lens shadow compensation parameter corresponding to the second image, to obtain a processed image of the target object corresponding to the second camera.

2. The method according to claim 1, characterized in that The determining of a grid position mapping relationship between the first image of the target object and the second image of the target object includes: Determining a zoom ratio parameter based on a field angle parameter of the first image and a field angle parameter of the second image; Determining an offset parameter based on a positional relationship between the first camera and the second camera; The grid position mapping relationship between the first image and the second image is determined based on the zoom ratio parameter and the offset parameter.

3. The method according to claim 1, characterized in that The determining, based on the grid position mapping relationship and the preset grid radius parameter, a compensation intensity adjustment parameter corresponding to the second image includes: For a grid position of any pixel point in the grid image of the second image, determining, based on the grid position mapping relationship, that the grid position corresponds to a mapped grid position of the grid image of the first image; Determining a first distance parameter of a grid image of the first image corresponding to the mapped grid position; Based on the first distance parameter, the preset grid radius parameter and the brightness ratio parameter corresponding to the first image, a compensation intensity adjustment parameter corresponding to the second image is determined.

4. The method according to claim 3, characterized in that The determining, based on the first distance parameter, the preset grid radius parameter and the brightness ratio parameter corresponding to the first image, a compensation intensity adjustment parameter corresponding to the second image includes: determining a first adjustment parameter and a second adjustment parameter based on the first distance parameter, the preset grid radius parameter, and a brightness ratio parameter corresponding to the first image; A compensation intensity adjustment parameter corresponding to the second image is determined based on the first adjustment parameter and / or the second adjustment parameter.

5. The method according to claim 4, characterized in that The determining the compensation intensity adjustment parameter corresponding to the second image based on the first adjustment parameter and / or the second adjustment parameter includes: determining the first adjustment parameter as the compensation intensity adjustment parameter corresponding to the second image; or, determining the second adjustment parameter as a compensation intensity adjustment parameter corresponding to the second image; or, Based on the preset weight, the first adjustment parameter and the second adjustment parameter, a compensation intensity adjustment parameter corresponding to the second image is determined.

6. The method according to any one of claims 1 to 5, characterized in that The determining, based on the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter corresponding to the second camera, a lens shadow compensation parameter corresponding to the second image includes: For a grid position of any pixel point in the grid image of the second image, multiplying the compensation intensity adjustment parameter corresponding to the second image and the initial shadow compensation gain parameter to obtain a product result corresponding to the grid position; A lens shading compensation parameter corresponding to the second image is determined based on the multiplication result.

7. The method according to claim 1, characterized in that The method further comprises: For a grid position of any pixel point in the grid image of the first image, determining a second distance parameter of the grid position corresponding to the grid image of the first image; determining a compensation intensity adjustment parameter corresponding to the first image based on the second distance parameter, the preset grid radius parameter, and a brightness ratio parameter corresponding to the first image; determining a lens shading compensation parameter corresponding to the first image based on a compensation intensity adjustment parameter corresponding to the first image and an initial shading compensation gain parameter corresponding to the first camera; Performing shading compensation processing on the first image based on the lens shading compensation parameters corresponding to the first image, to obtain a processed image of the target object corresponding to the first camera.

8. The method according to any one of claims 3 to 5 and 7, characterized in that: The method further comprises: Determine a first brightness parameter corresponding to a grid edge position in a grid image of the first image; Determine a second brightness parameter corresponding to a center position of a grid in a grid image of the first image; A brightness ratio parameter corresponding to the first image is determined based on the first brightness parameter and the second brightness parameter.

9. An image shading compensation device, characterized in that: The image shading compensation device comprises: A determination unit, configured to determine a grid position mapping relationship between a first image of the target object and a second image of the target object when switching from a first camera to a second camera to shoot the target object; wherein the first image is acquired by the first camera, and the second image is acquired by the second camera; based on the grid position mapping relationship and a preset grid radius parameter, determine a compensation intensity adjustment parameter corresponding to the second image; based on the compensation intensity adjustment parameter corresponding to the second image and an initial shadow compensation gain parameter corresponding to the second camera, determine a lens shading compensation parameter corresponding to the second image; The compensation unit is used to perform shadow compensation processing on the second image based on the lens shadow compensation parameters corresponding to the second image, so as to obtain a processed image of the target object corresponding to the second camera.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images