Image processing method, device, electronic device, and computer-readable storage medium

Abstract

The present application relates to an image processing method, apparatus, electronic device, and computer-readable storage medium. The method comprises: obtaining a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera; performing global correction processing on the second field of view image based on the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction; performing local correction processing on the globally corrected image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction; and fusing the intermediate image and the globally corrected image to obtain a target image. This method can reduce false colors in images and avoid image color distortion.

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of imaging technology, electronic devices often have multiple cameras for shooting. The final image is generated by fusing the images captured by the multiple cameras of the electronic device.

[0003] However, there are certain differences in the image quality of images captured by different cameras, which can easily lead to color distortion problems in the fused images. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can effectively reduce false colors and improve image quality.

[0005] An image processing method, comprising:

[0006] Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera;

[0007] Performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction;

[0008] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction;

[0009] The intermediate image and the global correction image are fused to obtain a target image.

[0010] An image processing device, comprising:

[0011] An acquisition module, configured to acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera;

[0012] a global correction module, configured to perform global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction;

[0013] a local correction module, configured to perform local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction;

[0014] A fusion module is used to fuse the intermediate image and the global correction image to obtain a target image.

[0015] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0016] Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera;

[0017] Performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction;

[0018] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction;

[0019] The intermediate image and the global correction image are fused to obtain a target image.

[0020] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0021] Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera;

[0022] Performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction;

[0023] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction;

[0024] The intermediate image and the global correction image are fused to obtain a target image.

[0025] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the following steps:

[0026] Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera;

[0027] Performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction;

[0028] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction;

[0029] The intermediate image and the global correction image are fused to obtain a target image.

[0030] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product obtain a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera, and the second field of view image is globally corrected according to the first field of view image to obtain a globally corrected image. The global correction includes at least one of brightness correction and color correction. The brightness correction can effectively reduce the brightness difference between images captured by multiple cameras, and the color correction can effectively reduce the color difference between images captured by multiple cameras, thereby effectively reducing the overall difference between each image. Based on the first field of view image, the globally corrected image is locally corrected to obtain an intermediate image. The local correction includes position correction. The position correction can further reduce the brightness and difference in the local area between the images captured by multiple cameras, making the correction more accurate, thereby further reducing false colors. The intermediate image and the globally corrected image are fused to obtain the target image, thereby avoiding the problem of color distortion of the fused image, thereby effectively improving the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A schematic diagram of an image processing circuit of an electronic device in one embodiment;

[0033] Figure 2 is a flowchart of an image processing method in one embodiment;

[0034] Figure 3is a flowchart of an image processing method in another embodiment;

[0035] Figure 4 Schematic diagram of the interface of wide view, narrow view and global correction image in one embodiment;

[0036] Figure 5 is a schematic diagram of an interface of a wide view, a narrow view, and a target image in one embodiment;

[0037] Figure 6 is a structural block diagram of an image processing device in one embodiment;

[0038] Figure 7 FIG. 4 is a structural block diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0040] The image processing method in the embodiments of the present application can be applied to electronic devices. The electronic devices can be various personal computers, laptops, smartphones, tablet computers, Internet of Things devices, and portable wearable devices with at least two cameras. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0041] In one embodiment, the electronic device may include an image processing circuit. The image processing circuit may be implemented using hardware and / or software components and may include various processing units that define an ISP (Image Signal Processing) pipeline. Figure 1 FIG. 1 is a schematic diagram of an image processing circuit in one embodiment. Figure 1 As shown, for ease of explanation, only various aspects of the image processing technology related to the embodiments of the present application are shown.

[0042] like Figure 1As shown, an image processing circuit for an electronic device with at least two cameras is provided. The image processing circuit includes a first ISP processor 130, a second ISP processor 140, and a control logic 150. The first camera 110 includes one or more first lenses 112 and a first image sensor 114. The first image sensor 114 may include a color filter array (such as a Bayer filter). The first image sensor 114 can obtain light intensity and wavelength information captured by each imaging pixel of the first image sensor 114 and provide a set of image data that can be processed by the first ISP processor 130. The second camera 120 includes one or more second lenses 122 and a second image sensor 124. The second image sensor 124 may include a color filter array (such as a Bayer filter). The second image sensor 124 can obtain light intensity and wavelength information captured by each imaging pixel of the second image sensor 124 and provide a set of image data that can be processed by the second ISP processor 140.

[0043] The first field of view image captured by the first camera 110 is transmitted to the first ISP processor 130 for processing. After processing the first field of view image, the first ISP processor 130 may send statistical data of the first image (such as image brightness, image contrast value, image color, etc.) to the control logic 150. The control logic 150 may determine the control parameters of the first camera 110 based on the statistical data, so that the first camera 110 may perform operations such as autofocus and autoexposure based on the control parameters. After being processed by the first ISP processor 130, the first field of view image may be stored in the image memory 160. The first ISP processor 130 may also read the image stored in the image memory 160 for processing. In addition, after being processed by the ISP processor 130, the first field of view image may be directly sent to the display 170 for display. The display 170 may also read the image in the image memory 160 for display.

[0044] The first ISP processor 130 processes image data on a pixel-by-pixel basis in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits. The first ISP processor 130 may perform one or more image processing operations on the image data and collect statistical information about the image data. The image processing operations may be performed at the same or different bit depths.

[0045] The image memory 160 may be a portion of a memory device, a storage device, or an independent dedicated memory within the electronic device, and may include a DMA (Direct Memory Access) feature.

[0046] When receiving image data from the first image sensor 114 interface, the first ISP processor 130 may perform one or more image processing operations, such as time domain filtering. The processed image data may be sent to the image memory 160 for further processing before being displayed. The first ISP processor 130 receives processed data from the image memory 160 and performs image data processing in RGB and YCbCr color spaces on the processed data. The image data processed by the first ISP processor 130 may be output to the display 170 for viewing by the user and / or further processed by a graphics engine or GPU (Graphics Processing Unit). In addition, the output of the first ISP processor 130 may also be sent to the image memory 160, and the display 170 may read the image data from the image memory 160. In one embodiment, the image memory 160 may be configured to implement one or more frame buffers.

[0047] The statistical data determined by the first ISP processor 130 may be sent to the control logic 150. For example, the statistical data may include first image sensor 114 statistics such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, and shading correction for the first lens 112. The control logic 150 may include a processor and / or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters for the first camera 110 and the first ISP processor 130 based on the received statistical data. For example, the control parameters for the first camera 110 may include gain, integration time for exposure control, anti-shake parameters, flash control parameters, control parameters for the first lens 112 (e.g., focal length for focus or zoom), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices used for auto-white balance and color adjustment (e.g., during RGB processing), as well as shading correction parameters for the first lens 112.

[0048] Similarly, the second field of view image captured by the second camera 120 is transmitted to the second ISP processor 140 for processing. After processing the second field of view image, the second ISP processor 140 may send statistical data of the second field of view image (such as image brightness, image contrast value, image color, etc.) to the control logic 150. The control logic 150 may determine the control parameters of the second camera 120 based on the statistical data, so that the second camera 120 can perform operations such as autofocus and autoexposure according to the control parameters. After being processed by the second ISP processor 140, the second field of view image may be stored in the image memory 160. The second ISP processor 140 may also read the image stored in the image memory 160 for processing. In addition, after being processed by the ISP processor 140, the second field of view image may be directly sent to the display 170 for display. The display 170 may also read the image in the image memory 160 for display. The second camera 120 and the second ISP processor 140 may also implement the processing process described for the first camera 110 and the first ISP processor 130.

[0049] In one embodiment, the field of view corresponding to the first camera 110 is different from the field of view corresponding to the second camera 120. The first ISP processor 130 and the second ISP processor 140 may be the same ISP processor.

[0050] The electronic device can capture images of any scene through the first camera 110 and the second camera 120 to obtain a first field of view image corresponding to the first camera 110 and a second field of view image corresponding to the second camera 120. The control logic 150 fuses the original compressed image with the first field of view image to obtain a fusion result. The control logic 150 performs global correction processing on the second field of view image based on the first field of view image to obtain a globally corrected image, where the global correction includes at least one of brightness correction and color correction. The control logic 150 performs local correction processing on the globally corrected image based on the first field of view image to obtain an intermediate image, where the local correction includes position correction. The intermediate image and the globally corrected image are fused to obtain a target image, thereby effectively reducing the brightness and color differences between images captured by multiple cameras by performing brightness correction and color correction on the images captured by multiple cameras, thereby effectively reducing the overall differences between the images, avoiding the problem of color distortion of the fused image, and effectively improving the image quality and visual effects.

[0051] In one embodiment, Figure 2 As shown, an image processing method is provided, which is applied to Figure 1 The electronic device in FIG is taken as an example to illustrate, including the following steps:

[0052] Step 202 : Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera.

[0053] The field of view (FOV) represents the maximum range a camera can observe, usually expressed in degrees. The larger the FOV, the wider the observation range. For example, some cameras have a FOV of 50 degrees, while others have a FOV of 58 degrees. Compared to a FOV of 50 degrees, a 58-degree FOV can capture a wider range of scenery at the same distance.

[0054] The first field of view image and the second field of view image can be any of the following: RGB (Red, Green, Blue) images, RAW images, grayscale images, depth images, YUV images, etc. Among them, the RAW image is the raw data that the image sensor converts the captured light source signal into a digital signal. The "Y" in the YUV image represents the brightness (Luminance or Luma), that is, the grayscale value, and "U" and "V" represent the chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0055] The first field of view image and the second field of view image can be images captured from any scene, such as, but not limited to, images of people, scenery, or industrial devices. It is understood that the first field of view image and the second field of view image are images captured by the first camera and the second camera, respectively, from the same scene.

[0056] It can be understood that the first field of view image and the second field of view image can be complete images or partial image areas in the image. For example, in other embodiments, the first field of view image is the first main area in the original field of view image captured by the first camera, and the second field of view image is the second main area in the original field of view image captured by the second camera.

[0057] Specifically, the electronic device captures an image of any scene using a first camera and a second camera to obtain a first field of view image corresponding to the first camera and a second field of view image corresponding to the second camera. If the field of view corresponding to the first camera is different from the field of view corresponding to the second camera, then the field of view corresponding to the first field of view image is different from the field of view corresponding to the second field of view image.

[0058] In one embodiment, the field of view corresponding to the first camera is different from the field of view corresponding to the second camera. For example, the field of view corresponding to the first camera may be larger than the field of view corresponding to the second camera, and the field of view corresponding to the first field of view image may be larger than the field of view corresponding to the second field of view image. For example, the first field of view image is a wide field of view image with a corresponding field of view of 58 degrees, while the second field of view image is a narrow field of view image with a corresponding field of view of 50 degrees.

[0059] Step 204 : performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction.

[0060] Specifically, the electronic device may perform at least one of brightness correction processing and color correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image.

[0061] In this embodiment, the electronic device may perform brightness correction processing on the second field of view image based on the first field of view image to obtain a globally corrected image. Further, the electronic device may perform brightness correction processing on the brightness information of the second field of view image based on the brightness information of the first field of view image to obtain a globally corrected image.

[0062] In this embodiment, the electronic device can perform color correction processing on the second field of view image based on the first field of view image to obtain a globally corrected image. Further, the electronic device can perform color correction processing on the color information of the second field of view image based on the color information of the first field of view image to obtain a globally corrected image.

[0063] In this embodiment, the electronic device may perform brightness correction and color correction on the second field of view image based on the first field of view image to obtain a globally corrected image. Furthermore, the electronic device may perform brightness correction on the second field of view image based on the first field of view image; and perform color correction on the image obtained by the brightness correction based on the first field of view image to obtain a globally corrected image. Alternatively, the electronic device may perform color correction on the second field of view image based on the first field of view image; and perform brightness correction on the image obtained by the color correction based on the first field of view image to obtain a globally corrected image.

[0064] Step 206 : Perform local correction processing on the global correction image based on the first field of view image to obtain an intermediate image. The local correction includes position correction.

[0065] Local correction refers to correcting a local area within a globally corrected image. Furthermore, local correction includes position correction, which refers to correcting a local area formed by multiple positions within the globally corrected image. Correction of a local area is achieved by correcting the pixels corresponding to the multiple positions within the local area.

[0066] Specifically, the electronic device can determine a second local area that needs to be corrected in the global correction image, determine a corresponding first local area from the first field of view image, and perform correction processing on the second local area based on the first local area to obtain an intermediate image.

[0067] When there are multiple second local areas in the global corrected image, the electronic device can determine the first local area corresponding to each second local area from the first field of view image, perform correction processing on the corresponding second local area based on the first local area, and after the correction of each second local area in the global corrected image is completed, an intermediate image can be obtained.

[0068] In one embodiment, after performing global correction processing on the second field of view image according to the first field of view image to obtain a global correction image, the method further includes: performing registration processing on the global correction image based on the first field of view image to obtain a corresponding registered image;

[0069] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, including: performing local correction processing on the registration image based on the first field of view image to obtain an intermediate image;

[0070] The intermediate image and the global correction image are fused to obtain the target image, including: the intermediate image and the registration image are fused to obtain the target image.

[0071] Step 208: Fusing the intermediate image and the global correction image to obtain a target image.

[0072] The fusion process can be global fusion or local fusion. Global fusion refers to fusing the global information of the intermediate image with the global information of the globally corrected image. Local fusion refers to fusing the local information of the intermediate image with the local information of the globally corrected image, or replacing the local information of the globally corrected image with the local information of the intermediate image.

[0073] Specifically, the electronic device may select local information from the intermediate image and select corresponding local information from the globally corrected image to generate a target image based on the selected local information. Furthermore, the electronic device may replace the selected local content in the globally corrected image with the corresponding local information selected from the intermediate image to obtain the target image.

[0074] In this embodiment, the local information may be low-frequency information. The electronic device may determine the low-frequency information corresponding to the intermediate image and the low-frequency information corresponding to the global corrected image, and generate a target image based on the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image.

[0075] In other embodiments, the global information includes pixel values ​​corresponding to all pixels in the image. The electronic device averages or weighted averages the pixel values ​​corresponding to matching pixels in the intermediate image and the global corrected image to obtain a target pixel value. The target image can be obtained based on the target pixel values ​​corresponding to each pixel.

[0076] In the above-mentioned image processing method, a first field of view image captured by a first camera and a second field of view image captured by a second camera are obtained. The field of view corresponding to the first camera is different from the field of view corresponding to the second camera. The second field of view image is globally corrected based on the first field of view image to obtain a globally corrected image. The global correction includes at least one of brightness correction and color correction. The brightness correction can effectively reduce the brightness difference between images captured by multiple cameras. The color correction can effectively reduce the color difference between images captured by multiple cameras, so as to effectively reduce the overall difference between each image. Based on the first field of view image, the globally corrected image is locally corrected to obtain an intermediate image. The local correction includes position correction. The position correction can further reduce the brightness and difference in the local area between the images captured by multiple cameras, making the correction more accurate, thereby further reducing false colors. The intermediate image and the globally corrected image are fused to obtain the target image, avoiding the problem of color distortion of the fused image, thereby effectively improving the image quality and visual effect.

[0077] In one embodiment, performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image includes:

[0078] Performing brightness correction processing on the second field of view image according to the first field of view image to obtain a brightness corrected image;

[0079] Color correction is performed on the brightness-corrected image according to the first field of view image to obtain a global-corrected image.

[0080] Specifically, the electronic device may obtain brightness information of the first field of view image and brightness information of the second field of view image, and perform brightness correction processing on the brightness information of the second field of view image based on the brightness information of the first field of view image to obtain a brightness-corrected image. Further, the electronic device may obtain brightness information of the first field of view image in a brightness channel and brightness information of the second field of view image in a brightness channel, and perform brightness correction processing on the brightness information of the second field of view image in a brightness channel based on the brightness information of the first field of view image in the brightness channel to obtain a brightness-corrected image.

[0081] The electronic device may obtain color information of the first field of view image and color information of the brightness correction image, and perform color correction processing on the color information of the brightness correction image based on the color information of the first field of view image to obtain a globally corrected image. Furthermore, the electronic device may obtain color information on color channels of the first field of view image and color information on color channels of the second field of view image, and perform color correction processing on the color information on color channels of the second field of view image based on the color information of the first field of view image to obtain a globally corrected image.

[0082] In this embodiment, the brightness information may be represented by the histogram information of the brightness channel, and the color information may be represented by the histogram information of the color channel. The brightness channel may include the "Y" channel in the YUV image, and the color channel may include at least one of the red channel, the green channel, and the blue channel.

[0083] In other embodiments, the second field of view image is globally corrected according to the first field of view image to obtain a globally corrected image, including: performing color correction on the second field of view image according to the first field of view image to obtain a color corrected image; and performing brightness correction on the brightness correction image according to the first field of view image to obtain a globally corrected image.

[0084] Specifically, the electronic device may obtain color information of the first field of view image and color information of the second field of view image, perform color correction processing on the color information of the second field of view image based on the color information of the first field of view image, and obtain a color-corrected image. The electronic device may obtain brightness information of the first field of view image and brightness information of the color-corrected image, and perform brightness correction processing on the brightness information of the brightness-corrected image based on the brightness information of the first field of view image, and obtain a globally corrected image.

[0085] Similarly, the electronic device may also perform color correction processing on the second field of view image first, and then process the brightness information after the color correction, which can also reduce the color and brightness differences between images captured by different cameras.

[0086] In this embodiment, using the brightness information of the first field of view image as a reference, brightness correction processing is performed on the second field of view image based on the first field of view image, effectively reducing the brightness difference between the two images captured by the first camera and the second camera, so that the brightness information of the second field of view image is adjusted to be consistent with the brightness information of the first field of view image. After brightness correction, using the color information of the first field of view image as a reference, color correction processing is performed on the brightness-corrected image based on the first field of view image, so that the color information of the brightness-corrected image is adjusted to be consistent with the color information of the first field of view image, effectively reducing the color difference between images captured by different cameras.

[0087] In one embodiment, performing brightness correction processing on the second field of view image according to the first field of view image to obtain a brightness corrected image includes:

[0088] First histogram information of a brightness channel of the first field of view image is counted, and second histogram information of a brightness channel of the second field of view image is counted; a brightness mapping relationship between the first field of view image and the second field of view image is determined based on the first histogram information and the second histogram information; and brightness correction processing is performed on the second field of view image based on the brightness mapping relationship to obtain a brightness-corrected image.

[0089] An image histogram is a histogram used to represent the brightness distribution in a digital image, plotting the number of pixels at each brightness value in the image. For example, if the grayscale value of an image is 0-255, the grayscale value range contains 256 values. This range can be divided into multiple sub-regions according to a certain pattern, each of which is called a bin. The brightness mapping relationship refers to the mapping relationship between the first field of view image and the second field of view image in the brightness channel.

[0090] Specifically, the electronic device may count the histogram information of the brightness channel of the first field of view image to obtain corresponding first histogram information, and the electronic device may count the histogram information of the brightness channel of the second field of view image to obtain corresponding second histogram information.

[0091] In this embodiment, determining the brightness mapping relationship between the first field of view image and the second field of view image according to the first histogram information and the second histogram information includes:

[0092] The first histogram information and the second histogram information are divided to obtain multiple first sub-regions corresponding to the first histogram information and multiple second sub-regions corresponding to the second histogram information; and a brightness mapping relationship between the first field of view image and the second field of view image is determined based on the multiple first sub-regions and the multiple second sub-regions.

[0093] Specifically, the electronic device may divide the first histogram information and the second histogram information in the same division manner to obtain a plurality of first sub-regions corresponding to the first histogram information and a plurality of second sub-regions corresponding to the second histogram information.

[0094] The first sub-region includes multiple brightness values ​​on the brightness channel of the first field of view image. For example, if 256 values ​​are divided into 8 first sub-regions, each first sub-region includes 32 brightness values. Alternatively, if 256 values ​​are divided into 32 first sub-regions, each first sub-region includes 8 brightness values. It can be understood that the number of divisions can be set according to specific needs and is not limited here. The second sub-region includes multiple brightness values ​​on the brightness channel of the second field of view image. The number of first sub-regions is the same as the number of second sub-regions. Each first sub-region corresponds to a second sub-region, that is, each first sub-region has a matching second sub-region.

[0095] The electronic device can determine the brightness mapping relationship between the first field of view image and the second field of view image based on multiple first sub-areas and multiple second sub-areas, including: selecting multiple first brightness values ​​from each first sub-area, and screening out second brightness values ​​that match the corresponding first brightness values ​​from the second sub-area. Based on the multiple first brightness values ​​and the matching second brightness values, the brightness mapping relationship between the brightness channels of the first field of view image and the brightness channels of the second field of view image can be calculated.

[0096] The first brightness value refers to the brightness value in the first subregion, and the second brightness value refers to the brightness value in the second subregion. The brightness values ​​of matching pixels in the first and second field of view images also match in the brightness channel. For example, the maximum brightness value in the first subregion matches the maximum brightness value in the first subregion.

[0097] For example, the maximum value and the minimum value in each first sub-region, and the maximum value and the minimum value in each second sub-region are selected; based on the maximum value and the minimum value in each first sub-region, and the maximum value and the minimum value in each second sub-region, the brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image is determined.

[0098] After obtaining the brightness mapping relationship, the electronic device can perform brightness correction processing on the brightness channel of the second field of view image using the brightness mapping relationship to obtain a brightness-corrected image. Further, the electronic device can perform brightness correction processing on the brightness value of each pixel point in the brightness channel of the second field of view image using the brightness mapping relationship to obtain a brightness-corrected image.

[0099] It is understandable that the electronic device can also select multiple second brightness values ​​from each second sub-area, and filter out first brightness values ​​that match the corresponding second brightness values ​​from the first sub-area. Based on the multiple first brightness values ​​and the matching second brightness values, the brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image can be calculated.

[0100] In this embodiment, the electronic device can calculate a brightness mapping relationship between the second sub-region and the first sub-region based on multiple second brightness values ​​and multiple first brightness values ​​selected from the second sub-region and the first sub-region. Based on this brightness mapping relationship, brightness correction processing can be performed on the channel values ​​of the brightness channel of the corresponding region in the second field of view image. Using the same processing method, a brightness mapping relationship can be obtained between each second sub-region and the corresponding first sub-region. Based on each brightness mapping relationship, brightness correction processing can be performed on the channel values ​​of the brightness channel of each region in the second field of view image, thereby obtaining a brightness-corrected image.

[0101] In this embodiment, the electronic device can perform guided filtering processing on the brightness channel of the first field of view image and the brightness channel of the second field of view image respectively, and then respectively count the first histogram information of the brightness channel of the first field of view image after filtering processing, and the second histogram information of the brightness channel of the second field of view image after filtering processing. The guided filtering processing can reduce the influence of noise.

[0102] In this embodiment, first histogram information of the brightness channel of the first field of view image is counted, and second histogram information of the brightness channel of the second field of view image is counted, so as to characterize the brightness distribution of the first field of view image and the second field of view image through the histogram, and thus determine the brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image based on the first histogram information and the second histogram information, so that the second field of view image can be brightness corrected through the brightness mapping relationship to adjust the brightness information of the second field of view image to be consistent with the brightness of the first field of view image to the greatest extent, thereby reducing the brightness difference between the two images.

[0103] In one embodiment, performing color correction processing on the color-corrected image according to the first field of view image to obtain a global correction image includes:

[0104] Counting the histogram information of the first field of view image and the second field of view image in each color channel of the first color space respectively; determining the color mapping relationship between the first field of view image and the second field of view image in the corresponding color channel according to the histogram information of the first field of view image and the second field of view image in each color channel;

[0105] Color correction processing is performed on the corresponding color channel of the second field of view image through the color mapping relationship corresponding to each color channel to obtain a global corrected image.

[0106] Among them, the histogram can also be used to represent the color distribution in a digital image, plotting the number of pixels in each color range in the image.

[0107] Specifically, the first color space corresponds to multiple color channels. The electronic device can separately calculate histogram information for each color channel of the first field of view image in the first color space, and separately calculate histogram information for each color channel of the second field of view image in the first color space. For example, the electronic device can separately calculate histogram information for the red channel, green channel, and blue channel of the first field of view image, and histogram information for the red channel, green channel, and blue channel of the second field of view image.

[0108] The electronic device determines the color mapping relationship between the first field of view image and the second field of view image on the same color channel based on the histogram information of the first field of view image and the second field of view image on the same color channel. For example, the electronic device determines the color mapping relationship between the first field of view image and the second field of view image on the red channel based on the histogram information of the first field of view image and the second field of view image on the red channel.

[0109] Following the same process, a color mapping relationship between the first and second field of view images for each color channel can be obtained, and color correction processing can be performed on the corresponding color channel of the second field of view image using the color mapping relationship corresponding to each color channel to obtain a globally corrected image. For example, color correction processing can be performed on the red channel of the second field of view image using the color mapping relationship corresponding to the red channel, color correction processing can be performed on the green channel of the second field of view image using the color mapping relationship corresponding to the green channel, and color correction processing can be performed on the blue channel of the second field of view image using the color mapping relationship corresponding to the blue channel to obtain a globally corrected image.

[0110] In this embodiment, determining the color mapping relationship between the first field of view image and the second field of view image in corresponding color channels according to the histogram information of the first field of view image and the second field of view image in each color channel includes:

[0111] The histogram information of the first field of view image and the second field of view image on each color channel is divided to obtain a plurality of third sub-regions corresponding to the first field of view image on each color channel, and a plurality of fourth sub-regions corresponding to the second field of view image on each color channel; and a color mapping relationship between the first field of view image and the second field of view image on the corresponding color channel is determined based on the plurality of third sub-regions and the plurality of fourth sub-regions of the same color channel.

[0112] Specifically, the electronic device may divide the histogram information of each color channel of the first field of view image in the same division method to obtain multiple third sub-regions corresponding to each color channel of the first field of view image. For example, multiple third sub-regions corresponding to the red channel of the first field of view image, multiple third sub-regions corresponding to the green channel, and multiple third sub-regions corresponding to the blue channel.

[0113] The electronic device may divide the histogram information of each color channel of the second field of view image in the same division method to obtain multiple third sub-regions corresponding to each color channel of the second field of view image. For example, multiple third sub-regions corresponding to the red channel of the second field of view image, multiple third sub-regions corresponding to the green channel, and multiple third sub-regions corresponding to the blue channel.

[0114] The third sub-region includes multiple color values ​​on a color channel of the first field of view image. For example, the color range of the red channel is 0-255, with a total of 256 color values. The 256 values ​​are divided into 32 third sub-regions, and each third sub-region corresponding to the red channel includes 8 color values. It can be understood that the number of divisions can be set according to specific needs and is not limited here. The fourth sub-region includes multiple color values ​​on the color channel of the second field of view image. The number of third sub-regions corresponding to the same color channel is the same as the number of fourth sub-regions. Each third sub-region of the same color channel corresponds to a fourth sub-region, that is, each third sub-region of the same color channel has a matching fourth sub-region.

[0115] In this embodiment, the color mapping relationship between the first field of view image and the second field of view image on the corresponding color channels is determined based on multiple third sub-areas and multiple fourth sub-areas of the same color channel, including: for each color channel, multiple first color values ​​are selected from each third sub-area of ​​the corresponding color channel, and second color values ​​matching each first color value are screened out from the fourth sub-area. Based on the multiple first color values ​​and the matching second color values, the color mapping relationship corresponding to the first field of view image and the second field of view image on the corresponding color channels can be calculated to obtain the color mapping relationship corresponding to the first field of view image and the second field of view image on each color channel.

[0116] The first color value refers to the color value in the third subregion, and the second color value refers to the color value in the fourth subregion. Matching pixels in the first and second field of view images also have matching color values ​​for the same color channel. For example, the maximum color value in the first third subregion corresponding to the red channel matches the maximum color value in the first fourth subregion.

[0117] For example, the maximum value and the minimum value in each third subregion of the red channel, as well as the maximum value and the minimum value in each fourth subregion of the red channel, are selected; based on the maximum value and the minimum value in each third subregion of the red channel, as well as the maximum value and the minimum value in each fourth subregion, the corresponding color mapping relationship between the first field of view image and the second field of view image on the red channel is determined.

[0118] After obtaining the color mapping relationship corresponding to each color channel, the electronic device can perform color correction processing on the corresponding color channel of the second field of view image using the color mapping relationship corresponding to each color channel to obtain a color-corrected image. Furthermore, the electronic device can perform color correction processing on the color value of each pixel in the second field of view image in the corresponding color channel using the color mapping relationship corresponding to each color channel to obtain a color-corrected image.

[0119] It can be understood that for each color channel, the electronic device can also select multiple second color values ​​from each fourth sub-area of ​​the corresponding color channel, and filter out first color values ​​that match each second color value from the third sub-area corresponding to the corresponding color channel. Based on the multiple first color values ​​and the matching second color values, the color mapping relationship between the first field of view image and the second field of view image on the corresponding color channel can be calculated to obtain the color mapping relationship between the first field of view image and the second field of view image on each color channel.

[0120] In this embodiment, the electronic device can calculate a color mapping relationship between the fourth subregion and the third subregion of the same color channel based on multiple second color values ​​and multiple first color values ​​selected from the fourth subregion and the third subregion of the same color channel. This color mapping relationship can be used to perform color correction processing on the color values ​​on the same color channel of the corresponding regions in the second field of view image. Using the same processing method, a color mapping relationship can be obtained between each fourth subregion and the corresponding third subregion, and thus, based on each color mapping relationship, color correction processing can be performed on the color values ​​on the same color channel of each region in the second field of view image. Furthermore, using the same processing method, the fourth subregion and the third subregion of each color channel are processed to perform color correction processing on the color values ​​on each color channel of each region in the second field of view image, thereby obtaining a globally corrected image.

[0121] In this embodiment, the electronic device can perform mean filtering on the histogram information of the first field of view image and the second field of view image on each color channel respectively, and then determine the color mapping relationship between the first field of view image and the second field of view image on the corresponding color channels based on the histogram information obtained by the respective mean filtering processes. The mean filtering process can avoid excessive mapping differences between similar pixel values.

[0122] In this embodiment, histogram information of the first field of view image and the second field of view image in each color channel of the first color space is respectively counted to characterize the color distribution of the first field of view image and the second field of view image through the histogram, so as to determine the color mapping relationship between the first field of view image and the second field of view image in each color channel based on the histogram information of the first field of view image and the second field of view image. This allows the color information of each color channel of the second field of view image to be color corrected through the color mapping relationship, so as to adjust the color information of the second field of view image to be consistent with the color of the first field of view image to the greatest extent, thereby reducing the color difference between the two images.

[0123] In one embodiment, performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image includes:

[0124] A difference image between the first field of view image and the global correction image is determined; and based on the first field of view image and the global correction image, a nonlinear mapping process is performed on the difference image to obtain a corresponding intermediate image.

[0125] The difference image is an image composed of the pixel differences between matching pixels in the first field of view image and the globally corrected image. Each pixel difference serves as a pixel value to generate the difference image. Nonlinear mapping involves mapping multiple pixels in the first field of view image and the globally corrected image onto the same image based on the pixel values ​​in the difference image to form an intermediate image.

[0126] Specifically, the electronic device performs matching processing on the first field of view image and the global correction image to determine mutually matching pixels in the first field of view image and the global correction image. The mutually matching pixels may form pixel pairs.

[0127] The electronic device can determine the pixel values ​​of the matching pixels in their respective images and, based on the pixel values ​​of the matching pixels in their respective images, determine the pixel differences between the matching pixels. For example, the difference or the absolute value of the difference between the pixel values ​​of the matching pixels in their respective images can be used as the pixel difference. Using the same processing method, the electronic device can determine the pixel differences corresponding to all matching pixel pairs and use each pixel difference as the corresponding pixel value to generate a difference image. Each pixel value in the difference image is the corresponding pixel difference.

[0128] The first field of view image, the global correction image, and the difference image have matching pixels. The electronic device performs nonlinear mapping processing on corresponding pixels in the difference image based on the matching pixels in the first field of view image and the global correction image to obtain a corresponding intermediate image.

[0129] Furthermore, for the pixel value corresponding to a single pixel in the difference image, the electronic device compares the pixel value with the first difference threshold and the second difference threshold, respectively, to obtain a comparison result corresponding to the pixel value. Based on the comparison result and the pixel pair corresponding to the pixel value, the electronic device generates an intermediate pixel corresponding to the single pixel. Following the same processing method, the electronic device can obtain the intermediate pixel corresponding to each pixel in the difference image, and all the intermediate pixels constitute the intermediate image.

[0130] In this embodiment, a corresponding difference image is formed based on the difference between the first field of view image and the globally corrected image. Each pixel in the difference image can intuitively reflect the pixel differences between the first field of view image and the globally corrected image at corresponding locations. Based on the first field of view image and the globally corrected image, the difference image is subjected to nonlinear mapping processing. Based on the pixel values ​​in the difference image, multiple pixels in the first field of view image and the globally corrected image are mapped into the same image to form an intermediate image. This generated intermediate image combines the clearer and richer information of the first field of view image with the clearer and richer information of the globally corrected image, further reducing false color and improving image visual quality.

[0131] In one embodiment, determining a difference image between the first field of view image and the global correction image includes:

[0132] The first field of view image and the global correction image are matched to obtain mutually matching pixel points in the first field of view image and the global correction image; the channel differences corresponding to the mutually matching pixel points on each preset channel of the second color space are determined; and based on the channel differences corresponding to each preset channel of the mutually matching pixel points, a difference image between the first field of view image and the global correction image is generated.

[0133] Specifically, the first field of view image and the global correction image are both images defined in a second color space. The electronic device performs matching processing on the first field of view image and the global correction image to determine matching pixels in the first field of view image and the global correction image. A pixel value in the second color space is formed by channel values ​​on multiple preset channels. For example, if the second color space is a YUV color space, the multiple preset channels are the Y channel, the U channel, and the V channel, and the pixel value is composed of the channel value of the Y channel, the channel value of the U channel, and the channel value of the V channel.

[0134] The electronic device can determine the channel values ​​corresponding to the matching pixels in each preset channel of the second color space, and calculate the channel difference in the corresponding preset channel based on the channel values ​​corresponding to each preset channel. The electronic device determines the pixel difference corresponding to the matching pixels based on the channel difference of each preset channel corresponding to the matching pixels. Using the same processing method, the electronic device can calculate the pixel difference corresponding to all matching pixels, and generate a difference image between the first field of view image and the global correction image based on the pixel differences corresponding to all matching pixels.

[0135] In this embodiment, the electronic device may use the difference in channel values ​​of the matching pixels on the same preset channel as the channel difference of the matching pixels on the same preset channel, thereby obtaining the channel difference of the matching pixels on each preset channel. Alternatively, the absolute value of the difference may be used as the channel difference.

[0136] In this embodiment, generating a difference image between the first field of view image and the global correction image based on the channel differences of the preset channels corresponding to the mutually matched pixel points includes: determining the pixel differences corresponding to the mutually matched pixel points based on the channel differences of the preset channels corresponding to the mutually matched pixel points;

[0137] Based on the pixel differences corresponding to the matching pixels, a difference image is generated between the first field of view image and the globally corrected image. Specifically, the electronic device sums the channel differences corresponding to each preset channel of the matching pixels to obtain the pixel differences corresponding to the matching pixels. Following the same processing method, the electronic device can calculate the pixel differences corresponding to all matching pixels, use the pixel differences as pixel values, and generate the difference image using all the obtained pixel values.

[0138] In one embodiment, when both the first field of view image and the global corrected image are images defined in a first color space, the electronic device converts the first field of view image and the global corrected image from the first color space to a second color space, obtaining the first field of view image and the global corrected image in the second color space. The electronic device performs matching processing on the first field of view image and the global corrected image in the second color space to obtain matching pixels in the first field of view image and the global corrected image. The first color space is different from the second color space. The first color space can be an RGB color space or an SRGB color space, etc., and the second color space can be a YUV color space.

[0139] In this embodiment, the first field of view image and the globally corrected image are matched to determine matching pixels in the first field of view image and the globally corrected image. Based on the channel differences corresponding to the determined matching pixels in each predetermined channel of the second color space, a difference image between the first field of view image and the globally corrected image can be generated. This difference image more intuitively and accurately demonstrates the differences between the first field of view image and the globally corrected image at corresponding locations, using each pixel in the difference image.

[0140] In one embodiment, the field of view corresponding to the first field of view image is larger than the field of view corresponding to the second field of view image; and based on the first field of view image and the global correction image, a nonlinear mapping process is performed on the difference image to obtain a corresponding intermediate image, including:

[0141] For each pixel difference in the difference image, if the pixel difference is less than a first difference threshold, select a pixel belonging to the first field of view image from the pixel pair corresponding to the pixel difference as the intermediate pixel; the pixel difference represents the pixel value corresponding to the pixel in the difference image, the pixel difference being obtained based on the channel difference of each preset channel, and the pixel pair represents the matching pixels in the first field of view image and the global corrected image;

[0142] If the pixel difference is greater than or equal to the first difference threshold and the pixel difference is less than the second difference threshold, a corresponding intermediate pixel point is generated based on the pixel point pair corresponding to the pixel difference; if the pixel difference is greater than or equal to the second difference threshold, a pixel point belonging to the global corrected image is selected from the pixel point pair corresponding to the pixel difference as the intermediate pixel point; and an intermediate image is generated based on each intermediate pixel point.

[0143] Specifically, the pixel difference represents the pixel value corresponding to the pixel in the difference image. The pixel difference is obtained based on the channel difference of each preset channel. The pixel pair represents the matching pixels in the first field of view image and the globally corrected image. The electronic device can obtain a first difference threshold and a second difference threshold, where the second difference threshold is greater than the first difference threshold. The first and second difference thresholds are obtained by coupling and adjusting multiple variables, such as the multi-camera system exposure.

[0144] For the pixel difference corresponding to a single pixel in the difference image, the pixel difference is compared with the first difference threshold and the second difference threshold respectively. If the pixel difference is less than the first difference threshold, a pixel pair corresponding to the pixel difference is determined, and the pixel in the pixel pair belonging to the first field of view image is determined, and the pixel belonging to the first field of view image is used as the intermediate pixel.

[0145] If the pixel difference is greater than or equal to the second difference threshold, a pixel pair corresponding to the pixel difference is determined, and a pixel in the pixel pair belonging to the global corrected image is determined, and the pixel belonging to the global corrected image is used as the intermediate pixel.

[0146] If the pixel difference is greater than or equal to the first difference threshold, and the pixel difference is less than the second difference threshold, the electronic device generates a corresponding intermediate pixel point based on the pixel point pair corresponding to the pixel difference. Furthermore, if the pixel difference is greater than or equal to the first difference threshold, and the pixel difference is less than the second difference threshold, the electronic device generates a corresponding intermediate pixel point based on the weight corresponding to each pixel point in the pixel point pair corresponding to the pixel difference and the respective pixel value. For example, the weight corresponding to each pixel point in the pixel point pair and the respective pixel value can be multiplied and summed to obtain the sum of the pixels. The position of each pixel point in the pixel point pair in the image to which it belongs is the position of the sum of the pixels in the difference image, and then the corresponding intermediate pixel point can be obtained.

[0147] By traversing each pixel difference in the difference image in the same processing manner, the electronic device can determine the intermediate pixel point corresponding to each pixel difference in the difference image, and generate an intermediate image based on all the intermediate pixel points.

[0148] In this embodiment, the comparison result between the pixel difference in the difference image and two difference thresholds is used as the condition for generating the intermediate pixel of the intermediate image. Each pixel difference in the difference image is traversed. If the pixel difference is less than the first difference threshold, indicating that the pixel difference at the corresponding position between the first field of view image and the globally corrected image is small, the pixel belonging to the first field of view image is selected from the pixel pairs corresponding to the pixel difference as the intermediate pixel. This allows the corresponding pixel in the first field of view image to be directly used for positions with small differences.

[0149] If the pixel difference is greater than or equal to the first difference threshold and less than the second difference threshold, a corresponding intermediate pixel is generated based on the pixel pair corresponding to the pixel difference. This indicates that the pixel difference between the first field of view image and the global corrected image at the corresponding location is moderate. The corresponding pixel pair can be weighted, that is, the intermediate region is weighted to fuse the information of the pixel pair. If the pixel difference is greater than or equal to the second difference threshold, it indicates that the pixel difference between the first field of view image and the global corrected image at the corresponding location is too large. Then, pixels belonging to the global corrected image are selected from the pixel pair corresponding to the pixel difference, and the corresponding pixels of the global corrected image are retained as supplementary information for the narrow field of view. This generates an intermediate image that fuses the clearer and richer information from the first field of view image and the global corrected image, and further reduces color differences in the intermediate image.

[0150] In one embodiment, after performing global correction processing on the second field of view image according to the first field of view image to obtain a global correction image, the method further includes: performing registration processing on the global correction image based on the first field of view image to obtain a corresponding registered image;

[0151] Performing local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, including: performing local correction processing on the registration image based on the first field of view image to obtain an intermediate image;

[0152] The intermediate image and the global correction image are fused to obtain the target image, including: the intermediate image and the registration image are fused to obtain the target image.

[0153] Among them, registration processing refers to the process of aligning two or more images in space to match and superimpose two or more images acquired at different times, with different imaging devices or under different conditions.

[0154] Specifically, the electronic device performs matching processing on the first field of view image and the globally corrected image to determine matching pixels in the first field of view image and the globally corrected image. Based on the matching pixels, an image mapping relationship between the first field of view image and the globally corrected image is determined, and the globally corrected image is registered using the image mapping relationship to obtain a corresponding registered image.

[0155] Next, the electronic device may determine a second local area that needs to be corrected in the registered image, determine a corresponding first local area from the first field of view image, and perform correction processing on the second local area based on the first local area to obtain an intermediate image.

[0156] The electronic device may select local information from the intermediate image and select corresponding local information from the registered image to generate a target image based on the selected local information. Furthermore, the electronic device may replace the selected local content in the registered image with the corresponding local information selected from the intermediate image to obtain the target image.

[0157] In this embodiment, the electronic device may use the calibration parameters of the first camera and the calibration parameters of the second camera to perform registration processing on the global correction image to obtain a corresponding registered image.

[0158] In this embodiment, after globally correcting the second field of view image based on the first field of view image to obtain a globally corrected image, the degree of fit and generalization can be improved, thereby increasing the performance of image registration. Registration processing is performed on the globally corrected image based on the first field of view image to obtain a corresponding registered image. The first field of view image can be used as a reference to convert the globally corrected image from the field of view of the second camera to the field of view of the first camera, so that after registration, the parallax of the images corresponding to different cameras is compensated. Local correction processing is then performed on the registered image based on the first field of view image, which can further reduce brightness and color differences and resolve color anomalies. The intermediate image and the registered image are fused to obtain the target image, thereby avoiding the problem of color distortion in the fused image and effectively improving image quality and visual effects.

[0159] In one embodiment, the intermediate image and the global correction image are fused to obtain a target image, including:

[0160] The intermediate image and the global corrected image are filtered to obtain low-frequency information corresponding to the intermediate image and low-frequency information corresponding to the global corrected image; and a target image is generated according to the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image.

[0161] Images contain both high-frequency and low-frequency information. High-frequency information represents image edges, where high frequencies refer to areas with high frequency changes. Low-frequency information represents the content within these edges, which constitutes the majority of the image information, or the approximate image information. Low-frequency information refers to areas where color changes slowly, or in other words, where grayscale changes slowly.

[0162] Specifically, the electronic device performs filtering processing on the intermediate image and the global corrected image, respectively, to obtain low-frequency information corresponding to the intermediate image and low-frequency information corresponding to the global corrected image. Furthermore, the electronic device performs mean filtering processing on the intermediate image and the global corrected image, respectively, to obtain their respective corresponding low-frequency information. Mean filtering, also known as linear filtering, replaces each pixel value in the original image with a mean. Specifically, a template is assigned to the target pixel in the image. The template includes the neighboring pixels surrounding the target pixel (i.e., the eight pixels surrounding the target pixel form a filtering template, excluding the target pixel itself), and then the original pixel value is replaced with the average value of all pixels in the template.

[0163] The electronic device replaces the low-frequency information in the global correction image with the low-frequency information corresponding to the intermediate image to obtain the target image.

[0164] In other embodiments, the electronic device may obtain the weights corresponding to the low-frequency information of the global corrected image and the low-frequency information of the intermediate image, and fuse the low-frequency information of the intermediate image into the low-frequency information of the global corrected image based on the respective weights to obtain the target image.

[0165] In this embodiment, the intermediate image and the global corrected image are filtered to obtain low-frequency information corresponding to the intermediate image and the global corrected image. A target image is generated based on the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image. The low-frequency information corresponding to the intermediate image can be fused into the global corrected image, thereby replacing the low-frequency information in the global corrected image, further reducing disparity and false color.

[0166] In one embodiment, the first field of view image is a first subject area in an original field of view image captured by a first camera, and the second field of view image is a second subject area in an original field of view image captured by a second camera; and fusing the intermediate image and the global correction image to obtain a target image includes:

[0167] The intermediate image, the global correction image and the non-subject area in the second field of view image are fused to obtain the target image.

[0168] Specifically, the electronic device captures an image of any scene through a first camera and a second camera to obtain an original field of view image corresponding to the first camera and an original field of view image corresponding to the second camera. The electronic device performs subject recognition on the original field of view image corresponding to the first camera to obtain a first subject area. The electronic device performs subject recognition on the original field of view image corresponding to the second camera to obtain a second subject area and a non-subject area. Among them, subject recognition is also known as subject detection, which means that when facing a scene, the area of ​​interest is automatically processed and the area of ​​no interest is selectively ignored. The area of ​​interest is called the subject area. The subject refers to various objects, such as people, flowers, cats, dogs, cows, blue sky, white clouds, background, etc., but is not limited to these.

[0169] The electronic device performs global correction processing on the second subject area according to the first subject area to obtain a globally corrected image, and then performs local correction processing on the globally corrected image based on the first subject area to obtain an intermediate image.

[0170] The electronic device performs filtering on the intermediate image and the globally corrected image to obtain low-frequency information corresponding to the intermediate image and the globally corrected image. The electronic device replaces the low-frequency information in the globally corrected image with the low-frequency information corresponding to the intermediate image to obtain a new subject area. The electronic device then synthesizes the new subject area with the non-subject area in the second field of view image to obtain a target image.

[0171] In this embodiment, the first field of view image is the first subject area in the original field of view image captured by the first camera, and the second field of view image is the second subject area in the original field of view image captured by the second camera. This allows the subject area in the image to be individually corrected to effectively reduce brightness and color differences in the subject area between images captured by multiple cameras, thereby effectively reducing the overall differences between the subject areas, and obtaining the subject area after global and local correction, i.e., the intermediate image. The corrected subject area is then fused with the non-subject area in the second field of view image. The non-subject area is not a critical area and may not be processed, thereby saving processing time and improving processing efficiency.

[0172] In other embodiments, the electronic device may perform subject recognition on the original field of view image captured by the first camera to obtain a first subject region and a first non-subject region. Subject recognition may be performed on the original field of view image captured by the second camera to obtain a second subject region and a second non-subject region. The second subject region may be globally corrected based on the first subject region, and the image obtained by the global correction may be locally corrected based on the first subject region to obtain a first intermediate image. The second non-subject region may be globally corrected based on the first non-subject region, and the image obtained by the global correction may be locally corrected based on the first non-subject region to obtain a second intermediate image. The first intermediate image may be the subject region, and the second intermediate image may be the non-subject region. The first intermediate image may be the subject region, and the second intermediate image may be the non-subject region. The target image may be obtained by fusing the first and second intermediate images. By separately correcting the subject region and the non-subject region in each image captured by multiple cameras, the brightness and color differences in the subject region and the brightness and color differences in the non-subject region may be separately corrected based on semantic segmentation. This separate processing can further improve the accuracy of the correction.

[0173] In one embodiment, an image processing method is provided, such as Figure 3 As shown, the wide field of view image is the first field of view image, referred to as the wide view, and the narrow field of view image is the second field of view image, referred to as the narrow view. Due to the difference in focal length of the asymmetric optical system, the cropped wide field of view image displays a field of view similar to the telescopic image. The two cropped images (wide view and tele view) are used as input. Global correction processing is performed on the narrow view, namely, global brightness and color correction, to obtain a globally corrected image. Specifically, histogram matching is performed on the brightness Y channels of the wide and narrow views, and then converted to RGB space for color histogram matching of the R, B, and B channels. These two nonlinear mappings improve the goodness of fit and generalization, enhancing image registration performance. The globally corrected image is then registered based on the wide view to obtain a registered globally corrected image, referred to as the registered image. After registration, the parallax between the dual-camera images is compensated. Local correction is then performed on the registered globally corrected image to further reduce brightness and color differences and resolve color anomalies. Specifically, local correction is based on the difference between the wide view and the registered global correction image. The registered global correction image is nonlinearly stretched, and then the high- and low-frequency information is decomposed through mean filtering. The low-frequency information of the stretched intermediate image is replaced with the globally corrected narrow field of view image to obtain the final corrected image, that is, the target image.

[0174] Specifically, the global correction process includes brightness correction and color correction. The brightness correction is brightness histogram matching, and the color correction is color histogram matching.

[0175] The brightness histogram matching process is as follows:

[0176] (1) First, the luminance channels of the wide view and the narrow view are subjected to guided filtering to reduce the influence of noise, and then the histogram information of the luminance channel is counted respectively;

[0177] (2) In order to improve the mapping fitting ability, the histogram is evenly distributed into 32 bins, each 32 bins constitutes a bin interval, and a quantity control threshold is set to avoid overexposure. The maximum and minimum values ​​in the corresponding bin intervals of the two images are matched, and the other pixel values ​​are interpolated to obtain the mapping relationship;

[0178] (3) Apply the mapping relationship to the brightness channel of the narrow view tele to obtain a global brightness mapping correction map, i.e., a global correction image.

[0179] Color histogram matching is processed as follows:

[0180] (1) First, convert the wide view and global correction images from YUV color space to RGB space, and then count the histogram information of the three color channels respectively. The histogram is also evenly distributed into 32 bins, and a certain amount is added to each histogram to avoid the effect of local overexposure or darkening.

[0181] (2) Perform mean filtering on the histogram to avoid large mapping differences between similar pixel values, and then Figure 3 The maximum pixel value and the minimum pixel value in each color channel bin are respectively corresponded, and the other pixel values ​​are also interpolated to obtain the mapping relationship;

[0182] (3) The mapping relationship of the three color channels is applied to the color channels corresponding to the narrow view tele to obtain the final global correction image.

[0183] As shown in Figure 4, the brightness and color information are represented by the filled background in the image. Figure 4 As can be seen in the figure, there are certain differences in the brightness and color information of the wide view and the narrow view. After performing global correction processing on the narrow view based on the wide view, the brightness and color information of the narrow view can be corrected. The difference between the brightness and color information of the global correction image obtained by global correction and the brightness and color information of the wide view is significantly reduced, making the brightness and color of the global correction image and the wide view as consistent as possible.

[0184] Image registration processing:

[0185] The global correction image is registered based on the wide view tele to obtain a registered global correction image, namely, the registered image tele′.

[0186] The local correction process is as follows:

[0187] (1) Calculate the difference image between the wide view wide and the registered image tele′

[0188] diff=|y wide -y tele ′|+|u wide -u tele ′|+|v wide -v tele ′|

[0189] Wherein, diff is the pixel value corresponding to the pixel point in the difference image.

[0190] (2) Use the difference image to perform nonlinear mapping to obtain the intermediate image P. The nonlinear mapping is as follows:

[0191]

[0192] w=(thres2-diff) / (thres2-thres1)

[0193] Where p is the pixel value of the pixel in the intermediate image P. The difference thresholds thres1 and thres2 are obtained by coupling and adjusting multiple variables, such as the dual-camera system exposure. For areas with small differences, the wide-field image information is directly used. Areas with large differences are treated as supplementary information from the narrow field of view and retained, with weighting applied to the intermediate areas.

[0194] (2) The long-range view with narrow field of view can mainly enhance the texture details of the wide field of view image. Therefore, the intermediate image P and the registered image tele′ are mean filtered to obtain their corresponding low-frequency information lowf P 、lowf tele′ , the low frequency information lowf in the registration image P Replaced with lowf in the intermediate image P tele′ , in order to further reduce false colors and reduce differences, we can get Figure 5 The final corrected image (i.e., the target image) shown is processed as follows:

[0195] tele″=tele′-lowf tele′ +lowf P

[0196] In this embodiment, image brightness and color correction is performed through multiple nonlinear fitting calculations based on histogram matching. This effectively reduces color and brightness differences between dual-camera images, improving the performance of dual-camera image registration and fusion. Furthermore, for dual-camera and multi-camera systems, multiple nonlinear fitting and low-frequency replacement are used to further reduce errors caused by imaging differences and enhance image visual quality.

[0197] In one embodiment, an image processing method is provided, which is applied to an electronic device and includes:

[0198] A first field of view image captured by a first camera and a second field of view image captured by a second camera are acquired, wherein the field of view corresponding to the first camera is larger than the field of view corresponding to the second camera.

[0199] Next, first histogram information of the brightness channel of the first field of view image is counted, and second histogram information of the brightness channel of the second field of view image is counted; the brightness channel is the Y channel corresponding to the YUV color space.

[0200] Next, a brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image is determined according to the first histogram information and the second histogram information.

[0201] Next, brightness correction processing is performed on the brightness channel of the second field of view image through the brightness mapping relationship to obtain a brightness corrected image.

[0202] Furthermore, histogram information of the first field of view image on the red channel, green channel and blue channel, and histogram information of the second field of view image on the red channel, green channel and blue channel are respectively counted.

[0203] Next, according to the histogram information of the first field of view image and the second field of view image on the red channel, green channel and blue channel, the color mapping relationship of the first field of view image and the second field of view image on the red channel, green channel and blue channel is determined respectively.

[0204] Furthermore, color correction processing is performed on the red channel, green channel, and blue channel of the second field of view image according to the color mapping relationships corresponding to the red channel, green channel, and blue channel, respectively, to obtain a globally corrected image.

[0205] Next, the global correction image is registered based on the first field of view image to obtain a corresponding registered image.

[0206] Next, the first field of view image and the registered image are matched to obtain mutually matching pixels in the first field of view image and the registered image; and the channel differences corresponding to the mutually matching pixels in the Y channel, U channel, and V channel of the YUV color space are determined.

[0207] Next, based on the channel differences corresponding to the matching pixels in the Y, U, and V channels, the pixel differences corresponding to the matching pixels are determined. Based on the pixel differences corresponding to all matching pixels, a difference image is generated between the first field of view image and the registered image. This pixel difference serves as the pixel value corresponding to the pixel in the difference image, and the matching pixels in the first field of view image and the registered image form pixel pairs.

[0208] Furthermore, for each pixel difference in the difference image, if the pixel difference is less than a first difference threshold, a pixel belonging to the first field of view image is selected from the pixel pair corresponding to the pixel difference as the middle pixel.

[0209] Optionally, if the pixel difference is greater than or equal to the first difference threshold and the pixel difference is less than the second difference threshold, a corresponding intermediate pixel point is generated according to the pixel point pair corresponding to the pixel difference.

[0210] Optionally, if the pixel difference is greater than or equal to a second difference threshold, a pixel point belonging to the registered image is selected from the pixel point pair corresponding to the pixel difference as the intermediate pixel point.

[0211] Furthermore, an intermediate image is generated based on each intermediate pixel point.

[0212] Next, the intermediate image and the registered image are filtered to obtain low-frequency information corresponding to the intermediate image and low-frequency information corresponding to the registered image.

[0213] Furthermore, the low-frequency information in the registered image is replaced with the low-frequency information corresponding to the intermediate image to obtain the target image.

[0214] In this embodiment, first histogram information of the brightness channel of the first field of view image is counted, and second histogram information of the brightness channel of the second field of view image is counted, so as to characterize the brightness distribution of the first field of view image and the second field of view image through the histogram, and thus determine the brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image based on the first histogram information and the second histogram information, so that the second field of view image can be brightness corrected through the brightness mapping relationship to adjust the brightness information of the second field of view image to be consistent with the brightness of the first field of view image to the greatest extent, thereby reducing the brightness difference between the two images.

[0215] Histogram information of the first field of view image and the second field of view image in each color channel of the first color space is respectively counted to characterize the color distribution of the first field of view image and the second field of view image through the histogram, so as to determine the color mapping relationship between the first field of view image and the second field of view image in each color channel according to the histogram information of the first field of view image and the second field of view image, so that color correction processing can be performed on the color information of each color channel of the second field of view image through the color mapping relationship, so as to adjust the color information of the second field of view image to be consistent with the color of the first field of view image to the greatest extent, thereby reducing the color difference between the two images.

[0216] The global correction image is registered based on the first field of view image to obtain a corresponding registration image. The first field of view image can be used as a reference to convert the global correction image from the field of view of the second camera to the field of view of the first camera, so that the two images are in the same field of view. After registration, the parallax of the images corresponding to different cameras is compensated.

[0217] The first field of view image and the registered image are matched to determine matching pixels in the first field of view image and the registered image. Based on the channel differences corresponding to the determined matching pixels in each predetermined channel of the second color space, a difference image between the first field of view image and the registered image can be generated. This difference image more intuitively and accurately demonstrates the difference at each position in the first field of view image and the registered image through each pixel in the difference image.

[0218] The comparison result of the pixel difference in the difference image and the two difference thresholds is used as the condition for generating the intermediate pixel point of the intermediate image. Each pixel difference in the difference image is traversed. If the pixel difference is less than the first difference threshold, it means that the pixel difference between the first field of view image and the registered image at the corresponding position is small. Then, the pixel point belonging to the first field of view image is selected from the pixel point pair corresponding to the pixel difference as the intermediate pixel point, so that the corresponding pixel point of the first field of view image can be directly used for the position with small difference. If the pixel difference is greater than or equal to the first difference threshold and the pixel difference is less than the second difference threshold, the corresponding intermediate pixel point is generated based on the pixel point pair corresponding to the pixel difference. This indicates that the pixel difference between the first field of view image and the registered image at the corresponding position is moderate. The corresponding pixel point pair can be weighted, that is, the intermediate area is weighted to fuse the information of the pixel point pair. If the pixel difference is greater than or equal to the second difference threshold, indicating that the pixel difference between the first field of view image and the registered image at the corresponding position is too large, the pixel points belonging to the registered image are selected from the pixel point pairs corresponding to the pixel difference, so that the corresponding pixel points of the registered image are retained as supplementary information of the narrow field of view, so that the generated intermediate image integrates clearer and richer information in the first field of view image and the registered image, and the obtained intermediate image further reduces the color difference.

[0219] The intermediate image and the registered image are filtered to obtain the low-frequency information corresponding to the intermediate image and the low-frequency information corresponding to the registered image, so as to generate the target image according to the registered image, the low-frequency information corresponding to the registered image and the low-frequency information corresponding to the intermediate image. The low-frequency information corresponding to the intermediate image can be fused into the registered image, thereby replacing the low-frequency information in the registered image, further reducing the difference and false color, thereby effectively improving the image quality and visual effect of the generated image.

[0220] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0221] Based on the same inventive concept, embodiments of the present application also provide an image processing device for implementing the aforementioned image processing method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following image processing device embodiments can be found in the above-described limitations on the image processing method and will not be further elaborated here.

[0222] In one embodiment, Figure 6 As shown, an image processing device 600 is provided, comprising: an acquisition module 602, a global correction module 604, a local correction module 606 and a fusion module 608, wherein:

[0223] The acquisition module 602 is configured to acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera.

[0224] The global correction module 604 is configured to perform global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction.

[0225] The local correction module 606 is configured to perform local correction processing on the global correction image based on the first field of view image to obtain an intermediate image, wherein the local correction includes position correction.

[0226] The fusion module 608 is used to fuse the intermediate image and the global correction image to obtain a target image.

[0227] In this embodiment, a first field of view image captured by a first camera and a second field of view image captured by a second camera are obtained, the field of view corresponding to the first camera is different from the field of view corresponding to the second camera, and the second field of view image is globally corrected based on the first field of view image to obtain a globally corrected image. The global correction includes at least one of brightness correction and color correction. The brightness correction can effectively reduce the brightness difference between images captured by multiple cameras, and the color correction can effectively reduce the color difference between images captured by multiple cameras, thereby effectively reducing the overall difference between the images. Based on the first field of view image, the globally corrected image is locally corrected to obtain an intermediate image. The local correction includes position correction. The position correction can further reduce the brightness and difference between the images captured by multiple cameras in the local area, making the correction more accurate, thereby further reducing false colors. The intermediate image and the globally corrected image are fused to obtain the target image, avoiding the problem of color distortion of the fused image, thereby effectively improving the image quality and visual effect.

[0228] In one embodiment, the global correction module 604 is further configured to perform brightness correction on the second field of view image according to the first field of view image to obtain a brightness corrected image; and perform color correction on the brightness corrected image according to the first field of view image to obtain a globally corrected image.

[0229] In this embodiment, using the brightness information of the first field of view image as a reference, brightness correction processing is performed on the second field of view image based on the first field of view image, effectively reducing the brightness difference between the two images captured by the first camera and the second camera, so that the brightness information of the second field of view image is adjusted to be consistent with the brightness information of the first field of view image. After brightness correction, using the color information of the first field of view image as a reference, color correction processing is performed on the brightness-corrected image based on the first field of view image, so that the color information of the brightness-corrected image is adjusted to be consistent with the color information of the first field of view image, effectively reducing the color difference between images captured by different cameras.

[0230] In one embodiment, the global correction module 604 is further used to count the first histogram information of the brightness channel of the first field of view image and the second histogram information of the brightness channel of the second field of view image; determine the brightness mapping relationship between the first field of view image and the second field of view image based on the first histogram information and the second histogram information; and perform brightness correction processing on the second field of view image through the brightness mapping relationship to obtain a brightness-corrected image.

[0231] In this embodiment, first histogram information of the brightness channel of the first field of view image is counted, and second histogram information of the brightness channel of the second field of view image is counted, so as to characterize the brightness distribution of the first field of view image and the second field of view image through the histogram, and thus determine the brightness mapping relationship between the brightness channel of the first field of view image and the brightness channel of the second field of view image based on the first histogram information and the second histogram information, so that the second field of view image can be brightness corrected through the brightness mapping relationship to adjust the brightness information of the second field of view image to be consistent with the brightness of the first field of view image to the greatest extent, thereby reducing the brightness difference between the two images.

[0232] In one embodiment, the global correction module 604 is further used to respectively count the histogram information of the first field of view image and the second field of view image on each color channel of the first color space; determine the color mapping relationship between the first field of view image and the second field of view image on the corresponding color channel based on the histogram information of the first field of view image and the second field of view image on each color channel; and perform color correction processing on the corresponding color channel of the second field of view image through the color mapping relationship corresponding to each color channel to obtain a globally corrected image.

[0233] In this embodiment, histogram information of the first field of view image and the second field of view image in each color channel of the first color space is respectively counted to characterize the color distribution of the first field of view image and the second field of view image through the histogram, so as to determine the color mapping relationship between the first field of view image and the second field of view image in each color channel based on the histogram information of the first field of view image and the second field of view image. This allows the color information of each color channel of the second field of view image to be color corrected through the color mapping relationship, so as to adjust the color information of the second field of view image to be consistent with the color of the first field of view image to the greatest extent, thereby reducing the color difference between the two images.

[0234] In one embodiment, the local correction module 606 is further configured to determine a difference image between the first field of view image and the global correction image; and perform nonlinear mapping processing on the difference image based on the first field of view image and the global correction image to obtain a corresponding intermediate image.

[0235] In this embodiment, a corresponding difference image is formed based on the difference between the first field of view image and the globally corrected image. Each pixel in the difference image can intuitively reflect the pixel differences between the first field of view image and the globally corrected image at corresponding locations. Based on the first field of view image and the globally corrected image, the difference image is subjected to nonlinear mapping processing. Based on the pixel values ​​in the difference image, multiple pixels in the first field of view image and the globally corrected image are mapped into the same image to form an intermediate image. This generated intermediate image combines the clearer and richer information of the first field of view image with the clearer and richer information of the globally corrected image, further reducing false color and improving image visual quality.

[0236] In one embodiment, the local correction module 606 is also used to match the first field of view image and the global correction image to obtain mutually matching pixel points in the first field of view image and the global correction image; determine the channel differences corresponding to the mutually matching pixel points on each preset channel of the second color space; and generate a difference image between the first field of view image and the global correction image based on the channel differences corresponding to the mutually matching pixel points on each preset channel.

[0237] In this embodiment, the first field of view image and the globally corrected image are matched to determine matching pixels in the first field of view image and the globally corrected image. Based on the channel differences corresponding to the determined matching pixels in each predetermined channel of the second color space, a difference image between the first field of view image and the globally corrected image can be generated. This difference image more intuitively and accurately demonstrates the differences between the first field of view image and the globally corrected image at corresponding locations, using each pixel in the difference image.

[0238] In one embodiment, the field of view corresponding to the first field of view image is larger than the field of view corresponding to the second field of view image; the local correction module 606 is further configured to, for each pixel difference in the difference image, if the pixel difference is less than a first difference threshold, select a pixel belonging to the first field of view image from the pixel pair corresponding to the pixel difference as an intermediate pixel; the pixel difference represents a pixel value corresponding to the pixel in the difference image, the pixel difference is obtained based on the channel difference of each preset channel, and the pixel pair represents matching pixels in the first field of view image and the global correction image; if the pixel difference is greater than or equal to the first difference threshold and the pixel difference is less than the second difference threshold, generate a corresponding intermediate pixel based on the pixel pair corresponding to the pixel difference; if the pixel difference is greater than or equal to the second difference threshold, select a pixel belonging to the global correction image from the pixel pair corresponding to the pixel difference as an intermediate pixel; and generate an intermediate image based on each intermediate pixel.

[0239] In this embodiment, the comparison result between the pixel difference in the difference image and two difference thresholds is used as the condition for generating the intermediate pixel of the intermediate image. Each pixel difference in the difference image is traversed. If the pixel difference is less than the first difference threshold, indicating that the pixel difference at the corresponding position between the first field of view image and the globally corrected image is small, the pixel belonging to the first field of view image is selected from the pixel pairs corresponding to the pixel difference as the intermediate pixel. This allows the corresponding pixel in the first field of view image to be directly used for positions with small differences.

[0240] If the pixel difference is greater than or equal to the first difference threshold and the pixel difference is less than the second difference threshold, a corresponding intermediate pixel point is generated based on the pixel point pair corresponding to the pixel difference, which means that the pixel difference between the first field of view image and the global correction image at the corresponding position is moderate. The corresponding pixel point pair can be weighted, that is, the intermediate area can be weighted to fuse the information of the pixel point pair.

[0241] If the pixel difference is greater than or equal to the second difference threshold, indicating that the pixel difference between the first field of view image and the global corrected image at the corresponding position is too large, then the pixel points belonging to the global corrected image are selected from the pixel point pairs corresponding to the pixel difference, so that the corresponding pixel points of the global corrected image are retained as supplementary information for the narrow field of view, so that the generated intermediate image integrates clearer and richer information in the first field of view image and the global corrected image, and the obtained intermediate image further reduces the color difference.

[0242] In one embodiment, the apparatus further comprises a registration module; the registration module is configured to perform registration processing on the global correction image based on the first field of view image to obtain a corresponding registered image;

[0243] The local correction module 606 is further configured to perform local correction processing on the registered image based on the first field of view image to obtain an intermediate image;

[0244] The fusion module 608 is further configured to fuse the intermediate image and the registered image to obtain a target image.

[0245] In this embodiment, after globally correcting the second field of view image based on the first field of view image to obtain a globally corrected image, the degree of fit and generalization can be improved, thereby enhancing image registration performance. By registering the globally corrected image based on the first field of view image to obtain a corresponding registered image, the globally corrected image can be converted from the second camera's field of view to the first camera's field of view, using the first field of view image as a reference. After registration, parallax between images corresponding to different cameras is compensated. Subsequently, local correction processing is performed on the registered image based on the first field of view image, further reducing brightness and color differences and resolving color anomalies.

[0246] In one embodiment, the fusion module 608 is further configured to perform filtering processing on the intermediate image and the global corrected image to obtain low-frequency information corresponding to the intermediate image and low-frequency information corresponding to the global corrected image; and generate a target image based on the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image.

[0247] In this embodiment, the intermediate image and the global corrected image are filtered to obtain low-frequency information corresponding to the intermediate image and the global corrected image. A target image is generated based on the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image. The low-frequency information corresponding to the intermediate image can be fused into the global corrected image, thereby replacing the low-frequency information in the global corrected image, further reducing disparity and false color.

[0248] In one embodiment, the first field of view image is the first main body area in the original field of view image captured by the first camera, and the second field of view image is the second main body area in the original field of view image captured by the second camera; the fusion module 608 is also used to fuse the intermediate image, the global correction image and the non-main body area in the second field of view image to obtain the target image.

[0249] In this embodiment, the first field of view image is the first subject area in the original field of view image captured by the first camera, and the second field of view image is the second subject area in the original field of view image captured by the second camera. This allows the subject area in the image to be individually corrected to effectively reduce brightness and color differences in the subject area between images captured by multiple cameras, thereby effectively reducing the overall differences between the subject areas, and obtaining the subject area after global and local correction, i.e., the intermediate image. The corrected subject area is then fused with the non-subject area in the second field of view image. The non-subject area is not a critical area and may not be processed, thereby saving processing time and improving processing efficiency.

[0250] Each module in the above-mentioned image processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0251] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0252] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0253] The present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of the image processing method.

[0254] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the image processing method.

[0255] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0256] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0257] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0258] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera; Performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction; Performing local correction processing on the globally corrected image based on the first field of view image to obtain an intermediate image, including: determining a difference image between the first field of view image and the globally corrected image, and performing nonlinear mapping processing on the difference image based on the first field of view image and the globally corrected image to obtain a corresponding intermediate image, wherein the local correction includes position correction; the nonlinear mapping processing refers to mapping multiple pixel points in the first field of view image and the globally corrected image into the same image based on pixel values ​​in the difference image; The intermediate image and the global correction image are fused to obtain a target image.

2. The method according to claim 1, characterized in that The performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image includes: performing brightness correction processing on the second field of view image according to the first field of view image to obtain a brightness corrected image; Color correction is performed on the brightness-corrected image according to the first field of view image to obtain a global-corrected image.

3. The method according to claim 2, characterized in that The performing brightness correction processing on the second field of view image according to the first field of view image to obtain a brightness corrected image includes: Counting first histogram information of the brightness channel of the first field of view image, and counting second histogram information of the brightness channel of the second field of view image; Determining a brightness mapping relationship between the first field of view image and the second field of view image according to the first histogram information and the second histogram information; Brightness correction processing is performed on the second field of view image according to the brightness mapping relationship to obtain a brightness corrected image.

4. The method according to claim 2, characterized in that The performing color correction processing on the brightness-corrected image according to the first field of view image to obtain a global correction image includes: respectively collecting statistical histogram information of the first field of view image and the second field of view image in each color channel of the first color space; Determining a color mapping relationship between the first field of view image and the second field of view image in corresponding color channels according to histogram information of the first field of view image and the second field of view image in each color channel; Color correction processing is performed on the corresponding color channel of the second field of view image using the color mapping relationship corresponding to each color channel to obtain a global corrected image.

5. The method according to claim 1, wherein Determining a difference image between the first field of view image and the global correction image includes: Performing matching processing on the first field of view image and the global corrected image to obtain mutually matching pixel points in the first field of view image and the global corrected image; Determining channel differences corresponding to the mutually matching pixel points in respective preset channels of the second color space; Based on the channel differences of the mutually matched pixel points corresponding to the preset channels, a difference image between the first field of view image and the global correction image is generated.

6. The method according to claim 5, characterized in that The field of view corresponding to the first field of view image is larger than the field of view corresponding to the second field of view image; and performing nonlinear mapping processing on the difference image based on the first field of view image and the global correction image to obtain a corresponding intermediate image includes: For each pixel difference in the difference image, if the pixel difference is less than a first difference threshold, selecting a pixel belonging to the first field of view image from the pixel pair corresponding to the pixel difference as the intermediate pixel; the pixel difference represents a pixel value corresponding to the pixel in the difference image, the pixel difference is obtained based on the channel difference of each of the preset channels, and the pixel pair represents matching pixels in the first field of view image and the globally corrected image; If the pixel difference is greater than or equal to a first difference threshold and the pixel difference is less than a second difference threshold, generating a corresponding intermediate pixel point according to the pixel point pair corresponding to the pixel difference; If the pixel difference is greater than or equal to the second difference threshold, selecting a pixel belonging to the global corrected image from the pixel pair corresponding to the pixel difference as an intermediate pixel; An intermediate image is generated based on each of the intermediate pixel points.

7. The method according to claim 1, characterized in that After performing global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image, the method further includes: Performing registration processing on the global correction image based on the first field of view image to obtain a corresponding registered image; The locally correcting the global corrected image based on the first field of view image to obtain an intermediate image includes: Performing local correction processing on the registered image based on the first field of view image to obtain an intermediate image; The fusing the intermediate image and the global correction image to obtain a target image includes: The intermediate image and the registered image are fused to obtain a target image.

8. The method according to any one of claims 1 to 7, characterized in that The fusing the intermediate image and the global correction image to obtain a target image includes: performing filtering processing on the intermediate image and the global corrected image to obtain low-frequency information corresponding to the intermediate image and low-frequency information corresponding to the global corrected image; A target image is generated according to the global corrected image, the low-frequency information corresponding to the global corrected image, and the low-frequency information corresponding to the intermediate image.

9. The method according to claim 1, characterized in that The first field of view image is a first subject area in the original field of view image captured by the first camera, and the second field of view image is a second subject area in the original field of view image captured by the second camera; The fusing the intermediate image and the global correction image to obtain a target image includes: The intermediate image, the global correction image and the non-subject area in the second field of view image are fused to obtain a target image.

10. An image processing device, characterized in that: include: An acquisition module, configured to acquire a first field of view image captured by a first camera and a second field of view image captured by a second camera, wherein the field of view corresponding to the first camera is different from the field of view corresponding to the second camera; a global correction module, configured to perform global correction processing on the second field of view image according to the first field of view image to obtain a globally corrected image; the global correction includes at least one of brightness correction and color correction; a local correction module, configured to perform local correction processing on the global corrected image based on the first field of view image to obtain an intermediate image, comprising: determining a difference image between the first field of view image and the global corrected image, and performing nonlinear mapping processing on the difference image based on the first field of view image and the global corrected image to obtain a corresponding intermediate image; the nonlinear mapping processing refers to mapping multiple pixels in the first field of view image and the global corrected image into the same image based on pixel values ​​in the difference image; the local correction includes position correction; A fusion module is used to fuse the intermediate image and the global correction image to obtain a target image.

11. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images