Image processing methods and apparatus, electronic devices, readable storage media

Abstract

This disclosure relates to an image processing method and apparatus, electronic device, and readable storage medium. The method includes: obtaining a texture mask representing the strength of image texture based on a first image captured by a first camera and a second image captured by a second camera; processing the first image and the second image respectively based on the texture mask to obtain a base image corresponding to the first image and a detail image corresponding to the second image; and fusing the base image corresponding to the first image and the detail image corresponding to the second image to obtain a target image. In this embodiment, a texture mask can be used to process the first and second images, meaning that the texture strength can be determined based on the texture mask, and whether filtering is performed can be decided, thereby avoiding ripples during image processing and achieving a unified style fusion effect between the first and second images, which is beneficial for improving the shooting experience.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method and apparatus, electronic device, and readable storage medium. Background Technology

[0002] Currently, smartphones and other electronic devices typically include multiple cameras, including a main camera and a telephoto camera. To improve the zoom capabilities of these cameras, current electronic devices use a method that combines the main camera and the telephoto camera. This involves merging the images captured by the main camera and the telephoto camera, so that the merged image has the field of view and color brightness of the main camera, while the overlapping areas of the images have the resolution of the telephoto camera. Summary of the Invention

[0003] This disclosure provides an image processing method and apparatus, an electronic device, and a readable storage medium to address the shortcomings of related technologies.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, applied to an electronic device, the electronic device including a first camera and a second camera, the method comprising:

[0005] A texture mask representing the strength of image texture is obtained based on the first image captured by the first camera and the second image captured by the second camera;

[0006] Based on the texture mask, the first image and the second image are processed respectively to obtain the base image corresponding to the first image and the detail image corresponding to the second image;

[0007] The target image is obtained by fusing the base image corresponding to the first image and the detail image corresponding to the second image.

[0008] Optionally, obtaining a texture mask representing the strength of image texture based on a first image captured by the first camera and a second image captured by the second camera includes:

[0009] Align the first image and the second image;

[0010] Using each pixel at the same position in the overlapping area of ​​the first image and the second image as a reference, the maximum and minimum values ​​of the pixel values ​​within a preset range of each pixel in each image are obtained, and the difference between the maximum and minimum values ​​is obtained.

[0011] When the difference between the first image and the second image is less than the set threshold, the pixel is determined to be a pixel in a weak texture region; when the difference between the first image and / or the second image exceeds the set threshold, the pixel is determined to be a pixel in a strong texture region.

[0012] Traverse each pixel at the same position within the overlapping region of the first image and the second image, and generate a texture mask representing the strength of the image texture. The texture mask is also used to characterize pixels in strong texture regions and pixels in weak texture regions.

[0013] Optionally, the first image and the second image are processed based on the texture mask to obtain a base image corresponding to the first image and a detail image corresponding to the second image, including:

[0014] The first image and the second image are decomposed into pyramids respectively, and the images decomposed in each step are filtered according to the texture mask to obtain the base image corresponding to the first image and the base image corresponding to the second image.

[0015] Obtain the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image corresponding to the second image.

[0016] Optionally, the pyramid decomposition includes:

[0017] The input image is downsampled m times to obtain a first image; the input image includes either the first image or the second image; m is greater than or equal to 2;

[0018] The first image is upsampled m times to obtain the base image corresponding to the input image; the scale of the base image is the same as the scale of the input image.

[0019] Optionally, filtering is performed on the decomposed image based on the texture mask, including:

[0020] Both the strong and weak texture regions of the image decomposed by the last upsampling are filtered; in each subsequent upsampling, the strong texture regions in the image decomposed by each downsampling and each upsampling are filtered according to the texture mask.

[0021] Optionally, the first image and the second image include YUV three-channel sub-images; filtering is performed on each decomposed image according to the texture mask, including:

[0022] The texture mask is used to filter the Y channel sub-image.

[0023] Optionally, the target image is obtained by fusing the base image corresponding to the first image and the detail image corresponding to the second image, including:

[0024] The sum of pixel values ​​of the same channel sub-images in the base image and the detail image is obtained to obtain the sub-images of each channel of the target image, and the sub-images of each channel constitute the target image.

[0025] According to a second aspect of the present disclosure, an image processing apparatus is provided, applied to an electronic device, the electronic device including a first camera and a second camera, the apparatus comprising:

[0026] The texture mask acquisition module is used to acquire a texture mask representing the strength of image texture based on the first image captured by the first camera and the second image captured by the second camera;

[0027] The detail image acquisition module is used to process the first image and the second image based on the texture mask to obtain the base image corresponding to the first image and the detail image corresponding to the second image.

[0028] The target image acquisition module is used to fuse the base image corresponding to the first image and the detail image corresponding to the second image to obtain the target image.

[0029] Optionally, the texture mask acquisition module includes:

[0030] An image alignment module is used to align the first image and the second image;

[0031] The difference acquisition module is used to obtain the maximum and minimum values ​​of pixel values ​​within a preset range for each pixel point, based on each pixel point at the same position in the first image and the second image, and to obtain the difference between the maximum and minimum values.

[0032] The texture acquisition module is used to determine that the pixel is a pixel in a weak texture region when the difference between the first image and the second image is less than the set threshold; and to determine that the pixel is a pixel in a strong texture region when the difference between the first image and / or the second image exceeds the set threshold.

[0033] The mask generation module is used to traverse each pixel in the overlapping area of ​​the first image and the second image and generate a texture mask that represents the strength of the image texture. The texture mask is also used to characterize pixels in strong texture areas and pixels in weak texture areas.

[0034] Optionally, the detailed image acquisition module includes:

[0035] The texture mask filtering submodule is used to perform pyramid decomposition on the first image and the second image respectively, and to filter the decomposed images according to the texture mask to obtain the base image corresponding to the first image and the base image corresponding to the second image.

[0036] The detail image acquisition submodule is used to acquire the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image corresponding to the second image.

[0037] Optionally, the texture mask filtering submodule includes:

[0038] The first image acquisition module is used to downsample the input image m times to obtain a first image; the input image includes either the first image or the second image; m is greater than or equal to 2;

[0039] The base image acquisition module is used to upsample the first image m times to obtain the base image corresponding to the input image; the scale of the base image is the same as the scale of the input image.

[0040] Optionally, the texture mask filtering submodule is further configured to:

[0041] Both the strong and weak texture regions of the image decomposed by the last upsampling are filtered; in each subsequent upsampling, the strong texture regions in the image decomposed by each downsampling and each upsampling are filtered according to the texture mask.

[0042] Optionally, the first image and the second image include YUV three-channel sub-images; the texture mask filtering submodule is used to filter the Y channel sub-image using the texture mask.

[0043] Optionally, the target image acquisition module is further configured to acquire the sum of pixel values ​​of the same channel sub-images in the base image and the detail image to obtain sub-images of each channel of the target image; and generate the target image based on the sub-images of each channel.

[0044] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor;

[0045] The memory is used to store computer programs that can be executed by the processor;

[0046] The processor is used to execute the computer program in the memory to implement the method as described above.

[0047] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the method described above.

[0048] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0049] As can be seen from the above embodiments, the solution provided in this disclosure can obtain a texture mask representing the strength of image texture based on a first image captured by a first camera and a second image captured by a second camera. Then, based on the texture mask, the first image and the second image are processed respectively to obtain a base image corresponding to the first image and a detail image corresponding to the second image. Afterwards, the base image corresponding to the first image and the detail image corresponding to the second image are fused to obtain the target image. In this way, this embodiment can use a texture mask to process the first image and the second image, which can avoid generating ripples during image processing and achieve a fusion effect with a unified style between the first image and the second image, which is beneficial to improving the shooting experience.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0052] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0053] Figure 2 This is a flowchart illustrating an example of obtaining a texture mask according to an exemplary embodiment.

[0054] Figure 3 This is a schematic diagram illustrating the effect of aligning a first image and a second image according to an exemplary embodiment.

[0055] Figure 4 This is a flowchart illustrating an embodiment of acquiring a detailed image of a second image.

[0056] Figure 5 This is a schematic diagram illustrating a pyramid decomposition according to an exemplary embodiment.

[0057] Figure 6 This is a flowchart illustrating a pyramid decomposition according to an exemplary embodiment.

[0058] Figure 7This is a schematic diagram illustrating another pyramid decomposition according to an exemplary embodiment.

[0059] Figure 8 This is a schematic diagram illustrating the acquisition of a target image according to an exemplary embodiment.

[0060] Figure 9 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0061] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described below by way of example do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatus consistent with some aspects of this disclosure as detailed in the appended claims. It should be noted that, without conflict, the following embodiments and features in the implementation methods can be combined with each other.

[0063] In related applications, due to the different configuration parameters of the main camera and the telephoto camera when capturing images, the images captured by the two cameras may differ in terms of color and brightness. Therefore, related technologies can extract the texture features of the two images, filter the texture features, and then fuse the texture features into the image captured by the main camera, thereby eliminating differences in brightness and color. However, in the process of repeatedly enhancing and filtering the texture features of the original image in related technologies, ripples may be generated in some scenes (such as under light sources), affecting the fusion effect.

[0064] To address the aforementioned technical problems, this disclosure provides an image processing method applicable to electronic devices including a first camera and a second camera. The first camera has a larger field of view than the second camera, and the color brightness of the image captured by the first camera is superior to that of the image captured by the second camera; the resolution of the image captured by the second camera is superior to that of the image captured by the first camera. In related applications, the first camera is often referred to as the main camera, and the second camera is often referred to as a telephoto camera. For ease of description, the image captured by the first camera will be referred to as the first image, and the image captured by the second camera will be referred to as the second image, for illustrative purposes.

[0065] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. See also... Figure 1 An image processing method includes steps 11 to 13.

[0066] In step 11, a texture mask representing the strength of image texture is obtained based on the first image captured by the first camera and the second image captured by the second camera.

[0067] In this embodiment, when a user needs to take photos, they can operate the electronic device, i.e., activate the camera function. Once the camera function is activated, the first and second cameras can start simultaneously. The user can adjust parameters such as the orientation of the electronic device and the focal length of the cameras. When the user determines that the preview image meets their needs, they can click the shutter button or use voice control to take the picture. The electronic device can detect the user's actions. When it detects that a button has been triggered or detects audio data containing preset keywords, the electronic device can control the camera to capture images. It is understood that the camera outputs at least two frames: the first image captured by the first camera and the second image captured by the second camera.

[0068] In this embodiment, the processor in the electronic device can communicate with the camera to acquire the first image and the second image sent by the camera. Of course, the camera can also store the first image and the second image in a specified location such as local memory, cache, or the cloud. The processor can read the first image and the second image from the specified location to obtain the first image and the second image.

[0069] In this embodiment, after obtaining the first image and the second image, the processor can obtain a texture mask representing the strength of image texture based on the first image and the second image, see [link to relevant documentation]. Figure 2 This includes steps 21 to 24.

[0070] In step 21, the processor can align the first image and the second image. For example, the processor can use a preset recognition algorithm to identify feature points in the first image and feature points in the second image. The preset recognition algorithm can include, but is not limited to, the Scale Invariant Feature Transform (SIFT) algorithm. A suitable recognition algorithm can be selected based on the specific scenario, and is not limited here. Feature points can be pre-defined during the training of the preset recognition algorithm, such as object edges, corners, connection points, etc. A suitable feature method can be selected based on the specific scenario, and is not limited here. Then, the processor can match at least three feature points in the first image and the second image, and use these at least three feature points as reference feature points. Then, the processor can align the at least three reference feature points in the first image and the second image one by one, and then align the other pixels besides the reference feature points sequentially, thus finally aligning the first image and the second image, as shown in the image. Figure 3As shown. In other words, aligning the first and second images can be understood as placing the second image in the coordinate system of the first image, and ensuring that some pixels of the second image overlap with some pixels of the first image, resulting in an overlapping area between the two images.

[0071] For ease of description, the second images appearing in the following embodiments are either aligned second images or images transformed from aligned second images.

[0072] In step 22, the processor can use pixels at the same position within the overlapping area of ​​the first image and the second image as a reference to obtain the maximum and minimum values ​​of the pixel values ​​within a preset range for each pixel in each image, and obtain the difference between the maximum and minimum values. Wherein, obtaining pixels at the same position in the first and second images can be understood as obtaining pixels within the overlapping area of ​​the first and second images.

[0073] Taking the first image as an example, the processor can focus on each pixel within the overlapping region of the first image. For each pixel, the processor forms a preset range centered on that pixel, such as a 3x3 sliding window. The processor can then obtain the pixel values ​​of each pixel within this preset range, thereby obtaining the maximum and minimum pixel values. Next, the processor can calculate the difference between the maximum and minimum values, i.e., the processor can obtain the difference corresponding to that pixel. Based on the above method, the processor can obtain the difference corresponding to all pixels within the overlapping region of the first image. It is understood that at this point, misaligned pixels within the overlapping region are ignored, or in other words, this step assumes that all pixels within the overlapping region of the first and second images are aligned.

[0074] It should be noted that the processor also obtains the difference between pixels in the overlapping area of ​​the second image. For details, please refer to the process of obtaining the difference between pixels in the first image, which will not be repeated here.

[0075] In step 23, the processor may determine that the pixel is a pixel in a weak texture region when the difference corresponding to the first image is less than a set threshold and the difference corresponding to the second image is less than the set threshold; and determine that the pixel is a pixel in a strong texture region when the difference corresponding to the first image exceeds the set threshold and / or the difference corresponding to the second image exceeds the set threshold.

[0076] In this step, the electronic device can store a set threshold value, which ranges from 1 to 10, and the unit is grayscale value. The processor can obtain the set threshold value and compare the difference between each pixel with the set threshold value to obtain the relationship between the difference value and the set threshold value.

[0077] After comparing the differences between all pixels in the first and second images, the processor can determine that the difference in the first image is less than a set threshold, and the difference in the second image is also less than a set threshold. When the differences between the two images are both less than the set threshold, the pixel corresponding to the difference is determined to be a pixel in a weak texture region, meaning that the corresponding pixel in both the first and second images is a pixel in a weak texture region.

[0078] When the difference between the first image and / or the difference between the second image exceeds the set threshold, that is, when one of the differences between the two images is greater than the set threshold, the pixel corresponding to the difference is determined to be a pixel in a strong texture region, that is, the corresponding pixel in the first image and the second image are both pixels in strong texture regions.

[0079] In practical applications, after determining the pixels in the weak texture region and the pixels in the strong texture region, the processor can add a type identifier to each pixel. For example, the type identifier of the pixel in the strong texture region is 1, and the type identifier of the pixel in the weak texture region is 0. If the type of the pixels in the strong and weak texture regions can be adjusted, a reasonable type identifier can be selected, and the corresponding scheme falls within the protection scope of this disclosure.

[0080] In step 24, the processor can traverse each pixel at the same position within the overlapping region of the first and second images and generate a texture mask representing the strength of the image texture. This texture mask is also used to characterize pixels in strong texture regions and pixels in weak texture regions. In this step, the texture mask can be represented by a matrix, where elements at different positions are the type identifiers of each pixel. Of course, the texture mask can also be implemented using a multi-valued image. Subsequent embodiments will use a matrix as an example for the texture mask. For instance, if the type identifier 'a' of pixel (x1, y1) within the overlapping region is 'a', then the element in the x1-th row and y1-th column of the matrix will have the value 'a'. That is, in this step, the texture mask can characterize whether each pixel within the overlapping region is located in a strong texture region or a weak texture region. Thus, the texture mask in this step can guide the filtering process in subsequent image processing, masking weak texture regions so that pixels in weak texture regions are not filtered or are filtered less, avoiding ripples during the filtering process. In other words, the texture mask can prevent ripples at the source, which is beneficial for improving subsequent visual effects.

[0081] In conjunction with step 24, the texture mask in this disclosure refers to a mask used to shield weak texture regions in the first and second images, preventing them from participating in processing or parameter calculation, or only processing or statistically processing the shielded regions. It is understood that the texture mask used in this disclosure is a weak texture mask.

[0082] In step 12, the first image and the second image are processed based on the texture mask to obtain the base image corresponding to the first image and the detail image corresponding to the second image.

[0083] See Figure 4 In step 41, the processor can perform pyramid decomposition on the first image and the second image respectively, and filter the decomposed images according to the texture mask to obtain the base image (baseimage layer) corresponding to the first image and the base image corresponding to the second image. In step 42, the processor can obtain the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image (detail image layer) corresponding to the second image.

[0084] It should be noted that, considering the need to fuse the base image and detail images to obtain the target image, the base image in this disclosure can also be referred to as the base image layer, and the detail image can also be referred to as the detail image layer. The base image mainly reflects the basic image quality and structure, while the detail image mainly reflects the texture and details.

[0085] It's important to note that pyramid decomposition refers to downsampling an image at progressively smaller scales to obtain downsampled images at different scales. In this step, scale is equivalent to the number of pixels in the image; the larger the scale, the more pixels, the clearer the image, and the richer the details; the smaller the scale, the fewer pixels, and the blurrier the image. Generally, the pyramid decreases in scale from the bottom up, resulting in fewer pixels and less detail.

[0086] In one embodiment, see Figure 5 The pyramid decomposition is configured with m+1 levels, with the unsampled input image (i.e., the first or second image) at the bottom. The scale of the first image is called the original scale. The first image is Gaussian filtered (other filtering methods can also be used, this disclosure is not limited to this), and then downsampled to the first scale to obtain the first-scale downsampled image (referred to as the first downsampled image). The first downsampled image is then filtered and downsampled to the second scale to obtain the second downsampled image; and so on, until the m-th downsampled image is obtained. The scales from the first to the m-th scale are progressively smaller, meaning the number of pixels in the first to the m-th downsampled image decreases sequentially. Thus, downsampled images at different scales have different detail information. Therefore, by extracting the difference information between downsampled images at different scales, the detail image corresponding to the input image can be obtained, called the detail image corresponding to the input image.

[0087] In this embodiment, the processor can upsample the downsampled image at scale i to scale i-1 to obtain the upsampled image at scale i-1, and determine the detail image at scale i-1 based on the difference between the downsampled image at scale i-1 and the upsampled image at scale i-1, where i is any positive integer in [2, m].

[0088] Here, the downsampled image at scale i can represent any image from the second downsampled image to the m-th downsampled image. Upsampling the i-th downsampled image to scale i-1 can recover the number of pixels in the image, but it cannot completely recover the detail information. That is, the upsampled image at scale i-1 (referred to as the i-1 upsampled image) lacks detail information compared to the i-1 downsampled image, and the difference between the two images is the detail information. For example, subtracting the i-1 downsampled image from the i-1 upsampled image at the pixel level yields the i-1 scale detail image (referred to as the i-1 detail image).

[0089] For the first downsampled image, it can be upsampled to the original scale of the input image to obtain an upsampled image at the original scale, which has the same number of pixels as the input image. Then, the detail image at the original scale is determined based on the difference between the input image and the upsampled image at the original scale.

[0090] See also Figure 5 The m-th downsampled image is upsampled to the (m-1)-th scale and Gaussian filtered to obtain the (m-1)-th upsampled image; the (m-1)-th downsampled image is upsampled to the (m-2)-th scale and Gaussian filtered to obtain the (m-2)-th upsampled image; and so on, until the 1-th downsampled image is obtained. Furthermore, the 1-th downsampled image can be upsampled to the original scale and Gaussian filtered to obtain the original scale upsampled image. Subtracting the (m-1)-th upsampled image from the (m-1)-th downsampled image yields the (m-1)-th detail image; similarly, the (m-2)-th detail images to the 1-th detail image can be obtained. Further, subtracting the image from the original scale upsampled image yields the original scale detail image.

[0091] In this embodiment, a texture mask is used when performing Gaussian filtering on the image during the pyramid decomposition process described above. Specifically, the acquisition... Figure 5 In the first column of the downsampled image, Gaussian filtering uses a texture mask, meaning pixels in weak texture areas are not filtered, while pixels in strong texture areas are filtered. (Acquisition) Figure 5 During the upsampling process in the second column, Gaussian filtering does not use a texture mask; that is, all pixels in the overlapping area are filtered.

[0092] In this embodiment, both the first image and the second image include sub-images with YUV three channels. The texture mask is used for the filtering process of the Y channel sub-image, i.e., for brightness correction, but not for the filtering process of the UV channel sub-image, i.e., not for color correction. This reduces the amount of data calculation and improves the correction efficiency while ensuring the brightness correction.

[0093] It should be noted that both the first and second images undergo the same pyramid decomposition process, yielding detail images from the (m-1)th to the first detail image and the original scale detail image. For ease of explanation, the (m-1)th to the first detail image and the original scale detail images obtained from the first image are referred to as the base images corresponding to the first image; similarly, the (m-1)th to the first detail image and the original scale detail images obtained from the second image are referred to as the detail images corresponding to the second image.

[0094] In another embodiment, see Figure 6 In step 61, the processor can downsample the input image m times to obtain the first image. The first image refers to the m-th downsampled image at the m-th scale; that is, after the input image is downsampled once, a downsampled image at the first scale is obtained. The downsampled image at the first scale is then downsampled a second time to obtain a downsampled image at the second scale, and so on, until the m-th downsampled image at the m-th scale, which is the first image. The effect is as follows: Figure 7 As shown. In step 62, the processor can perform m upsampling operations on the first image to obtain the base image corresponding to the input image, wherein the scale of the base image is the same as the scale of the input image, i.e., the original scale. Specifically, m upsampling operations refer to upsampling the downsampled image at scale m to scale (m-1), obtaining the (m-1)th upsampled image at scale (m-1); upsampling the upsampled image at scale (m-1) to scale (m-2), obtaining the (m-2)th upsampled image; and so on, to obtain the base image at the original scale.

[0095] In this embodiment, when the input image is a first image, the upsampled image at the original scale is used as the base image of the first image. When the input image is a second image, the upsampled image at the original scale is used as the base image of the second image, and then the detail image corresponding to the second image is obtained by subtracting the base image of the second image from the second image.

[0096] It should be noted that after the first and second images are aligned, a very small number of pixels in the overlapping area remain misaligned. These misaligned pixels, after multiple filtering processes, will have their original pixel values ​​quantized, resulting in new ripples during superposition. This ripple effect is particularly pronounced in weakly textured areas containing light sources within the shooting scene. In this embodiment... Figure 7The proposed scheme leverages the advantages of pyramid decomposition. Through multiple downsampling operations, it can separate the detailed information of the first and second images as much as possible, thereby avoiding ghosting when merging the misaligned areas of the base image corresponding to the first image and the detailed image corresponding to the second image. This makes the base image corresponding to the first image and the detailed image corresponding to the second image robust.

[0097] In this embodiment, the processor can perform filtering during upsampling and downsampling, such as mean filtering. Specifically, both strong and weak texture regions in the image decomposed by the last upsampling are filtered. In each subsequent downsampling and upsampling, the strong texture regions are filtered according to the texture mask, while the weak texture regions are not filtered. Thus, the final filtering in this example allows for a smooth transition of texture edges.

[0098] It should be noted that, in comparison Figure 5 and Figure 7 The pyramid decomposition process shown in the two examples differs in the following ways: First, Figure 5 The upsampling process occurs after each downsampling step; that is, the object of upsampling is the downsampled image. Figure 7 The upsampling process is based on the previous upsampled image. Therefore, the upsampling time and target are different in the two schemes. Second, Figure 5 The pyramid decomposition process shown yields multiple detailed images, while Figure 7 The pyramid decomposition process shown yields a base image, i.e., an upsampled image at the original scale. Third, Figure 5 In the pyramid decomposition process shown, downsampling and upsampling are performed simultaneously, while Figure 7 In the pyramid decomposition process shown, downsampling occurs before upsampling.

[0099] The comparison shows that, Figure 7 The pyramid decomposition process shown has a smaller data processing volume and less time overhead, making it suitable for electronic devices with limited computing resources. Figure 5 The data processing volume in the pyramid decomposition process shown is large, which can be adapted to electronic devices with abundant computing resources.

[0100] In step 13, the base image corresponding to the first image and the detail image corresponding to the second image are fused to obtain the target image.

[0101] In this embodiment, the processor can fuse the base image corresponding to the first image and the detail image corresponding to the second image.

[0102] by Figure 5 For example, see the pyramid decomposition scheme shown below. Figure 8The process involves fusing the base image corresponding to the first image and the detail image corresponding to the second image. Specifically, the m-th downsampled image of the first image is fused with the m-th downsampled image of the second image to obtain the m-th target detail image; the (m-1)-th detail image of the first image is fused with the (m-1)-th detail image of the second image to obtain the (m-1)-th target detail image; and so on. Finally, the original-scale detail image of the first image is fused with the detail images of the second image to obtain the target detail image at the original scale. Then, the processor can fuse the m-th target detail image, the (m-1)-th target detail image, ..., the first target detail image and the original-scale target detail image. Specifically, the m-th target detail image is upsampled to the (m-1)-th scale and added to the (m-1)-th target detail image to obtain the (m-1)-th target image; the (m-1)-th target image is upsampled to the (m-2)-th scale and added to the (m-2)-th target detail image to obtain the (m-2)-th target image; and so on, to obtain the first target image. The first target image is upsampled to the original scale and added to the target detail image at the original scale to obtain the target image at the original scale.

[0103] by Figure 7 Taking the pyramid decomposition scheme shown as an example, the base image corresponding to the first image and the detail image corresponding to the second image are added together to obtain the target image. In this example, there is one base image and one detail image, so the computational cost is much smaller than that of a typical pyramid decomposition scheme. Figure 5 The computational cost of the proposed scheme.

[0104] Therefore, the solution provided in this embodiment can obtain a texture mask representing the strength of image texture based on a first image captured by a first camera and a second image captured by a second camera. Then, based on the texture mask, the first image and the second image are processed respectively to obtain a base image corresponding to the first image and a detail image corresponding to the second image. Afterward, the base image corresponding to the first image and the detail image corresponding to the second image are fused to obtain the target image. In this way, this embodiment can use a texture mask to process the first image and the second image, that is, the texture strength can be determined according to the texture mask and whether filtering is performed. This can avoid generating ripples in the image processing process from the source, and can achieve a fusion effect with unified brightness style of the first image and the second image, which is beneficial to improving the shooting experience.

[0105] Based on the image processing method provided in the embodiments of this disclosure, the embodiments of this disclosure also provide an image processing apparatus applied to an electronic device, the electronic device including a first camera and a second camera, see [link to relevant documentation]. Figure 9 The device includes:

[0106] The texture mask acquisition module 91 is used to acquire a texture mask representing the strength of image texture based on the first image captured by the first camera and the second image captured by the second camera;

[0107] The detail image acquisition module 92 is used to process the first image and the second image based on the texture mask to obtain the base image corresponding to the first image and the detail image corresponding to the second image;

[0108] The target image acquisition module 93 is used to fuse the base image corresponding to the first image and the detail image corresponding to the second image to obtain the target image.

[0109] In one embodiment, the texture mask acquisition module includes:

[0110] An image alignment module is used to align the first image and the second image;

[0111] The difference acquisition module is used to obtain the maximum and minimum values ​​of pixel values ​​within a preset range for each pixel in the overlapping area of ​​the first image and the second image, respectively, based on each pixel in the same position. The difference between the maximum and minimum values ​​is then obtained.

[0112] The texture acquisition module is used to determine that the pixel is a pixel in a weak texture region when the difference between the first image and the second image is less than the set threshold; and to determine that the pixel is a pixel in a strong texture region when the difference between the first image and / or the second image exceeds the set threshold.

[0113] The mask generation module is used to traverse each pixel at the same position in the overlapping area of ​​the first image and the second image, and generate a texture mask that represents the strength of the image texture. The texture mask is also used to characterize pixels in strong texture areas and pixels in weak texture areas.

[0114] In one embodiment, the detail image acquisition module includes:

[0115] The texture mask filtering submodule is used to perform pyramid decomposition on the first image and the second image respectively, and to filter the decomposed images according to the texture mask to obtain the base image corresponding to the first image and the base image corresponding to the second image.

[0116] The detail image acquisition submodule is used to acquire the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image corresponding to the second image.

[0117] In one embodiment, the texture mask filtering submodule includes:

[0118] The first image acquisition module is used to downsample the input image m times to obtain a first image; the input image includes either the first image or the second image; m is greater than or equal to 2;

[0119] The base image acquisition module is used to upsample the first image m times to obtain the base image corresponding to the input image; the scale of the base image is the same as the scale of the input image.

[0120] In one embodiment, the texture mask filtering submodule is further configured to:

[0121] Both the strong and weak texture regions of the image decomposed by the last upsampling are filtered; in each subsequent upsampling, the strong texture regions in the image decomposed by each downsampling and each upsampling are filtered according to the texture mask.

[0122] In one embodiment, the first image and the second image include YUV three-channel sub-images; the texture mask filtering submodule is used to filter the Y-channel sub-image using the texture mask.

[0123] In one embodiment, the target image acquisition module is further configured to acquire the sum of pixel values ​​of the same channel sub-images in the base image and the detail image to obtain sub-images of each channel of the target image, wherein the sub-images of each channel constitute the target image.

[0124] It should be noted that the apparatus shown in this embodiment matches the content of the method embodiment, and the content of the above method embodiment can be referred to, which will not be repeated here.

[0125] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, the electronic device 1000 may be a smartphone, computer, digital broadcasting terminal, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0126] Reference Figure 10 The electronic device 1000 may include one or more of the following components: processing component 1002, memory 1004, power supply component 1006, multimedia component 1008, audio component 1010, input / output (I / O) interface 1012, sensor component 1014, communication component 1016, and image acquisition component 1018.

[0127] Processing component 1002 typically controls the overall operation of electronic device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute computer programs. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0128] Memory 1004 is configured to store various types of data to support the operation of electronic device 1000. Examples of such data include computer programs for any application or method operating on electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0129] Power supply component 1006 provides power to various components of electronic device 1000. Power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1000. Power supply component 1006 may include a power chip, and a controller may communicate with the power chip to control the power chip to turn on or off switching devices, enabling or disabling battery power supply to the motherboard circuitry.

[0130] Multimedia component 1008 includes a screen that provides an output interface between electronic device 1000 and target object. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input information from the target object. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0131] Audio component 1010 is configured to output and / or input audio file information. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio file information when electronic device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio file information may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio file information.

[0132] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc.

[0133] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of electronic device 1000. For example, sensor assembly 1014 can detect the on / off state of electronic device 1000, the relative positioning of components (e.g., the display screen and keypad of electronic device 1000), changes in position of electronic device 1000 or a component, the presence or absence of contact between a target object and electronic device 1000, the orientation or acceleration / deceleration of electronic device 1000, and temperature changes of electronic device 1000. In this example, sensor assembly 1014 may include a magnetic sensor, a gyroscope, and a magnetic field sensor, wherein the magnetic field sensor includes at least one of the following: a Hall sensor, a thin-film magnetoresistive sensor, and a magnetic fluid accelerometer.

[0134] Communication component 1016 is configured to facilitate wired or wireless communication between electronic device 1000 and other devices. Electronic device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast information or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0135] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital information processors (DSPs), digital information processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0136] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, wherein the executable computer program described above can be executed by a processor. The readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0137] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0138] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device, the electronic device including a first camera and a second camera, the method includes: Based on the first image captured by the first camera and the second image captured by the second camera, a texture mask representing the strength of image texture is obtained; the texture mask is used to characterize pixels in strong texture regions and pixels in weak texture regions. Based on the texture mask, the first image and the second image are processed respectively to obtain the base image corresponding to the first image and the detail image corresponding to the second image; the base image is used to represent the basic image quality and structure of the image, and the detail image is used to represent the texture and details. The target image is obtained by fusing the base image corresponding to the first image and the detail image corresponding to the second image; Based on the first image captured by the first camera and the second image captured by the second camera, a texture mask representing the intensity of image texture is obtained, including: Align the first image and the second image; Using each pixel at the same position in the overlapping area of ​​the first image and the second image as a reference, the maximum and minimum values ​​of the pixel values ​​within a preset range of each pixel in each image are obtained, and the difference between the maximum and minimum values ​​is obtained. When the difference between the first image and the second image is less than the set threshold, the pixel is determined to be a pixel in a weak texture region; when the difference between the first image and / or the second image exceeds the set threshold, the pixel is determined to be a pixel in a strong texture region. Traverse each pixel at the same position within the overlapping area of ​​the first image and the second image, and generate a texture mask representing the strength of the image texture. The texture mask is represented by a matrix, and the elements at different positions in the matrix are the type identifiers of each pixel.

2. The method according to claim 1, characterized in that, Based on the texture mask, the first image and the second image are processed respectively to obtain the base image corresponding to the first image and the detail image corresponding to the second image, including: The first image and the second image are decomposed into pyramids respectively, and the images decomposed in each step are filtered according to the texture mask to obtain the base image corresponding to the first image and the base image corresponding to the second image. Obtain the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image corresponding to the second image.

3. The method according to claim 2, characterized in that, The pyramid decomposition includes: The input image is downsampled m times to obtain a first image; the input image includes either the first image or the second image; m is greater than or equal to 2; The first image is upsampled m times to obtain the base image corresponding to the input image; the scale of the base image is the same as the scale of the input image.

4. The method according to claim 3, characterized in that, The image decomposed in each step is filtered according to the texture mask, including: Both the strong and weak texture regions of the image decomposed by the last upsampling are filtered; in each subsequent upsampling, the strong texture regions in the image decomposed by each downsampling and each upsampling are filtered according to the texture mask.

5. The method according to claim 2, characterized in that, The first image and the second image include sub-images with YUV three channels; The image decomposed in each step is filtered according to the texture mask, including: The texture mask is used to filter the Y channel sub-image.

6. The method according to claim 1, characterized in that, The target image is obtained by fusing the base image corresponding to the first image and the detail image corresponding to the second image, including: The sum of pixel values ​​of the same channel sub-images in the base image and the detail image is obtained to obtain the sub-images of each channel of the target image; The target image is generated based on the sub-images of each channel.

7. An image processing apparatus, characterized in that, Applied to an electronic device, the electronic device including a first camera and a second camera, the device includes: The texture mask acquisition module is used to acquire a texture mask representing the strength of image texture based on a first image captured by the first camera and a second image captured by the second camera; the texture mask is used to characterize pixels in strong texture regions and pixels in weak texture regions; The detail image acquisition module is used to process the first image and the second image based on the texture mask to obtain a base image corresponding to the first image and a detail image corresponding to the second image; the base image is used to represent the basic image quality and structure of the image, and the detail image is used to represent the texture and details; The target image acquisition module is used to fuse the base image corresponding to the first image and the detail image corresponding to the second image to obtain the target image; The texture mask acquisition module includes: An image alignment module is used to align the first image and the second image; The difference acquisition module is used to obtain the maximum and minimum values ​​of pixel values ​​within a preset range for each pixel in the overlapping area of ​​the first image and the second image, respectively, based on each pixel at the same position in the overlapping area of ​​the first image and the second image, and to obtain the difference between the maximum and the minimum values. The texture acquisition module is used to determine that the pixel is a pixel in a weak texture region when the difference between the first image and the second image is less than the set threshold; and to determine that the pixel is a pixel in a strong texture region when the difference between the first image and / or the second image exceeds the set threshold. The mask generation module is used to traverse each pixel at the same position in the overlapping area of ​​the first image and the second image, and generate a texture mask that represents the strength of the image texture. The texture mask is represented by a matrix, and the elements at different positions in the matrix are the type identifiers of each pixel.

8. The apparatus according to claim 7, characterized in that, The detailed image acquisition module includes: The texture mask filtering submodule is used to perform pyramid decomposition on the first image and the second image respectively, and to filter the decomposed images according to the texture mask to obtain the base image corresponding to the first image and the base image corresponding to the second image. The detail image acquisition submodule is used to acquire the difference image between the second image and the base image corresponding to the second image, and use the difference image as the detail image corresponding to the second image.

9. The apparatus according to claim 8, characterized in that, The texture mask filtering submodule includes: The first image acquisition module is used to downsample the input image m times to obtain a first image; the input image includes either the first image or the second image; m is greater than or equal to 2; The base image acquisition module is used to upsample the first image m times to obtain the base image corresponding to the input image; the scale of the base image is the same as the scale of the input image.

10. The apparatus according to claim 9, characterized in that, The texture mask filtering submodule is also used for: Both the strong and weak texture regions of the image decomposed by the last upsampling are filtered; in each subsequent upsampling, the strong texture regions in the image decomposed by each downsampling and each upsampling are filtered according to the texture mask.

11. The apparatus according to claim 10, characterized in that, The first image and the second image include YUV three-channel sub-images; the texture mask filtering sub-module is used to filter the Y channel sub-image using the texture mask.

12. The apparatus according to claim 7, characterized in that, The target image acquisition module is further configured to acquire the sum of pixel values ​​of the same channel sub-images in the base image and the detail image to obtain sub-images of each channel of the target image; and generate the target image based on the sub-images of each channel.

13. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs that can be executed by the processor; The processor is configured to execute a computer program in the memory to implement the method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, When the executable computer program in the storage medium is executed by a processor, it can implement the method as described in any one of claims 1 to 6.

Images