Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, electronic equipment and storage medium, comprising: acquiring a plurality of images collected by a plurality of image acquisition modules in the electronic equipment; wherein the field of view of different image acquisition modules is different; in the image preview process, determining a first image from the plurality of images according to an editing operation acting on the display screen of the electronic equipment; cutting off the part of the first image that has no overlap with the content of the editing range acted on by the editing operation to obtain a first image region, and displaying the first image region; in the image storage process after the end of the image preview, determining a second image from the plurality of images according to the editing operation acting on the display screen of the electronic equipment; wherein the image acquisition module of the second image is different from the image acquisition module of the first image region; cutting off the part of the second image that has no overlap with the content of the editing range to obtain a second image region; and fusing the first image region and the second image region to obtain a cropped target image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Mobile phone cameras are usually equipped with multiple lenses with different focal lengths to achieve the best shooting results under different FOV (Field of view).

[0003] Typically, mid-to-low-end phones are equipped with wide (wide-angle lens) and UW (ultra-wide) lenses; high-end phones are also equipped with telephoto lenses, with telephoto lenses having a magnification ratio of 2x to 10x relative to wide lenses.

[0004] Generally, when taking a photo, users actively zoom in and out of the preview screen to obtain the desired FOV (field of view / image size). The phone automatically switches to the matching lens based on the user's desired FOV and saves the image at that FOV. These photos have limited post-processing editability in the photo album. For example, they cannot be reduced to obtain a larger FOV; and when zoomed in sufficiently, the lens with better image quality cannot be used.

[0005] Currently, there are technical solutions available to capture a larger field of view (FOV) during zooming out. However, there are no related technical solutions to utilize images from higher-quality cameras during zooming in, mainly due to resolution limitations. For example, a mobile phone might have three 12MP cameras with focal lengths of 0.5x, 1.0x, and 2.0x respectively. Figure 1 As shown, if you want to use an image from a 2x camera during a zoom-in operation, and stitch the image from the 1x camera onto the image from the 0.5x camera without loss of image quality, you first need to upsample the 12M 0.5x camera image to 48M (12M*4). This upsampled 0.5x image will then match the resolution of the 12M 1.0x camera image for stitching. The user then scales the stitched 48M image, and upon saving, the image is downsampled to 12M based on the user-selected area. To align the FOV and resolution of the 2x camera, the 1x camera needs to be upsampled to 48M (12M*4), and the 0.5x camera needs to be upsampled to 192M (12M*4*4), then stitched together to form a 192M image before any further operations are performed. This method of image processing can easily cause lag in electronic devices. Summary of the Invention

[0006] This disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0007] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0008] Acquire multiple images from multiple image acquisition modules in an electronic device; wherein, different image acquisition modules have different field-of-view sizes;

[0009] During the image preview process, based on the editing operation performed on the display screen of the electronic device, a first image is determined from the plurality of images; the portion of the first image that does not overlap with the editing range affected by the editing operation is cropped to obtain a first image area, and the first image area is displayed.

[0010] During image storage after the image preview ends, a second image is determined from the plurality of images based on the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image is different from the image acquisition module of the first image region; the portion of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0011] In some embodiments, cropping the portion of the first image that does not overlap with the editing area affected by the editing operation to obtain a first image region and displaying the first image region includes:

[0012] During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device;

[0013] The first image area is displayed after the image resolution is adjusted to the display resolution.

[0014] In some embodiments, the method includes:

[0015] If the refresh rate of the content display in the editing area during the image preview process is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device.

[0016] The first image region and the second image region after image resolution adjustment are merged to obtain a fused image;

[0017] The fused image is displayed.

[0018] In some embodiments, fusing the first image region and the second image region to obtain the cropped target image includes:

[0019] During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

[0020] In some embodiments, fusing the first image region and the second image region to obtain the cropped target image includes:

[0021] During image storage after the image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device;

[0022] The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

[0023] In some embodiments, cropping the portion of the second image that does not overlap with the content of the editing area to obtain a second image region; fusing the first image region and the second image region to obtain the cropped target image includes:

[0024] If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0025] In some embodiments, the field of view of the image acquisition module in the second image region is smaller than the field of view of the image acquisition module in the first image region.

[0026] In some embodiments, when the multiple image acquisition modules of the electronic device are arranged in descending order of field of view from largest to smallest, the image acquisition module of the second image region is located one position after the image acquisition module of the first image region.

[0027] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0028] The acquisition module is used to acquire multiple images captured by multiple image acquisition modules in an electronic device; wherein, the focal length values ​​of different image acquisition modules are different;

[0029] The preview module is used to determine a first image from a plurality of images during the image preview process based on the editing operation performed on the display screen of the electronic device; to crop out the portion of the first image that does not overlap with the editing range affected by the editing operation to obtain a first image area, and to display the first image area;

[0030] A fusion module is used to determine a second image from the plurality of images during image storage after the image preview ends, based on the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image is different from the image acquisition module of the first image area; the portion of the second image that does not overlap with the content of the editing range is cropped to obtain a second image area; the first image area and the second image area are fused to obtain a cropped target image.

[0031] In some embodiments, the preview module is used for:

[0032] During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device.

[0033] The image region is displayed after the image resolution is adjusted to the display resolution. The fusion module includes:

[0034] If the refresh rate of the content display in the editing area during image preview is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device.

[0035] The first image region and the second image region after image resolution adjustment are merged to obtain a fused image;

[0036] The fused image is displayed.

[0037] In some embodiments, the fusion module is configured to:

[0038] During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

[0039] In some embodiments, the fusion module is configured to:

[0040] During image storage after the image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device;

[0041] The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

[0042] In some embodiments, the fusion module is configured to:

[0043] If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0044] In some embodiments, the field of view of the image acquisition module in the second image region is smaller than the field of view of the image acquisition module in the first image region.

[0045] In some embodiments, when the multiple image acquisition modules of the electronic device are arranged in descending order of field of view from largest to smallest, the image acquisition module of the second image region is located one position after the image acquisition module of the first image region.

[0046] According to a third aspect of the present disclosure, a terminal is provided, including a processor, a memory, and an executable program stored in the memory and executable by the processor, characterized in that the processor executes the steps of the image processing method of the first aspect when running the executable program.

[0047] According to a fourth aspect of the present disclosure, a storage medium is provided having an executable program stored thereon, wherein the executable program, when executed by a processor, implements the steps of the image processing method described in the first aspect.

[0048] This disclosure provides an image processing method, apparatus, electronic device, and storage medium. By cropping the portion of the image that does not overlap with the editing area, during image preview and image storage, it is unnecessary to process all multiple images captured by the image acquisition modules. Only the first and second image regions that overlap with the editing area are processed, i.e., only the portion of the image in the user-enlarged area is processed, reducing the amount of data processed and thus reducing the load on the electronic device's memory and CPU. Furthermore, during image preview, only the first image region is displayed, reducing the amount of data processing during preview and improving the real-time performance of the preview process. During image storage after the preview, the first and second image regions are merged. This utilizes images captured by different image acquisition modules, fully leveraging the image quality advantages of each module, and the merging operation makes the image quality of the first and second image regions more matched and optimized.

[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0051] Figure 1 This is a schematic diagram showing the relative positional relationship of images acquired by different image acquisition modules during the resolution processing process.

[0052] Figure 2 This is one of the schematic flowcharts illustrating an image processing method according to an exemplary embodiment;

[0053] Figure 3 This is a second schematic diagram of an image processing method according to an exemplary embodiment;

[0054] Figure 4 This is one of the schematic diagrams illustrating the relative positional relationship between the content of the editing area and multiple images according to an exemplary embodiment;

[0055] Figure 5 This is a second schematic diagram illustrating the relative positional relationship between the content of the editing area and multiple images according to an exemplary embodiment;

[0056] Figure 6 This is the third schematic diagram illustrating the relative positional relationship between the content of the editing area and multiple images according to an exemplary embodiment;

[0057] Figure 7This is a third schematic diagram of an image processing method according to an exemplary embodiment;

[0058] Figure 8 This illustrates the relative positional relationship of multiple images according to an exemplary embodiment;

[0059] Figure 9 This illustrates the relationship between multiple images and focal length values ​​stored in an electronic device according to an exemplary embodiment.

[0060] Figure 10 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment;

[0061] Figure 11 This is a block diagram illustrating the structural composition of an image processing apparatus according to an exemplary embodiment. Detailed Implementation

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this disclosure as detailed in the appended claims.

[0063] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0064] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0065] The execution entities involved in the embodiments disclosed herein include, but are not limited to, electronic devices such as mobile phones, tablets, laptops, and wearable devices.

[0066] Figure 2This is a flowchart illustrating an image processing method according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following specific steps:

[0067] Step S101: Acquire multiple images from multiple image acquisition modules in the electronic device; wherein, the field of view of different image acquisition modules is different;

[0068] Step S102: During the image preview process, based on the editing operation performed on the display screen of the electronic device, a first image is determined from the plurality of images; the portion of the first image that does not overlap with the editing range affected by the editing operation is cropped to obtain a first image area, and the first image area is displayed.

[0069] Step S103: During the image storage process after the image preview ends, a second image region is determined from the plurality of images according to the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image region is different from the image acquisition module of the first image region; the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0070] In step S101, when taking a picture, regardless of which image acquisition module's field of view is displayed in the user's preview (i.e., regardless of the selected focal length value in the preview), the electronic device will simultaneously retain multiple images captured by multiple image acquisition modules. For example, the electronic device includes three image acquisition modules with focal lengths of 0.5x, 1.0x, and 2.0x, respectively. Even if the user previews an image acquisition module with a focal length of 0.5x, when taking a picture, the 0.5x, 1.0x, and 2.0x image acquisition modules will simultaneously capture images. The field of view of the 1.0x image acquisition module is within the range of the 0.5x image acquisition module, and the field of view of the 2.0x image acquisition module is within the range of the 1.0x image acquisition module. The three images captured by the three image acquisition modules will also be saved for subsequent editing operations.

[0071] This system stores multiple images captured by multiple image acquisition modules, enabling post-capture editing operations to capture images from modules with a larger field of view when zooming out. In other words, during zooming out, it allows editing of images corresponding to objects outside the preview field of view. This enriches the application scenarios of post-capture editing and improves the user experience regarding composition.

[0072] In step S102, "image preview process" refers to the preview process in editing mode after taking a picture.

[0073] The electronic device can detect editing operations applied to the display screen via a touch sensor and determine a first image based on the editing operation. The first image at least partially overlaps with the content of the editing area affected by the editing operation.

[0074] It is understandable that if the entire first image overlaps with the content of the editing area, the cropped portion is 0, and the first image area is equal to the first image.

[0075] Generally, the content of the editing range corresponds to an image captured by at least one acquisition module. If the content of the editing range corresponds to an image captured by only one acquisition module, then that image is the first image. Since there is no second image at this time, the first image is displayed both during the image preview process and during the image storage process after the image preview ends, without the need for fusion operation.

[0076] If the content of the editing range corresponds to images acquired by two or more acquisition modules, the first image refers to the image with the greatest overlap with the content of the editing range among multiple images. That is, among the multiple image acquisition modules corresponding to the image that overlaps with the content of the editing range, the image acquisition module of the first image has the largest field of view and the smallest focal length.

[0077] like Figure 8 As shown, generally, the smaller the focal length value, the larger the field of view of the image acquisition module; conversely, the larger the focal length value, the smaller the field of view of the image acquisition module.

[0078] like Figure 9 As shown, if an electronic device is equipped with a wide (wide-angle lens), a UW (ultra-wide) lens, and a telephoto lens, where the telephoto lens has a magnification ratio of 2x to 10x relative to the wide lens, then step S101 can save the following: Figure 9 The images shown are full-size original images acquired by all the image acquisition modules shown.

[0079] During image preview, only cropping operations can be performed, reducing the amount of data calculated during the image preview process and improving the real-time correspondence between the image preview display and the actual editing operation.

[0080] In step S103, similar to the first image, the electronic device can detect the editing operation applied to the display screen via a touch sensor and determine the second image based on the editing operation. The second image at least partially overlaps with the content of the editing area affected by the editing operation. If the second image completely overlaps with the content of the editing area, the cropped portion is 0, and the second image area is equal to the second image.

[0081] It is understood that in this embodiment of the disclosure, the editing operation refers to the zoom-in operation relative to the preview image before taking the picture. Different zoom levels result in different editing ranges, different first images, and correspondingly, different second images. For example, when the zoom level is the first level, the first image is the image captured by the 0.5x image acquisition module, and the second image is the image captured by the 1.0x image acquisition module. When the zoom level is the second level, the first image is the image captured by the 1.0x image acquisition module, and the second image is the image captured by the 2.0x image acquisition module; wherein, the second level is greater than the first level.

[0082] In this embodiment of the disclosure, fusion refers to replacing a corresponding portion of a low-resolution image with a high-resolution image. For example, replacing a corresponding portion of an image acquired by a 1.0x image acquisition module with an image acquired by a 0.5x image acquisition module.

[0083] During image storage, the fused target image is stored. The target image is obtained by fusing the first image region and the second image region. Therefore, the target image combines the first image and the second image, improving image quality and enabling the retrieval of images acquired by different image acquisition modules during magnification operations.

[0084] The target image has a higher resolution than the image resolution of the first image region displayed during the preview process;

[0085] And / or,

[0086] The fidelity of the target image is higher than that of the first image region displayed during the preview process.

[0087] It is understandable that, since both the first and second image regions overlap with the content of the editing range, the merged target image and the image displayed during the image preview process both represent the same physical scene, ensuring the consistency between the image preview process and the image storage process.

[0088] As can be seen from steps 102 and 103, this embodiment of the present disclosure separates the image preview process and the image storage process, which ensures both the real-time performance of the image preview process and the image quality of the target image.

[0089] In this embodiment, by cropping the non-overlapping portions of the editing area, it is unnecessary to process all multiple images captured by the image acquisition modules during the image preview and image storage processes. Only the first and second image regions that overlap with the editing area are processed, i.e., only the portion of the image in the user-enlarged area is processed, reducing the amount of data processed and thus reducing the load on the electronic device's memory and CPU. Furthermore, during image preview, only the first image region is displayed, reducing the amount of data processing during the preview process and improving the real-time performance of the preview. During image storage after the image preview ends, the first and second image regions are merged. This utilizes images captured by different image acquisition modules, fully leveraging the image quality advantages of each module, and the fusion operation makes the image quality of the first and second image regions more matched and optimized.

[0090] In some other alternative embodiments, cropping the portion of the first image that does not overlap with the editing area affected by the editing operation to obtain a first image region and displaying the first image region includes:

[0091] During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device.

[0092] The first image area is displayed after the image resolution is adjusted to the display resolution.

[0093] Generally, the resolution of the first image area is larger than the display resolution of the screen. Adjusting the resolution of the first area to the display resolution required by the screen reduces the amount of data in the first image area, thereby improving display efficiency while ensuring the display effect of the screen.

[0094] The first preset threshold can be any value between 5 and 20 frames per second. For example, the first preset threshold can be 5 frames per second, 8 frames per second, 10 frames per second, or 12 frames per second, etc.

[0095] For ease of description, the process of "when the refresh rate of the content display of the editing range is greater than or equal to a first preset threshold during the image preview process, adjusting the image resolution of the first image area to the display resolution of the electronic device; and displaying the first image area after the image resolution is adjusted to the display resolution" is referred to as the "simplified display" process.

[0096] In some other alternative embodiments, the method includes:

[0097] If the refresh rate of the content display in the editing area during the image preview process is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device.

[0098] The first image region and the second image region after image resolution adjustment are merged to obtain a fused image;

[0099] The fused image is displayed.

[0100] For ease of description, the process of forming a fused image from the first image region and the second image region is referred to as the "fine display" process. In practical applications, the "simplified display" process and the "fine display" process can be performed simultaneously. If the refresh rate of the content displayed within the editing range is greater than or equal to a first preset threshold, the "simplified display" process is completed, but the "fine display" process is not. In this case, the result of the "simplified display" is displayed, i.e., the first image region after the image resolution has been adjusted to the specified display resolution. If the refresh rate of the content displayed within the editing range is less than the first preset threshold, both the "simplified display" process and the "fine display" process are completed. In this case, only the structure of the "fine display" process is displayed, i.e., the fused image is displayed.

[0101] Because the fused image combines the first and second image regions, it has better image quality than the first image region whose resolution has been adjusted to the display resolution, and it is also closer to the saved target image. Therefore, displaying the fused image is more conducive to maintaining consistency with the target image.

[0102] However, the first image area after adjusting the display resolution can achieve faster display efficiency, which is beneficial for ensuring real-time performance. Therefore, based on whether the refresh rate of the content within the editing range reaches a first preset threshold, preview display is selectively performed, balancing real-time performance and display quality.

[0103] In addition, by first adjusting the resolution of both the first and second image regions to the display resolution, the amount of data in the first and second image regions is reduced, which in turn reduces the computational load of the fusion process and helps improve display efficiency.

[0104] In some other alternative embodiments, cropping the portion of the second image that does not overlap with the content of the editing area to obtain a second image region; and fusing the first image region and the second image region to obtain the cropped target image, includes:

[0105] If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0106] Without limitation, the second preset threshold can be any value among 60%, 70%, 80%, 90%, or 100%, or a range between any two values.

[0107] When the second preset threshold is reached, it indicates that the second image region accounts for a relatively large proportion of the editing range. At this time, the second image region can be used for subsequent fusion to improve the image quality of the target image or the image quality of the fused image. Otherwise, the second image region is not used, and only the first image region is processed.

[0108] In some other alternative embodiments, fusing the first image region and the second image region to obtain the cropped target image includes:

[0109] During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

[0110] In this embodiment of the disclosure, if the resolution of the first image region is the same as that of the second image region, the first image region and the second image region can be fused based on the original resolution of the first image and the second image to ensure the image quality of the target image.

[0111] In some other alternative embodiments, fusing the first image region and the second image region to obtain the cropped target image includes:

[0112] During image storage after image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device;

[0113] The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

[0114] Without limitation, the target resolution can be the larger of the resolutions in the first and second image regions, or it can be a value greater than the larger of the resolutions in the first and second image regions.

[0115] In other alternative embodiments, the field of view of the image acquisition module in the second image region is smaller than the field of view of the image acquisition module in the first image region.

[0116] In some application scenarios, among multiple images that overlap with the editing area, besides the first and second images, there may be a third or even a fourth image. When the multiple image acquisition modules of the electronic device are arranged in descending order of field of view, the acquisition module for the second image can be located one or two positions after the acquisition module for the first image area.

[0117] It is understandable that the smaller the field of view of the image acquisition module, the smaller the corresponding image area, and the larger the amount of data in the image area. Taking into account the image quality of the target image and the complexity of the electronic device hardware, when the multiple image acquisition modules of the electronic device are arranged in descending order of field of view, the image acquisition module of the second image area is ranked after the image acquisition module of the first image area.

[0118] In one specific embodiment, the electronic device is equipped with three cameras (i.e., an image acquisition module). The three cameras are configured with focal lengths of 0.5 x 12M, 1 x 12M, and 2 x 12M, respectively, where 0.5x, 1.0x, and 2.0x refer to the focal length values ​​of the cameras, and 12M (12 million pixels) refers to the camera resolution. The screen resolution is 2778*1284. The image processing method includes:

[0119] Step S210: When a user takes a photo, regardless of the fov value used for previewing, in addition to saving the image of the user's current fov value in the album, an additional 0.5x-1x-2x full-size photo is saved, that is, three images taken by the three cameras at the same time are saved. In actual products, this can be reduced as appropriate according to the actual scenario, but it will correspondingly limit the adjustment range / effect of the post-processing function.

[0120] Step S220: After taking the photo, the backend needs to preprocess the stored 0.5x-1x-2x images to complete "stereo correction" and "color alignment" based on the captured fovea (fo-ov). Stereo correction refers to correction based on the image's feature points, a common method for aligning images captured by two cameras in different poses. Color alignment refers to calculating the color differences in the images using a Look-Up Table (LUT) to bring the colors of the original image closer to the target image, reducing the differences.

[0121] Step S230: Enter edit mode from the album and start the "simplified display" process.

[0122] like Figure 3 As shown, step S230 includes:

[0123] Step S231: Obtain the current fov (i.e., obtain the content of the editing range, where the content of the editing operation range refers to the content corresponding to the editing operation applied to the display screen of the electronic device); the editing range can be referenced as follows: Figures 4 to 6 The image shows a thickened rectangle (100).

[0124] Step S232: The algorithm determines the camera (also known as the image sensor) with the longest focal length that meets the focal length requirement, thus identifying the first image. In this example, such as... Figures 4 to 6 As shown, the editing range 100 is located at the center of the first image. The first image only partially overlaps with the content of the editing range, while the second image completely overlaps with the content of the editing range. Figure 4 As shown, if the first image is an image captured by a 0.5x camera, proceed to step S323; Figure 5 As shown, if the first image is an image captured by a 1.0x camera, proceed to step S324; Figure 6 As shown, if the first image is an image captured by a 2.0x camera, proceed to step S325;

[0125] Step S233: Based on the image captured by the 0.5x camera, crop the corresponding area. That is, crop the part of the image captured by the 0.5x camera that does not overlap with the content of the editing operation area to obtain the first image area corresponding to the image captured by the 0.5x camera, referred to as the 0.5x first image area.

[0126] Step S234: Based on the image captured by the 1.0x camera, crop the corresponding area. That is, crop the part of the image captured by the 1.0x camera that does not overlap with the content of the editing operation area to obtain the first image area corresponding to the image captured by the 1.0x camera, referred to as the 1.0x first image area.

[0127] Step S235: Based on the image captured by the 2.0x camera, crop the corresponding area. That is, crop the portion of the image captured by the 2.0x camera that does not overlap with the content of the editing operation area, obtaining the first image area corresponding to the image captured by the 2.0x camera, referred to as the 2.0x first image area. For example... Figures 4 to 6 As shown, the second image region is the same as the second image, and can be simply referred to as the 0.5x second image region, the 1.0x second image region, and the 2.0x second image region, respectively.

[0128] Step S236: Adjust (rescale) the resolution of the first image area to the display resolution of the electronic device screen;

[0129] Step S237: Display output, that is, display the first image area after the image resolution is adjusted to the display resolution.

[0130] In step S230, as the user zooms in and out, the image with the corresponding fov changes, and a switch occurs between 0.5x, 1x, and 2x, i.e., the image is switched to the corresponding fov.

[0131] Step S240: While performing "simplified display," the "refined display" algorithm is executed. Once the "refined display" calculation is complete, its output replaces the "simplified display" image to improve image quality. For example... Figure 7 As shown, the "fine display" algorithm flow, i.e., step S240, includes:

[0132] Step S241: Obtain the content of the editing operation range, that is, determine the currently displayed screen.

[0133] Step S242: If the first image area corresponds to the image captured by the 0.5x camera, determine whether the screen ratio of the 1.0x second image area in the display area exceeds 70%. If yes, proceed to step S243; otherwise, proceed to step S248.

[0134] Step S243: Adjust (rescale) the 0.5x first image region and the 1.0x second image region to the display resolution respectively;

[0135] Step S244: Merge the 1.0x second image region after adjusting the resolution into the 0.5x first image region, and use it for preview display output;

[0136] Step S245: If the first image area corresponds to the image captured by the 1.0x camera, determine whether the screen area corresponding to the 2.0x second image area exceeds 70%. If yes, proceed to step S246; if no, proceed to step S248. If the first image area corresponds to the image captured by the 2.0x camera, also proceed to step S248.

[0137] Step S246: Adjust (rescale) the 1.0x first image region and the 2.0x second image region to the display resolution respectively;

[0138] Step S247: Merge the 2.0x second image region after adjusting the resolution into the 1.0x first image region, and use it for preview display output;

[0139] Step S248: No processing is required, i.e., no fusion processing is needed.

[0140] In step S250, the user repeats steps S230 and S240 while adjusting the screen.

[0141] In step S260, after the user completes the screen adjustment and saves the result, the image fusion is still performed according to the strategy in step S240. If the resolution of the first image region and the second image region is the same, steps S243 and S246 need to be removed, and the fusion operation is performed on the original resolution of the image.

[0142] Step S260: If the resolutions of the first image region and the second image region are inconsistent, a unified target resolution needs to be defined, that is, the display resolution in steps S243 and S246 is replaced with the target resolution to achieve a unified resolution of the result image.

[0143] In summary, this example employs a separate preview and save approach. The preview only processes images at the display resolution (2778*1284, approximately 3.5M), resulting in low computational load and near real-time performance. A "simplified display" process is also introduced to ensure real-time accuracy. This separation of preview and save avoids the problem of generating a single, massively resolution image across the entire zoom range, making full-range zoom possible. Both preview and save utilize a fusion method, ensuring both image quality optimization and consistency between the preview and the actual saved image.

[0144] like Figure 10 As shown, this disclosure also provides an image processing apparatus 500, comprising:

[0145] The acquisition module 510 is used to acquire multiple images acquired by multiple image acquisition modules in an electronic device; wherein, the focal length values ​​of different image acquisition modules are different.

[0146] The preview module 520 is used to determine a first image from the plurality of images during the image preview process based on the editing operation performed on the display screen of the electronic device; to crop out the portion of the first image that does not overlap with the content of the editing area affected by the editing operation, thereby obtaining a first image area, and to display the first image area.

[0147] The fusion module 530 is used to determine a second image from the plurality of images during the image storage process after the image preview ends, based on the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image is different from the image acquisition module of the first image area; the part of the second image that does not overlap with the content of the editing range is cropped to obtain a second image area; the first image area and the second image area are fused to obtain a cropped target image.

[0148] In other alternative embodiments, the preview module is used for:

[0149] During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device.

[0150] The first image area is displayed after the image resolution is adjusted to the display resolution.

[0151] In other alternative embodiments, the fusion module is configured to:

[0152] If the refresh rate of the content display in the editing area during the image preview process is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device.

[0153] The first image region and the second image region after image resolution adjustment are merged to obtain a fused image;

[0154] The fused image is displayed.

[0155] In other alternative embodiments, the fusion module is configured to:

[0156] During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

[0157] In other alternative embodiments, the fusion module is configured to:

[0158] During image storage after the image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device;

[0159] The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

[0160] In other alternative embodiments, the fusion module is configured to:

[0161] If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; the first image region and the second image region are merged to obtain the cropped target image.

[0162] In other alternative embodiments, the field of view of the image acquisition module in the second image region is smaller than the field of view of the image acquisition module in the first image region.

[0163] In other alternative embodiments, when the multiple image acquisition modules of the electronic device are arranged in descending order of field of view from largest to smallest, the image acquisition module of the second image region is located one position after the image acquisition module of the first image region.

[0164] This disclosure also provides an electronic device, including a processor, a memory, and an executable program stored in the memory and executable by the processor, wherein the processor executes the steps of the image processing method as described in any of the above embodiments when running the executable program.

[0165] This disclosure also provides a storage medium storing an executable program thereon, which, when executed by a processor, implements the steps of the image processing method as described in any of the above embodiments.

[0166] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0167] In an exemplary embodiment, the acquisition module, preview module, and fusion module may be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), baseband processors (BPs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0168] Figure 11This is a block diagram illustrating an apparatus 800 for ambient light compensation according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0169] Reference Figure 11 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0170] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0171] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0172] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0173] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0174] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0175] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0176] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0177] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0178] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0180] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0181] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Acquire multiple images from multiple image acquisition modules in an electronic device; wherein, different image acquisition modules have different field-of-view sizes; During the image preview process, a first image is determined from the plurality of images based on the editing operation performed on the display screen of the electronic device; the portion of the first image that does not overlap with the editing range affected by the editing operation is cropped to obtain a first image area, and the first image area is displayed. During image storage after the image preview ends, a second image is determined from the plurality of images based on the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image is different from the image acquisition module of the first image area; the portion of the second image that does not overlap with the content of the editing range is cropped to obtain the second image area; the first image area and the second image area are merged to obtain the cropped target image.

2. The image processing method according to claim 1, characterized in that, The step of cropping out portions of the first image that do not overlap with the editing area affected by the editing operation to obtain a first image region, and displaying the first image region, includes: During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device. The first image area is displayed after the image resolution is adjusted to the display resolution.

3. The image processing method according to claim 2, characterized in that, The method includes: If the refresh rate of the content display in the editing area during the image preview process is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device. The first image region and the second image region after image resolution adjustment are merged to obtain a fused image; The fused image is displayed.

4. The image processing method according to claim 1, characterized in that, The process of fusing the first image region and the second image region to obtain the cropped target image includes: During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

5. The image processing method according to claim 1, characterized in that, The process of fusing the first image region and the second image region to obtain the cropped target image includes: During image storage after the image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device; The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

6. The image processing method according to claim 1, characterized in that, The portion of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; The cropped target image is obtained by fusing the first image region and the second image region, including: If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; The first image region and the second image region are merged to obtain the cropped target image.

7. The image processing method according to claim 1, characterized in that, The field of view of the image acquisition module in the second image region is smaller than that of the image acquisition module in the first image region.

8. The image processing method according to claim 7, characterized in that, When the multiple image acquisition modules of the electronic device are arranged in descending order of field of view from largest to smallest, the image acquisition module of the second image region is located one position after the image acquisition module of the first image region.

9. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire multiple images captured by multiple image acquisition modules in an electronic device; wherein, the focal length values ​​of different image acquisition modules are different; The preview module is used to determine a first image from a plurality of images during the image preview process based on the editing operation performed on the display screen of the electronic device; to crop out the portion of the first image that does not overlap with the editing range affected by the editing operation to obtain a first image area, and to display the first image area; A fusion module is used to determine a second image from the plurality of images during image storage after the image preview ends, based on the editing operation performed on the display screen of the electronic device; wherein the image acquisition module of the second image is different from the image acquisition module of the first image area; the portion of the second image that does not overlap with the content of the editing range is cropped to obtain a second image area; the first image area and the second image area are fused to obtain a cropped target image.

10. The image processing apparatus according to claim 9, characterized in that, The preview module is used for: During the image preview process, when the refresh rate of the content display in the editing area is greater than or equal to a first preset threshold, the image resolution of the first image area is adjusted to the display resolution of the electronic device. The first image area is displayed after the image resolution is adjusted to the display resolution.

11. The image processing apparatus according to claim 10, characterized in that, The fusion module includes: If the refresh rate of the content display in the editing area during image preview is less than the first preset threshold, the image resolution of both the first image area and the second image area will be adjusted to the display resolution of the electronic device. The first image region and the second image region after image resolution adjustment are merged to obtain a fused image; The fused image is displayed.

12. The image processing apparatus according to claim 9, characterized in that, The fusion module is used for: During image storage after the image preview ends, if the resolution of the first image region is the same as the resolution of the second image region, the first image region and the second image region are merged to obtain the cropped target image.

13. The image processing apparatus according to claim 9, characterized in that, The fusion module is used for: During image storage after the image preview ends, if the resolution of the first image region is different from the resolution of the second image region, the image resolution of both the first image region and the second image region will be adjusted to the target resolution; wherein, the target resolution is greater than the display resolution of the electronic device; The target image is obtained by fusing the first image region and the second image region after image resolution adjustment.

14. The image processing apparatus according to claim 9, characterized in that, The fusion module is used for: If the percentage of the area occupied by the image content corresponding to the second image region in the content of the editing range reaches a second preset threshold, the part of the second image that does not overlap with the content of the editing range is cropped to obtain the second image region; The first image region and the second image region are merged to obtain the cropped target image.

15. The image processing apparatus according to claim 9, characterized in that, The field of view of the image acquisition module in the second image region is smaller than that of the image acquisition module in the first image region.

16. The image processing apparatus according to claim 14, characterized in that, When the multiple image acquisition modules of the electronic device are arranged in descending order of field of view from largest to smallest, the image acquisition module of the second image region is located one position after the image acquisition module of the first image region.

17. An electronic device comprising a processor, a memory, and an executable program stored in the memory and executable by the processor, characterized in that, When the processor runs the executable program, it performs the steps of the image processing method as described in any one of claims 1 to 8.

18. A storage medium having an executable program stored thereon, characterized in that, When the executable program is executed by a processor, it implements the steps of the image processing method as described in any one of claims 1 to 8.

Images