Image processing methods, devices and storage media

Abstract

This application provides an image processing method, apparatus, and storage medium. In this method, a second image is obtained by super-resolution processing of a first image with a zoom ratio greater than or equal to a first zoom ratio. The second image obtained after super-resolution processing has more texture features than the first image, thereby improving the clarity of the image with an excessively large zoom ratio.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method, device, and storage medium. Background Technology

[0002] With technological advancements, the shooting capabilities of electronic devices (such as smartphones and tablets with cameras) are constantly improving, making photography with electronic devices a common choice. Currently, electronic devices typically support image zoom when capturing images. Image zoom refers to the ability of an electronic device to change the size of the subject in an image by altering the zoom ratio. However, when the zoom ratio is too high, the resulting image may become blurry. Summary of the Invention

[0003] This application provides an image processing method, apparatus, and storage medium. The image processing method can perform super-resolution processing on a first image captured at a zoom level greater than or equal to a first zoom level to obtain a clearer second image.

[0004] In a first aspect, embodiments of this application provide an image processing method, the method comprising: encoding a first image to obtain a feature extraction matrix of the first image; the first image being captured by a device with a zoom ratio greater than or equal to a first zoom ratio; dividing the feature extraction matrix into blocks to obtain N first feature matrix blocks, where N is a positive integer greater than 1; performing super-resolution processing on each first feature matrix block to obtain a second feature matrix block corresponding to each first feature matrix block; and processing the second feature matrix blocks corresponding to each first feature matrix block to obtain a second image.

[0005] Optionally, the second feature matrix block corresponding to each first feature matrix block is processed to obtain a second image, including: concatenating the second feature matrix blocks corresponding to each first feature matrix block to obtain a processed feature extraction matrix, and decoding the processed feature extraction matrix to obtain a second image.

[0006] In the method described in this aspect, the electronic device can perform super-resolution processing on a first image with a zoom ratio greater than or equal to a first zoom ratio to obtain a second image. The second image obtained after super-resolution processing has more texture features than the first image, thereby improving the clarity of the image with an excessively large zoom ratio.

[0007] Furthermore, in this embodiment, the feature extraction matrix of the first image is divided into blocks before super-resolution processing. This allows for block-based super-resolution processing even for large first images, thus making the method applicable to a wide range of image sizes. Additionally, by performing super-resolution processing after block division, this embodiment improves image quality compared to scaling the first image before processing. Moreover, compared to separately applying super-resolution processing to large and small first images, it effectively saves computational resources.

[0008] In addition, since the embodiments of this application divide the feature extraction matrix located in the latent space into blocks, compared with the scheme of directly dividing the first image into blocks, it can avoid the stitching marks between image blocks.

[0009] In conjunction with the first aspect, in one optional implementation, each of the N feature matrix blocks overlaps with its adjacent first feature matrix blocks; super-resolution processing is performed on the k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block, including: selecting a first feature matrix block adjacent to the k-th first feature matrix block from the 1st to the (k-1)th first feature matrix blocks; k is a positive integer greater than 1; based on the overlapping area between the adjacent first feature matrix blocks and the k-th first feature matrix block, selecting the corresponding overlapping area from the second feature matrix blocks corresponding to the adjacent first feature matrix blocks; concatenating the overlapping areas of the second feature matrix blocks corresponding to the adjacent first feature matrix blocks with the non-overlapping areas of the k-th first feature matrix block to obtain the processed k-th first feature matrix block; super-resolution processing is performed on the processed k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block.

[0010] In this embodiment, the electronic device can perform overlapping block segmentation of the feature extraction matrix, and replace the overlapping region of the k-th first feature matrix block with the overlapping region of the second feature matrix block corresponding to the first feature matrix block adjacent to it. This is beneficial for preserving the features obtained by the adjacent first feature matrix blocks after super-resolution processing when performing super-resolution processing on the k-th first feature matrix block, thereby ensuring that the texture features of the image block corresponding to the k-th first feature matrix block are the same as the texture features of the image blocks corresponding to its adjacent first feature matrix blocks. In this way, even if the size of the first image is large, the second image obtained by segmenting, performing super-resolution processing, and then stitching together has continuous and consistent texture features on adjacent image blocks.

[0011] In conjunction with the first aspect, in one optional implementation, processing the second feature matrix block corresponding to each first feature matrix block to obtain a second image includes: calculating the mean and standard deviation of each first feature matrix block, and the mean and standard deviation of each second feature matrix block; inputting the mean and standard deviation of each first feature matrix block, the mean and standard deviation of each second feature matrix block, and the second feature matrix block into a distribution transformation formula to obtain a third feature matrix block corresponding to each first feature matrix block; and processing the third feature matrix block corresponding to each first feature matrix block to obtain a second image.

[0012] Optionally, the third feature matrix block corresponding to each first feature matrix block is processed to obtain the second image, including: concatenating the third feature matrix blocks corresponding to each first feature matrix block to obtain the processed feature extraction matrix, and decoding the processed feature extraction matrix to obtain the second image.

[0013] In this embodiment, a third feature matrix block can be obtained by performing style transfer on the second feature matrix block after super-resolution processing based on the first feature matrix block before super-resolution processing in the latent space. The third feature matrix block obtained after style transfer represents a color that is closer to the color represented by the first feature matrix block than the second feature matrix block. Thus, when the style transfer step is performed for each second feature matrix block, the color difference between the third feature matrix blocks obtained after style transfer is small. This can effectively avoid the block artifacts caused by the color difference between different image blocks, thereby making it easier for the color of the second image obtained after super-resolution processing to be the same as the color of the first image before super-resolution processing.

[0014] In conjunction with the first aspect, in one optional implementation, each of the N feature matrix blocks overlaps with its adjacent first feature matrix blocks; super-resolution processing is performed on the k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block, including: selecting a first feature matrix block adjacent to the k-th first feature matrix block from the 1st to the (k-1)th first feature matrix blocks; k is a positive integer greater than 1; based on the overlapping area between the adjacent first feature matrix blocks and the k-th first feature matrix block, selecting the corresponding overlapping area from the third feature matrix blocks corresponding to the adjacent first feature matrix blocks; concatenating the overlapping areas of the third feature matrix blocks corresponding to the adjacent first feature matrix blocks with the non-overlapping areas of the k-th first feature matrix block to obtain the processed k-th first feature matrix block; super-resolution processing is performed on the processed k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block.

[0015] In conjunction with the first aspect, in one optional implementation, the distribution transformation formula is:

[0016]

[0017] Where X represents the second feature matrix block, μ2 represents the mean of the second feature matrix block, σ2 represents the standard deviation of the second feature matrix block, μ1 represents the mean of the first feature matrix block, σ1 represents the standard deviation of the first feature matrix block, and output represents the third feature matrix block.

[0018] In conjunction with the first aspect, in an optional implementation, before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: detecting whether the shooting mode of capturing the first image is a super-resolution mode; the super-resolution mode is used to trigger the super-resolution processing operation performed on the first image whose zoom ratio is greater than or equal to the first zoom ratio; if the shooting mode is a super-resolution mode, then the step of encoding the first image to obtain the feature extraction matrix of the first image is performed.

[0019] In conjunction with the first aspect, in an optional implementation, before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: displaying a first user interface, the first user interface including a first control; the first control being used to enable a super-resolution mode; enabling the super-resolution mode when a first operation on the first control is detected; and capturing the first image. In this implementation, the user can directly enable the super-resolution mode through the first control of the first user interface, which is simpler and more convenient.

[0020] In conjunction with the first aspect, in an optional implementation, before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: displaying a second user interface, the second user interface including a second control; the second control being used to trigger the display of a first control; when a second operation on the second control is detected, displaying a third user interface, the third user interface including the first control; when a first operation on the first control is detected, activating a super-resolution mode; and capturing the first image. In this implementation, the first control can be triggered by the second control, making the user interface simpler.

[0021] Secondly, embodiments of this application also provide an electronic device, which includes: a central processing unit, a graphics processor, an image signal processor, and one or more memories; the one or more memories are coupled to the central processing unit, the graphics processor, and the image signal processor, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which cause the electronic device to perform the method of the first aspect when the central processing unit, the graphics processor, or the image signal processor executes the computer instructions.

[0022] Thirdly, embodiments of this application further provide a computer-readable storage medium, including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of the first aspect.

[0023] Fourthly, embodiments of this application further provide a computer program product that, when run on an electronic device, causes the electronic device to execute the method described in the first aspect. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0025] Figure 1 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0026] Figure 2 This is a software structure block diagram of the electronic device provided in the embodiments of this application;

[0027] Figures 3A to 3E This is a schematic diagram of the user interface provided in an embodiment of this application;

[0028] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0029] Figure 5 This is a schematic flowchart of another image processing method provided in an embodiment of this application. Detailed Implementation

[0030] The following explanations of some terms used in this application are provided to facilitate understanding by those skilled in the art.

[0031] The applications (Apps) involved in the embodiments of this application, also known as application programs, are software programs capable of performing one or more specific functions. Typically, multiple applications can be installed on an electronic device, such as instant messaging applications, audio applications, image capture applications, etc. Instant messaging applications may include, for example, SMS applications, WeChat, DingTalk, etc. Audio applications may include, for example, Kugou Music, Xiami Music, etc. Image capture applications may include, for example, camera applications (the system camera set at the factory or a third-party camera application). The applications mentioned in the following embodiments may be applications pre-installed at the factory of the electronic device, or applications downloaded by the user from the network or obtained from other electronic devices during the use of the electronic device.

[0032] It should be understood that, unless otherwise stated, " / " in this application means "or," for example, A / B can mean A or B; "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone. "At least one" means one or more, and "more" means two or more.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0034] Currently, electronic devices typically support image zoom when capturing images. During shooting, the position of the subject relative to the electronic device remains unchanged. If the zoom ratio increases, the field of view (FOV) of the electronic device decreases, making the subject appear larger in the image. However, when the zoom ratio is too high, the zoomed image may be blurry. For example, when using digital zoom, the electronic device needs to crop an image from the original image corresponding to the zoom ratio, and then upsample the cropped image to create an image of the same size as the original image as the zoomed image. The cropped image has reduced resolution, resulting in a blurry zoomed image. It should be noted that the original image refers to the image captured by the electronic device at 1x zoom.

[0035] Based on this, embodiments of this application provide an image processing method. In this method, a second image can be obtained by super-resolution processing of a first image with a zoom ratio greater than or equal to a first zoom ratio. The second image has more texture features than the first image, and the second image is clearer than the first image.

[0036] For example, the image processing method mentioned in the embodiments of this application can run in an electronic device, which can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, in-vehicle device, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc. The embodiments of this application do not impose special limitations on the specific form of the electronic device.

[0037] like Figure 1 The diagram illustrates a hardware structure of an electronic device. This device may include: a processor, an external memory interface, internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, antenna 1, antenna 2, a mobile communication module, a wireless communication module, a sensor module, buttons, a motor, an indicator, a camera, a display screen, and a SIM card slot. The audio module may include a speaker, a receiver, a microphone, and a headphone jack. The sensor module may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, proximity sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0038] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0039] The processor may include one or more processing units, such as an application processor (AP), a modem, a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The processor is the nerve center and command center of the electronic device. For example, the CPU, GPU, and ISP in the electronic device can execute the image processing method of the embodiments of this application.

[0040] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, a mobile communication module, a wireless communication module, and a modem. In some embodiments, antenna 1 of the electronic device is coupled to the mobile communication module, and antenna 2 is coupled to the wireless communication module, enabling the electronic device to communicate with network-side devices and other electronic devices through wireless communication technology.

[0041] Touch sensors, also known as "touch devices," can be located on a display screen. A touch sensor and a display screen together form a touchscreen, also called a "touchscreen." Touch sensors detect touch operations applied to or near the device. The touch sensor transmits the detected touch operation to the application (AP) to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen. In other embodiments, the touch sensor may also be located on the surface of the electronic device, in a different position than the display screen. In this embodiment, the electronic device can detect whether there is a user operation applied to the display screen using a touch sensor. After the touch sensor detects a user operation applied to a camera application, the electronic device can capture an image.

[0042] In addition, an operating system runs on top of the aforementioned components. Examples include Apple's iOS operating system, Google's Android open-source operating system, and Microsoft's Windows operating system.

[0043] The operating system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to exemplify the software structure of an electronic device. It should be noted that although this application uses the Android system as an example, its basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0044] See Figure 2 This is a software structure block diagram of an electronic device provided in an embodiment of this application. The software structure adopts a layered architecture, which divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. Taking the Android system as an example, in some embodiments, the Android system is divided into the application layer, application framework layer (Framework), hardware abstraction layer (HAL), and kernel layer from top to bottom.

[0045] The application layer can include a series of application packages. These application packages can include apps such as camera, gallery, calendar, call, map, WLAN, Bluetooth, music, video, and SMS.

[0046] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. The application framework layer includes predefined functions. For example, it may include a window manager, content provider, view system, resource manager, notification manager, etc. This application embodiment does not impose any limitations on these. For instance, the window manager is used to manage window programs. The window manager can obtain the screen size, determine if a status bar exists, lock the screen, capture the screen, etc. The content provider is used to store and retrieve data, making this data accessible to applications. The data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc. The view system can be used to construct the application's display interface. Each display interface can consist of one or more controls. Generally, controls may include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other interface elements. The resource manager provides various resources to the application, such as localized strings, icons, images, layout files, video files, etc. The notification manager described above allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also appear as an icon or scrollbar text in the system's top status bar, such as notifications from background applications, or as a dialog window on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating, or flashing indicator lights. In some embodiments, the application framework layer may also include a Camera Service. The Camera Service can interact with the Hardware Abstraction Layer.

[0047] The Hardware Abstraction Layer (HAL) is an interface layer located between the kernel layer and the hardware, used to abstract the hardware. In some embodiments, the HAL includes a Hardware Abstraction Layer Interface Definition Language (HIDL) interface. The HAL may include camera HAL, display HAL, sensor HAL, etc. In some embodiments, the HAL may include an algorithm library. Optionally, the algorithm library provides one or more algorithm models, such as at least one of an encoding model, a decoding model, a super-resolution model, and a style transfer model. The encoding model performs encoding processing, the decoding model performs decoding processing, the super-resolution model performs super-resolution processing, and the style transfer model performs style transfer processing (e.g., the style transfer model performs style transfer processing based on a distribution transformation formula).

[0048] The kernel layer is the foundation of the Android operating system, responsible for hardware drivers, networking, power, system security, and memory management. It acts as an intermediary between hardware and software, relaying application requests to the hardware. The kernel layer includes at least camera drivers, sensor drivers, display drivers, and audio drivers.

[0049] The following is a detailed schematic diagram of the user interface for an electronic device implementing the image processing method provided in the embodiments of this application.

[0050] The image processing method provided in this application embodiment can be applied in conjunction with a photography method provided in this application embodiment. In this photography method, the electronic device can provide a zoom function, which will be discussed below. Figures 3A-3E The user interface of the photographing method involved in the embodiments of this application will be described in detail.

[0051] like Figure 3A As shown, the electronic device can detect a user's touch operation on the camera application 301 within the user interface 31, and in response to this operation, the electronic device can display as shown in the image. Figure 3B The user interface 32 shown is a camera application's shooting interface, where the electronic device can capture images in response to user operations within the user interface 32. The camera application is an image-capturing application on the electronic device; this application does not limit the name of such application. That is, the user can capture images by clicking on... Figure 3A The camera application 301 shown is used to open the camera application's user interface 32. However, users can also open the user interface 32 in other applications; for example, a user can click the shooting control in an instant messaging application (such as WeChat) to open the user interface 32.

[0052] Figure 3B An example is shown of a user interface 32 for a camera application on an electronic device. For example... Figure 3B As shown, the user interface 32 may include: a preview frame 311, a shooting mode list 312, a gallery shortcut key 313, a shutter control 314, a camera flip control 315, and a zoom progress bar 316. Among them:

[0053] The preview frame 311 can be used to display images captured in real time by the camera of an electronic device. The electronic device can refresh the displayed content in real time so that the user can preview the image currently captured by the camera.

[0054] The shooting mode list 312 may display one or more shooting mode options. These options may include, but are not limited to: Portrait mode option 312A, Video mode option 312B, Photo mode option 312C, Super-resolution mode option 312D, and More options 312E. These options may be displayed as text information on the interface, such as "Portrait," "Video," "Photo," "Super-resolution," and "More." However, they may also be displayed as icons or other interactive elements (IE). The Super-resolution mode option 312D is used to enable super-resolution mode. In response to user actions (such as clicks) on the Super-resolution mode option 312D, the electronic device can activate super-resolution mode. Super-resolution mode can be understood as the ability of the electronic device to perform super-resolution processing on images captured at a zoom level greater than the first zoom level. In other words, super-resolution mode is used to trigger super-resolution processing operations on images captured at a zoom level greater than or equal to the first zoom level. Optionally, the super-resolution mode option 312D can also be collapsed into more options 312E, such as... Figure 3C As shown, another user interface 33 for a camera application on an electronic device is illustrated. The shooting mode list 312 of user interface 33 includes a portrait mode option 312A, a video recording mode option 312B, a photo mode option 312C, and more options 312E. The electronic device can detect a user's touch operation on the more options 312E within user interface 33 and, in response to this operation, display as shown... Figure 3D The user interface 34 shown includes a super-resolution mode option 312D.

[0055] The Gallery shortcut key 313 can be used to open the Gallery application. In response to a user action (such as a click) on the Gallery shortcut key 313, the electronic device can open the Gallery application. This allows users to conveniently view captured photos and videos without having to exit the Camera application first and then open the Gallery application. The Gallery application is an image management application on the electronic device, also known as a "photo album," and this embodiment does not limit the name of the application. The Gallery application supports various operations on images stored on the electronic device, such as browsing, editing, deleting, and selecting.

[0056] The shutter control 314 can be used to monitor user actions that trigger image capture. The electronic device can detect user actions (such as clicks) applied to the shutter control 314, and in response to this action, the electronic device can save the image in the preview frame 311 as an image in the gallery application. Additionally, the electronic device can display a thumbnail of the saved image in the gallery shortcut key 313. That is, the user can click the shutter control 314 to trigger image capture. The shutter control 314 can be a button or other form of control.

[0057] The camera flip control 315 can be used to monitor user actions that trigger the camera flip. The electronic device can detect user actions (such as clicks) performed on the camera flip control 315, and in response to the action, the electronic device can flip the camera, for example, switch the rear camera to the front camera.

[0058] The zoom progress bar 316 is used to adjust the zoom ratio during shooting. For example, the zoom progress bar can provide zoom functionality between 1X and 100X. Optionally, the electronic device can detect user operations (e.g., dragging operations) applied to the zoom progress bar 316, and in response to this operation, the electronic device adjusts the zoom ratio during shooting. For example, as... Figure 3B As shown, the electronic device can detect an upward drag operation applied to the zoom progress bar 316, and in response to this operation, the electronic device increases the zoom magnification to 60X. Figure 3E The user interface 35 is shown. At different zoom levels, the size of the subject displayed in the preview frame 311 varies; for example, as shown... Figure 3B As shown, in a scene with a zoom ratio of 1X, the subject displayed in preview frame 311 is relatively small, such as... Figure 3EAs shown, in a 60x zoom scenario, the subject displayed in the preview frame 311 is relatively large. Optionally, the user interface 32 can also hide the zoom progress bar 316. When the electronic device detects a user operation (such as a double-click or drawing a rectangle) on the user interface 32, the user interface 32 displays the zoom progress bar 316. In other embodiments, the electronic device can also adjust the zoom level with other user operations, such as zoom gestures (e.g., sliding two fingers outward to increase the zoom level, and sliding two fingers inward to decrease the zoom level), which is not limited in this application.

[0059] Please see Figure 4 , Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application. The following is in conjunction with... Figures 3A-3E The user interface shown and Figure 2 The software architecture of the electronic device shown is illustrated, and the flow of the image processing method provided in the embodiments of this application is introduced.

[0060] S101, Electronic device activates super-resolution mode.

[0061] refer to Figure 3B The user interface 32 shown allows the electronic device to detect user operations applied to the super-resolution mode option 312D, and in response to these operations, the electronic device determines that the super-resolution mode is enabled. Optionally, the user interface 32 may be referred to as a first user interface, the super-resolution mode option 312D may be referred to as a first control, and the user operation applied to the super-resolution mode option 312D may be referred to as a first operation.

[0062] refer to Figure 3C The user interface 33 shown indicates that the electronic device can detect user actions on the more options 312E and, in response to such actions, display... Figure 3D The user interface 34 shown includes a super-resolution mode option 312D. The electronic device can detect user operations performed on the super-resolution mode option 312D, and in response to the operation, the electronic device determines that the super-resolution mode is enabled. Optionally, the user interface 33 can be referred to as a second user interface, the more options 312E can be referred to as a second control, the user operation performed on the more options 312E can be referred to as a second operation, and the user interface 34 can be referred to as a third user interface.

[0063] It should be noted that step S101 is optional. That is, in one scenario, the electronic device can directly perform super-resolution processing on the first image with a zoom ratio greater than or equal to the first zoom ratio. In another scenario, if the electronic device incorporates a super-resolution mode, it can perform super-resolution processing on the first image with a zoom ratio greater than or equal to the first zoom ratio when the super-resolution mode is enabled; otherwise, it will not perform super-resolution processing on the first image with a zoom ratio greater than or equal to the first zoom ratio.

[0064] S102, The electronic device adjusts the zoom level for shooting to the second zoom level.

[0065] refer to Figure 3E The user interface 34 shown allows the electronic device to detect user operations on the zoom progress bar 316 of the user interface 32. In response to this operation, the electronic device adjusts the zoom magnification to a second zoom magnification (e.g., ...). Figure 3E (60X shown in the image).

[0066] S103, The electronic device captures a first image at a second zoom level.

[0067] It should be noted that the embodiments in this application are illustrated using a single, independent first image. In other embodiments, the first image may be any frame from an image sequence. Here, an image sequence refers to a set of images composed of multiple consecutively acquired frames arranged in the order of acquisition; for example, the image sequence may be a video sequence.

[0068] Optionally, the format of the first image captured in this application embodiment is not limited. For example, the first image may be at least one of RGB image, YUV image, or YCrCb image.

[0069] Optionally, the size of the first image captured in this application embodiment is not limited. For example, the first image can be a 4K image or a 1080P image.

[0070] It should be noted that the second zoom ratio in this embodiment refers to the zoom ratio used to capture the first image.

[0071] S104. If the second zoom ratio is greater than or equal to the first zoom ratio, the electronic device calls the encoding model to encode the first image and obtains the feature extraction matrix of the first image.

[0072] The coding model is located in the system library of the hardware abstraction layer.

[0073] It should be noted that the value of the first zoom ratio is not limited in this application embodiment, and the first zoom ratio can be 20X, 40X, 60X, 80X, etc. For ease of explanation, a first zoom ratio of 40X is used as an example. For instance, when the second zoom ratio (60X) for capturing the first image is greater than the first zoom ratio (40X), the electronic device calls the encoding model to encode the first image and obtain the feature extraction matrix of the first image.

[0074] In one optional implementation, the electronic device may encode the first image based on an encoding model (or "encoder") to obtain a feature extraction matrix (or "hidden code") of the first image. Optionally, each element in the feature extraction matrix may be used to represent features such as shape and color of a corresponding region of the first image.

[0075] Optionally, the encoding model involved in the embodiments of this application has a corresponding decoding model (or "decoder"). The encoding model and decoding model are obtained by pairwise training of deep neural networks. The encoding model and decoding model may exist in an autoencoder model (or a variational autoencoder model), which will not be described in detail.

[0076] S105. The electronic device divides the feature extraction matrix into blocks to obtain N first feature matrix blocks, where N is a positive integer greater than 1.

[0077] For example, a feature extraction matrix is ​​an m-row, n-column feature extraction matrix containing m*n elements, where m and n are positive integers.

[0078] For example, the feature extraction matrix =

[0079] In one optional implementation, step S103 specifically includes: dividing the feature extraction matrix into blocks according to the number N of blocks in the first image to obtain N first feature matrix blocks. Optionally, the number of blocks N can be determined based on the size of the first image and the size of the input image of the super-resolution model. For example, when the first image is a 4K image (i.e., the size of the first image is 3840×2160) and the input image of the super-resolution model is a 1080P image (i.e., the size of the image is 1920×1080), the number of blocks N = 4.

[0080] For ease of explanation, the embodiments of this application use a feature extraction matrix =

[0081]

[0082] If N=2, the feature extraction matrix can be divided into the following two first feature matrix blocks:

[0083] The first feature matrix block =

[0084] The second first feature matrix block =

[0085] S106. The electronic device calls the super-resolution model to perform super-resolution processing on each first feature matrix block to obtain the second feature matrix block corresponding to each first feature matrix block.

[0086] Optionally, the second feature matrix block has the same dimension as the first feature matrix block. For example, when the first feature matrix block is a p-row * q-column matrix, the second feature matrix block is also a p-row * q-column matrix, where p and q are positive integers. The second feature matrix block after super-resolution processing represents more texture features than the first feature matrix block.

[0087] In one optional implementation, the super-resolution model may include an input layer, a processing layer, and an output layer. The input layer concatenates a first feature matrix block and a preset feature matrix block to obtain an intermediate feature matrix block. A convolutional layer performs convolution processing on the intermediate feature matrix block to obtain a second feature matrix block. The output layer outputs the second feature matrix block. The preset feature matrix block is a feature extraction matrix corresponding to a fixed-noise image. Optionally, the dimension of the preset feature matrix block is the same as the dimension of the first feature matrix block; for example, when the first feature matrix block is a p-row * q-column matrix, the preset feature matrix block is also a p-row * q-column matrix. Optionally, the first feature matrix block and the preset feature matrix block may be concatenated along channels. Optionally, the processing layer may be built based on the Unet model, without limitation.

[0088] S107. The electronic device processes the second feature matrix block corresponding to each first feature matrix block to obtain the second image.

[0089] In one optional implementation, the second feature matrix blocks corresponding to each first feature matrix block are concatenated to obtain a processed feature extraction matrix. The processed feature extraction matrix is ​​then decoded to obtain the second image. Optionally, the dimension of the processed feature extraction matrix is ​​the same as the dimension of the feature extraction matrix. For example, when the feature extraction matrix is ​​an s-row * z-column matrix, the processed feature extraction matrix is ​​also an s-row * z-column matrix, where s and z are positive integers. Compared to the feature extraction matrix, the processed feature extraction matrix represents more texture features.

[0090] Optionally, the splicing process in step S107 corresponds to the block segmentation process in step S105. Continuing with the example from step S105, if the second feature matrix block corresponding to the first feature matrix block =

[0091] The second feature matrix block corresponding to the second first feature matrix block =

[0092] Then, by concatenating the second feature matrix blocks corresponding to these two first feature matrix blocks, we obtain the processed feature extraction matrix.

[0093] Optionally, the decoding in step S107 corresponds to the encoding in step S104. For example, step S107 performs decoding based on a decoding model that corresponds to the encoding model. Optionally, this decoding model may also be located in the algorithm library of the hardware abstraction layer.

[0094] In another optional implementation, the mean and standard deviation of each first feature matrix block and the mean and standard deviation of each second feature matrix block are calculated respectively. The mean and standard deviation of each first feature matrix block, the mean and standard deviation of each second feature matrix block, and the second feature matrix block are then input into a distribution transformation formula to obtain a third feature matrix block corresponding to each first feature matrix block. The third feature matrix block corresponding to each first feature matrix block is then processed to obtain a second image. Optionally, this distribution transformation formula can be deployed in a style transfer model, which resides in the algorithm library of the hardware abstraction layer.

[0095] Optionally, the above distribution transformation formula can be:

[0096]

[0097] Where X represents the second feature matrix block, μ2 represents the mean of the second feature matrix block, σ2 represents the standard deviation of the second feature matrix block, μ1 represents the mean of the first feature matrix block, σ1 represents the standard deviation of the first feature matrix block, and output represents the third feature matrix block.

[0098] For example, the following detailed embodiments illustrate the relevant implementation of obtaining the third feature matrix block corresponding to the first feature matrix block.

[0099] Mean of the first feature matrix block

[0100] Standard deviation of the first feature matrix block

[0101] The mean of the second feature matrix block corresponding to the first feature matrix block

[0102] The standard deviation of the second feature matrix block corresponding to the first feature matrix block

[0103] The third feature matrix block corresponding to the first feature matrix block =

[0104] in 1≤i≤3, 1≤j≤3.

[0105] It should be noted that the style transfer in this application is based on the distribution transformation formula. Other style transfer methods may also be used in other implementations, and this application does not limit them.

[0106] It should be noted that the specific implementation method for processing the third feature matrix block corresponding to the first feature matrix block to obtain the second image can be found in the relevant embodiments for processing the second feature matrix block corresponding to the first feature matrix block to obtain the second image, and will not be repeated here.

[0107] In this embodiment, a third feature matrix block can be obtained by performing style transfer on the second feature matrix block after super-resolution processing based on the first feature matrix block before super-resolution processing in the latent space. The third feature matrix block obtained after style transfer represents a color that is closer to the color represented by the first feature matrix block than the second feature matrix block. Thus, when the style transfer step is performed for each second feature matrix block, the color difference between the third feature matrix blocks obtained after style transfer is small. This can effectively avoid the block artifacts caused by the color difference between different image blocks, thereby making it easier for the color of the second image obtained after super-resolution processing to be the same as the color of the first image before super-resolution processing.

[0108] To better illustrate the image processing procedures involved in steps S104 to S107, the following will combine... Figure 5 The process is illustrated using a flowchart. Figure 5 This is a schematic flowchart illustrating another image processing method provided in an embodiment of this application. Figure 5 As shown, the electronic device can encode the first image to obtain a feature extraction matrix of the first image. After dividing the feature extraction matrix into blocks to obtain N first feature matrix blocks, the electronic device can perform super-resolution processing on each first feature matrix block to obtain a second feature matrix block corresponding to each first feature matrix block. Then, style transfer processing is performed on each second feature matrix block to obtain a third feature matrix block corresponding to each first feature matrix block. After concatenating the N third feature matrix blocks, a processed feature extraction matrix can be obtained. This processed feature extraction matrix can be decoded to obtain the second image. For specific implementation details, please refer to steps S104 to S107, which will not be repeated here.

[0109] S108, The electronic device stores the second image.

[0110] S109. If the second zoom ratio is less than the first zoom ratio, the electronic device stores the first image.

[0111] It should be noted that the above embodiment is illustrated by the example of an electronic device performing super-resolution processing on a first image captured by itself with a zoom ratio greater than or equal to the first zoom ratio. In other embodiments, the electronic device can also perform super-resolution processing on a first image captured by another device with a zoom ratio greater than or equal to the first zoom ratio. For example, the electronic device can obtain the first image from another device through instant messaging applications, Bluetooth applications, or device sharing functions. When the electronic device detects that the first image was captured by a device with a zoom ratio greater than or equal to the first zoom ratio, the electronic device can execute steps S104 to S108, which will not be described in detail. Optionally, the electronic device can determine the zoom ratio of the device that captured the first image based on the focal length of the first image. For example, when the focal length of the first image is 542mm, the zoom ratio of the device that captured the first image is 20X; as another example, when the focal length of the first image is 810mm, the zoom ratio of the device that captured the first image is 30X.

[0112] It should be noted that before the electronic device performs super-resolution processing on the first image captured by other devices with a zoom ratio greater than or equal to the first zoom ratio, the method further includes: the electronic device detecting whether the shooting mode of capturing the first image is a super-resolution mode.

[0113] In this embodiment of the application, the electronic device can perform super-resolution processing on a first image with a zoom ratio greater than or equal to a first zoom ratio to obtain a second image. The second image obtained after super-resolution processing has more texture features than the first image, thereby improving the clarity of the image with an excessively large zoom ratio.

[0114] Furthermore, in this embodiment, the feature extraction matrix of the first image is divided into blocks before super-resolution processing. This allows for block-based super-resolution processing even for large first images, thus making the method applicable to a wide range of image sizes. Additionally, by performing super-resolution processing after block division, this embodiment improves image quality compared to scaling the first image before processing. Moreover, compared to separately applying super-resolution processing to large and small first images, it effectively saves computational resources.

[0115] In addition, since the embodiments of this application divide the feature extraction matrix located in the latent space into blocks, compared with the scheme of directly dividing the first image into blocks, it can avoid the stitching marks between image blocks.

[0116] In addition, this application embodiment also provides another image processing method. In this image processing method, step S105 includes: the electronic device divides the feature extraction matrix into blocks according to a preset overlap ratio based on the number N blocks of the first image, obtaining N first feature matrix blocks. Each of the N first feature matrix blocks overlaps with its adjacent first feature matrix blocks. It should be noted that the preset overlap ratio refers to the overlap ratio between any two adjacent first feature matrix blocks in the N first feature matrix blocks, and this preset overlap ratio can be an overlap ratio input by the user as needed.

[0117] Optionally, the overlap ratio can refer to the ratio between the elements included in the overlapping region and the total number of elements included in the first feature matrix block. For example, with an overlap ratio of 50%, the feature extraction matrix can be divided into blocks to obtain two first feature matrix blocks.

[0118] The first feature matrix block =

[0119] The second first feature matrix block =

[0120] The first feature matrix block includes a 13 a 14 a 23 a 24 a 33 and a 34 Elements, the second first feature matrix block also includes a 13 a 14 a 23 a 24 a 33 and a 34 Elements, the first first feature matrix block and the second first feature matrix block overlap, a 13 a 14 a 23 a 24 a 33 and a 34 The elements are the overlapping elements between the first and second first feature matrix blocks, or in other words, the overlapping region (the solid-line rectangular region) between the first and second first feature matrix blocks includes a. 13 a 14 a 23 a 24 a 33 and a 34 Elements such as...

[0121] Optionally, the overlapping area of ​​the first feature matrix block varies depending on its position within the feature extraction matrix. For example, when the first feature matrix block is located on the left side of the feature extraction matrix, it overlaps with the first feature matrix block located on the right side of the feature extraction matrix; or, for another example, when the first feature matrix block is located on the upper left side of the feature extraction matrix, it overlaps with the first feature matrix block located on the upper right side and the first feature matrix block located on the lower left side of the feature extraction matrix.

[0122] For example, taking N=4 as an example, the feature extraction matrix can be divided into blocks to obtain 4 first feature matrix blocks.

[0123] The first feature matrix block =

[0124] The second first feature matrix block =

[0125] The third first feature matrix block =

[0126] The 4th first feature matrix block =

[0127] Specifically, for the first feature matrix block, there is overlap between the first feature matrix block and the second and third feature matrix blocks. The overlapping area (solid-lined rectangular area) between the first and second feature matrix blocks includes a. 13 a 14 a 23 and a 24 Elements; the overlapping region (dotted-line rectangle region) between the first and third first feature matrix blocks includes a 21 a 22 a 23 and a 24 Elements such as...

[0128] Regarding the second first feature matrix block, there is overlap between the second first feature matrix block and the first and fourth first feature matrix blocks. The overlapping area (solid-lined rectangular area) between the second and first first feature matrix blocks includes a. 13 a 14 a 23 and a 24 Elements; the overlapping region (dotted-line rectangle region) between the second and fourth first feature matrix blocks includes a 23 a 24 a25 and a 26 Elements such as...

[0129] Regarding the third first feature matrix block, there is overlap between the third first feature matrix block and the first and fourth first feature matrix blocks. The overlapping area (dotted-line rectangle area) between the third and first first feature matrix blocks includes a. 21 a 22 a 23 and a 24 Elements; the overlapping region (long dashed rectangle region) between the third and fourth first feature matrix blocks includes a 23 a 24 a 33 and a 34 Elements such as...

[0130] Regarding the fourth first feature matrix block, there is overlap between the fourth first feature matrix block and the second and third first feature matrix blocks. The overlapping area (the dotted-line rectangle area) between the fourth and second first feature matrix blocks includes a. 23 a 24 a 25 and a 26 Elements; the overlapping region (long dashed rectangle region) between the 4th first feature matrix block and the 3rd first feature matrix block includes a 23 a 24 a 33 and a 34 Elements such as...

[0131] Correspondingly, in this image processing method, step S106 includes: performing super-resolution processing on the k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block. Specifically: from the 1st to the (k-1)th first feature matrix blocks, select the first feature matrix block adjacent to the k-th first feature matrix block; based on the overlapping area between the adjacent first feature matrix blocks and the k-th first feature matrix block, select the corresponding overlapping area from the second feature matrix blocks corresponding to the adjacent first feature matrix blocks; concatenate the overlapping areas of the second feature matrix blocks corresponding to the adjacent first feature matrix blocks with the non-overlapping areas of the k-th first feature matrix block to obtain the processed k-th first feature matrix block; perform super-resolution processing on the processed k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block. Here, k is a positive integer greater than 1.

[0132] For example, taking k=2, for the second first feature matrix block, the electronic device can obtain the second feature matrix block corresponding to the first first feature matrix block adjacent to the second first feature matrix block. Then, the electronic device can select the corresponding overlapping area from the second feature matrix block corresponding to the first first feature matrix block based on the overlapping area between the second first feature matrix block and the first first feature matrix block.

[0133] For example, the second feature matrix block corresponding to the first feature matrix block = For example, the overlapping region (i.e., the solid line distance box region) corresponding to the second feature matrix block corresponding to the first first feature matrix block includes b. 13 b 14 b 23 and b 24 Elements such as the first first feature matrix block and the second feature matrix block are concatenated. The overlapping region of the first first feature matrix block and the non-overlapping region of the second first feature matrix block are then joined to obtain the processed second first feature matrix block.

[0134] The processed second first feature matrix block =

[0135] For example, taking k=4, for the fourth first feature matrix block, the electronic device can obtain the second feature matrix block corresponding to the second first feature matrix block adjacent to the fourth first feature matrix block, and the second feature matrix block corresponding to the third first feature matrix block adjacent to the fourth first feature matrix block. Then, the electronic device can select the corresponding overlapping region from the second feature matrix block corresponding to the second first feature matrix block based on the overlapping region between the fourth and second first feature matrix blocks. Similarly, it can select the corresponding overlapping region from the second feature matrix block corresponding to the third first feature matrix block based on the overlapping region between the fourth and third first feature matrix blocks.

[0136] For example, the second feature matrix block corresponding to the second first feature matrix block = For example, the overlapping region (i.e., the distance box region between the dotted and dashed lines) corresponding to the second feature matrix block corresponding to the first feature matrix block includes b. 23 b 24 b 25 and b 26 Elements such as...

[0137] For example, the second feature matrix block corresponding to the third first feature matrix block = For example, the overlapping region (i.e., the long dashed distance box region) corresponding to the second feature matrix block corresponding to the third first feature matrix block includes b. 23b 24 b 33 and b 34 Elements such as...

[0138] The overlapping regions corresponding to the second and third first feature matrix blocks, as well as the non-overlapping regions corresponding to the second and third first feature matrix blocks, are concatenated with the non-overlapping regions of the fourth first feature matrix block to obtain the processed fourth first feature matrix block.

[0139] The fourth first feature matrix block after processing =

[0140] In this embodiment, the electronic device can perform overlapping block division on the feature extraction matrix (i.e., each of the N first feature matrix blocks overlaps with its adjacent first feature matrix blocks), and use the overlapping area of ​​the second feature matrix block corresponding to the first feature matrix block adjacent to the kth first feature matrix block to replace the overlapping area of ​​the kth first feature matrix block. This is beneficial to retain the features obtained by the adjacent first feature matrix blocks after super-resolution processing when performing super-resolution processing on the kth first feature matrix block. This is beneficial to ensure that the texture features of the image block corresponding to the kth first feature matrix block are the same as the texture features of the image blocks corresponding to its adjacent first feature matrix blocks. In this way, even if the size of the first image is large, the second image obtained by stitching together the blocks after super-resolution processing has continuous and consistent texture features on the adjacent image blocks.

[0141] It should be noted that when the image processing method involved in the embodiments of this application includes a style transfer operation, step S106 performs super-resolution processing on the k-th first feature matrix block to obtain the second feature matrix block corresponding to the k-th first feature matrix block, including: selecting a first feature matrix block adjacent to the k-th first feature matrix block from the 1st to the (k-1)th first feature matrix blocks; selecting a corresponding overlapping region from the third feature matrix blocks corresponding to the adjacent first feature matrix blocks based on the overlapping region between the adjacent first feature matrix blocks and the k-th first feature matrix block; concatenating the overlapping region of the third feature matrix blocks corresponding to the adjacent first feature matrix blocks with the non-overlapping region of the k-th first feature matrix block to obtain the processed k-th first feature matrix block; and performing super-resolution processing on the processed k-th first feature matrix block to obtain the third feature matrix block corresponding to the k-th first feature matrix block. Here, k is a positive integer greater than 1. Further details are omitted.

[0142] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image processing method, characterized in that, The method includes: The first image is encoded to obtain the feature extraction matrix of the first image; the first image is taken by a device with a zoom ratio greater than or equal to the first zoom ratio. The feature extraction matrix is ​​divided into blocks to obtain N first feature matrix blocks, where N is a positive integer greater than 1; each of the N first feature matrix blocks overlaps with its adjacent first feature matrix blocks; Super-resolution processing is performed on each processed first feature matrix block corresponding to the first feature matrix block to obtain a second feature matrix block corresponding to each first feature matrix block; the processed first feature matrix block is obtained by splicing the non-overlapping region of the first feature matrix block with the processed overlapping region, and the processed overlapping region is determined based on the second feature matrix blocks corresponding to the adjacent first feature matrix blocks of the first feature matrix block; The second feature matrix block corresponding to each first feature matrix block is processed to obtain the second image.

2. The method as described in claim 1, characterized in that, The processed overlapping region is determined based on the second feature matrix block corresponding to the first feature matrix block adjacent to the first feature matrix block, including: The processed overlapping region includes the overlapping region of the second feature matrix block corresponding to the adjacent first feature matrix block.

3. The method as described in claim 1, characterized in that, The step of processing the second feature matrix block corresponding to each first feature matrix block to obtain the second image includes: Calculate the mean and standard deviation of each first feature matrix block, and the mean and standard deviation of each second feature matrix block; The mean and standard deviation of each first feature matrix block, the mean and standard deviation of each second feature matrix block, and the second feature matrix block are respectively input into the distribution transformation formula to obtain the third feature matrix block corresponding to each first feature matrix block; The third feature matrix block corresponding to each first feature matrix block is processed to obtain the second image.

4. The method as described in claim 3, characterized in that, The processed overlapping region is determined based on the second feature matrix block corresponding to the first feature matrix block adjacent to the first feature matrix block, including: The processed overlapping region includes the overlapping region of the third feature matrix block corresponding to the adjacent first feature matrix block.

5. The method as described in claim 3, characterized in that, The distribution transformation formula is: Where X represents the second feature matrix block. Used to represent the mean of the second feature matrix block, Used to represent the standard deviation of the second feature matrix block Used to represent the mean of the first feature matrix block, Used to represent the standard deviation of the first feature matrix block Used to represent the third feature matrix block.

6. The method according to any one of claims 1 to 5, characterized in that, Before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: Detect whether the shooting mode for capturing the first image is super-resolution mode; the super-resolution mode is used to trigger the super-resolution processing operation performed on the first image whose zoom ratio is greater than or equal to a first zoom ratio. If the shooting mode is super-resolution mode, then the step of encoding the first image to obtain the feature extraction matrix of the first image is performed.

7. The method according to any one of claims 1 to 5, characterized in that, Before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: A first user interface is displayed, the first user interface including a first control; the first control is used to enable super-resolution mode; When a first operation is detected targeting the first control, super-resolution mode is activated; Take the first image.

8. The method according to any one of claims 1 to 5, characterized in that, Before encoding the first image to obtain the feature extraction matrix of the first image, the method further includes: A second user interface is displayed, the second user interface including a second control; the second control is used to trigger the display of the first control; When a second operation is detected on the second control, a third user interface is displayed, the third user interface including the first control; When a first operation is detected targeting the first control, super-resolution mode is activated; Take the first image.

9. An electronic device, characterized in that, include: A central processing unit, a graphics processing unit, an image signal processor, and one or more memories; the one or more memories are coupled to the central processing unit, the graphics processing unit, and the image signal processor, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the central processing unit, the graphics processing unit, or the image signal processor, cause the electronic device to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

11. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-8.

Images