Thumbnail display method and related device

Abstract

This application provides a thumbnail display method and related device, relating to the field of terminal technology. The method includes: after taking a photo, acquiring a first image from a preview stream, dividing the first image into N sub-images; processing the data format of all N sub-images into an interleaved format, and during the process of processing the data format of the N sub-images into an interleaved format, after processing the data format of each sub-image, inputting the processed sub-image into a super-resolution network model for super-resolution processing; after the super-resolution network model has performed super-resolution processing on all N processed sub-images, obtaining a second image; and displaying the second image in thumbnail form. In this way, the electronic device can obtain a high-resolution thumbnail, reducing the difference in image quality when switching from a thumbnail to a captured image.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a thumbnail display method and related equipment. Background Technology

[0002] Electronic devices can support photo-taking functions, allowing users to take pictures of objects using the device's camera app.

[0003] When an electronic device launches its camera application, a thumbnail control can be displayed in the camera application interface. When a trigger action is received on the thumbnail control, the electronic device can display a thumbnail of the captured image.

[0004] However, in some implementations, the thumbnail image quality is often poor. Summary of the Invention

[0005] This application provides a thumbnail display method and related device, applied in the field of terminal technology, which obtains high-quality thumbnails by performing super-resolution processing on images in a preview stream; and enables the saving and sharing of thumbnails.

[0006] In a first aspect, embodiments of this application propose a thumbnail display method. The method includes: responding to a photo-taking operation (e.g., in...) Figure 7 In the interface shown in Figure a, the click operation of the camera control 102 is performed to obtain a first image (e.g., image 1), which is an image in the preview stream; the first image is divided into N sub-images (e.g., image 51, image 52... image 5N); N is a positive integer greater than 1; the data format of the N sub-images is processed into an interleaved format, and during the process of processing the data format of the N sub-images into an interleaved format, after the data format of each sub-image is processed, the processed sub-image is input into the super-resolution network model for super-resolution processing; after the super-resolution network model has performed super-resolution processing on all N processed sub-images, a second image is obtained; the second image is displayed in thumbnail form.

[0007] The process involves processing the data format of each sub-image, then inputting the processed sub-image into the super-resolution network model for super-resolution processing. This can be understood as follows: At the first time step, the data format of the first sub-image is processed into an interleaved format. At the second time step, the processed first sub-image is input into the super-resolution network model for super-resolution processing, and the data format of the second sub-image is processed into an interleaved format. At the third time step, the processed second sub-image is input into the super-resolution network model for super-resolution processing, and the data format of the third sub-image is processed into an interleaved format. This continues until the Nth time step, where the processed (N-1)th sub-image is input into the super-resolution network model for super-resolution processing, and the data format of the Nth sub-image is processed into an interleaved format. At the N+1th time step, the processed Nth sub-image is input into the super-resolution network model for super-resolution processing.

[0008] In this way, electronic devices can obtain higher resolution thumbnails, reducing the difference in image quality when switching from thumbnails to captured images; and by using parallel processing, the super-resolution processing of thumbnails is accelerated, reducing waiting time.

[0009] Optionally, the first image is divided into N sub-images, including: obtaining the resolution of the first image; and dividing the first image into N sub-images if the resolution of the first image meets preset conditions; wherein the preset conditions include: the resolution of the first image is greater than a resolution threshold; the resolution threshold is used to characterize the super-resolution network model's ability to process image resolution; and the ratio between the resolution of the first image and the resolution threshold is positively correlated with the value of N.

[0010] In this way, when the performance of electronic devices is poor, parallel processing can be used to perform super-resolution processing on images to improve image resolution.

[0011] Optionally, after the super-resolution network model performs super-resolution processing on all N processed sub-images, a second image is obtained; this includes: stitching the N super-resolution sub-images into a third image; the third image is in RGB format; color processing is performed on the third image; color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; converting the color-processed third image into a YUV format third image; and downsampling the YUV format third image to obtain the second image.

[0012] The post-processing module is optional. When the electronic device does not include a post-processing module, the electronic device can perform the above steps to obtain a thumbnail of a preset size.

[0013] In this way, the electronic device can process the super-resolution image into a thumbnail that meets the preset size, so that the image can be displayed in thumbnail form later.

[0014] Optionally, after inputting the processed sub-images into the super-resolution network model for super-resolution processing, the method further includes: during the process of inputting the processed sub-images into the super-resolution network model for super-resolution processing, after each processed sub-image is super-resolution completed, image enhancement processing is performed on the super-resolution sub-image; the image enhancement processing includes one or more of the following: denoising processing and sharpening processing; obtaining the second image includes: stitching together N image-enhanced sub-images into a fourth image; the fourth image is in RGB format; color processing is performed on the fourth image; the color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; converting the color-processed fourth image into a YUV format fourth image; downsampling the YUV format fourth image to obtain the second image.

[0015] The post-processing module is optional. When the electronic device includes a post-processing module, the electronic device can perform the above steps to obtain a thumbnail of a preset size.

[0016] The process is as follows: At the first time step, the data format of the first sub-image is processed into an interleaved format. At the second time step, the processed first sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the second sub-image is processed into an interleaved format. At the third time step, image enhancement processing is performed on the super-resolution first sub-image; the processed second sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the third sub-image is processed into an interleaved format. At the fourth time step, image enhancement processing is performed on the super-resolution second sub-image; the processed third sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the fourth sub-image is processed into an interleaved format.

[0017] And so on...

[0018] At time N, image enhancement processing is performed on the (N-2)th sub-image after super-resolution; the processed (N-1)th sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the Nth sub-image is processed into an interleaved format. At time N+1, image enhancement processing is performed on the (N-1)th sub-image after super-resolution; the processed Nth sub-image is input into the super-resolution network model for super-resolution processing. At time N+2, image enhancement processing is performed on the Nth sub-image after super-resolution.

[0019] In this way, the super-resolution sub-images can be further processed to improve their image quality; parallel processing can improve the processing efficiency of the thumbnail super-resolution module; and the electronic device can process the super-resolution image into a thumbnail that meets the preset size so that the image can be displayed in thumbnail form later.

[0020] Optionally, the electronic device includes a first module, a super-resolution network model, and a second module; the first module is used to process the data format of the sub-image into an interleaved format; the second module is used to perform image enhancement processing on the sub-image after super-resolution; the first module runs on a first processor, the super-resolution network model runs on a second processor, and the second module runs on a third processor; the first processor, the second processor, and the third processor are different processors.

[0021] In this way, the preprocessing module, the super-resolution network model, and the post-processing module can run in parallel, thereby improving the processing efficiency of the thumbnail super-resolution module.

[0022] Optionally, the first processor is a CPU, the second processor is an NPU, and the third processor is a GPU; or, the first processor is a GPU, the second processor is an NPU, and the third processor is a CPU.

[0023] Alternatively, the first processor, the second processor, and the third processor may be any combination of three processors among CPU1, NPU1, GPU1, CPU2, NPU2, GPU2, CPU3, NPU3, and GPU3. This application does not limit this.

[0024] This allows for the parallel operation of various preprocessing modules, super-resolution network models, and post-processing modules.

[0025] Optionally, after obtaining the second image, the method also includes: saving the second image in a gallery application.

[0026] In this way, electronic devices can achieve two photos in one shot, storing both the thumbnail and the captured image simultaneously in the gallery application; making it convenient to view and share the thumbnail later.

[0027] Optionally, after saving the second image in the gallery application, the process includes: displaying the first interface of the gallery application (e.g., ...). Figure 8 The interface shown in Figure a); the first interface includes a second image (e.g., thumbnail 701) and a fifth image (e.g., captured image 702), the fifth image being a captured image obtained in response to a capture operation, the resolution of the fifth image being greater than or equal to the resolution of the second image; the second image has a corresponding thumbnail identifier (e.g., thumbnail identifier 801); in response to a trigger operation on the second image, the second interface of the gallery application (e.g., Figure 8 The interface shown in b); the second interface displays a second image and a sharing control (e.g., sharing control 804); the sharing control is used to share the second image.

[0028] This allows you to store thumbnails in the gallery app and share them.

[0029] Optionally, after saving the second image in the gallery app, the method also includes: displaying a third interface of the gallery app (e.g., ...). Figure 8 The third interface includes a thumbnail album (e.g., thumbnail album 803); in response to a trigger operation on the thumbnail album, the fourth interface of the gallery application (e.g., ...) is displayed. Figure 8 The fourth interface displays one or more thumbnails, one or more of which include a second image (e.g., thumbnail 701); in response to a triggering operation on the second image, the second interface of the gallery application (e.g., [image not provided]) is displayed. Figure 8 (The interface shown in b).

[0030] This allows you to store thumbnails in the gallery app and share them.

[0031] Optionally, the electronic device supports beautification and / or blurring functions. The first image is an image from the preview stream that has not undergone beautification and / or blurring processing. After downsampling the third image in YUV format, the method further includes: if the beautification and / or blurring functions are enabled, performing beautification and / or blurring processing on the downsampled third image to obtain the second image. This allows for beautification and / or blurring of the thumbnail to make it fit the captured image, reducing the differences between the two and thus minimizing image differences when switching between thumbnails and captured images.

[0032] Optionally, the electronic device supports beautification and / or blurring functions. The first image is an image from the preview stream that has not undergone beautification and / or blurring processing. After downsampling the fourth image in YUV format, the method further includes: if the beautification and / or blurring functions are enabled, performing beautification and / or blurring processing on the downsampled fourth image to obtain the second image. This allows for beautification and / or blurring of the thumbnail to make it fit the captured image, reducing the differences between the two and thus minimizing image differences when switching between thumbnails and captured images.

[0033] Secondly, embodiments of this application provide an electronic device, which may also be referred to as a terminal device, terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0034] The electronic device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the electronic device to perform the method as described in the first aspect.

[0035] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method as described in the first aspect.

[0036] Fourthly, embodiments of this application provide a computer program product, which includes a computer program that, when run, causes a computer to perform the method as described in the first aspect.

[0037] This application provides a chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a line. The at least one processor is used to run computer programs or instructions to perform the method as described in the first aspect.

[0038] It should be understood that the second to fifth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the interface for a possible photo-taking scenario.

[0040] Figure 2 A schematic diagram of the interface for viewing captured images in a possible implementation;

[0041] Figure 3 This is a schematic diagram of the image processing flow in a possible implementation;

[0042] Figure 4 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application;

[0043] Figure 5 A schematic diagram of the software structure of the electronic device 100 provided in the embodiments of this application;

[0044] Figure 6 A flowchart illustrating a thumbnail display method provided in an embodiment of this application;

[0045] Figure 7 A schematic diagram of an interface for taking a picture in an embodiment of this application;

[0046] Figure 8 A schematic diagram of an interface for viewing thumbnails provided in an embodiment of this application;

[0047] Figure 9 A flowchart illustrating a thumbnail display method provided in an embodiment of this application;

[0048] Figure 10 A schematic diagram illustrating an interleaving format and a deinterleaving format provided in an embodiment of this application;

[0049] Figure 11 A flowchart illustrating a thumbnail display method provided in an embodiment of this application;

[0050] Figure 12 A flowchart illustrating another thumbnail display method provided in an embodiment of this application;

[0051] Figure 13 A flowchart illustrating another thumbnail display method provided in an embodiment of this application;

[0052] Figure 14 A flowchart illustrating a thumbnail display method provided in an embodiment of this application;

[0053] Figure 15 This is a schematic diagram of the structure of a thumbnail display device provided in an embodiment of this application. Detailed Implementation

[0054] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0055] 1. Electronic equipment

[0056] The electronic devices in this application embodiment may include handheld devices with image processing functions, vehicle-mounted devices, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices (such as vehicle infotainment systems), wearable devices, electronic devices in 5G, or future evolutions of public land mobile communications. The embodiments of this application do not limit the scope of electronic devices in a network (PLMN).

[0057] By way of example and not limitation, in this embodiment, the electronic device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as hearing aids, glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or attached to the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0058] Furthermore, in this application embodiment, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects through communication technology, thereby realizing the intelligent interconnection of humans and machines and the interconnection of things.

[0059] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0060] 2. Preview images and take photos

[0061] A preview image can be data captured in real time by the camera of an electronic device and displayed in a preview screen. For example, when an electronic device receives a user's command to open a camera app, it can capture a preview image from the camera and display it in real time in the camera app's preview screen.

[0062] A photographed image can be data acquired based on a camera control within an electronic device. For example, when an electronic device receives a user's click on the camera control, it can acquire a photographed image taken at the moment the photo is taken.

[0063] 3. Other terms

[0064] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0065] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0066] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation; the embodiments of this application do not specifically limit this. Furthermore, the display interface provided in the embodiments of this application is merely an example, and the display interface may include more or less content.

[0067] The following is combined with Figure 1 as well as Figure 2 The corresponding embodiments illustrate the photo-taking process and the image processing process.

[0068] Figure 1 A schematic diagram of the interface for a possible photo-taking scenario is shown. For example... Figure 1 As shown:

[0069] After launching the camera app, the electronic device can display the camera app interface. The camera app offers various shooting modes, such as aperture mode, night mode, portrait mode, still mode, video mode, and movie mode. Taking portrait mode as an example, the camera app interface for portrait mode might look like this: Figure 1 In the interface shown in Figure 'a', the human portrait mode control is selected.

[0070] Figure 1 In the interface shown in Figure a, the electronic device can display shooting function controls in the top and bottom areas. For example, the top area displays flash controls, HDR controls, filter controls, and settings controls. The bottom area displays thumbnail controls 101, photo controls 102, lens switching controls 103, bokeh controls 104, skin smoothing controls 105, and one or more shooting mode controls. Figure 1 The interface shown in 'a' can also include a framing area, which can display a preview image, such as a girl.

[0071] Among them, the thumbnail control 101 can be used to quickly preview the photos that have been taken; the photo control 102 can be used to take a picture; the lens switching control 103 can be used to switch between the front camera and the rear camera; the bokeh control 104 can be used to adjust the degree of background blur to highlight the subject; and the skin beautification control 105 can be used to adjust the skin tone, smoothness and other effects of the people in the photo to achieve the purpose of beautifying the portrait.

[0072] exist Figure 1 In the interface shown in Figure a, when a trigger operation is received for the camera control 102, the electronic device can take a picture of the object being photographed and display the image as shown below. Figure 1 The interface shown in b is as follows. After obtaining the image of the girl, the icon of the thumbnail control 101 can display a thumbnail of the girl's image accordingly.

[0073] Users can view the previously captured image using the thumbnail control 101. Understandably, during the photo-taking process, electronic devices perform a series of processing steps to improve image quality, such as resolution and clarity. For lower-performance electronic devices, image processing takes longer. For example, a lower-performance device might use an AIGC (Artificial Intelligence Generated Content) link for image processing. While AIGC links offer good image processing results, their poor performance leads to longer processing times, such as 6 seconds. In some cases, users might click the thumbnail control 101 immediately after clicking the photo-taking control 101; in this case, the electronic device can first display the thumbnail. After image processing is complete, the electronic device will then remove the thumbnail and display the captured image.

[0074] For example, within a certain period of time after a trigger operation on the camera control 102 (before the electronic device has finished processing the captured image), upon receiving a trigger operation on the thumbnail control 101, the electronic device may display something like... Figure 1 The interface shown in c is shown in the image. Figure 1 The interface shown in c includes thumbnail 106.

[0075] After a period of time, electronic devices can be... Figure 1 Switching to the interface shown in c Figure 1 The interface shown in d is shown in the image. Figure 1 The interface shown in d displays a photographed image 107. The thumbnail 106 has a lower resolution than the photographed image 107.

[0076] The difference in clarity between thumbnail 106 and photographed image 107 is because the electronic device displays thumbnail 106 between the moment the user clicks thumbnail control 101 and the moment the electronic device generates the photographed image. In other words, the electronic device spends less time processing thumbnail 106 than it spends processing photographed image 107. To compress the processing time of thumbnail 106, the processing steps for thumbnail 106 can be reduced, resulting in lower clarity for thumbnail 106 compared to photographed image 107.

[0077] Figure 1 This demonstrates how to view a captured image using the thumbnail control 101. The following section combines... Figure 2 This demonstrates another way to view photographed images, such as Figure 2 As shown:

[0078] In response to an action used to launch the Gallery app (also known as the Photo Gallery app), the electronic device can display the Gallery app interface, such as... Figure 2 The interface shown in Figure 'a' can display captured images, such as photographed image 107. When a trigger operation is received for photographed image 107, the electronic device can zoom in to display the captured image 107, such as... Figure 2 The interface shown in b is shown in the image.

[0079] It is understandable that during the photo-taking process, the electronic device may temporarily store the thumbnail 106 before processing the captured image 107; after processing the captured image 107, the electronic device may delete the thumbnail 106. The electronic device's gallery application includes the captured image 107 but does not include the thumbnail 106. Alternatively, when the electronic device subsequently uses, for example... Figure 1 When viewing the image as shown in operation b, the electronic device displays the photographed image 107.

[0080] The following is combined with Figure 3 The image processing flow in a possible implementation is described. For example... Figure 3 As shown:

[0081] The electronic device may include a camera, an image front-end processing module, an image processing engine 1, an image processing engine 2, a functional feature module, and a format conversion module. The camera can be used to capture raw images, such as RAW images. The camera may include an image sensor. The image sensor can be used to convert captured light into electrical signals, thereby acquiring an image of the object being photographed.

[0082] The image front end (IFE) module can be used to receive raw images from the image sensor and perform preliminary processing on the raw images, such as color correction, downsampling, depigmentation, and statistical analysis of 3A data.

[0083] The image processing engine (IPE) is responsible for performing more complex image processing tasks, such as hardware noise reduction, image cropping, color processing, and detail enhancement. Image processing engine 1 processes images in the preview stream to obtain the preview image. Image processing engine 2 processes images in the capture stream to obtain the captured image. Some submodules in image processing engine 1 may be the same as or different from some submodules in image processing engine 2.

[0084] Understandably, the resolution of the captured image is higher than that of the preview image. Therefore, image processing engine 1 requires less complex image processing than image processing engine 2, ensuring high image quality for the captured image and smoothness for the preview image. Furthermore, image processing engine 1 processes the image captured by the camera in real time, while image processing engine 2 processes the image only after the capture operation has been performed; their processing times differ.

[0085] Functionality modules can be used to process the functionalities activated by the electronic device during the photo-taking process. For example, functionality modules may include beautification and bokeh modules. During the photo-taking process, the electronic device displays, for example... Figure 1 As shown in the interface 'a', the blur control 104 and the skin smoothing control 105 are both enabled; therefore, the electronic device can use the functional feature module to perform skin smoothing and blurring processing on the image.

[0086] The format conversion module can be used to convert the format of images, for example, converting a YUV format image to a format that meets thumbnail requirements.

[0087] Specifically, after launching the camera application, the electronic device can display a preview image on the screen in real time. For example, the camera captures the raw image and transmits it to the image front-end processing module. The image front-end processing module performs color correction, downsampling, de-mosaicing, and statistical 3A data processing on the raw image to obtain a front-end processed image, which is then transmitted to the image processing engine 1. The image processing engine 1 can perform hardware noise reduction, image cropping, noise reduction, color processing, and detail enhancement on the front-end processed image to obtain a processed image, which is then transmitted to the functional feature module.

[0088] When a function is enabled (e.g., skin smoothing and / or blurring), the function module can perform skin smoothing and / or blurring on the processed image 1 to obtain a smoothed and blurred image, and then send the image to the display screen for display, as shown below. Figure 1 The image shown in 'a' is a preview of the interface.

[0089] During the display of the preview image, when a trigger operation is received for the camera control 102, the electronic device can transmit the beautified and blurred image of the current frame to the format conversion module to process the image into a thumbnail. In this process, the thumbnail can be a copied preview image of the current frame. The electronic device temporarily stores the thumbnail and prompts the user to complete the photo capture via the thumbnail control 101. Alternatively, in response to a trigger operation for the camera control 102, the electronic device can also transmit the front-end processed image to the format conversion module to process the image into a thumbnail (this method is not shown in the figure).

[0090] Simultaneously, in response to a trigger operation on the camera control 102, the electronic device transmits the front-end processed image to the image processing engine 2. The image processing engine 2 performs hardware noise reduction, image cropping, noise reduction, color processing, and detail enhancement on the image to obtain a processed image, which is then transmitted to the functional features module. The functional features module performs skin smoothing and / or blurring processing on the processed image to obtain a smoothed and blurred image, which is then saved to the image library to obtain the captured image. At the same time, the electronic device can clear the temporary cache of thumbnails.

[0091] It should be noted that preview images require high fidelity, while photographed images require high image quality. Therefore, within the same frame, the image processed by image processing engine 1 differs from the image processed by image processing engine 2. Furthermore, the sharpness of the preview image within the same frame is lower than that of the photographed image; the sharpness of the thumbnail is less than or equal to that of the preview image, resulting in lower image quality for the thumbnail.

[0092] When a low-performance electronic device displays... Figure 1 The interface shown in c and Figure 1 In the interface shown by d, users will see the image change from blurry to clear, which affects the user experience.

[0093] In view of this, embodiments of this application provide a thumbnail display method, which obtains an image processed by an image front-end processing module and an image processing engine 1 from a preview stream, and performs preprocessing, super-resolution network (SRNet, also known as super-resolution), and postprocessing on the image to obtain a high-resolution thumbnail. To reduce the processing time of the thumbnail, one or more processing steps can be set up in the super-resolution process. For example, when the performance of the electronic device meets the image processing requirements, one processing step can be set up, such as preprocess 1, SRNet 1, and postprocess 1; when the performance of the electronic device does not meet the image processing requirements, multiple processing steps can be set up, such as: preprocess 1, SRNet 1, and postprocess 1; preprocess 2, SRNet 2, and postprocess 2, etc. The three processing steps run on different processors to process the image data in parallel, improve the efficiency of the super-resolution process, and make the thumbnail generation time less than the generation time of the captured image. In this way, the image quality of thumbnails can be improved without affecting the thumbnail generation time; at the same time, electronic devices can also save high-quality thumbnails for later use.

[0094] To better understand the embodiments of this application, the structure of the electronic device of this application is described below:

[0095] Figure 4A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a subscriber identification module (SIM) card interface 195, and an embedded secure element (eSE) chip 196, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0096] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0097] Processor 110 may include one or more processing units, such as a central processing unit (CPU), application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. In this embodiment, during image processing, the electronic device can run relevant modules on different processors to enable concurrent processing of multiple modules, improving processing efficiency; the processors may be, for example, CPUs, NPUs, and GPUs.

[0098] Electronic device 100 implements display functions through GPU, display screen 194, and application processor.

[0099] A GPU is a microprocessor for image processing, connected to a display screen 194 and an application processor. The GPU performs mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information. In this embodiment, the GPU can be used to run algorithms related to image processing to achieve image processing and image display.

[0100] The display screen 194 is used to display images, videos, etc. In this embodiment, the display screen 194 can be used to display a camera application interface and a gallery application interface, for example... Figure 7 and Figure 8 The relevant interface in the program.

[0101] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0102] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193. In this application embodiment, the ISP can achieve image processing based on software and hardware collaborative processing; for example, the ISP is equipped with modules such as IPE and IFE.

[0103] Camera 193 is used to capture still images or videos.

[0104] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices 100, such as image recognition, face recognition, speech recognition, and text understanding. In this embodiment, the NPU can be used to run algorithms related to image processing to achieve image processing and image display.

[0105] Internal memory 121 can be used to store executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located within the processor.

[0106] In this embodiment of the application, the electronic device can store the captured thumbnails and photographed images based on the internal memory 121.

[0107] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc. This application embodiment uses the layered architecture Android system as an example to illustrate the software structure of electronic device 100.

[0108] For example, Figure 5 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: the application (APP) layer, the application framework (or FWK) layer, the hardware abstraction layer (HAL), and the kernel layer, etc. This application does not limit the specific layers.

[0109] The application layer may include a series of application packages. The application layer may include applications such as camera applications, but this embodiment does not limit the scope of the application.

[0110] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined interfaces. It may include features such as a camera firmware.

[0111] The camera FWK can be used to transfer images between the application layer and the HAL. For example, the camera FWK can send requests generated by the camera application to the HAL, or return images generated by the HAL to the camera application.

[0112] In possible implementations, the application framework layer may also include one or more of the following: a display compositor, a window manager, a content provider, a resource manager, a view system, or a notification manager, etc. Figure 5 (Not shown in the image).

[0113] The purpose of the hardware abstraction layer is to abstract hardware, providing a unified interface for upper-layer applications to query hardware devices, or to provide data storage services for upper-layer applications.

[0114] The hardware abstraction layer may include: a camera algorithm library.

[0115] The camera algorithm library can be used to implement image front-end processing, beautification, skin smoothing, blurring, and thumbnail processing. For details on how the camera algorithm library implements each processing step, please refer to [link to documentation / reference]. Figures 6-13 The description in the text.

[0116] The kernel layer is the layer between hardware and software. It drives the hardware to function. The kernel layer can include one or more of the following: camera driver, display driver, or sensor driver, etc.

[0117] The camera driver can capture raw images via the camera when it detects that an electronic device has launched a camera application.

[0118] In possible implementations, the electronic device may include a hardware layer, which may include one or more of the following: a camera, a liquid crystal display (LCD), a GPU, an NPU, or a CPU, etc. Figure 5 Not shown in the image.

[0119] In this application embodiment, no specific limitations are made on the software layers involved in the software architecture, the modules contained in the layers, and the functions of the modules.

[0120] The following is combined with Figure 6 The flowchart of the thumbnail display method provided in the embodiments of this application will be described, taking the example of not enabling the skin beautification and bokeh functions during the photo taking process. Figure 6 As shown:

[0121] S601. Acquire the original image and preview it to obtain image 1.

[0122] The original image can be in RAW format. Preview processing may include image processing using an image front-end processing module and image processing engine 1. The IFE processing involved in the image front-end processing module includes one or more of the following: color correction, downsampling, demosaicing, and processing of statistical 3A (Autofocus (AF), Automatic Exposure (AE), Automatic White Balance (AWB)) data. The IPE processing involved in the image processing engine includes one or more of the following: hardware noise reduction (MFNR, MFSR), image cropping, image noise reduction, color processing, and image enhancement. Image 1 is an image processed by both IFE and IPE.

[0123] After launching the camera application, the electronic device can capture and process the preview stream. The processing can be, for example: the camera captures the original image; the electronic device uses an image front-end processing module to process the original image, obtaining image 3; image processing engine 1 processes image 3, obtaining image 1. The processing procedures of the front-end processing module and image processing engine 1 can be found in the aforementioned embodiments and will not be repeated here.

[0124] It should be noted that when beautification or bokeh effects are not enabled during photo taking, the electronic device does not need to use the feature modules to process the image. "Not enabling beautification or bokeh effects" can be understood as follows: in some shooting modes, the electronic device does not provide beautification and bokeh functions, such as still mode, night mode, and aperture mode. In these shooting modes, the preview stream processing does not involve the feature modules. Alternatively, in some shooting modes, the electronic device provides beautification and bokeh functions, but the corresponding controls are not enabled; in this case, the preview stream processing also does not involve the feature modules.

[0125] Without involving the processing flow of functional feature modules, Image 1 can be a preview image for display on the screen.

[0126] S602. Display image 1 in the camera application interface.

[0127] The camera application interface can correspond to Figure 1 The interface shown in image 'a' corresponds to [image 1]. Figure 1 The preview image shown in the interface is shown in Figure 'a'. The camera application interface can be set to any shooting mode; and when the shooting mode is portrait mode, both the bokeh control 104 and the beautification control 105 are turned off.

[0128] S603, In response to a trigger operation on the camera control, obtain image 2 based on image 1 and the super-resolution network model; the resolution of image 2 is greater than the resolution of image 1.

[0129] The trigger operation for the camera control can correspond to... Figure 1 The image shown (a) illustrates a click operation on the camera control 102 within the interface. The super-resolution network model, also known as the SRNet model, has the ability to improve image resolution.

[0130] In this embodiment, the super-resolution network model can be set in the thumbnail processing module. See also Figure 6 Upon receiving a trigger operation for the camera control, the electronic device can respond to the camera request by acquiring image 1 from the preview stream. Image 1 can be an image processed by image processing engine 1. The electronic device then transmits image 1 to the thumbnail processing module, which processes image 1 to obtain and save a thumbnail. The resolution of the thumbnail is greater than that of image 1.

[0131] S604. Save image 2 in the gallery application and display image 2 in the thumbnail control in the camera application interface.

[0132] It is understandable that, since the image quality of the thumbnail is improved and the resolution of image 2 is higher, the electronic device can store image 2 in this embodiment of the application for subsequent viewing and / or sharing of image 2.

[0133] In the camera application interface, the thumbnail control can be understood as... Figure 1 The thumbnail control 101 is located in the interface shown in b.

[0134] S605. Obtain image 4 from the original image and store image 4 in the gallery application; the resolution of image 4 is greater than the resolution of image 2; the content of images 1, 2 and 4 is the same.

[0135] Image 4 can be a captured image. Simultaneously with generating the thumbnail, the electronic device can also capture a captured image in response to a triggering action on the camera control. The process of obtaining Image 4 from the original image can be found in [link to documentation]. Figure 6 Image 3 is obtained; Image 3 is processed using image processing engine 2 to obtain image 4. The processing procedure of image processing engine 2 can be found in the aforementioned embodiments, and will not be repeated here.

[0136] It is understood that Images 1, 2, 3, and 4 are images resulting from different processing of the same original image frame, and the content of Images 1, 2, 3, and 4 is the same. Furthermore, Image 4 is a photographed image, Image 1 is a preview image, and Image 2 is a thumbnail; in this embodiment, the clarity of the photographed image is greater than or equal to the clarity of the thumbnail, and the clarity of the thumbnail is greater than the clarity of the preview image.

[0137] Furthermore, in some implementations, when the thumbnail has low resolution, its resolution may be insufficient to meet the needs of high-quality display, printing, or online sharing. However, in the embodiments of this application, the thumbnails are richer in detail, resulting in improved visual effects. This facilitates viewing on high-resolution devices and lossless sharing on online platforms, thus meeting the needs of visual communication.

[0138] Meanwhile, in this embodiment, the resolution of the thumbnail is lower than that of the photographed image, which means that the file size of the thumbnail is smaller than that of the photographed image. When the electronic device shares the thumbnail, the thumbnail can be uploaded and downloaded faster, saving time and network resources and facilitating rapid image sharing.

[0139] This allows electronic devices to obtain and save high-resolution thumbnails, reducing the difference in image quality when switching from thumbnails to photographed images and improving the user experience.

[0140] The following is combined with Figure 7 and Figure 8The interface of the thumbnail display method provided in the embodiments of this application will be described. Figure 7 This illustrates the process of displaying an image based on the thumbnail control 101 provided in an embodiment of this application, such as... Figure 7 As shown:

[0141] After launching the camera app, the electronic device can display something like this: Figure 7 The interface shown in Figure 'a' can display a camera control 102. In response to a trigger operation on the camera control 102, the electronic device can display as shown in Figure 'a'. Figure 7 The interface shown in b is shown in the image. Figure 7 In the interface shown in b, the thumbnail control 101 can display image 2. Figure 7 The interface shown in 'a' and Figure 7 Other controls in the interface shown in b can be found by referring to... Figure 1 The interface shown in 'a' and Figure 1 The relevant description of the interface shown in b is not repeated here.

[0142] Within a certain time period, when a trigger operation is received for the thumbnail control 101, the electronic device can display as follows: Figure 7 The interface shown in 'c' illustrates this. It's understandable that, for a short period after taking a photo, the electronic device can prioritize displaying the thumbnail, and then switch to the actual photographic image once it's generated. Figure 7 In the interface shown in c, the electronic device can display thumbnail 701 (e.g., image 2).

[0143] After a period of time, the electronic device can display something like this. Figure 7 The interface shown in d is... Figure 7 The interface shown as d in the figure can display the captured image 702 (e.g., image 4).

[0144] In this embodiment, the sharpness of thumbnail 701 is less than that of photographed image 702. However, because the resolution of thumbnail 701 is improved, the difference in sharpness between thumbnail 701 and photographed image 702 in this embodiment is smaller than the difference in sharpness between thumbnail 106 and photographed image 107 in a possible implementation. Therefore, from Figure 7 The interface shown in c is directed towards Figure 7 When switching between interfaces as shown in d, the screen displayed on the electronic device will not change noticeably, thus improving the user experience.

[0145] Figure 8 This application illustrates the process of viewing thumbnails using a gallery application, as provided in an embodiment of this application. Figure 8 As shown:

[0146] In one possible implementation, the electronic device can display [the image] after the gallery application is launched. Figure 8The interface shown in Figure 'a'. Figure 8 The interface shown in 'a' can be the photo interface under the gallery application. This interface displays a photo control, an album control 802, a memory control, a creation control, and multiple images. For example, the photo control is selected, and the multiple images can be, for example, thumbnails 701 and captured images 702.

[0147] Electronic devices can enlarge the thumbnail based on a triggered action. For example, in Figure 8 In the interface shown in Figure a, when a click operation is received on thumbnail 701, the electronic device can display as follows: Figure 8 The interface shown in b is shown in the image. Figure 8 In the interface shown in b, the electronic device displays a magnified thumbnail 701.

[0148] In the photo gallery, thumbnails and photographed images can be stored together; for example, thumbnail 701 and photographed image 702 can be adjacent. To easily distinguish between thumbnails and photographed images, the electronic device can display a thumbnail identifier, for example... Figure 8 The thumbnail identifier 801 in the interface shown in Figure a; users can distinguish between the captured image and the thumbnail based on the thumbnail identifier 801.

[0149] This application embodiment does not restrict the order of the captured images and thumbnails during storage. In practical applications, electronic devices can also... Figure 8 In the interface shown in Figure a, the thumbnail 701 is displayed first, followed by the photographed image 702. Furthermore, the thumbnail identifier 801 can also be represented by other shapes or text identifiers; this embodiment does not impose any limitations on this.

[0150] In another possible implementation, the electronic device can display [the image] after the gallery app is launched. Figure 8 The interface shown in c is shown in the image. Figure 8 The interface shown in c can be the album interface under the gallery application. This interface displays photo controls, album controls 802, memory controls, creation controls, and multiple albums. For example, album control 802 is selected. The multiple albums can be, for example, camera album, all albums, video album, thumbnail album 803, and screenshot / screen recording album, etc. In this embodiment of the application, the thumbnails that can be captured in one shot can be stored in the thumbnail album.

[0151] When a trigger operation is received for the thumbnail album 803, the electronic device can display all thumbnails in the thumbnail album, such as... Figure 8 The interface shown in d is an example of this. This interface includes, for example, a thumbnail 701. When a click is received on the thumbnail 701, the electronic device can display, as shown in Figure d. Figure 8 The interface shown in b is shown in the image.

[0152] exist Figure 8 In the interface shown in b, the electronic device can display a sharing control 804; after receiving a trigger operation on the sharing control 804, the electronic device can share the thumbnail 701 to other applications (such as social applications).

[0153] In this way, electronic devices can obtain both the captured image and a thumbnail after taking a picture. The thumbnail has higher resolution and a smaller file size than the captured image, allowing for quick sharing and receiving, saving sharing time and network resources.

[0154] Optionally, in this embodiment, the camera application may include a control for enabling or disabling thumbnail saving. When the control is enabled, the electronic device can save both the thumbnail and the captured image simultaneously when taking a picture. When the control is disabled, the electronic device only saves the captured image. This embodiment does not impose any limitations on this.

[0155] The above provides a brief overview of the flow and interface of the thumbnail display method according to embodiments of this application. The following section details the process. Figure 9 and Figure 11 The structure of the thumbnail processing module in the embodiments of this application will be further described. See first... Figure 9 :

[0156] The thumbnail processing module may include: a color space conversion module, a thumbnail super-resolution module, and a format conversion module. The color space conversion (YUV2RGB) module converts YUV format images to RGB format images. The thumbnail super-resolution module performs super-resolution reconstruction on the image to improve its resolution. The format conversion module converts the image format to a preset format to suit thumbnail output. The thumbnail super-resolution module may include a preprocessing module, a super-resolution network model, and a post-processing module. The preprocessing module performs data rearrangement and data segmentation on the image; the super-resolution network model performs super-resolution processing on the image; and the post-processing module performs data stitching and / or image enhancement processing on the image.

[0157] For example, the process by which the thumbnail processing module processes image 1 into a thumbnail may include the following steps:

[0158] S901. Image 1 is obtained. The color space conversion module converts Image 1 into Image 5 and then transmits Image 5 to the thumbnail super-resolution module. Image 1 is a YUV format image, and Image 5 is an RGB format image.

[0159] The super-resolution network model can be executed in the RGB color space to more accurately simulate and enhance image details. Here, the YUV format image 1 obtained from the preview stream can be converted to the RGB format image 5 for subsequent super-resolution processing. This application does not limit the method of converting YUV format to RGB format.

[0160] S902, the preprocessing module in the thumbnail super-resolution module obtains the resolution and data format of image 5. The data format includes interleaving and deinterleaving formats.

[0161] The resolution of image 5 can be compared with the processing power of the super-resolution network model. If the resolution of image 5 is greater than the upper limit of the resolution that the super-resolution network model can handle, image 5 is divided into blocks. For example, for high-performance electronic devices, the super-resolution network model has a high processing power and can perform super-resolution processing on image 5 well; for low-performance electronic devices, the super-resolution network model has a low processing power, so image 5 can be divided into blocks; then the super-resolution network is used to perform super-resolution processing on each sub-image.

[0162] An image may include multiple pixels, and the value of any pixel may include R, G, and B values. The data format can be the R, G, and B values ​​of multiple pixels.

[0163] For both interleaving and uninterleaving formats, please refer to Figure 10 :

[0164] In image data storage and transmission, pixel values ​​can be stored using interleaving or deinterleaving formats. Interleaving arranges the red, green, and blue color channels in a specific order, such as: R, G, and B values ​​for pixel 1; R, G, and B values ​​for pixel 2; R, G, and B values ​​for pixel 3, and so on. Deinterleaving, on the other hand, decomposes an image containing multiple color channels back into its original, independent color channel form, such as: R values ​​for pixel 1, R values ​​for pixel 2, R values ​​for pixel 3, ..., G values ​​for pixel 1, G values ​​for pixel 2, G values ​​for pixel 3, ..., B values ​​for pixel 1, B values ​​for pixel 2, B values ​​for pixel 3, ...

[0165] S903. If the data format of image 5 is deinterleaved, the preprocessing module will process the data in the deinterleaved format. Figure 5 Image 5 was processed into an interlaced format.

[0166] To simplify the implementation of super-resolution network algorithms, electronic devices can convert images from deinterleaved to interleaved formats. This ensures that the super-resolution algorithm can efficiently process the entire image region while reducing the demand for data bandwidth and computational resources. Furthermore, this conversion helps maintain the consistency of the image data, allowing the algorithm to seamlessly access all necessary information during processing.

[0167] In one possible implementation, the electronic device can process the deinterleaved image 5 into an interleaved image 5 in the following manner: Create an array whose length is greater than or equal to the data length of the deinterleaved image 5; iterate through the data of the deinterleaved image 5, processing 3n consecutive data points in each iteration (n is a positive integer, which can be understood as the red, green, and blue components of n pixels; each 3n consecutive data points is, for example, R1, R2…Ri…Rn, G1, G2…Gi…Gn, B1, B2…Bi…Bn); assign Ri the value 3i, assign Gi the value 3i+1, and assign Bi the value 3i+2; after reordering, the interleaved image 5 is obtained. Other methods can also be used to convert the deinterleaved image in this application embodiment. This application embodiment does not limit this.

[0168] If the data format of image 5 is processed into an interleaved format, or if the data format of image 5 was originally an interleaved format, the electronic device may execute step S904.

[0169] S904. If the resolution of image 5 is less than or equal to the resolution threshold, input image 5 into the super-resolution network model to obtain image 6; wherein the resolution of image 6 is greater than the resolution of image 5.

[0170] The super-resolution network model is a pre-trained model that learns the mapping relationship between low-resolution sample images and their corresponding high-resolution sample images. The super-resolution network model has the ability to process low-resolution (LR) images into high-resolution (HR) images. A resolution threshold can be used to characterize the processing capability of the super-resolution network model. If the resolution of image 5 is less than or equal to the resolution threshold, the super-resolution network model supports processing image 5.

[0171] Optionally, S905 and the post-processing module process image 6 to obtain image 7.

[0172] The post-processing module can be used to stitch image data together. In this embodiment, in step S904, since the model's processing capability is sufficient to process image 5, image 5 is not segmented (the segmentation process can be found in subsequent step S1103). In step S905, the process of stitching the segmented images can be omitted.

[0173] In this embodiment of the application, to further improve image quality, the post-processing module can also be used to perform image enhancement processing on image 6; for example: noise reduction processing, sharpening processing, etc. That is, step S905 is an optional step.

[0174] S906, the format conversion module performs format conversion on image 7 to obtain image 2.

[0175] The format conversion module can be used to process image 7 into an image suitable for thumbnail output. Image 2 can be a thumbnail.

[0176] The format conversion module may include an RGB processing module, a YUV processing module, and an image resampling module. The RGB processing module adjusts the colors of image 7; the YUV processing module converts the RGB format image to the YUV format; and the image resampling module upsamples or downsamples the image to obtain an image conforming to the thumbnail output format.

[0177] For details on the processing steps of the format conversion module, please refer to [link / reference]. Figure 9 After obtaining image 7, image 7 is transferred to the RGB processing module. The RGB processing module performs color processing on image 7, including one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction. The RGB processing module then transfers the processed image 7 to the YUV processing module, which converts the RGB format image 7 to the YUV format image 7, where the YUV format can be YUVNV12. The YUV processing module then transfers the YUV format image 7 to the image resampling module, which resizes the YUV format image 7 to obtain image 2. The resizing process can be downsampling or upsampling. In this embodiment, the super-resolution image has a high resolution, and downsampling is generally used to adapt the image to the preset size of the thumbnail.

[0178] S907: Display image 2 in the camera application interface and save image 2 to the gallery application.

[0179] The thumbnail processing module can report image 2 to both the camera application and the gallery application. Electronic devices can then... Figure 7 Image 2 is displayed at thumbnail control 101 in the interface shown in b. Image 2 is also saved to the gallery application.

[0180] In this embodiment of the application, the AIGC link for image processing is deployed under a dump framework so that thumbnails under the AIGC link can be dumped to the gallery application.

[0181] It should be noted that, in order to reduce the processing time of the preview stream images, in this embodiment of the application, images 1, 2, 5, 6, and 7 can be single-frame images. Optionally, these images can also be multiple-frame images.

[0182] At this point, electronic devices can obtain relatively clear thumbnails, and these thumbnails can then be shared via gallery applications.

[0183] Figure 11 This application illustrates another image processing flow provided by an embodiment of the present application, such as... Figure 11 As shown:

[0184] S1101. Image 1 is obtained. The color space conversion module converts Image 1 into Image 5 and then transmits Image 5 to the thumbnail super-resolution module. Image 1 is a YUV format image, and Image 5 is an RGB format image.

[0185] S1102, the preprocessing module in the thumbnail super-resolution module obtains the resolution and data format of image 5.

[0186] Steps S1101 and S1102 can be referred to the relevant descriptions in steps S901-S902, and will not be repeated here.

[0187] S1103. If the resolution of image 5 is greater than the resolution threshold, the preprocessing module performs block processing on image 5 to obtain multiple sub-images.

[0188] Multiple sub-images, for example, image 51, image 52, image 53... image 5N (N is a positive integer greater than 1).

[0189] Image 5 has a resolution greater than the resolution threshold, which can be understood as the super-resolution network model's processing power being insufficient to process image 5. In this case, the electronic device performs block processing on image 5, dividing it into multiple sub-images. The number of sub-images is related to the resolution of image 5 and the resolution threshold; for example, if the resolution of image 5 is 1280×960 and the resolution threshold is 640×480, then the preprocessing module can divide image 5 into 4 equal sub-images.

[0190] The processing procedure for any sub-image can be described as follows:

[0191] S1104. If the data format of image 51 is deinterleaved, the preprocessing module will process the deinterleaved image 51 into an interleaved image 51.

[0192] The process of converting the deinterleaved image 51 into an interleaved image 51 can be found in the relevant description in step S903, and will not be repeated here.

[0193] S1105, the preprocessing module inputs image 51 into the super-resolution network model to obtain image 61; wherein, the resolution of image 61 is greater than the resolution of image 51.

[0194] The process of using a super-resolution network model to perform super-resolution on image 51 can be found in the relevant description in step S904, and will not be repeated here.

[0195] S1106 The post-processing module processes image 61 to obtain image 71.

[0196] This step can be referred to in the relevant description in step S1106, and will not be repeated here. At this point, the thumbnail super-resolution module has completed the super-resolution process for the sub-image (image 51).

[0197] Following step S1104, the method further includes:

[0198] S1107. If the data format of image 52 is deinterleaved, the preprocessing module will process the deinterleaved image 52 into an interleaved image 52.

[0199] S1108, the preprocessing module inputs image 52 into the super-resolution network model to obtain image 62; wherein, the resolution of image 62 is greater than the resolution of image 52.

[0200] S1109 The post-processing module processes image 62 to obtain image 72.

[0201] Steps S1107-S1109 can be referred to the relevant descriptions in steps S1104-S1106, and will not be repeated here.

[0202] In this embodiment of the application, the processing of image 51 and the processing of image 52 can be performed in parallel. For example, see... Figure 11 Figure a shows that the super-resolution network model's processing of image 51 can be synchronized with the preprocessing module's processing of image 52, and the post-processing module's processing of image 61 can be synchronized with the super-resolution network model's processing of image 52...

[0203] Specifically, after the preprocessing module processes image 51, it can input the interleaved image 51 into the super-resolution network model for image processing. While the super-resolution network model is processing the image, the preprocessing module is idle and can be used to process image 52. After the super-resolution network model performs super-resolution on image 51, it is idle and can be used to perform super-resolution on image 52, and so on…

[0204] It should be noted that in this embodiment, the thumbnail super-resolution module achieves parallel processing of multiple sub-modules by having multiple modules run on different processors. Specifically, the preprocessing module can run on the first processor, the super-resolution network model can run on the second processor, and the post-processing module can run on the third processor. The first, second, and third processors are three different processors. For example, the first processor can be a CPU, the second processor an NPU, and the third processor a GPU; or, the first processor can be a GPU, the second processor an NPU, and the third processor a CPU. Alternatively, the electronic device can also be equipped with multiple GPUs, CPUs, and / or NPUs. For example, the first processor can be a GPU, the second processor an NPU, and the third processor another GPU; this embodiment will not list all such instances. In this way, after the first processor finishes processing the previous sub-image, it can immediately begin processing the next sub-image, reducing processing waiting time.

[0205] Figure 11 Figure b illustrates another image processing flow, where the three modules in the thumbnail super-resolution module run on the same processor. This flow can be as follows: After obtaining image 5, the preprocessing module processes image 5 to obtain an interlaced arrangement format; then, image 5 is divided into multiple sub-images and passed to the super-resolution network model. The super-resolution network model performs super-resolution processing on the multiple sub-images to obtain multiple high-resolution character images; finally, these high-resolution sub-images are passed to the post-processing module to obtain image 7.

[0206] from Figure 11 As shown in Figure b, in this method, the electronic device needs to preprocess all sub-images using the preprocessing module before the super-resolution network module can perform super-resolution on the sub-images; and the post-processing module can only process the sub-images after the super-resolution network module has completed super-resolution processing of all sub-images.

[0207] Figure 11 Figure a in the middle can Figure 11 A comparison with graph b shows that: Figure 11As shown in Figure a, in the parallel processing process, the three modules run on different processors, which can save waiting time in the image processing process and speed up the generation of thumbnails.

[0208] Similarly, after step S1107, the following steps are also included:

[0209] S1110. If the data format of image 53 is deinterleaved, the preprocessing module will process the deinterleaved image 53 into an interleaved image 53.

[0210] S1111 The preprocessing module inputs image 53 into the super-resolution network model to obtain image 63; wherein, the resolution of image 63 is greater than the resolution of image 53.

[0211] S1112, the post-processing module processes image 63 to obtain image 73.

[0212] Steps S1110-S1112 can be referred to the relevant descriptions in steps S1107-S1109, and will not be repeated here.

[0213] ...

[0214] S1113. After traversing all sub-images, the post-processing module stitches images 71 to 7N together to obtain image 7.

[0215] Image 7N can be the image after the above processing of image 5N; image 7 can be the image obtained by stitching together image 71 and image 7N.

[0216] In some embodiments, the post-processing module may stitch the currently processed sub-image with the previously processed sub-image after obtaining each processed sub-image, until the last sub-image is stitched together. For example, after obtaining image 71, image 71 is cached; after obtaining image 72, image 71 is stitched together with image 72 to obtain the stitched image; after obtaining image 73, image 73 is stitched together with the stitched image, and so on, until image 7N is stitched together to obtain image 7.

[0217] In other embodiments, the post-processing module may stitch all processed sub-images together after obtaining all processed sub-images. For example, after obtaining image 71, image 71 is cached; after obtaining image 72, image 72 is cached... After obtaining image 7N, images 71 to 7N are stitched together to obtain image 7. This application does not limit the process of stitching multiple processed sub-images into one image.

[0218] S1117, the format conversion module performs format conversion on image 7 to obtain image 2.

[0219] S1118. Display image 2 in the camera application interface and save image 2 to the gallery application.

[0220] Steps S1117 and S1118 can be referred to the relevant descriptions in steps S906 and S907, and will not be repeated here.

[0221] Therefore, for electronic devices with poor performance, parallel processing can be used to speed up the thumbnail processing and reduce waiting time; at the same time, clear thumbnails can be obtained for easy saving and sharing.

[0222] The above examples illustrate the image processing workflow with bokeh and skin-smoothing functions disabled during shooting. The following section will combine these examples with... Figure 12 and Figure 13 This document explains how to display thumbnails for the blurring and skin-smoothing functions when they are enabled.

[0223] See Figure 12 :

[0224] S1201. Acquire the original image and preview it to obtain image 1.

[0225] This process can be referred to in the relevant description in step S601. Image 1 can be an image processed by image processing engine 1.

[0226] S1202. With the skin smoothing control and / or blurring control in the open state, perform skin smoothing and / or blurring processing on image 1 to obtain image 8.

[0227] With the skin smoothing and / or blurring controls enabled, the electronic device can detect identifiers indicating skin smoothing and / or blurring processing. The electronic device uses the functional feature module to perform skin smoothing and / or blurring algorithm processing on image 1, resulting in image 8.

[0228] The skin smoothing algorithm process may include the following steps: performing skin detection on image 1 to obtain the face and skin regions in image 1; identifying the texture in the face and skin regions and smoothing the texture; adjusting the color temperature, saturation, and brightness of the face and skin regions.

[0229] The blurring algorithm process may include the following steps: subject recognition in image 1; creating a depth map to obtain the foreground and background, wherein the foreground includes the subject; blurring the background using methods such as Gaussian blur; and processing the edges of the foreground and background to weaken them.

[0230] Other methods can also be used to beautify and blur image 1 in the embodiments of this application, and the embodiments of this application do not limit this.

[0231] S1203. Display image 8 in the camera application interface.

[0232] This step can be referred to in the relevant description in step S602, and will not be repeated here.

[0233] Following step S1201, the method further includes:

[0234] S1204. In response to a trigger operation on the camera control, obtain image 2 based on image 1 and the super-resolution network model; the resolution of image 2 is greater than the resolution of image 1.

[0235] This step can be referred to in the relevant description in step S603, and will not be repeated here.

[0236] S1205. Perform skin smoothing and / or blurring processing on image 2 to obtain image 9.

[0237] The skin-smoothing and blurring processes can be referred to in the relevant description in step S1202, and will not be repeated here.

[0238] S1206. Save image 9 in the gallery application and display image 9 in the thumbnail control in the camera application interface.

[0239] S1207. Obtain image 10 from the original image and store image 10 in the gallery application; the resolution of image 10 is greater than the resolution of image 9.

[0240] Steps S1206-S1207 can be referred to the relevant descriptions in steps S604-S605, and will not be repeated here.

[0241] With the skin smoothing and / or blurring functions enabled, image 8 is a preview, image 9 is a thumbnail, and image 10 is the captured image. The resolution of image 8 is lower than that of image 9, and the resolution of image 9 is lower than that of image 10.

[0242] This allows electronic devices to obtain and save high-resolution thumbnails, reducing the difference in image quality when switching from thumbnails to photographed images and improving the user experience.

[0243] Specifically, see below. Figure 13 For further explanation of how the blurring and skin-smoothing functions are displayed as thumbnails when enabled, please refer to [link to documentation]. Figure 13 :

[0244] S1301. Image 1 is obtained. The color space conversion module converts Image 1 into Image 5 and then transmits Image 5 to the thumbnail super-resolution module. Image 1 is a YUV format image, and Image 5 is an RGB format image.

[0245] S1302, the preprocessing module in the thumbnail super-resolution module obtains the resolution and data format of image 5.

[0246] S1303. If the data format of image 5 is deinterleaved, the preprocessing module will process the data in the deinterleaved format. Figure 5 Image 5 was processed into an interlaced format.

[0247] S1304. If the resolution of image 5 is less than or equal to the resolution threshold, input image 5 into the super-resolution network model to obtain image 6; wherein the resolution of image 6 is greater than the resolution of image 5.

[0248] Optionally, S1305, the post-processing module, processes image 6 to obtain image 7.

[0249] S1306 The format conversion module performs format conversion on image 7 to obtain image 2.

[0250] Steps S1301-S1306 can be referred to the relevant descriptions in steps S901-S906, and will not be repeated here.

[0251] S1307, the skin-smoothing module performs skin-smoothing algorithm processing on image 2 to obtain the skin-smoothed image 2.

[0252] The functional modules may include a skin-smoothing module and a blurring module. The skin-smoothing module can be used to process images using skin-smoothing algorithms; the blurring module can be used to process images using blurring algorithms.

[0253] The skin smoothing module can be referred to in step S1202 for skin smoothing processing of image 2, which will not be repeated here.

[0254] S1308 The blurring module performs blurring algorithm processing on the beautified image 2 to obtain image 9.

[0255] The blurring module can be referred to in step S1202 for blurring image 2, which will not be repeated here.

[0256] It should be noted that in this embodiment, when the beautification control is enabled and the blurring control is disabled, the electronic device can execute step S1307 but not step S1308, and image 9 can be the beautified image. When the beautification control is disabled and the blurring control is enabled, the electronic device can execute step S1308 but not step S1307, and image 9 can be the blurred image.

[0257] With both the skin smoothing and blurring controls enabled, the electronic device can execute step S1307 first, followed by step S1308. Image 9 shows the image after skin smoothing followed by blurring. This is because skin smoothing primarily focuses on improving skin texture and reducing blemishes, and these effects require a clear outline of the subject to maintain a natural appearance. If blurring is performed first, it might blur the edges of the subject, affecting the accuracy and naturalness of the subsequent skin smoothing effect. Therefore, step S1307 precedes S1308.

[0258] In this embodiment, the blurring step is performed relatively late in the processing flow because: after blurring, the background of the image is relatively blurred; blurring the image too early will affect the processing effect of some preliminary processing steps, such as noise removal, color correction, and contrast enhancement. Furthermore, blurring relies on existing depth information in the image or simulates depth-of-field effects through algorithms; the higher the image quality, the better the blurring effect.

[0259] In this embodiment, the skin-smoothing process is placed relatively late in the processing flow because preliminary processing steps, such as color correction, exposure adjustment, and noise removal, provide a foundation for subsequent skin-smoothing processing. These preliminary processing steps ensure that the image has good lighting and color balance, enabling the skin-smoothing algorithm to more accurately identify and process skin areas, avoiding processing distortion due to poor image quality.

[0260] S1309. Display image 9 in the camera application interface and save image 9 to the gallery application.

[0261] Step S1309 can be referred to the relevant description in step S907, and will not be repeated here.

[0262] At this point, electronic devices can obtain relatively clear thumbnails, and these thumbnails can then be shared via gallery applications.

[0263] Optionally, with the blurring and / or beautification functions enabled, after step S1302, the electronic device may also employ the following... Figure 11 As shown in Figure a, multiple sub-images of image 5 are processed to obtain image 7. That is, step S1303 can also be replaced by S1103-S1113. After obtaining image 7 in step S1113, steps S1306-S1309 are then executed to perform format conversion, skin smoothing, and blurring on image 7. This embodiment of the application will not elaborate further on these aspects.

[0264] It should be noted that in the embodiments of this application, "greater than, less than or equal to" can also be written as "greater than or equal to, less than". Intermediate values ​​can be categorized as either greater than or less than. The embodiments of this application do not impose any restrictions on this.

[0265] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0266] Based on the above embodiments, this application provides a thumbnail display method. For example, Figure 14 This is a flowchart illustrating a thumbnail display method provided in an embodiment of this application.

[0267] like Figure 14 As shown, the method may include the following steps:

[0268] S1401. In response to the photo capture operation, acquire the first image, which is an image in the preview stream.

[0269] Taking a photo can be done, for example, in Figure 7 The click operation on the camera control 102 in the interface shown in Figure 'a'; the first image can be an image obtained from the preview stream, for example, the first image can correspond to Figure 6 Image 1, or the first image, in the illustrated embodiment may correspond to... Figure 12 Image 1 in the illustrated embodiment.

[0270] S1402. Divide the first image into N sub-images; N is a positive integer greater than 1.

[0271] This process can be referred to the relevant description in step S1103. The N sub-images can correspond to images 51, 52, ..., 5N. It should be noted that dividing the first image into N sub-images can be done by converting the first image (image 1) to RGB format (image 5), and then performing block processing on the RGB format image (image 5).

[0272] S1403. Process the data format of N sub-images into interleaved format. During the process of processing the data format of N sub-images into interleaved format, after processing the data format of each sub-image, input the processed sub-image into the super-resolution network model for super-resolution processing.

[0273] For data format and data format processing procedures, please refer to [reference needed]. Figure 10 The relevant descriptions in steps S1104, S1107, and S1110 are provided. The super-resolution processing procedure can be found in the relevant descriptions in steps S1105, S1108, and S1111.

[0274] After processing the data format of each sub-image, the processed sub-image is input into the super-resolution network model for super-resolution processing. This can be understood as:

[0275] At the first moment, the data format of the first sub-image is processed into an interleaved format.

[0276] At the second time step, the processed first sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the second sub-image is processed into an interleaved format.

[0277] At the third time step, the processed second sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the third sub-image is processed into an interleaved format.

[0278] Similarly, at time N, the processed (N-1)th sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the Nth sub-image is processed into an interleaved format.

[0279] At time N+1, the processed Nth sub-image is input into the super-resolution network model for super-resolution processing.

[0280] S1404. After the super-resolution network model has performed super-resolution processing on all N processed sub-images, the second image is obtained.

[0281] This process can be referred to the relevant descriptions in steps S1113-S1117. After performing data format processing and super-resolution processing on multiple sub-images respectively, the multiple high-resolution sub-images can be stitched together to obtain a second image. The second image can correspond to... Figure 9 Image 2 in the illustrated embodiment, or Figure 13 Image 9 in the illustrated embodiment.

[0282] S1405. Display the second image as a thumbnail.

[0283] This process can be referred to the relevant description of step S1118. Displaying the second image as a thumbnail can be understood as: in... Figure 8 In the interface shown in b, the second image is displayed in the area where the thumbnail control 101 is located.

[0284] In this way, electronic devices can obtain higher resolution thumbnails, reducing the difference in image quality when switching from thumbnails to captured images; and by using parallel processing, the super-resolution processing of thumbnails is accelerated, reducing waiting time.

[0285] Optionally, the first image is divided into N sub-images, including:

[0286] Obtain the resolution of the first image; if the resolution of the first image meets the preset conditions, divide the first image into N sub-images; wherein, the preset conditions include: the resolution of the first image is less than or equal to a resolution threshold; the resolution threshold is used to characterize the super-resolution network model's ability to process image resolution; the ratio between the resolution of the first image and the resolution threshold is positively correlated with the value of N.

[0287] This process can be referred to the relevant description in step S1102. It is understood that when the resolution of the first image is less than or equal to the resolution threshold, the electronic device may not segment the first image, as shown in step S904. When the resolution of the first image is greater than the resolution threshold, the first image is segmented.

[0288] It should be noted that when the electronic device has good performance, it can also use this method to process thumbnails. For example, when the resolution of the first image is less than the resolution threshold, the electronic device can also use parallel processing to shorten the time required for super-resolution.

[0289] In this way, when the performance of electronic devices is poor, parallel processing can be used to perform super-resolution processing on images to improve image resolution.

[0290] Optionally, after the super-resolution network model performs super-resolution processing on all N processed sub-images, a second image is obtained; this includes: stitching the N super-resolution sub-images into a third image; the third image is in RGB format; color processing is performed on the third image; color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; converting the color-processed third image into a YUV format third image; and downsampling the YUV format third image to obtain the second image.

[0291] This process can be referred to in the relevant descriptions in steps S1113 and S1117. It is understood that the post-processing module is optional. If post-processing of images 61-6N is not required, and the stitching process involves caching N super-resolution processed sub-images before stitching, the post-processing module may not participate in the parallel process as described in step S1403. In this case, the electronic device can use the pre-processing module and the super-resolution network model as follows... Figure 11 The process shown in step a (excluding the post-processing module within the dashed box) processes the sub-image.

[0292] Among them, the N super-resolution completed sub-images can correspond to images 61-6N; the third image can be the image after stitching images 61-6N; the color processing process can correspond to the processing process of the RGB processing module; the process of converting RGB format to YUV format can correspond to the processing process of the YUV processing module; and the image downsampling process can correspond to the processing process of the image resampling module.

[0293] In this way, the electronic device can process the super-resolution image into a thumbnail that meets the preset size, so that the image can be displayed in thumbnail form later.

[0294] Optionally, after inputting the processed sub-image into the super-resolution network model for super-resolution processing, the method further includes: during the process of inputting the processed sub-image into the super-resolution network model for super-resolution processing, after each processed sub-image is super-resolution completed, image enhancement processing is performed on the super-resolution sub-image; the image enhancement processing includes one or more of the following: denoising processing and sharpening processing.

[0295] Understandably, electronic devices can be configured with a post-processing module to further process the super-resolution sub-images, such as image enhancement. In this case, the pre-processing module, super-resolution module, and post-processing module can refer to... Figure 11 The process shown in step 'a' (including the post-processing module within the dashed box) processes the sub-image.

[0296] This process is, for example:

[0297] At the first moment, the data format of the first sub-image is processed into an interleaved format.

[0298] At the second time step, the processed first sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the second sub-image is processed into an interleaved format.

[0299] At the third time step, image enhancement processing is performed on the first sub-image after super-resolution; the processed second sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the third sub-image is processed into an interleaved format.

[0300] At the fourth time step, image enhancement processing is performed on the second sub-image after super-resolution; the processed third sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the fourth sub-image is processed into an interleaved format.

[0301] And so on...

[0302] At time N, image enhancement processing is performed on the (N-2)th sub-image after super-resolution; the processed (N-1)th sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the Nth sub-image is processed into an interleaved format.

[0303] At time N+1, image enhancement processing is performed on the N-1th sub-image after super-resolution; the processed Nth sub-image is then input into the super-resolution network model for super-resolution processing.

[0304] At time N+2, image enhancement processing is performed on the Nth sub-image after super-resolution is completed.

[0305] This allows for preprocessing, super-resolution processing, and post-processing of all N sub-images. This enables further processing of the super-resolution sub-images to improve their image quality; simultaneously, parallel processing enhances the efficiency of the thumbnail super-resolution module.

[0306] Optionally, obtaining the second image includes: stitching together the N enhanced sub-images into a fourth image; the fourth image is in RGB format; performing color processing on the fourth image; the color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; converting the color-processed fourth image into a YUV format fourth image; and downsampling the YUV format fourth image to obtain the second image.

[0307] Among them, the N sub-images that have been enhanced can correspond to images 71-7N; the fourth image can be image 7 after stitching images 71-7N; the color processing process can correspond to the processing process of the RGB processing module; the process of converting RGB format to YUV format can correspond to the processing process of the YUV processing module; and the image downsampling process can correspond to the processing process of the image resampling module.

[0308] In this way, the electronic device can process the super-resolution image into a thumbnail that meets the preset size, so that the image can be displayed in thumbnail form later.

[0309] Optionally, after inputting the processed sub-image into the super-resolution network model for super-resolution processing, the method further includes: during the process of inputting the processed sub-image into the super-resolution network model for super-resolution processing, after each processed sub-image is super-resolutiond, the super-resolution sub-image is stitched together with the previously super-resolution sub-image.

[0310] Understandably, in another scenario, the electronic device can be equipped with a post-processing module to further process the sub-images after super-resolution, such as image stitching. In this case, the pre-processing module, super-resolution processing module, and post-processing module can refer to... Figure 11 The process shown in step 'a' (including the post-processing module within the dashed box) processes the sub-image.

[0311] This process is, for example:

[0312] At the first moment, the data format of the first sub-image is processed into an interleaved format.

[0313] At the second time step, the processed first sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the second sub-image is processed into an interleaved format.

[0314] At the third time step, the first sub-image that has completed super-resolution is cached; the processed second sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the third sub-image is processed into an interleaved format.

[0315] At the fourth time step, the second sub-image that has been super-resolution completed is stitched together with the first sub-image that has been super-resolution completed; the processed third sub-image is input into the super-resolution network model for super-resolution processing; and the data format of the fourth sub-image is processed into an interleaved format.

[0316] And so on...

[0317] At time N, the (N-2)th sub-image that has been super-resolutiond is stitched together with the other sub-images that have been stitched together; the (N-1)th sub-image that has been processed is input into the super-resolution network model for super-resolution processing; and the data format of the Nth sub-image is processed into an interleaved format.

[0318] At time N+1, the (N-1)th sub-image that has been super-resolutiond is stitched together with the other sub-images that have been stitched together; the processed Nth sub-image is then input into the super-resolution network model for super-resolution processing.

[0319] At time N+2, the Nth sub-image that has been super-resolutiond is stitched together with the other sub-images that have been stitched together.

[0320] Thus, preprocessing, super-resolution processing, and post-processing are performed on all N sub-images.

[0321] It should be noted that the post-processing module can also simultaneously perform image enhancement and image stitching on the sub-images after super-resolution. This embodiment of the application does not impose any limitations on this. In this way, parallel processing can be used to improve the processing efficiency of the thumbnail super-resolution module.

[0322] Optionally, the electronic device includes a first module, a super-resolution network model, and a second module; the first module is used to process the data format of the sub-image into an interleaved format; the second module is used to perform image enhancement processing on the sub-image after super-resolution; the first module runs on a first processor, the super-resolution network model runs on a second processor, and the second module runs on a third processor; the first processor, the second processor, and the third processor are different processors.

[0323] The first module can correspond to the preprocessing module, and the second module can correspond to the postprocessing module. It is understood that the first module, the second module, and the super-resolution network model can run on different processors. For example, the first processor, the second processor, and the third processor are not the same processor.

[0324] In this way, the preprocessing module, the super-resolution network model, and the post-processing module can run in parallel, thereby improving the processing efficiency of the thumbnail super-resolution module.

[0325] Optionally, the first processor is a CPU, the second processor is an NPU, and the third processor is a GPU; or, the first processor is a GPU, the second processor is an NPU, and the third processor is a CPU.

[0326] Alternatively, the first processor, the second processor, and the third processor may be any combination of three processors among CPU1, NPU1, GPU1, CPU2, NPU2, GPU2, CPU3, NPU3, and GPU3. This application does not limit this.

[0327] This allows for the parallel operation of various preprocessing modules, super-resolution network models, and post-processing modules.

[0328] Optionally, after obtaining the second image, the method also includes: saving the second image in a gallery application.

[0329] In this embodiment of the application, the electronic device can achieve dual capture, storing both the thumbnail and the captured image simultaneously in the gallery application; this facilitates subsequent viewing and sharing of the thumbnail.

[0330] Optionally, after saving the second image in the gallery application, the process includes: displaying a first interface of the gallery application; the first interface includes the second image and a fifth image, the fifth image being a captured image obtained in response to a photo-taking operation, the resolution of the fifth image being greater than or equal to the resolution of the second image; the second image having a corresponding thumbnail identifier; in response to a triggering operation on the second image, displaying a second interface of the gallery application; the second interface displaying the second image and a sharing control; the sharing control being used to share the second image.

[0331] This scenario can correspond to Figure 8 The illustrated embodiment. The first interface of the gallery application may correspond to... Figure 8 The interface shown in Figure 'a'; the second image can correspond to thumbnail 701; the fifth image can correspond to the captured image 702. The thumbnail identifier can correspond to thumbnail identifier 801. The trigger operation for the second image can correspond to the click operation for thumbnail 701. The second interface can correspond to Figure 8 The interface shown in b; the sharing control can correspond to sharing control 804.

[0332] This allows you to store thumbnails in the gallery app and share them.

[0333] Optionally, after saving the second image in the gallery application, the method further includes: displaying a third interface of the gallery application, the third interface including a thumbnail album; displaying a fourth interface of the gallery application in response to a triggering operation on the thumbnail album; the fourth interface displaying one or more thumbnails, one or more thumbnails including the second image; and displaying the second interface of the gallery application in response to a triggering operation on the second image.

[0334] This scenario can correspond to Figure 8 The illustrated embodiment. The third interface of the gallery application may correspond to... Figure 8 The interface shown in c; the thumbnail album can correspond to thumbnail album 803; the trigger operation for the thumbnail album can correspond to the click operation for thumbnail album 803.

[0335] The fourth interface can correspond to Figure 8 The interface shown in 'd'; the second image can correspond to thumbnail 701; the trigger operation for the second image can correspond to the click operation for thumbnail 701. The second interface can correspond to Figure 8 The interface shown in b is shown in the image.

[0336] This allows you to store thumbnails in the gallery app and share them.

[0337] Optionally, the electronic device supports skin smoothing and / or blurring functions, and the first image is an image from the preview stream that has not undergone skin smoothing and / or blurring processing; after downsampling the third image in YUV format, it also includes:

[0338] With the skin smoothing and / or blurring functions enabled, the downsampled third image is processed with skin smoothing and / or blurring to obtain the second image.

[0339] This process can be referenced. Figure 12 and Figure 13 The embodiment shown may exclude the post-processing module from the thumbnail super-resolution module.

[0340] This allows for skin smoothing and / or blurring of the thumbnail to make it fit the photographed image, reducing the differences between the two and thus minimizing the image differences when switching between thumbnails and photographed images.

[0341] Optionally, the electronic device supports skin smoothing and / or blurring functions, and the first image is an image from the preview stream that has not undergone skin smoothing and / or blurring processing; after downsampling the fourth image in YUV format, it also includes:

[0342] With the skin smoothing and / or blurring functions enabled, the downsampled fourth image is processed with skin smoothing and / or blurring to obtain the second image.

[0343] This process can be referenced. Figure 12 and Figure 13 The embodiment shown. The thumbnail super-resolution module may include a post-processing module.

[0344] This allows for skin smoothing and / or blurring of the thumbnail to make it fit the photographed image, reducing the differences between the two and thus minimizing the image differences when switching between thumbnails and photographed images.

[0345] The thumbnail display method of the present application has been described above. The apparatus for performing the above-described thumbnail display method provided in the embodiments of the present application will now be described. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced in each other, and the related apparatus provided in the embodiments of the present application can perform the steps in the above-described thumbnail display method.

[0346] like Figure 15 As shown, the thumbnail display device 1500 can be used in communication equipment, circuits, hardware components, or chips. The thumbnail display device includes a display unit 1501 and a processing unit 1502. The display unit 1501 supports the display steps performed by the thumbnail display device 1500; the processing unit 1502 supports the information processing steps performed by the thumbnail display device 1500.

[0347] In a possible implementation, the thumbnail display device 1500 may also include a communication unit 1503. Specifically, the communication unit supports the thumbnail display device 1500 in performing data transmission and data reception steps. The communication unit 1503 may be an input or output interface, pins, or circuitry, etc.

[0348] In a possible embodiment, the thumbnail display device may further include a storage unit 1504. The processing unit 1502 and the storage unit 1504 are connected via a line. The storage unit 1504 may include one or more memories, which may be devices in one or more devices or circuits used for storing programs or data. The storage unit 1504 may exist independently and be connected to the processing unit 1502 of the thumbnail display device via a communication line. Alternatively, the storage unit 1504 may be integrated with the processing unit 1502.

[0349] Storage unit 1504 may store computer-executable instructions for the methods in the terminal device, so that processing unit 1502 executes the methods in the above embodiments. Storage unit 1504 may be a register, cache, or RAM, etc., and storage unit 1504 may be integrated with processing unit 1502. Storage unit 1504 may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, and storage unit 1504 may be independent of processing unit 1502.

[0350] The thumbnail display method provided in this application can be applied to electronic devices with communication functions. Electronic devices include electronic devices in general; the specific device form of an electronic device can be referred to the above-described related features, and will not be repeated here.

[0351] This application provides an electronic device, which includes a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the electronic device to perform the above-described method.

[0352] This application provides a chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to execute the above-described method. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0353] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0354] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0355] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0356] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0357] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for displaying thumbnails, characterized in that, Applied to electronic devices, including: In response to a photo capture operation, a first image is acquired, wherein the first image is an image in the preview stream; The first image is divided into N sub-images; where N is a positive integer greater than 1. The data format of the N sub-images is processed into an interleaved format. During the process of processing the data format of the N sub-images into an interleaved format, after processing the data format of each sub-image, the processed sub-image is input into the super-resolution network model for super-resolution processing. After the super-resolution network model performs super-resolution processing on all N processed sub-images, a second image is obtained; In response to a trigger action on the thumbnail control, the second image is displayed; Upon completion of saving the fifth image, the system switches from the second image to the fifth image; the fifth image is a photographed image obtained in response to the photographing operation, and the resolution of the fifth image is greater than or equal to the resolution of the second image.

2. The method according to claim 1, characterized in that, The step of dividing the first image into N sub-images includes: Obtain the resolution of the first image; If the resolution of the first image meets a preset condition, the first image is divided into N sub-images; wherein, the preset condition includes: the resolution of the first image is greater than a resolution threshold; the resolution threshold is used to characterize the super-resolution network model's ability to process image resolution; the ratio between the resolution of the first image and the resolution threshold is positively correlated with the value of N.

3. The method according to claim 1 or 2, characterized in that, After the super-resolution network model performs super-resolution processing on all N processed sub-images, a second image is obtained; including: The N super-resolution sub-images are stitched together to form a third image; the third image is in RGB format. The third image is subjected to color processing; the color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; Convert the color-processed third image into a YUV format third image; The third image in YUV format is downsampled to obtain the second image.

4. The method according to claim 1 or 2, characterized in that, After the processed sub-images are input into the super-resolution network model for super-resolution processing, the process also includes: During the process of inputting the processed sub-image into the super-resolution network model for super-resolution processing, after each processed sub-image is super-resolution completed, image enhancement processing is performed on the super-resolution sub-image; the image enhancement processing includes one or more of the following: denoising processing and sharpening processing; The process of obtaining the second image includes: The N enhanced sub-images are stitched together to form a fourth image; the fourth image is in RGB format. The fourth image is subjected to color processing; the color processing includes one or more of the following: hue adjustment, saturation adjustment, brightness adjustment, contrast adjustment, gamma correction, and color correction; Convert the color-processed fourth image into a YUV format fourth image; The fourth image in YUV format is downsampled to obtain the second image.

5. The method according to any one of claims 1-4, characterized in that, The electronic device includes a first module, the super-resolution network model, and a second module; the first module is used to process the data format of the sub-image into an interleaved format; the second module is used to perform image enhancement processing on the sub-image after super-resolution. The first module runs on the first processor, the super-resolution network model runs on the second processor, and the second module runs on the third processor; the first processor, the second processor, and the third processor are different processors.

6. The method according to claim 5, characterized in that, The first processor is a CPU, the second processor is an NPU, and the third processor is a GPU; or, the first processor is a GPU, the second processor is an NPU, and the third processor is a CPU.

7. The method according to any one of claims 1-6, characterized in that, After obtaining the second image, the following is also included: Save the second image in the gallery application.

8. The method according to claim 7, characterized in that, After saving the second image in the gallery application, the following steps are included: The first interface of the gallery application is displayed; the first interface includes the second image and the fifth image, and the second image has a corresponding thumbnail icon; In response to a trigger operation on the second image, a second interface of the gallery application is displayed; the second interface displays the second image and a sharing control; the sharing control is used to share the second image.

9. The method according to claim 7 or 8, characterized in that, After saving the second image in the gallery application, the following is also included: The third interface of the gallery application is displayed, which includes a thumbnail album; In response to a trigger operation on the thumbnail album, a fourth interface of the gallery application is displayed; the fourth interface displays one or more thumbnails, the one or more thumbnails including the second image; In response to a trigger operation on the second image, a second interface of the gallery application is displayed.

10. The method according to claim 3, characterized in that, The electronic device supports skin smoothing and / or blurring functions, and the first image is an image in the preview stream that has not undergone skin smoothing and / or blurring processing; After downsampling the third image in YUV format, the process also includes: With the skin-smoothing and / or blurring functions enabled, the downsampled third image is processed with skin-smoothing and / or blurring to obtain the second image.

11. The method according to claim 4, characterized in that, The electronic device supports skin smoothing and / or blurring functions, and the first image is an image in the preview stream that has not undergone skin smoothing and / or blurring processing; After downsampling the fourth image in YUV format, the following steps are also included: With the skin smoothing and / or blurring functions enabled, the downsampled fourth image is subjected to skin smoothing and / or blurring processing to obtain the second image.

12. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1-11.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

14. A chip system, characterized in that, It includes at least one processor and a communication interface, the communication interface and the at least one processor being interconnected via a line, the at least one processor being configured to run a computer program or instructions to perform the method as described in any one of claims 1-11.

15. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the method as described in any one of claims 1-11.

Images