Image display method, electronic device and storage medium

Abstract

The embodiments of the present application provide an image display method, an electronic device, and a storage medium, and relate to the technical field of terminal devices. The image display method provided in the embodiments of the present application can obtain a third image with a quality close to that of the second image based on a first image processing model, and in the process of switching from the third image to the second image, avoid the phenomenon of image jumps perceived by the user's naked eye, thereby improving the user experience. The method includes: in response to a photo-taking operation of a target scene, acquiring a first image, and performing an image processing process on the second image; inputting the first image into a first image processing model to obtain a third image, and displaying the third image; after the second image completes the image processing process, switching from displaying the third image to displaying the second image to make the display process smooth.

Description

Technical Field

[0001] The present application relates to the technical field of terminal devices, and in particular to an image display method, an electronic device and a storage medium. Background Art

[0002] With the development of terminal technology, electronic devices with photography functions have become an indispensable part of daily life, allowing users to capture wonderful moments at any time and any place. When a user takes a photo, the electronic device obtains a high-quality original image through its photo algorithm link, such as an original image with enhanced brightness for a dark scene. In order to facilitate users to quickly understand the photo effect, after the user takes a photo, the electronic device will quickly generate an image corresponding to a thumbnail. While the original image is being generated, the electronic device uses the image corresponding to the thumbnail as a temporary image for the user to view. When the original image is generated, the electronic device switches the temporary image to the original image for the user to view. Compared with the original image, the image corresponding to the thumbnail (temporary image) has lower image quality.

[0003] Since the camera algorithm chain of the electronic device is relatively long, it takes a long time for the user to trigger the camera operation and generate the original image, such as at least 2 seconds. When the user performs the camera operation and then views the captured image, in the process of switching from the image corresponding to the thumbnail to the original image, the user will perceive that the image displayed by the electronic device jumps, resulting in a poor user experience. Summary of the invention

[0004] In order to solve the above problems, an embodiment of the present application provides an image display method, which can effectively avoid the user's naked eye from perceiving the image jump during the process of switching from the third image to the second image, thereby improving the user experience.

[0005] In order to achieve the above-mentioned purpose, in a first aspect, in response to a photographing operation of a target scene, a first image is acquired, and an image processing process is performed on a second image, wherein the first image is used to make the photographed target scene have a first image quality, and the second image is used to make the photographed target scene have a second image quality, and the first image quality is less than the second image quality; the first image is input into a first image processing model to obtain a third image, and the third image is sent for display; wherein the third image is used to make the photographed target scene have a third image quality, and the third image quality and the second image quality have a first image quality difference, and the absolute value of the first image quality difference is less than a first preset threshold; after the second image completes the image processing process, the display of the third image is switched to the display of the second image to make the display process smooth.

[0006] According to the image display method provided in the embodiment of the present application, the electronic device can shoot a target scene to obtain a first image with low image quality and a second image with high image quality. Furthermore, the electronic device can input the first image into the first image processing model according to the first image processing model to obtain a third image. The image quality of the third image is higher than that of the first image, and is close to that of the second image. In this way, when the second image has not completed the processing process, the electronic device can use the third image as a temporary image for the user to view. Since the image quality of the third image is close to that of the second image, after the second image completes the image processing process, when the electronic device switches from the third image to the second image, the user's naked eye cannot perceive the image switching, thereby avoiding the phenomenon of image jumps and improving the user experience.

[0007] In a feasible implementation, in response to a photographing operation of a target scene, a first image is acquired, including: in response to a start-up operation of a camera application, a first photographing interface is displayed, and the target scene is displayed in the first photographing interface, wherein the first photographing interface includes a first control; in response to a click operation on the first control, the target scene is photographed to generate a first thumbnail in the first photographing interface, and the first image is acquired, wherein the first thumbnail is used to call up a third image corresponding to the first image. In this way, when the user clicks the first control and then checks the photographing effect of the photographed target scene, the electronic device displays the third image as a temporary image for the user to view. Since the image quality of the third image is close to that of the second image, after the second image completes the image processing process, in the process of switching the third image to the second image, the user's naked eye cannot perceive the image switching, thereby avoiding the phenomenon of image jumps and improving the user experience.

[0008] In a feasible implementation, in response to a click operation on a first control, a target scene is photographed to generate a first thumbnail on a first shooting interface, and after acquiring the first image, the step of inputting the first image into a first image processing model in response to a click operation on the first thumbnail to obtain a third image and display the third image is also included. In this way, the electronic device can display the third image as a temporary image for the user to view. Since the image quality of the third image is close to that of the second image, after the second image completes the image processing process, in the process of switching from the third image to the second image, the user's naked eye cannot perceive the image switching, thereby avoiding the phenomenon of image jumps and improving the user experience.

[0009] In a feasible implementation, in response to a click operation on a first control, a target scene is photographed to generate a first thumbnail on a first shooting interface, and after acquiring the first image, it also includes: storing the first image in a gallery application; in response to a start operation on the gallery application, displaying a first image interface, and displaying at least one thumbnail on the first image interface, the thumbnail includes a second thumbnail, and the second thumbnail is used to evoke a third image corresponding to the first image; in response to a click operation on the second thumbnail, the step of inputting the first image into a first image processing model is executed to obtain a third image, and the third image is sent for display. In this way, the electronic device can display the third image as a temporary image for the user to view. Since the image quality of the third image is close to that of the second image, after the second image completes the image processing process, in the process of switching the third image to the second image, the user's naked eye cannot perceive the image switching, avoiding the phenomenon of image screen jumps, and improving the user experience.

[0010] In a feasible implementation, inputting the first image into the first image processing model to obtain the third image includes: inputting the first image into the first image processing model having a first model structure to obtain the third image, wherein the first model structure is used to make the input image and the output image of the first image processing model have the same resolution. In this way, based on the first image processing model having the first model structure, the electronic device can obtain the first image and the third image with the same resolution, so that the first image processing model can provide higher image quality and more accurate results when processing images.

[0011] In a feasible implementation, the first image processing model includes a first sub-model and a second sub-model, and the first image is input into the first image processing model to obtain a third image, including: inputting the first image into the first sub-model to obtain a fourth image, the first sub-model is used to improve the image quality of the first image on the basis of increasing the resolution of the first image, so as to generate the fourth image; inputting the fourth image into the second sub-model to obtain the third image, and the second sub-model is used to reduce the resolution of the fourth image to the same as the resolution of the first image, so as to generate the third image. In this way, the electronic device uses the second sub-model to adjust the resolution of the fourth image to the same resolution as the first image, thereby ensuring that the resolution of the third image is the same as the resolution of the first image, providing a basis for subsequent image processing.

[0012] In a feasible implementation, inputting the first image into the first sub-model to obtain the fourth image includes: inputting the first image into the first sub-model having a second model structure to obtain the fourth image, wherein the second model structure is used to make the resolution of the input image and the output image of the first sub-model different. In this way, the structure of the first sub-model makes the resolution of the input first image and the output fourth image of the first sub-model different, so that the first sub-model can process and generate images more efficiently and flexibly.

[0013] In a feasible implementation, before obtaining the first image in response to a photographing operation of a target scene, the method further includes: obtaining a training set, the training set including at least one first sample image and a first target image corresponding to the first sample image; wherein the image quality of the first target image is higher than the image quality of the first sample image; cropping each first sample image to obtain at least one second sample image, and cropping each first target image to obtain at least one second target image; using at least one second sample image as the input of a second image processing model and at least one second target image as the output of the second image processing model to train the second image processing model; and updating the parameters of the second image processing model to obtain the first image processing model. In this way, based on the online parameter update of the second image processing model after offline training, the first image processing model is obtained, and the first image processing model can cover all shooting scenes (target scenes), so that the final output image has a tendency of different target scenes, and the output image can be better adapted to each target scene.

[0014] In a feasible implementation, cropping each first sample image to obtain at least one second sample image includes: cropping each first sample image based on a first preset size to obtain at least one second sample image. In this way, cropping the first sample image into multiple second sample images can significantly increase the number of training sample images of the first image processing model, and effectively improve the generalization ability of the first image processing model.

[0015] In a feasible implementation, based on the first preset size, each first sample image is cropped to obtain at least one second sample image, including: based on the first preset size, at least one first preset area is set, and each first preset area is arranged in parallel; based on the first preset area, each first sample image is cropped to obtain a second sample image corresponding to each first preset area. In this way, cropping the first sample image into multiple second sample images can significantly increase the number of training sample images of the first image processing model, and effectively improve the generalization ability of the first image processing model.

[0016] In a feasible implementation, each first sample image is cropped based on a first preset size to obtain at least one second sample image, including: based on the first preset size, at least one second preset area is set, and each two adjacent second preset areas are overlapped; based on the second preset area, each first sample image is cropped to obtain a second sample image corresponding to each second preset area. In this way, cropping the first sample image into multiple second sample images can significantly increase the number of training sample images of the first image processing model and effectively improve the generalization ability of the first image processing model. At the same time, the overlapping setting of each two adjacent second preset areas can capture image information from different angles, perspectives and local areas, thereby increasing the diversity of training sample images.

[0017] In a feasible implementation, the parameters of the second image processing model are updated to obtain the first image processing model, including: obtaining a first update image group each time in response to a photo-taking operation, each first update image group including a first image and a second image; storing the first update image group in an image database; when the number of first update image groups is greater than a second preset threshold, cropping each first update image group to obtain at least one first cropped image group, the first cropped image group including a first cropped image corresponding to the first image and a second cropped image corresponding to the second image; using at least one first cropped image as the input of the first image processing model and using at least one first cropped image as the output of the first image processing model to train the first image processing model. In this way, based on the first update image group, the first image processing model is continuously updated and iterated, which can effectively improve the performance of the first image processing model and allow the first image processing model to adapt to more new scenes.

[0018] In a feasible implementation, the image database includes at least one first database, each of which corresponds to a different image scene; the image scene may include at least one of the lighting conditions of the image, the weather conditions of the image, and the characteristics of the camera. In this way, the image database includes first databases applied to different image scenes, and the setting of different image scenes can help the subsequent first image processing model to apply to multiple shooting scenes, thereby improving the generalization ability of the first image processing model.

[0019] In a feasible implementation, storing the first updated image group in the image database includes: in response to an operation for starting a camera application, obtaining preview interface data of the current scene; based on the preview interface data, storing the first updated image group in the first database of the image database. In this way, based on the preview interface data, the electronic device determines to which image scene the first image update group photographed for the target scene belongs, and stores it in the corresponding first database, providing a basis for the subsequent first image processing model to be applicable to multiple shooting scenes, thereby improving the generalization ability of the first image processing model.

[0020] In a feasible implementation, in response to a photographing operation of a target scene, obtaining a first image includes: receiving a light signal in response to the photographing operation; converting the light signal into a continuous electronic signal; sampling the continuous electronic signal to obtain a discrete electronic signal; quantizing the discrete electronic signal to obtain raw data; and generating a first image based on the raw data. In this way, the electronic device can convert the target scene obtained by the camera into the first image, which facilitates the subsequent extraction of information from the first image.

[0021] In a feasible implementation, before switching the third image to the second image, the method further includes: performing an image processing process on the second image to obtain the second image. In this way, the electronic device performs image processing on the second image to obtain a second image with high image quality, thereby enhancing the user's visual experience.

[0022] In a feasible implementation, an image processing process is performed on a second image to obtain a second image, including: in response to a photographing operation, obtaining a first image to be processed; preprocessing the first image to be processed to obtain a second image to be processed; the image quality of the second image to be processed is higher than that of the first image to be processed; the preprocessing includes at least one of de-mosaicing, black level correction, and noise suppression; inputting the second image to be processed into a third image processing model to obtain a third image to be processed; the image quality of the third image to be processed is higher than that of the second image to be processed; the second image processing model is used to optimize at least one of automatic exposure, automatic white balance, automatic focus, and sharpening of the second image to be processed to generate a third image to be processed; and performing high dynamic range synthesis HDR processing on the third image to be processed to obtain a second image. In this way, the electronic device performs image processing on the second image to obtain a second image with high image quality, thereby enhancing the user's visual experience.

[0023] In a feasible implementation, the first image, the second image, the third image, and the fourth image are all images in RGB format. Thus, based on the image in RGB format, the color in the image can be processed by the first image processing model to obtain an optimized image, thereby reducing the computational cost.

[0024] In a feasible implementation, in response to a photographing operation of a target scene, a first image is obtained, including: a camera application development toolkit SDK in an application layer of an electronic device responds to the photographing operation of the target scene, the camera SDK sends a photographing request to a camera service in an application framework layer of the electronic device; the camera service sends a photographing request to a camera HAL in a hardware abstraction HAL layer of the electronic device; the camera HAL sends a photographing request to a camera driver module in a driver layer of the electronic device; in response to the photographing request, the camera driver module drives a camera in a hardware layer of the electronic device to obtain a first image; the camera sends the first image to a memory in the hardware layer; a memory driver module of the electronic device obtains the first image from the memory in the hardware layer of the electronic device; the camera driver module sends the first image to a memory HAL in a HAL layer; the memory HAL sends the first image to the camera service; the camera service sends the first image to the camera HAL. In this way, the electronic device can convert the target scene obtained by the camera into a first image, which is convenient for subsequent information extraction of the first image.

[0025] In a feasible implementation, the first image is input into the first image processing model to obtain the third image, including: the camera HAL in the hardware abstraction HAL layer of the electronic device obtains the first image; the camera HAL inputs the first image into the image processing model to obtain the third image; the camera HAL sends the third image to the camera service in the application framework layer of the electronic device; the camera service sends the third image to the memory driver module in the driver layer of the electronic device; the memory driver module sends the third image to the memory of the hardware layer of the electronic device. In this way, after the second image completes the image processing process, since the image quality of the third image is close to that of the second image, in the process of switching from the third image to the second image, the user's naked eye cannot perceive the image switching, avoiding the phenomenon of image jumps, and improving the user experience.

[0026] In a second aspect, the present application provides an image display device, which has the function of implementing the electronic device behavior in the image display method of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0027] In a third aspect, the present application provides an electronic device comprising: a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; wherein the memory stores computer program code, the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the image display method provided in the first aspect above.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the image display method provided in the first aspect above.

[0029] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the image display method provided in the first aspect above.

[0030] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fifth aspects provided above can refer to the beneficial effects in the first aspect and any feasible implementation manner thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of image switching provided by an embodiment of the present application;

[0032] Figure 2 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a layered architecture of a software system of an electronic device provided in this embodiment;

[0034] Figure 4 is a schematic diagram of software modules and interactions between modules involved in an image display method provided in an embodiment of the present application;

[0035] Figure 5 This is a first flow chart of an image display method provided in an embodiment of the present application;

[0036] Figure 6 is a schematic diagram of an interface for starting a camera application provided in an embodiment of the present application;

[0037] Figure 7 is a timing diagram for generating a first image and a second image provided by an embodiment of the present application;

[0038] Figure 8 is a schematic diagram of switching images provided in an embodiment of the present application;

[0039] Fig. 9 is a structural schematic diagram of a first image processing model provided in an embodiment of the present application;

[0040] Fig.10 This is a second flow chart of an image display method provided in an embodiment of the present application;

[0041] Fig.11 It is a first flow chart of a first image processing model provided in an embodiment of the present application;

[0042] Fig.12 is a second flow chart of the first image processing model provided in an embodiment of the present application;

[0043] Fig.13 is a schematic diagram of a cropping method for a first sample image provided in an embodiment of the present application;

[0044] Fig.14 is a third flow chart of a first image processing model provided in an embodiment of the present application;

[0045] Fig.15 is a fourth flow chart of a first image processing model provided in an embodiment of the present application;

[0046] Fig.16 This is a third flow chart of an image display method provided in an embodiment of the present application;

[0047] Fig.17 This is a schematic diagram of a first interface of an image display method provided by an embodiment of the present application;

[0048] Fig.18 This is a third flow chart of an image display method provided in an embodiment of the present application;

[0049] Fig.19 is a schematic diagram of a second interface of an image display method provided in an embodiment of the present application;

[0050] Fig. 20 This is a fourth flow chart of an image display method provided in an embodiment of the present application;

[0051] Fig.21 It is a structural block diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, other embodiments obtained by ordinary technicians in this field without making creative work all belong to the protection scope of the present application.

[0053] In the following, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0054] In addition, in the present application, directional terms such as "upper", "lower", "inner" and "outer" are defined relative to the orientation of the components schematically placed in the drawings. It should be understood that these directional terms are relative concepts. They are used for relative description and clarification, and they can change accordingly according to the changes in the orientation of the components placed in the drawings.

[0055] In order to facilitate technical personnel to understand the technical solutions of the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained below.

[0056] Image transition refers to the process of smoothly replacing one image to another. Image transition can be divided into two categories, namely hard cut and soft cut. Hard cut refers to the process of directly replacing one image with another image without any transition effect in between. Soft cut refers to the process of switching from one image to another using various visual effects, such as visual effects that may include fade in and fade out. In the embodiment of the present application, image transition adopts hard cut.

[0057] Frame jumps refer to abrupt visual changes during image switching. In the embodiment of the present application, two images taken for the same target scene have different image qualities, so abrupt visual changes will occur during the hard switching from one image to another.

[0058] Image resolution refers to the fineness of details in an image, usually measured by the number of pixels (pi×els per inch, PPI). The higher the image resolution, the richer the details in the image, but it also means that the image size will be larger.

[0059] Image quality refers to the clarity of the image, the degree of detail retention, and the overall satisfaction of the visual effect.

[0060] De-mosaicing refers to the process of restoring the original details of the pixelated (mosaic effect) part of an image.

[0061] Image sensor refers to the basic device that constitutes an image. In the image sensor, it is responsible for converting the incoming light into electrical signals, which are then processed into digital image data.

[0062] Black level refers to the signal level recorded by the image sensor when it does not receive any light. Even in a completely dark environment, the pixels of the image sensor will still generate a certain amount of electrical signals, which can be called black level.

[0063] Black level correction, also called dark current correction or offset correction, is mainly aimed at eliminating or reducing the inherent signal generated by the image sensor when there is no external light, thereby avoiding encoding unwanted brightness or color deviations in the image.

[0064] Noise reduction, also known as denoising, is used to improve image quality by reducing noise in the image, which may be caused by imaging system hardware defects, signal transmission errors, lighting changes, or other factors during the data acquisition process.

[0065] Automatic exposure refers to the camera or camcorder automatically adjusting its exposure parameters to adapt to the lighting conditions of the shooting environment to ensure the correct exposure of the image.

[0066] Auto white balance (AWB) is used to automatically adjust the color balance of an image to ensure natural colors in photos and videos taken in various lighting conditions.

[0067] Auto focus (AF) refers to the ability of the camera system in photographic or video equipment to automatically detect and adjust the focus of the lens so that the subject is clearly presented in the image.

[0068] Sharpening is used to enhance the clarity and details of an image, making it appear clearer and more contrasty.

[0069] Software development kit (SDK): In a broad sense, it refers to a collection of related documents, examples, and tools that assist in the development of a certain type of software.

[0070] The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0071] With the advancement of electronic device technology, the photography experience has become more convenient. When users use electronic devices with photography functions to take pictures, the software and hardware inside the electronic devices can work together to ensure that users obtain high-quality original images and a good user experience. Since multiple steps and complex algorithms are involved in the process of generating original images, this usually means that this process requires a long processing time, such as 2 seconds (S). Therefore, the image quality of the original image generated by the complex photography algorithm chain is higher. For example, the image is obtained by suppressing the highlights of the overexposed areas of the original image, enhancing the brightness of the original image obtained in the dark scene, and selecting the appropriate output color for the style of the target scene.

[0072] In order to facilitate users to quickly understand the photo effect, when the user triggers the photo operation to generate the original image, an image corresponding to the thumbnail with the same image size as the original image will be quickly generated. Among them, the image corresponding to the thumbnail is not the thumbnail itself, but a larger version of the thumbnail. The image quality of the image corresponding to the thumbnail is the same as the image quality of the thumbnail. When the user triggers the photo operation to generate the original image, the electronic device will provide a quickly loaded image (the image corresponding to the thumbnail), so there is no need to wait until the original image is fully loaded before quickly viewing the image content.

[0073] While the original image is being generated, the image corresponding to the thumbnail is used as a temporary image for the user to view. Compared with the original image, the image corresponding to the thumbnail (temporary image) has a lower image quality. For example, there is a significant difference in color and brightness between the original image and the image corresponding to the thumbnail.

[0074] When the original image is generated, the temporary image is switched to the original image for the user to view.

[0075] Figure 1 This is a schematic diagram of image switching provided in an embodiment of the present application.

[0076] like Figure 1 As shown in (a) in FIG. 1 , in a scenario where a user takes a photo of a target scene and then views the captured image, Figure 1 (a) in the figure is that the electronic device quickly generates an image corresponding to a thumbnail for the target scene. The electronic device presents the image corresponding to the thumbnail to the user to meet the user's need to view it at any time.

[0077] like Figure 1 As shown in (b), Figure 1 (b) in the figure is the original image after being processed by a complex camera algorithm chain. Figure 1 The image corresponding to the thumbnail shown in (a) is switched to Figure 1 The original image shown in (b).

[0078] exist Figure 1 (a) in the switch to Figure 1 In the process (b) of FIG. 1 , since the image quality of the image corresponding to the thumbnail is low and the image quality of the original image is high, there is a certain difference in the image quality between the image corresponding to the thumbnail and the original image. Figure 1 (a) in the figure) to switch to the original image (as shown in Figure 1 As shown in (b) in the figure, the user will perceive that the image displayed on the electronic device has a screen jump phenomenon, and the user experience is poor.

[0079] In order to solve the above problems, an embodiment of the present application provides an image display method, during the image switching process, the user's naked eyes cannot perceive the image switching, thereby avoiding the phenomenon of image jumps and improving the user experience.

[0080] The image display method provided in this embodiment can be applied to electronic devices. In some embodiments, the electronic device can be a mobile phone, a tablet computer, a handheld computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0081] For example, Figure 2 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, taking a mobile phone as an example of the electronic device.

[0082] like Figure 2 As shown, the mobile phone may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, a sensor module 280, a display screen 293, a subscriber identification module (SIM) card interface 294, and a camera 295, etc. Among them, the sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, an ambient light sensor 280C, a gyroscope sensor 280D, a temperature sensor 280E, etc.

[0083] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0084] The controller can be the nerve center and command center of the mobile phone. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0085] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0086] In some embodiments, processor 210 may include one or more interfaces.

[0087] The external memory interface 220 can be used to connect to an external non-volatile memory to expand the storage capacity of the mobile phone. The external non-volatile memory communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0088] The internal memory 221 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0089] The charging management module 240 is used to receive charging input from a power supply device (such as a charger, a laptop power supply, etc.). The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 can receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 can receive wireless charging input through a wireless charging coil of the mobile phone.

[0090] While charging management module 240 is charging battery 242, it can also power the mobile phone through power management module 241. Battery 242 can be composed of multiple batteries connected in series. Power management module 241 is used to connect battery 242, charging management module 240 and processor 210.

[0091] The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, and supplies power to the processor 210, the internal memory 221, the display screen 293, the camera 295, and the wireless communication module 260. The power management module 241 can also be used to monitor parameters such as the battery voltage, current, battery cycle number, and battery health status (leakage, impedance). In some other embodiments, the power management module 241 can also be set in the processor 210.

[0092] The wireless communication function of the mobile phone can be realized through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem and baseband processor.

[0093] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas.

[0094] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied on mobile phones. The mobile communication module 250 can receive electromagnetic waves through the antenna 1, and filter, amplify and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the same device as at least some of the modules of the processor 210.

[0095] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to mobile phones. The wireless communication module 560 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0096] In some embodiments, the sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, an ambient light sensor 280C, a gyroscope sensor 280D, a temperature sensor 280E, and the like.

[0097] Among them, the pressure sensor 280A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be set on the display screen 293. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 280A, the capacitance between the electrodes changes. The mobile phone determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 293, the mobile phone detects the intensity of the touch operation based on the pressure sensor 280A. The mobile phone can also calculate the position of the touch based on the detection signal of the pressure sensor 280A. In an embodiment of the present application, the pressure sensor 280A is used to detect the user's operation on the first control or the second control to obtain the second image and the third image.

[0098] The fingerprint sensor 280B is used to collect fingerprints. The mobile phone 200 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint payment, etc.

[0099] The ambient light sensor 280C is used to sense the brightness of the ambient light. The ambient light sensor 280C can also cooperate with the proximity light sensor to detect whether the electronic device is in the pocket to prevent accidental touch.

[0100] The gyro sensor 280D can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 280D. The gyro sensor 280D can be used for anti-shake shooting.

[0101] The temperature sensor 280E is used to detect temperature. In some embodiments, the mobile phone 200 uses the temperature detected by the temperature sensor 280C to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 280C exceeds a threshold, the mobile phone 200 reduces the performance of the processor located near the temperature sensor 280E to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the mobile phone 200 heats the battery 242 to avoid abnormal shutdown of the mobile phone 200 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the mobile phone 200 performs a boost on the output voltage of the battery 242 to avoid abnormal shutdown caused by low temperature.

[0102] In some embodiments, the mobile phone may include 1 or N cameras 295, where N is a positive integer greater than 1. The type of camera 295 may be distinguished according to hardware configuration and physical location. In the embodiment of the present application, the camera may be used to capture images.

[0103] The mobile phone realizes the display function through a GPU, a display screen 293, and an application processor. The GPU is a microprocessor for image editing, connecting the display screen 293 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0104] The mobile phone can realize the shooting function through the ISP, the camera 295, the video codec, the GPU, the display screen 293 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0105] The ISP is used to process the data fed back by the camera 295. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 295. The camera 295 is used to capture static images or videos. In an embodiment of the present application, the ISP can be used to adjust the parameters in the shooting scene.

[0106] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when a mobile phone selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0107] The display screen 293 is used to display images, videos, etc. The display screen 293 includes a display panel. In some embodiments, the mobile phone may include 1 or N display screens 293, where N is a positive integer greater than 1. In the embodiment of the present application, the display screen 293 may be used to display a page required by the mobile phone (e.g., a first shooting interface, etc.).

[0108] The SIM card interface 294 is used to connect the SIM card. The SIM card can be connected to and separated from the mobile phone by inserting it into the SIM card interface 294 or pulling it out from the SIM card interface 294. The mobile phone can support one or more SIM card interfaces. The SIM card interface 294 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 594 at the same time. The SIM card interface 294 is also compatible with external memory cards. The mobile phone interacts with the network through the SIM card to realize functions such as calls and data communications. One SIM card corresponds to one user number.

[0109] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the mobile phone. In other embodiments of the present application, the mobile phone may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0110] Of course, it is understandable that the above Figure 2 The electronic device shown is only an exemplary description of the form of a mobile phone. If the electronic device is a tablet computer, a handheld computer, a PC, a PDA, a wearable device (such as a smart watch, a smart bracelet) or other device forms, the structure of the electronic device may include more Figure 2 The structure shown in the figure is less than Figure 2 More structures are shown in the figure, which are not limited here.

[0111] It is understandable that, in general, the realization of electronic device functions requires not only hardware support but also software cooperation. The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android® system with a layered architecture as an example to illustrate the software structure of the electronic device.

[0112] Figure 3 It is a schematic diagram of the layered architecture of the software system of the electronic device provided in this embodiment.

[0113] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces (such as APIs).

[0114] In some examples, reference Figure 3 As shown, in the embodiment of the present application, the software of the electronic device is divided into five layers, from top to bottom, namely, the application layer, the framework layer (or called the application framework layer), the system library and the Android runtime (android runtime), the HAL layer (hardware abstraction layer, hardware abstraction layer) and the driver layer (or called the kernel layer). Among them, the system library and the Android runtime can also be called the local framework layer or the native layer.

[0115] The application layer can include a series of applications. Figure 3 As shown, the application layer may include applications (APP) such as camera, gallery, calendar, map, WLAN, Bluetooth, music, short message, and call.

[0116] The framework layer provides application programming interface (API) and programming framework for applications in the application layer.

[0117] The application framework layer includes some predefined functions or services. For example, the application framework layer may include an activity manager, a window manager, a content provider, an audio service, a view system, a phone manager, a resource manager, a notification manager, a camera service, etc., and the present application embodiment does not impose any restrictions on this.

[0118] The system library can include multiple functional modules. For example: surface manager, media library, etc. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0119] The HAL layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. The HAL layer provides a standard interface to display the device hardware functions to the higher-level Java API framework (i.e., the framework layer). The HAL layer contains multiple library modules, each of which implements an interface for a specific type of hardware component, such as: audI / O HAL audio module, camera HAL camera HAL (also called camera HAL or camera hardware abstract module), where the camera HAL includes a thumbnail corresponding image processing module and an image processing module, and sensors HAL sensor module (or isensor service, sensor service). In an embodiment of the present application, the processing module can be used to receive instructions from the framework layer and send the instructions to the driver layer to start the camera.

[0120] The driver layer is a layer between hardware and software. The driver layer at least includes display driver, camera driver, audio driver, sensor driver, battery driver, etc., which are not limited in this application. In the embodiment of this application, the camera driver and hardware are used to interact to achieve image capture.

[0121] The hardware layer includes display, camera, and memory, etc.

[0122] The memory refers to a memory with a security protection function, which can be used to store the original image data collected by the camera.

[0123] The following describes the software modules and the interactions between the modules involved in the image display method provided in the embodiment of the present application.

[0124] Figure 4 is a schematic diagram of software modules and interactions between modules involved in an image display method provided in an embodiment of the present application;

[0125] like Figure 4As shown, the camera application in the application layer can interact with the camera SDK, and the camera SDK can interact with the camera service in the framework layer by calling a preset application programming interface (API) interface, and the camera service can interact with the camera HAL in the hardware abstraction layer (HAL) layer. The camera HAL can interact with the camera driver module in the driver layer, and the camera driver module can be used to drive the camera of the hardware layer to collect image data. The camera driver module can send the image data to the camera HAL. The camera HAL can include an image processing node, and the image processing node includes an image processing node corresponding to the thumbnail. The image processing node corresponding to the thumbnail processes the image data to obtain a processed image (such as a third image). The image processing node can process the image data to obtain a processed image (a second image). The image processing node corresponding to the thumbnail and the image processing node can store the processed images in the memory respectively. The image processing node corresponding to the thumbnail can send the image to the camera service, and the camera service calls the camera SDK to send the image to the camera application to display the image. Similarly, the gallery application in the application layer can also use the above-mentioned camera application interaction process to obtain the image stored in the memory.

[0126] Figure 5 This is a first flow chart of an image display method provided in an embodiment of the present application.

[0127] like Figure 5 As shown, the image display method provided in the embodiment of the present application may include the following steps:

[0128] Step S501: the electronic device starts a camera application and displays a first shooting interface.

[0129] Optionally, the camera application starts the camera application in response to a user's start request. In one embodiment, the camera application can start the camera application in response to the user clicking an icon of the camera application on the electronic device screen or using a voice command.

[0130] In one implementation, the camera application responds to the user's start request, calls the camera SDK, and sends a request to the camera service to access the camera resources. In response to the camera service accessing the camera resources, the camera HAL sends a request to start the camera. The camera service is used to manage the resource allocation of the camera hardware to ensure that the camera is correctly shared between multiple applications. When the camera service detects that there are idle camera resources, it sends an instruction to the camera HAL to start the camera preview mode.

[0131] The camera HAL sends an instruction to start the camera preview mode to the camera driver. Exemplarily, the camera HAL can establish communication with the camera driver module through an API interface and send an instruction to start the camera preview mode.

[0132] Next, the camera driver module responds to the instruction to start the camera preview mode, configures the camera parameters, such as exposure, white balance, focus, etc., and further drives the camera to continuously obtain raw image data. The camera continuously obtains raw image data based on the instruction to start the camera preview mode.

[0133] Exemplarily, based on an instruction to start the camera preview mode, the camera starts the image sensor in the camera to continuously capture the light signal of the target scene, and converts the continuous light signal into a continuous electronic signal; further, the continuous electronic signal is sampled and quantized to continuously obtain the original image data.

[0134] The camera driver module continuously obtains raw image data from the camera and sends continuous raw image data to the camera HAL.

[0135] The camera HAL processes the continuous raw image data to obtain continuous raw images, and continuously sends the raw images to the camera service. The camera service continuously sends raw images to the camera application through the camera SDK. The camera application renders the continuously received raw images to the preview interface (the first shooting interface) so that users can view the images captured by the camera in real time.

[0136] Figure 6 This is a schematic diagram of an interface for starting a camera application provided in an embodiment of the present application.

[0137] like Figure 6 As shown in (a), Figure 6 (a) is the main interface 10 of the electronic device, and the main interface 10 of the electronic device can display multiple application icons, such as an icon of a clock application, an icon of a calendar application, an icon of a memo application, and an icon 100 of a camera application, an icon 200 of a setting application, and an icon 300 of a gallery application. The electronic device can start the application in response to a user clicking on the application icon. The main interface 10 of the electronic device can also display a status bar, which can include: one or more signal strength indicators of mobile communication signals, one or more signal strength indicators of Wi-Fi signals, a battery indicator of the electronic device, a time indicator, etc.

[0138] In one implementation, the electronic device can launch a camera application in the main interface 10 .

[0139] For example, the electronic device may jump to the first shooting interface 11 (eg, Figure 6 (f) in the figure).

[0140] In another implementation, the electronic device may launch a camera application in a settings application.

[0141] like Figure 6 As shown, the electronic device can respond to the user on the main interface 10 (such as Figure 6 In (a) of FIG. 1 , a click operation 02 on the icon 200 of the setting application is performed to jump to the setting interface 12 (as shown in FIG. 1 ). Figure 6 As shown in (b) in FIG. 1 , the setting interface 12 may include multiple setting items, such as a WLAN setting item, a Bluetooth setting item, a mobile network setting item, an application setting item 400, etc. The electronic device may jump to the application interface 13 (such as FIG. 14 ) in response to the user clicking operation 04 on the application setting item 400 in the setting interface 12. Figure 6 In the application interface 13, multiple setting items may be included, such as an application management setting item 500, an application clone setting item, a default application setting item, an application usage time setting item, etc. The electronic device may jump to the application management interface 14 (such as Figure 6 As shown in (d) in FIG. 1 ), the application management interface 14 may include multiple application setting items, such as a calendar setting item, a clock setting item, a video setting item, a camera setting item 600, etc. The electronic device may jump to the application details interface 15 (such as FIG. 15 ) in response to the user clicking operation 06 on the camera setting item 600 in the application management interface 14. Figure 6 As shown in (e) in FIG. 1 ), the application details interface 15 may include an open control 700, a forced stop control, a deactivation control, a notification management setting item, etc. The electronic device may jump to the first shooting interface 11 (such as FIG. 11 ) in response to the user clicking operation 07 on the open control 700 in the application details interface 15. Figure 6 (f) in the figure).

[0142] It should be noted that Figure 6 The icons in the interfaces (main interface 10, first shooting interface 11, setting interface 12, etc.) are only shown as schematic diagrams and are not specifically limited.

[0143] Step S502: In response to a click operation on a target scene, the camera application calls the camera SDK and sends a first request and a second request to the camera service.

[0144] The first request is used to obtain a first image corresponding to the target scene, and the second request is used to obtain a second image corresponding to the target scene.

[0145] Optionally, the first request may carry the resolution size and data stream format of the image, and the second request may carry the resolution size and data stream format of the image.

[0146] Exemplarily, the resolution size of the images carried in the first request and the second request is 600×800 pixels (pixel), and the data stream format may be an image format.

[0147] The first image is a thumbnail image with the same resolution as the target image, and the second image is the target image processed by a complex camera algorithm chain. The processing time of the first image is less than the processing time of the second image.

[0148] Step S503: The camera service sends a first request and a second request to the camera HAL.

[0149] The first request and the second request may refer to the description of step S501 and will not be elaborated here.

[0150] Step S504: The camera HAL sends a first request and a second request to the camera driver module.

[0151] Step S505 : the camera driving module drives the camera to collect image data (raw image format, RAW data).

[0152] Among them, RAW data refers to the data directly output from the image sensor in the camera and is unprocessed data.

[0153] Step S506: the camera acquires RAW data.

[0154] Optionally, the image sensor in the camera captures the light signal of the target scene and converts the light signal into an electronic signal; further, the electronic signal is sampled and quantized to obtain RAW data.

[0155] As mentioned in step S505, RAW data is unprocessed data, and therefore, RAW data is different from an image. RAW data is data directly output from an image sensor and requires a series of subsequent processing to be converted into an image in an image format, such as JPEG.

[0156] Step S507: the camera sends RAW data to the camera driving module.

[0157] Step S508: The camera driver module sends RAW data to the image processing node in the camera HAL.

[0158] The camera HAL includes an image processing node, which is used to process the RAW data and convert it into an image format.

[0159] Step S509: the image processing node in the camera HAL generates a first image of the target scene based on the RAW data, and the image processing node performs a processing process on the second image based on the RAW data to generate a second image of the target scene.

[0160] In one implementation, after the RAW data is acquired, the RAW data is parsed to obtain parsed data, which can be used to construct the original image.

[0161] Exemplarily, firstly, RAW data is loaded to obtain binary data corresponding to the RAW data. The binary data is decoded and converted into pixel values, and the pattern of the color filter matrix is ​​applied to obtain the original color information of each pixel. Further, the original color information of each pixel is processed to obtain a first image.

[0162] In another embodiment, after acquiring the RAW data, the RAW data is preprocessed to obtain a first image to be processed, and the first image to be processed is input into a third image processing model to obtain a second image to be processed. Further, the second image to be processed is post-processed to obtain a second image. The image quality of the second image to be processed is higher than the image quality of the first image to be processed.

[0163] The pre-processing may include at least one of de-mosaicing, black level correction, and noise suppression. The third image processing model is used to optimize at least one of automatic exposure, automatic white balance, automatic focus, and sharpening of the second image to be processed to generate the second image to be processed. The post-processing may include high dynamic range synthesis.

[0164] It should be noted that the third image processing model can be a traditional image processing algorithm, such as the Canny edge detection algorithm, a model based on machine learning, such as a random forest, or a deep learning model, such as a convolutional neural network, without specific limitation here.

[0165] In the above, the first image has a first image quality, and the second image has a second image quality. The first image quality is less than the second image quality.

[0166] Figure 7 It is a timing diagram for generating an image provided in an embodiment of the present application. Figure 8 It is a schematic diagram of switching images provided in an embodiment of the present application.

[0167] Combination Figure 7 and Figure 8As shown, since the image quality of the second image is higher, this usually means that a longer processing time, such as two seconds, is required in the process of generating the second image. Therefore, the time required to generate the second image is greater than the time required to generate the first image.

[0168] Exemplarily, at time t0, the image processing node acquires the RAW data, and generates a first image of the target scene and a second image of the target scene based on the RAW data. Figure 8 As shown in (a) in FIG. 1 , at time t1, the first image processing is completed and the second image is being processed.

[0169] Step S510: the image processing node in the camera HAL sends the first image to the image processing node of the image corresponding to the thumbnail in the image processing nodes in the camera HAL.

[0170] Step S511, after generating the first image and before generating the second image, the image processing node in the camera HAL inputs the first image to the first image processing model in the image processing node corresponding to the thumbnail to obtain the third image.

[0171] For example, continue to combine Figure 7 and Figure 8 As shown in (b), at t2, the third image processing is completed.

[0172] The first image processing model has a first model structure, and the first model structure is used to make the resolution of the input image and the output image of the first image processing model the same. Optionally, the number of downsampling and upsampling of the first image is the same.

[0173] Fig. 9 It is a structural schematic diagram of a first image processing model provided in an embodiment of the present application.

[0174] like Fig. 9 As shown, the first image processing model may include a first encoder (dVAE Encoder) and a first decoder (dVAE Decoder). Using the first encoder, the first image is downsampled to obtain a first image feature. Using the first decoder, the first image feature is upsampled to obtain a third image, so that the third image is close to the second image, such as visually perceived. In this embodiment of the present application, close means that the two images are the same or the user perceives the images to be the same. In the above, the number of downsampling and the number of upsampling are not specifically limited.

[0175] Optionally, the first image processing model can be a pre-trained traditional image processing algorithm, such as the Canny edge detection algorithm, a pre-trained model based on machine learning, such as a random forest, or a pre-trained deep learning model, such as a convolutional neural network, without specific limitation herein.

[0176] Exemplarily, the first encoder is used to downsample the first image three times to obtain the first image feature, and the first decoder is used to upsample the first image feature three times to obtain the third image, so that the third image is close to the second image, such as close in visual perception.

[0177] For example, the image resolution of the first image is 224×224 pixels, after the first image is downsampled for the first time, the image resolution of the first image is 112×112 pixels, after the first image is downsampled for the second time, the image resolution of the first image is 56×56 pixels, after the first image is downsampled for the third time, the image resolution of the first image is 28×28 pixels, and the first image feature with a resolution of 28×28 pixels is obtained. Further, the resolution of the image corresponding to the first image feature is 28×28 pixels, after the first image feature is upsampled for the first time, the resolution of the image corresponding to the first image feature is 56×56 pixels, after the first image feature is upsampled for the second time, the resolution of the image corresponding to the first image feature is 112×112 pixels, and after the first image feature is upsampled for the third time, the resolution of the image corresponding to the first image feature is 224×224 pixels, and the third image is obtained, so that the third image is close to the second image, such as close in visual perception.

[0178] The third image is used to make the captured target scene have a third image quality, and the third image quality is higher than the first image quality. The third image has the same image resolution as the first image. In one embodiment, it can be determined whether the two images are close based on the image quality of the two images.

[0179] When comparing the image quality of two images, the similarity between the two images can be measured by calculating the pixel difference of the images, where the pixel difference refers to the difference between the pixels at the same position of the two images.

[0180] In one embodiment, the third image quality and the second image quality have a first image quality difference. When the absolute value of the first image quality difference is less than a first preset threshold, the second image quality is close to the third image quality, so as to determine that the third image is relatively close to the second image.

[0181] For example, the electronic device may calculate the absolute difference between the corresponding pixels of the third image and the second image, and then add the differences to obtain the first image quality difference. The first preset threshold may be set to 10 or 5, which is not specifically limited here.

[0182] For example, the first image is an image with 3 rows and 3 columns of pixels (3×3), and the pixel values ​​of the first image are [[110, 102, 104], [110, 112, 114], [120, 122, 124]]. The third image is a 3×3 image, and the pixel values ​​of the third image are [[100, 103, 104], [110, 112, 114], [120, 122, 124]]. Assume that the first preset threshold is set to 10 pixels. Based on the pixels of the first image and the second image, the absolute difference of each pixel on each color channel is calculated. For example, the absolute difference of the R channel, the G channel, and the B channel is summed to obtain the first image quality difference. Assume that the first image quality difference is 7. When the absolute value of the first image quality difference is less than 10 pixels, the quality of the second image is close to that of the third image, so as to determine that the third image is close to the second image.

[0183] Optionally, the calculation of the absolute difference between corresponding pixels of the third image and the second image may adopt a method of sum of pixel differences, a method of cumulative difference, or a method of root mean square error, which is not specifically limited here.

[0184] Step S512: the image processing node of the image corresponding to the thumbnail in the image processing node in the camera HAL sends the third image to the image processing node in the camera HAL.

[0185] Step S513: The image processing node in the camera HAL stores the third image and displays it to the user.

[0186] For example, continue to combine Figure 7 and Figure 8 As shown, at time t2, the third image is displayed to the user.

[0187] In one implementation, storing the third image may include steps S5131 to S5134.

[0188] Step S5131, the camera HAL sends the third image to the buffer center.

[0189] Step S5132: the buffer center sends the third image to the memory allocation HAL.

[0190] Step S5133: the memory allocation HAL sends the third image to the memory driver module.

[0191] Step S5134: the memory driving module writes the third image into the memory.

[0192] In another implementation, displaying the third image to the user may include steps S5135 and S5136.

[0193] Step S5135: The camera HAL sends the third image to the camera service.

[0194] Step S5136: The camera service sends the third image to the display application for display to the user.

[0195] Step S514, when the processing of the second image is completed, the display of the third image is switched to the display of the second image, so that the display process is smooth and the second image is stored and displayed to the user.

[0196] Exemplarily, when the processing of the second image is completed, the image processing node in the camera HAL stores and displays the second image to the user to replace the third image.

[0197] Continue to combine Figure 7 and Figure 8 As shown in (c) in FIG. 3 , at time t3, the second image is processed. At time t3, the second image is replaced by the third image, which is stored and displayed to the user. The second image is compared with the third image, and the image quality of the two images is relatively close. For example, the image quality of the second image is exactly the same as that of the third image. Alternatively, the image quality of the second image is similar to that of the third image, so that the user cannot distinguish the difference with the naked eye.

[0198] The specific content of step S514 can refer to the similar step S511 and will not be repeated here.

[0199] In summary, since the quality of the third image is similar to that of the second image, when the third image is switched to the second image, the user's naked eye can effectively avoid the phenomenon of image jumps being perceived, thereby improving the user experience.

[0200] Fig.10 This is a second flow chart of an image display method provided in an embodiment of the present application.

[0201] like Fig.10 As shown, the image display method provided in the embodiment of the present application may include the following steps:

[0202] Step S1001: The electronic device starts a camera application and displays a first shooting interface.

[0203] Step S1002 : In response to a photo-taking operation on a target scene, the camera application calls the camera SDK and sends a first request and a second request to the camera service.

[0204] Step S1003: In response to the first request and the second request, the camera service sends the first request and the second request to the camera HAL.

[0205] Step S1004: The camera HAL sends a first request and a second request to the camera driver module.

[0206] Step S1005: the camera driving module drives the camera to collect RAW data.

[0207] Step S1006: The camera obtains RAW data.

[0208] Step S1007: the camera sends RAW data to the camera driving module.

[0209] Step S1008: The camera driver module sends RAW data to the image processing node in the camera HAL.

[0210] Step S1009: the image processing node in the camera HAL generates a first image of the target scene based on the RAW data, and the image processing node performs a processing process on the second image based on the RAW data to generate a second image of the target scene.

[0211] The specific contents of steps S1001 to S1009 may refer to the above steps S501 to S509.

[0212] Step S1010: The image processing node in the camera HAL sends a first image to an image processing node of an image corresponding to a thumbnail in the image processing nodes in the camera HAL.

[0213] Step S1011, after generating the first image and before generating the second image, the image processing node in the camera HAL inputs the first image to the first image processing model in the image processing node corresponding to the thumbnail to obtain the third image.

[0214] Optionally, the first processing model includes a first sub-model and a second sub-model.

[0215] Step S1011 includes steps S91 to S93.

[0216] Step S91: the image processing node of the thumbnail corresponding image in the camera HAL inputs the first image into the first sub-model to obtain a fourth image.

[0217] The first sub-model is used to improve the image quality of the first image on the basis of increasing the resolution of the first image, so as to generate the fourth image.

[0218] The first sub-model has a second model structure, and the second model structure is used to make the resolution of the input image and the output image of the first sub-model different. In other words, the number of downsampling and the number of upsampling of the first image are different.

[0219] Continue to combine Fig. 9 As shown, the first image is downsampled using the first encoder to obtain the first image feature, and the first image feature is upsampled using the first decoder to obtain the fourth image.

[0220] In the above, there is no specific limitation on the number of downsampling and the number of upsampling.

[0221] Exemplarily, the first image is downsampled three times using the first encoder to obtain the first image feature, and the first image feature is upsampled four times using the first decoder to obtain the fourth image.

[0222] For example, the image resolution of the first image is 224×224 pixels. After the first image is downsampled for the first time, the image resolution of the first image is 112×112 pixels. After the first image is downsampled for the second time, the image resolution of the first image is 56×56 pixels. After the first image is downsampled for the third time, the image resolution of the first image is 28×28 pixels, and a fourth image with an image resolution of 28×28 pixels is obtained. Further, the fourth image has a resolution of 28×28 pixels. After the fourth image is upsampled for the first time, the resolution of the fourth image is 56×56 pixels. After the first image is upsampled for the second time, the resolution of the first image feature is 112×112 pixels. After the fourth image is upsampled for the third time, the resolution of the first image feature is 224×224 pixels. After the fourth image is upsampled for the fourth time, the resolution of the first image feature is 448×448 pixels, and a fourth image is obtained.

[0223] Step S92: The first sub-model in the image processing node corresponding to the thumbnail image sends the fourth image to the second sub-model in the image processing node corresponding to the thumbnail image.

[0224] Step S93: input the fourth image into the second sub-model to obtain the third image.

[0225] The second sub-model is used to reduce the resolution of the fourth image to the same as the resolution of the first image to generate the third image, so that the third image is closer to the second image, such as closer in visual perception.

[0226] Exemplarily, the fourth image with a resolution of 448×448 pixels is input into the second sub-model to obtain a third image with a resolution of 224×224 pixels.

[0227] Step S1012: the image processing node of the image corresponding to the thumbnail in the image processing node in the camera HAL sends the third image to the image processing node in the camera HAL.

[0228] Step S1013: The image processing node in the camera HAL stores the third image and displays it to the user.

[0229] Step S1014, when the processing of the second image is completed, the second image is replaced with the third image, which is stored and displayed to the user.

[0230] Exemplarily, when the processing of the second image is completed, the image processing node in the camera HAL stores and displays the second image to the user to replace the third image.

[0231] The first image, the second image, the third image and the fourth image are all images in RGB format.

[0232] The specific contents of steps S1010 to S1014 may refer to the above steps S510 to S514.

[0233] In summary, since the quality of the third image is similar to that of the second image, when the third image is switched to the second image, the user's naked eye can effectively avoid the phenomenon of image jumps being perceived, thereby improving the user experience.

[0234] Fig.11 It is a first flow chart of a first image processing model provided in an embodiment of the present application. Fig.12 This is a second flow chart of the first image processing model provided in an embodiment of the present application.

[0235] like Fig.11 and Fig.12 As shown, the training method of the first image processing model provided in the embodiment of the present application may include the following steps:

[0236] Step S111: the electronic device obtains a training set.

[0237] The training set includes at least one first sample image and a first target image corresponding to the first sample image. The image quality of the first target image is higher than the image quality of the first sample image. The first sample image may be an image corresponding to a thumbnail, and the second target image may be an image of image quality generated by a complex camera algorithm chain.

[0238] Exemplarily, for different target scenes, a training set is obtained. For example, for different target scenes, 100 first sample images and first target images corresponding to the first sample images are collected.

[0239] Step S112: the electronic device crops each first sample image to obtain at least one second sample image, and crops each first target image to obtain at least one second target image.

[0240] Optionally, each first sample image is cropped based on a first preset size to obtain at least one second sample image. The first preset size may be set to 96 pixels×96 pixels, or other sizes, which are not specifically limited herein.

[0241] Fig.13 It is a schematic diagram of a cropping method for a first sample image provided in an embodiment of the present application.

[0242] Combination Fig.13 As shown in (a), in one embodiment, based on a first preset size, at least one first preset area is set, and each first preset area is arranged in parallel; based on the first preset area, each first sample image is cropped to obtain a second sample image corresponding to each first preset area.

[0243] Exemplarily, taking the first sample image with an image size of 480 pixels × 480 pixels as an example, the first preset size is set to 96 pixels × 96 pixels, and at least one first preset area is set. The first preset area is a square area. The area of ​​the first preset area is 96 pixels × 96 pixels. Five first preset areas are arranged side by side. Based on the edge lines of the first preset areas, the first sample image is cropped to obtain second sample images corresponding to twenty-five first preset areas.

[0244] It should be understood that the first preset area may be a square area (eg Fig.13 The area shown in (a) is 96 pixels × 96 pixels), or it can be a rectangular area (such as 48 pixels × 96 pixels), or a quadrilateral area, such as a diamond area, which is not specifically limited here.

[0245] Combination Fig.13 As shown in (b), in another embodiment, based on the first preset size, at least one second preset area is set, and every two adjacent second preset areas are overlapped; based on the second preset area, each first sample image is cropped to obtain a second target image corresponding to each second preset area.

[0246] Exemplarily, taking the first sample image with an image size of 480 pixels × 480 pixels as an example, the first preset size is set to 96 pixels × 96 pixels, and at least one second preset area is set. The second preset area is a square area with an area of ​​96 pixels × 96 pixels. The overlapping area of ​​each two adjacent second preset areas is set to 96 pixels × 48 pixels. Based on the edge lines of the second preset areas, the first sample image is cropped to obtain a second target image corresponding to forty-one second preset areas.

[0247] It should be understood that the second preset area may be a square area (such as Fig.13 As shown in (b) in FIG. 1 ), it may also be a rectangular area or a quadrilateral area, such as a diamond area, which is not specifically limited here.

[0248] Step S113, using at least one second sample image as input of the second image processing model and using at least one second target image as output of the second image processing model to train the second image processing model.

[0249] Continue to combine Fig.12 As shown, a second sample image is used as the input of the second image processing model, and a second target image is used as the output of the second image processing model as an example for explanation.

[0250] Exemplarily, first, the second sample image is input into the initial image processing model to obtain a result image corresponding to the second sample image.

[0251] The initial image processing model is a general model of initial parameters.

[0252] Secondly, based on the second target image and the result image, a loss value is determined using a loss function.

[0253] The loss function is a function used to measure the difference between the result image (the result image corresponding to the second sample image) and the target image (the second target image). The loss function may be a mean square error, a cross entropy loss, a structural similarity index, a perceptual loss, etc., which are not specifically limited here.

[0254] Next, the loss value is reversely input into the initial image processing model to obtain the second image processing model.

[0255] Exemplarily, the loss value is input into the initial image processing model, and the model parameters in the initial image processing model are updated so that the result image output by the initial image processing model gradually fits the target image to obtain the second image processing model.

[0256] Step S114, updating the parameters of the second image processing model to obtain the first image processing model.

[0257] Continue to combine Fig.12 As shown, for the image processing scenario corresponding to the thumbnail, the second image processing model is a universal training model. However, since the camera in the electronic device will shoot for different target scenes, new target scenes may appear. If the training set does not contain images corresponding to the new training scenes, the output of the first image processing model will not meet the needs of users. For example, the new target scene may be a dark light scene, and the electronic device needs to enhance the brightness. For example, the new target scene may be a highlight or overexposed scene, and the electronic device needs to suppress the brightness of the image, etc. Therefore, based on the universal first image processing model, the final image effect is not good. Based on this, it is necessary to update the parameters of the first processing model to obtain an adaptive second image processing model.

[0258] Fig.14 It is a third flow chart of a first image processing model provided in an embodiment of the present application. Fig.15 It is a fourth flow chart of a first image processing model provided in an embodiment of the present application.

[0259] Combination Fig.14 As shown, step S114 includes steps S1141 to S1148.

[0260] Step S1141: acquiring a first updated image group each time in response to a photographing operation, wherein each first updated image group includes a first image and a second image.

[0261] For example, in combination Fig.15 As shown, the electronic device starts the camera operation to obtain the first image and the second image.

[0262] The specific content of step S1141 can refer to the above steps S501 to S509 and will not be repeated here.

[0263] Step S1142: store the first updated image group in a first database in the image databases.

[0264] The image database includes at least one first database, each of which corresponds to a different image scene; the image scene may include at least one of the lighting conditions of the image, the weather conditions of the image, and the characteristics of the camera.

[0265] Exemplarily, the lighting conditions of the image may include direct light (such as daytime light), scattered light (such as nighttime light), etc., the weather conditions of the image may include sunny light, rainy light, etc., and the characteristics of the camera may include a telephoto lens, etc.

[0266] For example, the first first database is for image scenes of daytime light, the second first database is for image scenes of dark day light, the third first database is for image scenes of sunny day light, the fourth first database is for image scenes of rainy day light, and the fifth first database is for image scenes taken with a telephoto lens.

[0267] Optionally, step S1142 may include step S421 - step S422.

[0268] Step S421, in response to the operation for starting the camera application, obtaining preview interface data of the current scene.

[0269] The specific content of step S421 can refer to the above step S501 and will not be repeated here.

[0270] Exemplarily, the electronic device obtains preview interface data of a "desert" photographed at 12 o'clock on a sunny day when the current scene is taken.

[0271] Step S422: based on the preview interface data, store the first updated image group in a first database of the image databases.

[0272] Continue to combine Fig.15 As shown, following the above example, the first updated image is an image group of the target scene of "desert" taken at 12 o'clock on a sunny day. The first updated image is stored as the third first database.

[0273] Step S1143: when the number of first update image groups is greater than or equal to a second preset threshold, crop each first update image group to obtain at least one first cropped image group.

[0274] The first cropped image group includes a first cropped image corresponding to the first image and a second cropped image corresponding to the second image.

[0275] The second preset threshold may be set to 10, or may be set to 9, 8, 7, etc., and is not specifically limited here.

[0276] Continue to combine Fig.15 As shown, assuming that the number of first updated images is 10, the first image in the first updated image is cropped to obtain a plurality of first cropped images, and the second image in the first updated image is cropped to obtain a plurality of second cropped images.

[0277] The specific content of step S1143 can refer to the above step S112 and will not be repeated here.

[0278] Step S1144, training the first image processing model using at least one first cropped image as input of the first image processing model and using at least one first cropped image as output of the first image processing model.

[0279] The specific content of step S1144 can refer to the above step S113 and will not be repeated here.

[0280] In summary, the adaptive first image processing model can handle the shooting requirements of different target scenes and meet the characteristics of scene differentiation.

[0281] Fig.16 This is the third flow chart of an image display method provided in an embodiment of the present application. Fig.17 This is a schematic diagram of a first interface of an image display method provided in an embodiment of the present application.

[0282] like Fig.16 As shown, the image display method provided in the embodiment of the present application may include the following steps:

[0283] Step S161, in response to the start-up operation of the camera application, displaying a first shooting interface and displaying a target scene in the first shooting interface.

[0284] The specific content of step S161 can refer to the above step S501 and will not be repeated here.

[0285] For example, in combination Fig.17 As shown, Figure 6 (f) is a first shooting interface 11 of the electronic device. An area 800 of the target scene is displayed in the first shooting interface 11. The first shooting interface 11 also includes a first control 900.

[0286] Step S162, in response to a click operation on the first control, photographing the target scene to generate a first thumbnail on the first photographing interface, and acquiring a first image.

[0287] For example, in combination Fig.17 As shown, the electronic device responds to the user on the main interface 10 (such as Fig.17 In (a) of FIG. 1 , a click operation 01 on the camera application icon 100 is performed to jump to the first shooting interface 11 (as shown in FIG. 1 ). Fig.17 As shown in (f) in FIG. 1 ), the electronic device responds to the user at the first shooting interface 11 (as shown in FIG. 1 ). Fig.17 In (f) of FIG. 1 ), a click operation 09 on the first control is performed to jump to the second shooting interface 16 (as shown in FIG. 1 ). Fig.17 As shown in (g) in the figure), a first thumbnail 901 is generated in the first shooting interface, and a first image is acquired. The first image is an image corresponding to the first thumbnail 901, and the first thumbnail is used to call up a third image corresponding to the first image.

[0288] Step S163, in response to a click operation on the first thumbnail, execute the step of inputting the first image into the first image processing model to obtain the third image, and display the third image.

[0289] Following the above example, continue to combine Fig.17 As shown in (g) and (k) in FIG. 1 , the electronic device responds to the user in the second shooting interface 16 (such as Fig.17 In (g) of FIG. 1 ), a click operation 010 on the first thumbnail 901 jumps to the image display interface 17 (as shown in FIG. 1 ). Fig.17 The third image is displayed on the image display interface 17.

[0290] In the above step S163, the first image is input into the first image processing model to obtain the specific content of the third image, which can be referred to the above step S513 and will not be repeated here.

[0291] In summary, since the quality of the third image is similar to that of the second image, when the third image is switched to the second image, the user's naked eye can effectively avoid the phenomenon of image jumps being perceived, thereby improving the user experience.

[0292] Fig.18 This is the third flow chart of an image display method provided in an embodiment of the present application. Fig.19 It is a schematic diagram of the second interface of an image display method provided in an embodiment of the present application.

[0293] like Fig.18 As shown, the image display method provided in the embodiment of the present application may include the following steps:

[0294] Step S181, in response to the start-up operation of the camera application, displaying a first shooting interface and displaying a target scene in the first shooting interface.

[0295] For example, in combination Fig.19 As shown, the electronic device responds to the user on the main interface 10 (such as Fig.19 In (a) of FIG. 1 , a click operation 01 on the camera application icon 100 is performed to jump to the first shooting interface 11 (as shown in FIG. 1 ). Fig.19 (f) in the figure).

[0296] Step S182, in response to the click operation on the first control, the target scene is photographed to generate a first thumbnail on the first photographing interface, and a first image is acquired, wherein the first thumbnail is used to call up a third image corresponding to the first image.

[0297] For example, continue to combine Fig.19 As shown, the electronic device responds to the user at the first shooting interface 11 (such as Fig.19In (f) of FIG. 1 ), a click operation 09 on the first control is performed to jump to the second shooting interface 16 (as shown in FIG. 1 ). Fig.19 As shown in (g) in the figure), a first thumbnail 901 is generated in the first shooting interface, and a first image is acquired. The first image is an image corresponding to the first thumbnail 901, and the first thumbnail is used to call up a third image corresponding to the first image.

[0298] The specific contents of step S181-step S182 can refer to the above-mentioned step S151-step S152 and will not be repeated here.

[0299] Step S183: store the first image in the gallery application.

[0300] The specific content of step S183 can refer to the above step S513 and will not be repeated here.

[0301] Step S184, in response to the start-up operation of the gallery application, displaying a first image interface, and displaying at least one thumbnail on the first image interface.

[0302] The thumbnails include a second thumbnail, and the second thumbnail is used to evoke a third image corresponding to the first image.

[0303] Following the above example, continue to combine Fig.19 As shown, the electronic device responds to the user in the second shooting interface 16 (such as Fig.19 In (g) of FIG. 1 , the user clicks the exit control 08 to jump to the main interface 10 (as shown in FIG. 1 ). Fig.19 Then, the electronic device responds to the user on the main interface 10 (as shown in (h)). Fig.19 In (h) of FIG. 1 , the startup operation 03 of the gallery application 300 is performed to jump to the gallery interface 18 (as shown in FIG. 1 ). Fig.19 As shown in (i) in the figure), in the gallery interface 18 (as shown in Fig.19 As shown in (i) in FIG. 1 ), multiple selection areas may be included, and the multiple selection areas are divided by time, such as “today”, “yesterday”, etc. Each selection area may include a thumbnail of at least one image, and the thumbnail includes the second thumbnail 902 .

[0304] Step S185, in response to a click operation on the second thumbnail, execute the step of inputting the first image into the first image processing model to obtain the third image, and display the third image.

[0305] Following the above example, combined with Fig.19 As shown, the electronic device responds to the user in the gallery interface 18 (such as Fig.19 (i) in FIG. 1 ) clicks the second thumbnail 902 to jump to the image display interface 17 (as shown in FIG. 1 ). Fig.19In the image display interface 17 (as shown in (k)). Fig.19 The third image is shown in (k)).

[0306] In summary, since the quality of the third image is similar to that of the second image, when the third image is switched to the second image, the user's naked eye can effectively avoid the phenomenon of image jumps being perceived, thereby improving the user experience.

[0307] Fig. 20 This is the fourth flow chart of an image display method provided in an embodiment of the present application.

[0308] like Fig. 20 As shown, the image display method provided in the embodiment of the present application may include the following steps:

[0309] Step S201, in response to a photographing operation of a target scene, a first image is acquired, and an image processing process is performed on a second image.

[0310] The photo-taking operation includes a start operation and a click operation. The start operation is used to start the camera application to obtain a first shooting interface. The click operation is used to click a first control in the first shooting interface to trigger an event of a first image of a target scene. The first image is used to make the captured target scene have a first image quality, and the second image is used to make the captured target scene have a second image quality, and the first image quality is less than the second image quality.

[0311] Step S202, input the first image into the first image processing model to obtain the third image, and send the third image for display.

[0312] The third image is used to make the captured target scene have a third image quality, the third image quality and the second image quality have a first image quality difference, and the absolute value of the first image quality difference is less than a first preset threshold.

[0313] Step S203, after the image processing process of the second image is completed, switching from displaying the third image to displaying the second image to make the display process smooth.

[0314] In this embodiment, the above method further includes:

[0315] In response to a photographing operation of a target scene, before acquiring a first image, the method further includes: acquiring a training set, the training set including at least one sample image and a first target image corresponding to the sample image; wherein the image quality of the first target image is higher than the image quality of the first sample image; cropping each first sample image to obtain at least one second sample image, and cropping each first target image to obtain at least one second target image; training the second image processing model using at least one second sample image as input of the second image processing model and using at least one second target image as output of the second image processing model; and updating parameters of the second image processing model to obtain the first image processing model.

[0316] Specifically, cropping each first sample image to obtain at least one second sample image includes: cropping each first sample image based on a first preset size to obtain at least one second sample image.

[0317] In one embodiment, based on a first preset size, at least one first sample image is cropped to obtain at least one second sample image, including: based on the first preset size, at least one first preset area is set, and each first preset area is arranged in parallel; based on the first preset area, each first sample image is cropped to obtain a second sample image corresponding to each first preset area.

[0318] In another embodiment, based on the first preset size, each first sample image is cropped to obtain at least one second sample image, including: based on the first preset size, at least one second preset area is set, and each two adjacent second preset areas are overlapped; based on the second preset area, each first sample image is cropped to obtain a second sample image corresponding to each second preset area.

[0319] Before switching the third image to the second image, the method further includes: performing an image processing process on the second image to obtain the second image.

[0320] Specifically, an image processing process is performed on the second image to obtain the second image, including: in response to a photo-taking operation, image data is obtained; pre-processing the image data to obtain a first image to be processed; the pre-processing includes at least one of de-mosaicing, black level correction, and noise suppression; the first image to be processed is input into a third image processing model to obtain a second image to be processed; the image quality of the second image to be processed is higher than the image quality of the first image to be processed; the third image processing model is used to optimize at least one of automatic exposure, automatic white balance, automatic focus and sharpening of the second image to be processed to generate a second image to be processed; and the second image to be processed is subjected to high dynamic range synthesis HDR processing to obtain a second image.

[0321] In summary, since the quality of the third image is similar to that of the second image, when the third image is switched to the second image, the user's naked eye can effectively avoid the phenomenon of image jumps being perceived, thereby improving the user experience.

[0322] The present application embodiment provides an electronic device, which may include: a display screen (such as a touch screen or a non-touch screen), a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the various functions or steps performed by the electronic device in the above method embodiment. The structure of the electronic device can refer to Figure 2 The structure of the electronic device shown.

[0323] Fig.21 It is a structural block diagram of a chip system provided in an embodiment of the present application.

[0324] The present application embodiment also provides a chip system 2100, such as Fig.21 As shown, the chip system 2100 includes at least one processor 2101 and at least one interface circuit 2102. The processor 2101 and the interface circuit 2102 can be interconnected through lines. For example, the interface circuit 2102 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 2102 can be used to send signals to other devices (such as a processor 2101 or a touch screen of an electronic device). Exemplarily, the interface circuit 2102 can read instructions stored in the memory and send the instructions to the processor 2101. When the instruction is executed by the processor 2101, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0325] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device in the above-mentioned method embodiment.

[0326] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the electronic device in the above method embodiment.

[0327] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0328] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application to obtain other embodiments based on the several embodiments provided in the present application, and these embodiments do not exceed the protection scope of the present application.

[0329] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0330] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0331] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0332] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code. It should be noted that those skilled in the art will easily think of other implementation schemes of the present application after considering the specification and practicing the application disclosed here. The present application is intended to cover any variants, uses or adaptive changes of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the technical field that are not disclosed in the present application. It is intended that the specification and examples be considered as exemplary only, with the true scope of the application being indicated by the claims.

[0333] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image display method, characterized in that: The image display method comprises: In response to a photographing operation of a target scene, a first image is generated, and an operation of generating a second image is performed, wherein the first image is used to make the photographed target scene have a first image quality, and the second image is used to make the photographed target scene have a second image quality, and the first image quality is less than the second image quality; Before the operation of generating the second image is completed, the first image is input into a first image processing model to obtain a third image, and the third image is sent for display; wherein the third image is used to make the photographed target scene have a third image quality, the third image quality and the second image quality have a first image quality difference, and the absolute value of the first image quality difference is less than a first preset threshold; After the operation of generating the second image is completed, the display of the third image is switched to the display of the second image to make the display process smooth.

2. The image display method according to claim 1, characterized in that: The step of generating a first image in response to a photographing operation of the target scene includes: In response to a start-up operation on a camera application, a first shooting interface is displayed, and the target scene is displayed in the first shooting interface, wherein the first shooting interface includes a first control; In response to a click operation on the first control, the target scene is photographed to generate a first thumbnail on the first shooting interface, and the first image is generated, wherein the first thumbnail is used to evoke the third image corresponding to the first image.

3. The image display method according to claim 2, characterized in that: In response to the click operation on the first control, the target scene is photographed to generate a first thumbnail on the first photographing interface, and after the first image is generated, the method further includes: In response to a click operation on the first thumbnail, the steps of inputting the first image into a first image processing model to obtain a third image and displaying the third image are performed.

4. The image display method according to claim 2, characterized in that: In response to the click operation on the first control, the target scene is photographed to generate a first thumbnail on the first photographing interface, and after the first image is generated, the method further includes: storing the first image in a gallery application; In response to a start-up operation on the gallery application, a first image interface is displayed, and at least one thumbnail is displayed on the first image interface, wherein the thumbnail includes a second thumbnail, and the second thumbnail is used to call up the third image corresponding to the first image; In response to a click operation on the second thumbnail, the steps of inputting the first image into a first image processing model to obtain a third image and displaying the third image are performed.

5. The image display method according to claim 1, characterized in that: The step of inputting the first image into a first image processing model to obtain a third image comprises: The first image is input into the first image processing model having a first model structure to obtain the third image, wherein the first model structure is used to make the resolution of the input image and the output image of the first image processing model the same.

6. The image display method according to claim 1, characterized in that: The first image processing model includes a first sub-model and a second sub-model, and inputting the first image into the first image processing model to obtain the third image includes: Inputting the first image into a first sub-model to obtain a fourth image, wherein the first sub-model is used to improve the image quality of the first image on the basis of increasing the resolution of the first image, so as to generate the fourth image; The fourth image is input into the second sub-model to obtain a third image, and the second sub-model is used to reduce the resolution of the fourth image to the same as the resolution of the first image to generate the third image.

7. The image display method according to claim 6, characterized in that: The step of inputting the first image into a first sub-model to obtain a fourth image comprises: The first image is input into the first sub-model having a second model structure to obtain the fourth image, wherein the second model structure is used to make the resolution of the input image and the output image of the first sub-model different.

8. The image display method according to claim 1, characterized in that: Also includes: Each time in response to a photographing operation, a first updated image group is acquired, each of the first updated image groups includes a first image and a second image; storing the first updated image group in an image database; When the number of the first update image groups is greater than a second preset threshold, cropping each of the first update image groups to obtain at least one first cropped image group, where the first cropped image group includes a first cropped image corresponding to the first image and a second cropped image corresponding to the second image; The first image processing model is trained by taking at least one of the first cropped images as input of the first image processing model and taking at least one of the first cropped images as output of the first image processing model.

9. The image display method according to claim 8, characterized in that: The image database includes at least one first database, each of which corresponds to a different image scene; the image scene includes at least one of the lighting conditions of the image, the weather conditions of the image, and the characteristics of the camera.

10. The image display method according to claim 9, characterized in that: The step of storing the first updated image group in an image database comprises: In response to an operation for starting a camera application, obtaining preview interface data of a current scene; Based on the preview interface data, the first update image group is stored in the first database in the image database.

11. The image display method according to claim 1, characterized in that: The step of generating a first image in response to a photographing operation of the target scene includes: In response to the photographing operation, receiving a light signal; converting the optical signal into a continuous electronic signal; Sampling the continuous electronic signal to obtain a discrete electronic signal; quantifying the discrete electronic signal to obtain raw data; Based on the original data, the first image is generated.

12. The image display method according to claim 1, characterized in that: Before switching from displaying the third image to displaying the second image, the method further includes: In response to the photographing operation, acquiring image data; Preprocessing the image data to obtain a first image to be processed; the preprocessing includes at least one of demosaicing, black level correction, and noise suppression; Inputting the first image to be processed into a third image processing model to obtain a second image to be processed; the image quality of the second image to be processed is higher than the image quality of the first image to be processed; the third image processing model is used to optimize at least one of automatic exposure, automatic white balance, automatic focus and sharpening of the second image to be processed to generate the second image to be processed; The second image to be processed is subjected to high dynamic range synthesis (HDR) processing to obtain the second image.

13. The image display method according to claim 6, characterized in that: The first image, the second image, the third image and the fourth image are all images in RGB format.

14. The image display method according to claim 1, characterized in that: The step of generating a first image in response to a photographing operation of the target scene includes: A camera application development kit SDK in an application program layer of the electronic device responds to the photographing operation of the target scene, and the camera SDK sends a photographing request to a camera service in an application program framework layer of the electronic device; The camera service sends a photo request to the camera HAL in the hardware abstraction HAL layer of the electronic device; The camera HAL sends a photo-taking request to a camera driver module in a driver layer of the electronic device; In response to the photo-taking request, the camera driving module drives the camera of the hardware layer of the electronic device to acquire the first image; The camera sends the first image to the memory of the hardware layer; The memory driving module of the electronic device acquires the first image from the memory of the hardware layer of the electronic device; The camera driving module sends the first image to the memory HAL in the HAL layer; The memory HAL sends the first image to the camera service; The camera service sends the first image to the camera HAL.

15. The image display method according to claim 1, characterized in that: The step of inputting the first image into a first image processing model to obtain a third image comprises: The camera HAL in the hardware abstraction HAL layer of the electronic device acquires the first image; The camera HAL inputs the first image into the image processing model to obtain the third image; The camera HAL sends the third image to a camera service in an application framework layer of the electronic device; The camera service sends the third image to a memory driver module in a driver layer of the electronic device; The memory driving module sends the third image to the memory of the hardware layer of the electronic device.

16. An electronic device, characterized in that: include: A display screen, a memory and one or more processors; the display screen and the memory are coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the image display method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that: It comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the image display method as described in any one of claims 1 to 15.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / the instructions are executed on an electronic device, the electronic device executes the image display method according to any one of claims 1 to 15.

Images