Image processing method and device

Abstract

The embodiments of the present application provide an image processing method and device, which relate to the field of terminal technology, so that an electronic device can use the brightness of a preview image to constrain the brightness of a photographed image, so that the brightness of a low-brightness area in the photographed image is almost the same as the low-brightness area in the preview image, thereby reducing the brightness jump when the electronic device switches from a preview image to a photographed image, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method and device. Background Art

[0002] With the popularization and development of the Internet, people's functional requirements for electronic devices are becoming more and more diverse. For example, electronic devices can support users to preview the captured images after taking them.

[0003] Typically, in response to a shooting operation, the electronic device may display a thumbnail, and in response to a triggering operation on the thumbnail, the electronic device may display a photographed image after displaying a preview image corresponding to the thumbnail.

[0004] However, since there is a brightness jump between the preview image and the photographed image, the user's experience is affected. Summary of the invention

[0005] The embodiments of the present application provide an image processing method and device, so that an electronic device can use the brightness of a preview image to constrain the brightness of a photographed image, reduce the brightness jump caused by the electronic device switching from a preview image to a photographed image, and improve the user experience.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, comprising: in response to a photographing operation, displaying a thumbnail, wherein the thumbnail is obtained by processing a preview image; in response to an operation on the thumbnail, displaying a preview image and a target image in sequence; wherein the target image is obtained by performing brightness processing on the photographed image based on the brightness of the preview image.

[0007] The preview image may be the target preview image described in the embodiments of the present application.

[0008] The electronic device can use the brightness of the preview image to constrain the brightness of the photographed image, thereby reducing the brightness jump caused by the electronic device switching from the preview image to the photographed image.

[0009] In one possible implementation, after responding to an operation on a thumbnail, the method further includes: performing brightness processing on the photographed image based on the brightness of the preview image to obtain a first image; fusing medium and low brightness areas in the first image and highlight areas in the photographed image to obtain a target image, wherein the highlight areas are areas with brightness greater than a first threshold, and the medium and low brightness areas are areas with brightness less than a second threshold, and the first threshold is greater than the second threshold.

[0010] The electronic device can fuse the medium and low brightness areas in the first image and the highlight area in the photographed image to achieve consistency between the medium and low brightness areas and the preview image while preventing overexposure of the highlight area, thereby obtaining an image with higher image quality.

[0011] In a possible implementation, the target image is obtained by fusing the medium and low brightness areas in the first image and the highlight area in the photographed image, including: fusing the medium and low brightness areas in the first image whose quality meets the requirements and the highlight area in the photographed image whose quality meets the requirements to obtain the target image.

[0012] The image quality meeting the requirement can be understood as fusing the medium and low brightness areas in the clear first image and the high brightness areas in the clear photographed image, avoiding fusing the blurred image into the target image, and improving the quality of the target image.

[0013] In a possible implementation, brightness processing is performed on a photographed image based on the brightness of the preview image to obtain a first image, including: dividing the preview image and the photographed image into images including N*N ranges, respectively, calculating the mean and the standard deviation within any range, and obtaining a first mean corresponding to the preview image, a first standard deviation corresponding to the preview image, a second mean corresponding to the photographed image, and a second standard deviation corresponding to the photographed image; upsampling the preview image divided into the N*N ranges and the photographed image divided into the N*N ranges, respectively, to obtain a preview image at a preset resolution and a photographed image at a preset resolution; and performing brightness processing on the photographed image at the preset resolution using the first mean, the first standard deviation, the second mean, and the second standard deviation to obtain the first image.

[0014] Since the mean value can represent the brightness of the pixel and the standard deviation can represent the contrast of the pixel, the electronic device can use brightness constraints to ensure that the brightness and contrast of the first image are consistent with the brightness and contrast in the preview image.

[0015] In a possible implementation, the first mean, the first standard deviation, the second mean, and the second standard deviation may satisfy the following formula:

[0016]

[0017] Among them, H ′ (x, y) is the grayscale value of the pixel in the first image, H(x, y) is the grayscale value of the pixel in the photographed image at a preset resolution, meanH(x, y) is the mean corresponding to the pixel in the first mean, meanP(x, y) is the mean corresponding to the pixel in the second threshold, stdH(x, y) is the standard deviation corresponding to the pixel in the first standard deviation, and stdP(x, y) is the standard deviation corresponding to the pixel in the second standard deviation.

[0018] In one possible implementation, the preview image includes an overexposed area, the photographed image or the target image does not include the overexposed area, the photographed image or the target image both include a highlight area corresponding to the overexposed area, and the photographed image is obtained by image fusion based on at least two images with different exposure times.

[0019] When there is an overexposed area in the preview image, the electronic device can replace the overexposed area in the target image after brightness migration with the brightness area in the photographed image through measures such as brightness migration and image brightness fusion, thereby ensuring that the target image is not overexposed.

[0020] In a possible implementation manner, the method further includes: saving the target image in a gallery application; and displaying the target image in response to an operation of opening the gallery application.

[0021] The electronic device not only reduces the brightness difference between the preview image and the target image by displaying the target image, but also can store the target image.

[0022] In a possible implementation, the grayscale value of the first threshold may be 220, and the grayscale value of the second threshold may be 190.

[0023] In a second aspect, an embodiment of the present application provides an image processing device, which includes a display unit and a processing unit. The display unit is used to process the steps of data display in the image processing device, and the processing unit is used to process the steps of data processing in the image processing device.

[0024] In a possible implementation, the image processing apparatus may further include a storage unit, which may include one or more memories, and the memory may be a device in one or more devices or circuits for storing programs or data.

[0025] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, the memory being used to store code instructions; the processor being used to run the code instructions so that the electronic device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0026] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed, the computer executes the method described in the first aspect or any one of the implementations of the first aspect.

[0027] In a fifth aspect, a computer program product includes a computer program, and when the computer program is executed, the computer executes the method described in the first aspect or any one of the implementations of the first aspect.

[0028] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0030] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of the software structure of the electronic device provided in the embodiment of the present application;

[0032] Figure 4 A flowchart of an image processing method provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of a process of image brightness migration provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the relationship between weight and brightness provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of an interface for displaying a target image provided in an embodiment of the present application;

[0036] Figure 8 A flowchart of another image processing method is provided for an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the hardware structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to clearly describe the technical solution of the embodiment of the present application, the terms involved in the embodiment of the present application are explained. It can be understood that the explanation is for a clearer explanation of the embodiment of the present application and does not necessarily constitute a limitation on the embodiment of the present application.

[0040] 1. Preview images and take photos

[0041] The preview image can be data collected in real time by the camera of the electronic device and allowed to be displayed in the preview screen. For example, when the electronic device receives an operation of the user to open the camera application, the electronic device can collect the preview image captured by the camera and display it in real time in the preview screen of the camera application.

[0042] The photographed image may be data obtained based on a photographing button in the electronic device. For example, when the electronic device receives a trigger operation of the user on the photographing button, the electronic device may obtain a photographed image obtained based on a camera at the time of photographing.

[0043] 2. High dynamic range (HDR)

[0044] HDR is a processing technology that improves image brightness and contrast. Compared with ordinary images, HDR can provide more dynamic range and image details. The final HDR image is synthesized using the image with the best details corresponding to each exposure time, which can better reflect the visual effects in the real environment.

[0045] A possible implementation of an electronic device determining whether the current scene is an HDR scene may be: the electronic device performs 4 times downsampling on the preview image to obtain a preview thumbnail, and determines whether the ratio of the number of highlighted pixels in the preview thumbnail to the number of all pixels in the preview thumbnail is greater than a preset pixel threshold. The preview thumbnail may be based on the picture corresponding to the preview image, retaining one row of pixels in every two rows of pixels, and storing each row of pixels in alternate columns. The highlighted pixels may be determined based on the grayscale threshold of the pixels, and the pixel threshold may be used to determine whether the current scene is a high dynamic scene. The electronic device may determine whether the current scene is a high dynamic scene based on one frame of data or multiple frames of data, which is not limited in the embodiments of the present application.

[0046] 3. Electronic equipment

[0047] Electronic devices may also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices may be mobile phones with touch screens, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technologies and specific device forms used by the electronic devices.

[0048] 4. Others

[0049] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first value and the second value are only used to distinguish different values, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0050] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0051] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0052] For example, Figure 1 A schematic diagram of a scenario provided in an embodiment of the present application. Figure 1 In the corresponding embodiments, an electronic device is taken as a mobile phone as an example for illustration, and this example does not constitute a limitation on the embodiments of the present application.

[0053] When the electronic device detects that the user has opened the camera application, the Figure 1 The interface shown in a in FIG. 1 may include: a photo button 101, a thumbnail 102, a preview screen 103, a control for switching cameras, and a control for indicating the use of a photo function. The camera application may also have other functions besides the photo function, such as supporting night scene photography, portrait photography, video recording, and HDR photography functions.

[0054] The preview screen 103 may display a preview image acquired by the electronic device in real time based on the camera, for example, a girl may be displayed in the preview screen 103 , and the thumbnail 102 may be a thumbnail of the previous image acquired by the electronic device based on the camera application, for example, a boy may be displayed in the thumbnail 102 .

[0055] In response to the user's triggering operation on the camera button 101, the electronic device displays the following Figure 1 The interface shown in b in the figure may include: a preview screen and a thumbnail 104, in which a girl may be displayed. Other contents displayed in the interface are similar to those in the Figure 1 The interface shown in a is similar and will not be described here.

[0056] The user can display the preview image corresponding to the thumbnail 104 and the photographed image corresponding to the thumbnail 104 by triggering the thumbnail 104. For example, in response to the user's triggering operation on the thumbnail 104, the electronic device can display the preview image corresponding to the thumbnail 104 and the photographed image corresponding to the thumbnail 104. Figure 1 After switching to the interface shown in c Figure 1 The interface shown in d.

[0057] Figure 1 The interface shown in c in FIG. 1 may display a preview image 105 corresponding to the thumbnail 104, and the preview image 105 may include an overexposed area 106 and a medium-low brightness area 107. The overexposed area 106 may include sunlight or other bright objects, and the medium-low brightness area 107 may include a shadow caused by low brightness, which may be on the lower side of the arm.

[0058] Figure 1 The interface shown in d in the figure can display: the photographed image 108 corresponding to the thumbnail 104. The photographed image 108 can be obtained by image fusion based on at least two image frames with different exposure times. Therefore, the photographed image 108 can suppress the brightness of the overexposed area in the preview image 105 and enhance the brightness of the low-brightness area in the preview image 106.

[0059] Although the photographed image 108 has a good dynamic range, the electronic device is Figure 1 The interface shown in c in the figure switches to Figure 1 There is a brightness jump in the process of the interface shown in d, which affects the user's experience of viewing the photographed images.

[0060] In view of this, an embodiment of the present application provides an image processing method, which can use the brightness of the preview image to constrain the brightness of the photographed image, so that the brightness of the low-brightness area in the photographed image is almost the same as the low-brightness area in the preview image, thereby reducing the brightness jump when the electronic device switches from the preview image to the photographed image, thereby improving the user experience.

[0061] Therefore, in order to better understand the embodiments of the present application, the structure of the electronic device of the embodiments of the present application is introduced below. Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0062] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, an indicator 192, a camera 193, and a display screen 194, etc.

[0063] The sensor module 180 may include one or more of the following: a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, or a bone conduction sensor ( Figure 2 (not shown) etc., which is not specifically limited in the embodiments of the present application.

[0064] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0065] The processor 110 may include one or more processing units. Among them, different processing units may be independent devices or integrated into one or more processors. A memory may also be provided in the processor 110 for storing instructions and data. For example, the processor 110 is used to implement the steps performed in the image processing method provided in the embodiment of the present application, and to store instructions and data related to the image processing method.

[0066] The charging management module 140 is used to receive charging input from a charger, which may be a wireless charger or a wired charger. The power management module 141 is used to connect the charging management module 140 to the processor 110 .

[0067] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0068] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. For example, the GPU is used to perform the graphics rendering process in the image processing method.

[0069] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1. For example, the display 194 is used to display a shooting interface in a camera application, and to display a preview image and a photographed image in response to a user triggering a thumbnail.

[0070] The electronic device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0071] The camera 193 is used to capture static images or videos. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1. For example, in response to a user's camera operation, the camera 193 may be used to obtain a raw image sequence.

[0072] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. For example, the internal memory 121 can be used to store executable program codes in the image processing method.

[0073] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0074] The touch sensor can be disposed on the display screen 194, and the touch sensor and the display screen 194 form a touch screen, or "touch screen". For example, the touch sensor is used to receive a user's trigger operation on a photo button, and receive a user's trigger operation on a thumbnail.

[0075] The key 190 includes a power button, a volume button, etc. The key 190 may be a mechanical key. It may also be a touch key. The electronic device may receive key input and generate key signal input related to the user settings and function control of the electronic device. In some scenarios, the electronic device may also implement a photo operation in response to the user pressing one or more keys in the key 190. The specific method of the photo operation is not limited in the embodiments of the present application.

[0076] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0077] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0078] Figure 3 It is a schematic diagram of the software structure of the electronic device provided in the embodiment of the present application.

[0079] The layered architecture divides the system into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom: application (APP), application framework layer (FWK), hardware abstraction layer (HAL), driver layer, and hardware layer.

[0080] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include: a camera, a gallery, etc. The camera may realize the shooting of images and the display of photographed images. The gallery may also be called an album, etc., and the gallery may realize the storage and access of images.

[0081] The application framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0082] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, and provides a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0083] The camera hardware abstraction layer may provide virtual hardware for camera device 1, camera device 2 or more camera devices. The camera algorithm library may include operating codes and data for implementing the image processing method provided in the embodiment of the present application.

[0084] The driver layer is a layer between hardware and software. The driver layer includes drivers for various hardware. The driver layer may include camera device drivers, digital signal processor drivers, and image processor drivers.

[0085] The camera device driver is used to drive the camera sensor to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0086] The following is a detailed description of the image processing method in the embodiment of the present application in combination with the above system structure:

[0087] In response to the user's operation of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends the instruction to the camera device driver of the kernel layer. The camera device driver can start the corresponding camera sensor and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0088] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0089] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores the program code for implementing the image processing method provided in the embodiment of the present application. Based on the digital signal processor and the image processor, the camera algorithm library executes the above code to implement the image processing method described in the embodiment of the present application.

[0090] The camera algorithm library can send the raw image captured by the camera to the camera hardware abstraction layer, which can then display it.

[0091] It should be understood that the software architecture provided in the embodiments of the present application is merely an example and does not constitute a limitation on the embodiments of the present application.

[0092] For example, Figure 4 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. Figure 4 In the corresponding embodiment, the electronic device may include: a camera, a camera access interface, a camera algorithm library, a camera hardware abstraction layer, and a camera device driver. The function of any module can be found in Figure 3 The corresponding embodiments will not be described in detail here.

[0093] like Figure 4 As shown, the image processing method may include the following steps:

[0094] S401: In response to a photo-taking operation, a camera device driver acquires a raw image sequence.

[0095] The original image sequence may include one or more of the following: a long frame sequence, a short frame sequence or a normal frame sequence, the exposure time of any frame in the long frame sequence is greater than the exposure time of any frame in the normal frame sequence, and the exposure time of any frame in the normal frame sequence is greater than the exposure time of any frame in the short frame sequence.

[0096] For example, the original image sequence may include: a short frame sequence and a normal frame sequence. The short frame sequence and the normal frame sequence can be used to achieve image fusion between at least two images with different exposure times at a certain moment to generate an HDR photographic image, and the image frames in the normal frame sequence can be used to generate preview images and thumbnails.

[0097] Or the original image sequence may include: a long frame sequence, a short frame sequence and a normal frame sequence, which is not limited in the embodiments of the present application.

[0098] It can be understood that the original image sequence may include at least two image frames captured at the same time, and the exposure times of the at least two image frames are different, so that the electronic device can generate a photographic image with HDR effect based on the image fusion of at least two image frames with different exposure times in the HDR photographic scene.

[0099] In response to the photo-taking operation, the method includes: Figure 1 The triggering operation of the interface photo button 101 shown in a, or the response to the user's photo operation in the HDR photo function.

[0100] S402: The camera algorithm library obtains a raw image sequence from the camera device driver.

[0101] Specifically, the camera hardware abstraction layer can obtain the original image sequence from the camera device driver, and the camera algorithm library can obtain the original image sequence from the camera hardware abstraction layer.

[0102] S403: The camera algorithm library processes the original image sequence into a preview image sequence and a photo image sequence.

[0103] The process of the camera algorithm library processing the original image sequence into the preview image sequence can be: the camera algorithm library can obtain a normal frame sequence in the original image sequence, and perform image pre-processing and other processes on any frame image in the normal frame sequence to obtain a preview image sequence.

[0104] The process of the camera algorithm library processing the original image sequence into the photographed image sequence may be: the camera algorithm library may obtain at least two image sequences in the original image sequence, and perform image preprocessing and other processes on any frame image in the at least two image sequences. Furthermore, the camera algorithm library may perform image fusion on at least two frames of images obtained at the same time in the at least two image sequences after image preprocessing to obtain the photographed image sequence.

[0105] It can be understood that any image in the photographed image sequence can be obtained by performing image fusion based on at least two images with different exposure times.

[0106] Among them, image pre-processing may include one or more of the following: bad pixel correction processing, RAW domain noise reduction processing, black level correction processing, optical shading correction processing / automatic white balance processing, color interpolation processing, color correction processing, or gamma correction processing, etc., which is not limited in the embodiments of the present application.

[0107] S404: The camera algorithm library generates a thumbnail based on the target preview image in the preview image sequence.

[0108] The preview image sequence can obtain the target preview image at the shooting time from the preview image sequence, and process the target preview image into a thumbnail by sampling and other methods. The shooting time can be the time when the photo-taking operation is triggered.

[0109] S405: The camera obtains a thumbnail from a camera algorithm library.

[0110] Specifically, the camera access interface can obtain the thumbnail from the camera algorithm library, and then the camera can obtain the thumbnail from the camera access interface.

[0111] S406: The camera calls the display to display thumbnails.

[0112] For example, the camera may display Figure 1 In the interface shown in b, the generated thumbnail can be displayed in the lower left corner of the interface.

[0113] S407: The camera algorithm library obtains from the camera an operation for viewing the photographed image corresponding to the thumbnail.

[0114] It is understandable that, in response to the operation for viewing the photographed image corresponding to the thumbnail, the camera can realize sequentially displaying the target preview image corresponding to the thumbnail (see S408-S409), and the target image corresponding to the thumbnail (see S410-413). Among them, the camera algorithm library can send the target preview image and the target image in sequence according to the steps shown in S408-S413, or the camera algorithm library can send the target image and the target preview image to the camera together after acquiring the target image, which is not limited in the embodiments of the present application.

[0115] S408: The camera obtains a target preview image from a camera algorithm library.

[0116] The process of the camera acquiring data from the camera algorithm library can be found in the description of S405 and will not be repeated here.

[0117] S409: The camera calls the display to display a preview image of the target.

[0118] For example, the camera may display Figure 1The interface shown in c in FIG. 1 may be a target preview image.

[0119] S410: The camera algorithm library obtains a target photographed image from a photographed image sequence, and performs brightness processing on the target photographed image based on the brightness of the target preview image to obtain a first image.

[0120] The target photographed image may be an image at the time of photographing, and the target photographed image may be obtained by image fusion based on at least two images with different exposure times. The target photographed image and the target preview image may be obtained by processing the original image generated at the same time.

[0121] In one possible implementation, when the target preview image includes an overexposed area, the target photographed image may not include the overexposed area. Since the target photographed image is obtained by image fusion based on at least two images with different exposure times, the target photographed image can suppress the overexposed area in the target preview image and enhance the brightness of the low-brightness area in the target preview image, and has a better dynamic range.

[0122] Exemplarily, in response to an operation for viewing a photographed image corresponding to a thumbnail, the camera algorithm library performs brightness processing on the target photographed image based on the brightness of the target preview image. The process of obtaining the first image can be seen in Figure 5 The corresponding embodiment, Figure 5 A schematic diagram of a process of image brightness migration provided in an embodiment of the present application.

[0123] like Figure 5 As shown, the camera algorithm library can process the target photographed image into an 8-bit grayscale image H, and process the target preview image into an 8-bit grayscale image P. The camera algorithm library can map a high-bit-number image to a low-bit-number image by tone mapping (tonemapping). In the embodiment of the present application, the low-bit-number image can be 8 bits or the like, and the tonemapping methods for the target photographed image and the target preview image can be different, which is not limited in the embodiment of the present application.

[0124] The camera algorithm library divides the grayscale image H and the grayscale image P into images with N*N grids, and calculates the mean of the grayscale values ​​in each grid and the standard deviation (or variance) of the grayscale values ​​in each grid, and obtains the mean map corresponding to the grayscale image H and the standard deviation map corresponding to the grayscale image H, and obtains the mean map corresponding to the grayscale image P and the standard deviation map corresponding to the grayscale image P. Wherein, N can be an integer greater than or equal to 1 such as 16, which is not limited to this in the embodiment of the present application. Wherein, the mean can be used to characterize the brightness of the pixel points in each grid, and the standard deviation is used to characterize the contrast of the pixel points in each grid.

[0125] The camera algorithm library upsamples the mean image corresponding to the grayscale image H, the standard deviation image corresponding to the grayscale image H, the mean image corresponding to the grayscale image P, and the standard deviation image corresponding to the grayscale image P, respectively, to obtain the upsampled mean image corresponding to the grayscale image H, the upsampled standard deviation image corresponding to the grayscale image H, the upsampled mean image corresponding to the grayscale image P, and the upsampled standard deviation image corresponding to the grayscale image P.

[0126] Among them, the camera algorithm library can upsample the image according to a preset resolution. The preset resolution can be 3648*2736, etc., which is not limited in the embodiments of the present application. The upsampling method may include: bilinear difference, nearest neighbor difference, lanzcos difference and other methods, which are not limited in the embodiments of the present application.

[0127] The camera algorithm library uses the upsampled mean map corresponding to the grayscale image P and the upsampled standard deviation map corresponding to the grayscale image P to perform brightness migration on the grayscale image H to obtain the first image H' after brightness migration. The local mean and standard deviation of H' are basically consistent with those in the grayscale image P.

[0128] It is understandable that due to

[0129] Then the brightness migration formula can be:

[0130] Among them, (x, y) can be understood as the coordinates of the pixel, H(x, y) can be understood as the grayscale value of the pixel in the grayscale image H, meanH(x, y) can be understood as the mean of the pixel in the grayscale image H, stdH(x, y) can be understood as the standard deviation of the pixel in the grayscale image H, H′(x, y) can be understood as the grayscale value of the pixel in the first image H′, meanP(x, y) can be understood as the mean of the pixel in the grayscale image P, and stdP(x, y) can be understood as the standard deviation of the pixel in the grayscale image P.

[0131] In a possible implementation, the electronic device may not perform the N*N grid division and up-sampling process on the grayscale image H and the grayscale image P respectively, which is not limited in the embodiments of the present application.

[0132] It can be understood that the camera algorithm library divides the grayscale image into N*N grid images, obtains the mean of the grayscale value in each grid and the standard deviation of the grayscale, upsamples, and then realizes brightness migration, so that the edge transition in the first image after brightness migration is smoother.

[0133] S411, the camera algorithm library fuses the highlight area in the target photographed image and the medium and low brightness areas in the first image to obtain the target image.

[0134] The camera algorithm library can perform image fusion processing on the first image and the target photographed image based on the weights pre-set by the weight generation module, so as to achieve fusion of the medium and low brightness areas in the first image and the highlight areas in the target photographed image.

[0135] For example, the camera algorithm library can implement image fusion based on AlphaBlend. For example, the formula for image fusion can be: target image = A*first image + (1-A) target photographed image, where A can be a preset weight.

[0136] Among them, the corresponding relationship between the value of A and the brightness can be as follows: Figure 6 As shown, Figure 6 A schematic diagram of the relationship between weight and brightness provided in an embodiment of the present application. Figure 6 As shown, the horizontal axis can be understood as the brightness value of the pixel point, and the vertical axis can be the value of A. There can be a corresponding relationship between the preset weight and the preset brightness, which can be understood as the higher the brightness value, the smaller the value of A, and the smaller the brightness value, the larger the value of A, which is not limited in the embodiments of the present application.

[0137] The highlight area may be an area with a brightness greater than the first threshold, and the medium and low brightness area may be an area with a brightness less than the second threshold, and the first threshold is greater than the second threshold. For example, the threshold of the first threshold may be 220, and the value of the second threshold may be 190, etc., which is not limited in the embodiments of the present application.

[0138] It is understandable that the electronic device can reduce the brightness shear between the target image and the target preview image by fusing the highlight area in the target photographed image and the medium and low brightness areas in the first image. In addition, since the high dynamic range in the first image replaces the overexposed area in the target preview image, there may be no overexposed area in the target image, which has better image quality.

[0139] In a possible implementation, the camera algorithm library can also obtain the target image by fusing the highlight area in the target photographed image whose quality meets the requirements, and the medium and low brightness areas in the first image whose quality meets the requirements. Among them, the image quality meeting the requirements can be understood as the area being a clear area. In order to ensure the fusion of the clear area, the embodiment of the present application can also remove the fuzzy area, that is, only the clear highlight area and the clear medium and low brightness area are fused. The embodiment of the present application does not specifically limit other requirements for image quality.

[0140] S412: The camera obtains a target image from a camera algorithm library.

[0141] The process of the camera acquiring the target image from the camera algorithm library can be referred to the description in S405, which will not be repeated here.

[0142] S413: The camera calls the display to display the target image.

[0143] For example, the camera may display Figure 7 The interface shown, Figure 7 This is a schematic diagram of an interface for displaying a target image provided by an embodiment of the present application. Figure 7 As shown, the interface may include: a high dynamic area 701 obtained by processing the overexposed area, and a medium and low brightness area 702 that is consistent with the brightness of the target preview image.

[0144] After S413 , the camera may save the target image into a gallery, and display the target image in response to a user operation of opening the gallery.

[0145] In a possible implementation, the camera may also save the target photographed image before brightness migration into a gallery to achieve storage of high-dynamic images, so as to facilitate subsequent use of the target photographed image to perform other image algorithm processing steps.

[0146] Thus, in electronic devices Figure 1 The interface shown in c in the figure switches to Figure 8 There will not be excessive brightness jumps during the interface shown, which improves the user experience of viewing photographed images.

[0147] Based on this, an embodiment of the present application provides an image processing method, which can use the brightness of the target preview image to constrain the brightness of the target photographed image, so that the brightness of the target image after the brightness constraint is consistent with the brightness of the target preview image, reducing the brightness jump when the electronic device switches from the target preview image to the target image, and improving the user experience.

[0148] It is understandable that in the embodiments of the present application Figure 4 The order of the steps in the process is not specifically limited.

[0149] In order to explain this application more clearly, Figure 8 A flowchart of another image processing method is provided for the embodiment of the present application. Figure 8 As shown, the electronic device can obtain a target preview image and a target photographed image, and perform tonemapping processing on the target photographed image and the target preview image under different algorithms to obtain a grayscale image H corresponding to the target photographed image and a grayscale image P corresponding to the target preview image.

[0150] The electronic device calculates the mean and standard deviation of the grayscale image H, and calculates the mean and standard deviation of the grayscale image P, and performs upsampling and brightness migration on the images after the mean and standard deviation are calculated to obtain a first image H'.

[0151] Furthermore, the electronic device may fuse the grayscale image H and the first image H' to obtain a target image.

[0152] The specific process of calculating the mean and standard deviation, upsampling, and image fusion can be found in the description of S410-S411, which will not be repeated here.

[0153] Combined with the above Figure 4-Figure 8 , the method provided in the embodiment of the present application is described, and the device for executing the above method provided in the embodiment of the present application is described below. Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device may be an electronic device in an embodiment of the present application, or a chip or chip system in an electronic device.

[0154] like Fig. 9 As shown, the image processing device 900 can be used in a communication device, a circuit, a hardware component or a chip, and the image processing device 900 includes: a display unit 901 and a processing unit 902. The display unit 901 is used to support the display step performed by the display method; the processing unit 902 is used to support the image processing device 900 to perform the information processing step.

[0155] In a possible implementation, the image processing apparatus 900 may further include a communication unit 903, and the communication unit 903 is used to support the image processing apparatus 900 to perform steps such as receiving or sending a message.

[0156] The image processing devices described in the embodiments of the present application may include Fig. 9 The units described in the corresponding embodiments.

[0157] Specifically, the processing unit 902 and the display unit 901 may be integrated together, and the processing unit 902 and the display unit 901 may communicate with each other.

[0158] In a possible implementation, the image processing apparatus 900 may further include: a storage unit 904. The storage unit 904 may include one or more memories, and the memory may be a device in one or more devices or circuits for storing programs or data.

[0159] The storage unit 904 may exist independently and be connected to the processing unit 902 via a communication bus. The storage unit 904 may also be integrated with the processing unit 902.

[0160] Taking the image processing device 900 as an example, which can be a chip or chip system of an electronic device in an embodiment of the present application, the storage unit 904 can store computer-executable instructions of the method of the electronic device, so that the processing unit 902 executes the method of the electronic device in the above embodiment. The storage unit 904 can be a register, a cache, or a random access memory (RAM), etc. The storage unit 904 can be integrated with the processing unit 902. The storage unit 904 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 904 can be independent of the processing unit 902.

[0161] In a possible implementation, the image processing device 900 may further include: a communication unit 903. The communication unit 903 is used to support the image processing device 900 to interact with other devices. Exemplarily, when the image processing device 900 is an electronic device, the communication unit 903 may be a communication interface or an interface circuit. When the image processing device 900 is a chip or a chip system in an electronic device, the communication unit 903 may be a communication interface. For example, the communication interface may be an input / output interface, a pin or a circuit, etc.

[0162] The device of this embodiment can be used to execute the steps executed in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0163] Fig.10 A schematic diagram of the hardware structure of another electronic device provided in an embodiment of the present application.

[0164] The electronic device includes a processor 1001, a communication circuit 1004 and at least one communication interface ( Fig.10 The communication interface 1003 is used as an example for explanation).

[0165] The processor 1001 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0166] Communications link 1004 may include circuitry to transmit information between the above-described components.

[0167] The communication interface 1003 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.

[0168] Possibly, the electronic device may further include a memory 1002 .

[0169] The memory 1002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a communication line 1004. The memory may also be integrated with the processor.

[0170] The memory 1002 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the computer-executable instructions stored in the memory 1002, thereby implementing the method provided by the embodiment of the present application.

[0171] Possibly, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, and the embodiments of the present application do not specifically limit this.

[0172] In a specific implementation, as an embodiment, the processor 1001 may include one or more CPUs, such as Fig.10 CPU0 and CPU1 in.

[0173] In a specific implementation, as an embodiment, the electronic device may include multiple processors, such as Fig.10 1 and 1005. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0174] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0175] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium can include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (digital versatiledisc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0176] The embodiments of the present application also provide a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0177] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while optical discs reproduce data optically using lasers.

[0178] The above combinations should also be included in the scope of computer-readable media. The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

Claims

1. An image processing method, It is characterized in that The method comprises: In response to a photographing operation, displaying a thumbnail, wherein the thumbnail is obtained by processing the preview image; In response to an operation on the thumbnail, displaying the preview image and the target image in sequence; After responding to the operation on the thumbnail, the method further includes: Performing brightness migration processing on the photographed image based on the brightness of the preview image to obtain a first image; The target image is obtained by fusing the medium and low brightness areas in the first image and the highlight area in the photographed image, wherein the highlight area is an area with brightness greater than a first threshold, and the medium and low brightness areas are areas with brightness less than a second threshold, and the first threshold is greater than the second threshold.

2. The method according to claim 1, It is characterized in that The fusing of the medium and low brightness areas in the first image and the highlight area in the photographed image to obtain the target image includes: The target image is obtained by fusing the medium and low brightness areas of the first image whose quality meets the requirements and the high brightness areas of the photographed image whose quality meets the requirements.

3. The method according to claim 1 or 2, It is characterized in that The step of performing brightness migration processing on the photographed image based on the brightness of the preview image to obtain a first image includes: Divide the preview image and the photographed image into images containing N*N ranges respectively, calculate the mean and standard deviation in any range, and obtain a first mean corresponding to the preview image, a first standard deviation corresponding to the preview image, a second mean corresponding to the photographed image, and a second standard deviation corresponding to the photographed image; Upsampling the preview image divided into N*N ranges and the photographed image divided into N*N ranges respectively to obtain a preview image at a preset resolution and a photographed image at a preset resolution; The first image is obtained by performing brightness migration processing on the photographed image at the preset resolution using the first mean, the first standard deviation, the second mean, and the second standard deviation.

4. The method according to claim 3, It is characterized in that The first mean, the first standard deviation, the second mean, and the second standard deviation may satisfy the following formula: Among them, H ′ (x, y) is the grayscale value of the pixel in the first image, H(x, y) is the grayscale value of the pixel in the photographed image at the preset resolution, meanH(x, y) is the mean corresponding to the pixel in the first mean, meanP(x, y) is the mean corresponding to the pixel in the second mean, stdH(x, y) is the standard deviation corresponding to the pixel in the first standard deviation, and stdP(x, y) is the standard deviation corresponding to the pixel in the second standard deviation.

5. The method according to claim 1, It is characterized in that The preview image includes an overexposed area, the photographed image or the target image does not include the overexposed area, the photographed image or the target image both include the highlight area corresponding to the overexposed area, and the photographed image is obtained by image fusion based on at least two images with different exposure times.

6. The method according to claim 1, It is characterized in that The method further comprises: Saving the target image in a gallery application; In response to an operation of opening a gallery application, the target image is displayed.

7. The method according to claim 1, It is characterized in that The grayscale value of the first threshold may be 220, and the grayscale value of the second threshold may be 190.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the computer is caused to perform the method according to any one of claims 1 to 7.

10. A computer program product, It is characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 7.

11. A chip, It is characterized in that include: A processor is used to read instructions stored in a storage memory, and when the processor executes the instructions, the chip implements the method described in any one of claims 1 to 7.

Images