Image processing method and related device

Abstract

The embodiment of the invention provides an image processing method and a related device, and relates to the technical field of terminals. The method comprises the following steps: acquiring an image; detecting the quality of the image to obtain a first detection result; under the condition that the image comprises the target object, feature extraction is conducted on the target object in the image, N first images are obtained, any first image comprises a preset feature of the target object, and N is an integer larger than 0; detecting the quality of the N first images to obtain N second detection results; and for the target image, the target image is an image in the N first images, under the condition that the image quality indicated by the second detection result corresponding to the target image is lower than the image quality indicated by the first detection result, image enhancement processing is performed on a target region in the image, and the target region is a region corresponding to the target image. Therefore, the image quality is improved or the brightness is adjusted for all or part of the features of the target object, the calculation amount is small, the time consumption of image processing is shortened, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method and related devices. Background Art

[0002] Mobile phones and other electronic devices have camera functions to facilitate users to record their lives, etc. Many factors may affect the quality of images taken by cameras in electronic devices, such as scene lighting conditions, human motion, camera parameters, and camera position when shooting.

[0003] Currently, electronic devices can use underlying visual methods to denoise, deblur, and brighten dark areas of images to improve image quality.

[0004] However, there may be cases where the quality of the processed image is poor. Summary of the invention

[0005] The embodiment of the present application provides an image processing method and related devices, which are applied to the field of terminal technology. Feature extraction is performed on an image through one or more preset features of a target object to obtain one or more images related to the target object. The quality of the one or more images and the quality of the image as a whole are detected. When the quality of the image related to the target object is less than the quality of the image as a whole, the area related to the target object in the image is optimized to improve the quality of the target object, thereby improving the quality of the image.

[0006] In addition, the area related to the target object in the image is optimized to improve the user experience. In addition, the area related to the target object in the image is optimized in a targeted manner, and some areas may not be optimized, which reduces the amount of calculation, shortens the image processing time, increases the image optimization rate, and improves the user experience.

[0007] In the first aspect, the embodiment of the present application proposes an image processing method. The method includes: acquiring an image; detecting the quality of the image to obtain a first detection result; when the image includes a target object, extracting features of the target object in the image to obtain N first images, any of which includes a preset feature of the target object, where N is an integer greater than 0; detecting the quality of the N first images to obtain N second detection results; for any of the N second detection results, when the quality of the target image indicated by any of the second detection results is lower than the quality of the image indicated by the first detection result, performing image enhancement processing on the target area in the image, the target image being the first image corresponding to any of the second detection results in the N first images, and the target area being the area corresponding to the target image.

[0008] In this way, the image quality of a partial area of ​​the target object is judged. When the image quality of a partial area of ​​the target object is low, image enhancement processing is performed on the area to improve the clarity or texture details of the area. When the image quality of a partial area of ​​the target object is high, image enhancement processing is not performed on the area, reducing the processing of the image, improving the processing speed of the entire image, and shortening the processing time of the image.

[0009] In one possible implementation, the first detection result includes: a first score; the image quality indicated by the first score is positively correlated with the first score; the second detection result includes: a second score; the image quality indicated by the second score is positively correlated with the second score; for any second detection result among the N second detection results, when the quality of the target image indicated by any second detection result is lower than the quality of the image indicated by the first detection result, image enhancement processing is performed on the target area in the image, including: for any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, image enhancement processing is performed on the target area in the image, and the target image is a first image corresponding to any second score among the N first images.

[0010] In this way, by quantifying the detection results into scores, it is convenient to compare the image qualities indicated by the detection results later.

[0011] In one possible implementation, the value range of the first score is the same as the value range of the second score; for any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, image enhancement processing is performed on the target area in the image, including: for any second score among the N second scores, when any second score is less than the first score, image enhancement processing is performed on the target area in the image.

[0012] In this way, the value range is the same, which facilitates subsequent score comparison and calculation and other subsequent processing.

[0013] In a possible implementation, detecting the quality of N first images to obtain N second detection results includes: detecting the quality of the N first images by using a pre-trained image quality assessment network to obtain N second detection results.

[0014] In this way, the image quality detection of the first image is realized through the image quality evaluation network, which is convenient, fast and easy to implement.

[0015] In a possible implementation, detecting the quality of N first images to obtain N second detection results includes: performing expansion processing on the N first images to obtain N expanded first images, wherein the sizes of the N expanded first images are all greater than or equal to a preset size; and detecting the quality of each expanded first image through a pre-trained image quality evaluation network to obtain N second detection results.

[0016] The preset size may be any size, for example, 224 mm*224 mm, 24 cm*24 cm, etc. No specific limitation is made here.

[0017] In this way, before quality detection is performed on the first image, expansion processing is performed so that the expanded first image can meet the size requirement of image quality detection, and then the image quality evaluation network can detect the first image.

[0018] In one possible implementation, when the image includes a target object, a brightness detection result of the target object is also obtained; the method also includes: when the image quality indicated by the brightness detection result is lower than the image quality of the image indicated by the first detection result, adjusting the brightness of the area corresponding to the target object in the image.

[0019] In this way, the brightness of the target object is detected, and when the image quality indicated by the brightness of the target object is low, the brightness of the area is adjusted to improve the brightness effect of the target object and the image quality. When the image quality indicated by the brightness of the target object is high, the brightness of the area is not adjusted, the image processing is reduced, the processing speed of the entire image is improved, and the image processing time is shortened.

[0020] In a possible implementation, before the brightness adjustment, the method further includes: when the image includes the target object, extracting features of the target object in the image by using an image segmentation algorithm to obtain a second image; and detecting the brightness of the second image to obtain a brightness detection result.

[0021] In this way, the target object is segmented from the image, making it easier to perform brightness detection on the target image.

[0022] In a possible implementation, extracting features of a target object in an image by using an image segmentation algorithm to obtain a second image includes: extracting features of a target object in an image by using an image segmentation algorithm and taking a margin expansion approach to obtain a second image.

[0023] In this way, when segmenting an image, the boundary filling method is used to retain more information of the target object, improve the integrity of the target object, and improve the detection effect of the target object.

[0024] In one possible implementation, the brightness detection result is related to the brightness mean, which is the average value of the brightness of each pixel in the second image; the image quality indicated by the brightness detection result is positively correlated with the first value, which is the absolute value of the difference between the brightness mean and the first brightness threshold.

[0025] In this way, by performing brightness detection through the average brightness, the overall brightness of the target object can be better evaluated and the accuracy of the brightness detection result can be improved.

[0026] In a possible implementation, the first brightness threshold is 127, and the brightness detection result includes: a third score, and the image quality indicated by the third score is positively correlated with the third score; In the case of Positive correlation: In the case of Positive correlation; is the mean brightness.

[0027] In this way, the closer the average brightness value is to the first brightness threshold, the higher the image quality indicated by the brightness detection result.

[0028] In a possible implementation, the brightness detection result also includes: a first deviation degree and a second deviation degree, the first deviation degree is the deviation degree between the brightness mean and the second brightness threshold, the second deviation degree is used to indicate the deviation degree between the brightness of all pixels corresponding to the target object and the brightness mean, and the brightness mean is the mean of the brightness of all pixels corresponding to the target object; when the image quality indicated by the brightness detection result is lower than the image quality indicated by the first detection result, the brightness of the area corresponding to the second image in the image is adjusted, including: when the ratio of the first deviation degree to the second deviation degree is less than the first threshold, and the third score is less than the first score, the target object is brightened; the absolute values ​​corresponding to the first threshold and the second threshold are the same, and the first threshold is less than the second threshold; when the ratio of the first deviation degree to the second deviation degree is greater than the second threshold, the target object is reduced in brightness.

[0029] The first threshold value may be 1 or any value; the second threshold value may be -1 or any value, which is not specifically limited here.

[0030] In this way, the underexposure or overexposure of the target object can be judged through the two deviation degrees, thereby achieving brightening adjustment or lowering brightness adjustment of the target object, thereby improving the brightness adjustment effect.

[0031] In a possible implementation, the second brightness threshold is 128, and the first deviation degree satisfies: S is the number of pixels in the second image, xi is the brightness value corresponding to each pixel; the second deviation degree satisfies: Among them, Hist[j] is the ordinate corresponding to each grayscale value in the histogram corresponding to all pixels of the second image, and j is the grayscale value.

[0032] In this way, the method is simple and easy to implement.

[0033] In a possible implementation, before obtaining the first detection result, the method further includes: displaying an interface of a gallery application, the interface displaying an image and a first control; acquiring the image, including: acquiring the image in response to a trigger operation on the first control.

[0034] The first control may correspond to the image optimization 104 and the image optimization 206 mentioned above. Figure 1 The interface shown in c or Figure 2 In some embodiments, the electronic device triggers display of the first control when detecting that "more controls" are triggered. No specific limitation is made here.

[0035] In this way, image quality detection can be triggered in the gallery application, or image optimization can be performed based on the image quality detection.

[0036] In a possible implementation manner, the method further includes: performing weighted processing on the first detection result, the N second detection results, and the brightness detection result to obtain a detection result of the image; and displaying the detection result of the image.

[0037] In the embodiment of the present application, weighted processing of the first detection result and the N second detection results can be achieved in any manner, which is not specifically limited here.

[0038] The electronic device can display the detection result of the image to prompt and / or guide the user. The displayed detection result is used to indicate the quality of the image. The detection result can be a score corresponding to the image (for example, 7.58, etc.), or a grade corresponding to the image (for example, good, etc.), or any form of detection result. In some embodiments, the detection result can also include: prompt information. The prompt information is used to guide the user to take pictures. For example, the prompt information can guide the user to adjust the light for taking pictures, the light and shadow of the face, the choice of background, and the composition of the picture.

[0039] In this way, the quality of the area corresponding to the target object and the quality of the entire image can be weighted, thereby increasing the impact of the target object on the quality and the attention paid to the target object, so that the detection results after weighted processing meet user expectations.

[0040] In a second aspect, an embodiment of the present application provides an image processing device, which may be an electronic device, or a chip or chip system in an electronic device. The image processing device may include a display unit and a processing unit. When the image processing device is an electronic device, the display unit may be a display screen. The display unit is used to perform the display step so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing device is an electronic device, the processing unit may be a processor. The image processing device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit within the chip (eg, a register, a cache, etc.), or a storage unit within the electronic device that is located outside the chip (eg, a read-only memory, a random access memory, etc.).

[0041] Exemplarily, a display unit is used to display an image.

[0042] The processing unit is used to detect the quality of the image to obtain a first detection result; the processing unit is also used to, when the image includes the target object, perform feature extraction on the target object in the image to obtain N first images, any one of the first images includes a preset feature of the target object, and N is an integer greater than 0; the processing unit is also used to detect the quality of the N first images to obtain N second detection results; the processing unit is also used to, for the target image, when the image quality indicated by the second detection result corresponding to the target image is lower than the quality of the image indicated by the first detection result, perform image enhancement processing on a target area in the image, the target image is an image among the N first images, and the target area is an area corresponding to the target image.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0045] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0046] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0047] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0048] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of an interface of an electronic device provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of an interface of another electronic device provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0053] Figure 5 A flowchart of an image processing method provided in an embodiment of the present application;

[0054] Figure 6A schematic diagram of a brightness adjustment process provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of an image processing process provided in an embodiment of the present application;

[0056] Figure 8 A flowchart of an image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0058] 1. Grayscale: refers to the number of color levels represented by each pixel in the image. Grayscale can also be called gray value, grayscale value, etc.

[0059] In the embodiment of the present application, when the color level corresponding to the image is 8 bits, the grayscale value may range from 0 to 255. Among them, the value 0 is black and the value 255 is white. When the color level corresponding to the image is 8 bits, the grayscale value may range from 0 to 65535. The embodiment of the present application does not specifically limit the range of the grayscale level.

[0060] 2. Pixel: refers to the smallest image unit. Each image consists of multiple pixels. For example, taking an image of 500 pixels * 338 pixels as an example, the image is composed of a 500 * 338 pixel matrix. The length of the image is 500 pixels and the width is 338 pixels.

[0061] 3. Other terms

[0062] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0063] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0064] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0065] 4. Electronic equipment

[0066] The electronic devices of the embodiments of the present application may include handheld devices, vehicle-mounted devices, etc. with image processing functions. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MID), wearable devices (e.g., smart watches, smart glasses, smart bracelets or smart jewelry, etc.), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, terminal devices in Internet of Things (IoT) systems, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), and wireless terminals in smart grids. The present invention relates to a user interface, a user interface, a user digital assistant, a PDA, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a terminal device in a 5G network, or a terminal device in a public land mobile communication network (PLMN) to be evolved in the future, etc. The embodiments of the present application are not limited to this.

[0067] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0068] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0069] Mobile phones and other electronic devices have camera functions to facilitate users to record their lives, etc. Many factors may affect the quality of images taken by cameras in electronic devices, such as scene lighting conditions, human motion, camera parameters, and camera position when shooting.

[0070] Currently, electronic devices can use underlying visual methods to denoise, deblur, and brighten dark areas of the entire image to improve image quality.

[0071] However, the user may pay more attention to the target object in the image. When the quality of the target object in the processed image is poor, the user's satisfaction with the processed image is low and the user experience is poor.

[0072] For example, taking an image containing a face as an example, the quality of the image content other than the face area in the processed image is good. When the brightness of the processed face area is low or the details of the face are not clear, the user thinks that the quality of the processed image is poor, the user is less satisfied with the processed image, and the user experience is poor.

[0073] In addition, the entire image is enhanced, which requires a lot of calculations and may make the image processing time-consuming and result in a poor user experience.

[0074] In view of this, the embodiments of the present application provide an image processing method and a related device. By comparing the quality of the entire image with the quality of the region of the target object in the image, when the quality of part or all of the region of the target object is poor, the image is optimized for the part or all of the region. In this way, the image is optimized for the target object, the effect of the target object in the image is improved, and the optimization effect of the image is improved. In addition, the region related to the target object in the image is optimized, and some regions may not be optimized, thereby improving the optimization rate of the image and shortening the time consumed for image optimization processing.

[0075] In the embodiment of the present application, the target object may be a living being such as a person or an animal, or a non-living being such as a vehicle, or a part of a living being (e.g., a face, eyes, eyebrows, nose, etc.) or a part of a non-living being. No specific limitation is made here.

[0076] For example, taking the target object as a human face, Figure 1 A schematic diagram of an interface of an electronic device provided in an embodiment of the present application.

[0077] When electronic devices are Figure 1 In the main interface shown in a of FIG. 1 , when a trigger operation (eg, a click operation) for the gallery application 101 is received, the electronic device can enter the Figure 1 The gallery interface shown in b in FIG. The gallery interface may include at least one thumbnail. The at least one thumbnail may be a thumbnail of an image or a thumbnail of a video, which is not specifically limited here.

[0078] When the electronic device receives a trigger operation (eg, a click operation) for the image thumbnail 102, the electronic device enters a state such as Figure 1 The image interface shown in c in FIG. The image interface includes: an image 103 corresponding to the thumbnail 102 and setting items. The setting items include but are not limited to: sharing, collection, editing, deletion, image optimization 104, more or other types of editing options.

[0079] When the electronic device receives a trigger operation (eg, a click operation) for image optimization 104, the electronic device optimizes the image 104 using the image processing method provided in the embodiment of the present application, and enters Figure 1 The interface shown in d. Figure 1 The interface optimized image 105 is shown in d.

[0080] In some embodiments, Figure 1 The image interface shown in c in FIG. 1 may not display “image optimization 104 ”. When the electronic device receives a click operation for “more”, it displays “image optimization 104 ”. No specific limitation is made here.

[0081] The "image optimization 104" may be located at the upper right corner of the image, or at the lower right corner of the image, or at any other position. The embodiment of the present application does not limit the specific position of the "image optimization 104".

[0082] For example, Figure 2 A schematic diagram of an interface of another electronic device provided in an embodiment of the present application.

[0083] When electronic devices are Figure 2 In the main interface shown in a of FIG. 1 , when receiving the operation of the user opening the camera application 201, the electronic device can enter the Figure 2 The interface shown in b in FIG. The interface may include: a preview area 202, a photo control 203, a thumbnail area 204, and a shooting mode selection item. The preview area 202 displays a preview image in real time. The photo control 203 is used to trigger the camera to take a photo. The thumbnail area 204 displays a thumbnail of the image taken by the camera. The shooting mode selection items include, but are not limited to: portrait, photo, video, professional or more, which are not specifically limited here.

[0084] When electronic devices are Figure 2 When the interface shown in b detects a trigger operation (eg, a click operation) for the photo control 203, the electronic device captures an image and enters Figure 2 The interface shown in c in FIG. The interface includes: a preview area 202, a photo control 203, and a thumbnail area 204. The thumbnail area 204 displays a thumbnail of the captured image 205.

[0085] When electronic devices are Figure 2 When the interface shown in c detects a trigger operation on the thumbnail area 204, the electronic device enters Figure 2 The interface shown in d in FIG. 1 displays image 205 .

[0086] When electronic devices are Figure 2 When the interface shown in d detects a trigger operation for image 205, the electronic device enters Figure 2 The interface shown in e in FIG. 1 shows an image 205 and setting items. The setting items include but are not limited to: share, collect, edit, delete, image optimization 206, more or other types of editing options.

[0087] When the electronic device receives a trigger operation (eg, a click operation) for image optimization 206, the electronic device optimizes the image 205 using the image processing method provided in the embodiment of the present application and displays the optimized image. Figure 1 The interface shown in f.

[0088] In some embodiments, Figure 2 The image interface shown in e in FIG. 2 may not display “image optimization 206 ”. When the electronic device receives a click operation for “more”, it displays “image optimization 206 ”. This is not specifically limited here.

[0089] The "image optimization 206" may be located at the upper right corner of the image, or at the lower right corner of the image, or at any other position. The embodiment of the present application does not limit the specific position of the "image optimization 206".

[0090] It is understandable that the above Figure 1 and Figure 2 In the scenes shown, image optimization is achieved through user operation. In other embodiments, the electronic device can also optimize the image by the image processing method provided by the embodiment of the present application when it is in an idle scene. The electronic device can save the optimized image separately in the gallery, or replace the image before optimization in the gallery with the optimized image.

[0091] It is understandable that the electronic device can confirm whether it is in an idle scene through a preset time period or any method (for example, load usage, power usage, etc.). The preset time period can be from 24:00 to 6:00 or any time period, which is not specifically limited here. Exemplarily, when the electronic device detects that it is in the screen-off state for a preset time, it confirms that the electronic device is in an idle scene. Alternatively, when the electronic device detects a preset time period and the electronic device is connected to the charger, it confirms that the electronic device is in an idle scene. The preset duration can be 30 minutes or 10 minutes, which is not specifically limited here. The screen-off state can be understood as the display screen being powered off. Power off can be understood as the current of the display screen being less than the preset current value.

[0092] In order to better understand the embodiments of the present application, the structure of the electronic device of the embodiments of the present application is introduced below. Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0093] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0094] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0095] The processor 110 may include one or more processing units. Among them, different processing units may be independent devices or integrated into one or more processors. A memory may also be provided in the processor 110 for storing instructions and data. For example, the processor 110 is used to store instructions and data involved in the image processing method provided in the embodiment of the present application.

[0096] For example, in the embodiment of the present application, after the electronic device can detect the trigger operation through the touch sensor 180K, the display screen of the electronic device can realize Figure 1 to Figure 2 When the electronic device can detect a trigger operation for “image optimization” through the touch sensor 180K, the processor 110 can optimize the image through the image processing method provided in the embodiment of the present application and save the optimized image.

[0097] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0098] For example, Figure 4The software structure diagram of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom, including the application (APP) layer, the application framework (framework) layer, the Android runtime (Android runtime) and the system library, and the kernel layer (kernel), etc., which is not limited in the embodiments of the present application.

[0099] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0100] For example, Figure 4 The software structure diagram of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown,

[0101] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0102] The application layer can include a series of application packages. Figure 4 As shown, the application package can include camera, gallery, video, SMS, image processing plug-ins, etc.

[0103] The image processing plug-in may include: an image quality detection module, an image optimization module, etc. Among them, the image quality detection module can detect the image quality, and perform quality detection on one or more regions of the target object in the image to obtain the detection result of the image and the detection result of one or more regions of the target object. The image optimization module can optimize the image according to the multiple detection results output by the image quality detection module.

[0104] For example, Figure 4 As shown, the image quality detection module may include: feature extraction, image quality assessment (IQA) network module, etc.

[0105] Taking image A as an example, the feature extraction module is used to extract features from image A to obtain image B corresponding to the target object, and image B includes at least one preset feature of the target object. The image quality assessment network module is used to detect the overall quality of image A and output detection result A. The image quality assessment network module is also used to detect the image quality of image B and output detection result B. The subsequent image optimization module can confirm whether to perform image enhancement processing on the area in image A corresponding to image B based on detection results A and B.

[0106] In some embodiments, the image quality detection module may further include: a brightness detection module (not shown in the figure). The feature extraction module may perform feature extraction on image A to obtain image C corresponding to the target object. The brightness detection module may obtain a brightness detection result corresponding to image C. The subsequent image optimization module may determine whether to adjust the brightness of the target object in image A based on the detection result A and the brightness detection result.

[0107] In some embodiments, the image quality detection module may further include: a weighting module (not shown in the figure). The weighting module is used to perform weighted processing on the detection result A, the detection result B and the brightness detection result to obtain the final detection result of the image A. In this way, considering the impact of the image quality of the target object on the overall quality of the image A, and considering the impact of the brightness of the target object on the overall quality of the image A, the overall quality of the image A is adjusted based on the image quality of the target object and the brightness of the target object, so that the final detection result is consistent with the image quality expected by the user.

[0108] The image optimization module is used to optimize the image according to the detection results of the image (for example, detection result A, detection result B, brightness detection result). Exemplarily, taking the above-mentioned image A as an example, when the quality of the image indicated by the detection result B is less than the quality of the image indicated by the detection result A, the image enhancement processing can be performed on the area corresponding to the image B in the image A. When the quality of the image indicated by the brightness detection result is less than the quality of the image indicated by the detection result A, the brightness adjustment can be performed on the area corresponding to the image C.

[0109] In the embodiment of the present application, the image optimization module may include one or more modules provided in the embodiment of the present application: a brightness adjustment module, an image enhancement module, etc. The brightness adjustment module may be used to brighten the image or reduce the brightness of the image. The image enhancement module may perform image enhancement processing on the image.

[0110] In the embodiment of the present application, image enhancement processing is used to improve image quality (for example, improve clarity), enrich information (for example, improve texture details, etc.), etc. Image enhancement processing can be implemented by frequency domain method, space domain method, etc.

[0111] The frequency domain method refers to the signal enhancement of the image based on the two-dimensional Fourier transform. Specifically, the low-pass filtering method (that is, only allowing low-frequency signals to pass) can remove the noise in the image; the high-pass filtering method can enhance high-frequency signals such as edges to make the blurred image clear.

[0112] The spatial domain method can be used to remove or reduce noise. Exemplarily, the spatial domain method includes: local averaging method and median filtering (taking the middle pixel value in the local neighborhood) method, etc. No specific limitation is made here.

[0113] The structure of the above-mentioned image processing plug-in is only an example, and the image processing plug-in may also include more or fewer modules, which is not specifically limited here.

[0114] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0115] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0116] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0117] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0118] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0119] The kernel layer is a layer between hardware and software. The kernel layer is used to drive the hardware to make the hardware work. The kernel layer at least includes display driver, audio driver, camera driver, sensor driver, and graphics processing unit (GPU) driver, etc., which are not limited in the embodiments of the present application.

[0120] The following is an illustrative description of the workflow of electronic device software and hardware in conjunction with the interface of the electronic device.

[0121] When the touch sensor in the electronic device receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into raw input events (including touch coordinates, touch force, timestamp of touch operation, etc.). The raw input events are stored in the kernel layer. The application framework layer obtains the raw input events from the kernel layer and identifies the control corresponding to the input event.

[0122] Taking the touch operation as a touch single-click operation, and the control corresponding to the single-click operation as a control for indicating image optimization, the gallery application calls the image processing plug-in, and detects the quality of the image through the image quality detection module to obtain the detection result, and performs image enhancement processing and / or brightness adjustment processing according to the detection result through the image optimization module to obtain the optimized image. The gallery application calls the view system of the application framework layer, and calls the image rendering library in the system library to draw and render the image, so as to display the interface of the gallery application on the screen (display), and the interface displays the optimized image.

[0123] In some embodiments, the electronic device may also call the image processing plug-in to optimize the image in the gallery application when it detects that the electronic device is in an idle scene (for example, the load is small). The specific optimization process can refer to the optimization process corresponding to the above touch operation, and is not specifically limited here. The judgment of the idle scene can refer to the above corresponding description, which will not be repeated here.

[0124] Combine the following Figures 5 to 8 The image processing method provided in the embodiment of the present application is described.

[0125] For example, Figure 5 Schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. Assume that the target object is a human face and image A is an image to be processed. Figure 5 As shown, the method includes:

[0126] S501: Acquire image A.

[0127] Image A may be any image, for example, an image saved in a gallery application or an image captured by a camera.

[0128] In some embodiments, the electronic device responds to the photo-taking operation, and a camera in the electronic device captures an image. The electronic device processes the image captured by the camera as image A, and executes S502-S518.

[0129] In some other embodiments, when the electronic device detects that it is in an idle scene, the electronic device obtains image A from the gallery application for image processing and executes S502 to S518. Image A is any image in the gallery application that has not been image processed.

[0130] In some other embodiments, in response to a trigger operation for “image optimization”, the electronic device processes the image displayed on the display screen as image A and executes S502-S518. Figure 1 Taking the interface shown in d in FIG. 1 as an example, when the electronic device detects a trigger operation for “image optimization 104”, it processes image 103 as image A and executes S502-S518.

[0131] For example, Figure 4 Taking the software structure shown as an example, the image processing plug-in in the application layer obtains image A. Exemplarily, after the camera in the electronic device captures the image, the image processing plug-in performs image processing on the image captured by the camera, and executes S502-S518; when the electronic device detects that it is in an idle scene, the image processing plug-in can obtain image A from the gallery application for image processing, and execute S502-S518; when the electronic device responds to the trigger operation for "image optimization", the image processing plug-in performs image processing on the image displayed on the display screen, and executes S502-S518.

[0132] S502: Input image A into IQA network a to obtain a basic image quality score.

[0133] IQA network a is used to perform image quality detection on the entire image A and evaluate the overall quality (image distortion level) of image A. The basic image quality score is used to indicate the overall quality of image A.

[0134] In the embodiment of the present application, the IQA network a can be: a blindly assess image quality in the wild guided by a self-adaptive hypernetwork (hyperIQA) network or any type of image quality assessment network. Any type of image quality assessment network can be an attention-based hybrid image quality assessment network (AHIQ), a convolutional neural network (CNN), etc., which is not specifically limited here.

[0135] For example, taking IQA network a as a hyperIQA network, IQA network a may include: ResNet50 network 51, hypernetwork 52, and quality prediction network 53. ResNet50 network 51 is used to extract semantic features; hypernetwork 52 is used to formulate quality perception rules based on semantic features; and quality prediction network 53 is used to predict image quality. Alternatively, it can be understood that ResNet50 network 51 is a ResNet50 network used for content understanding, hypernetwork 52 is used for perceptual rule learning, and quality prediction network 53 is used for image quality prediction.

[0136] Specifically, the hyperIQA network extracts semantic features from the image through the ResNet50 network 51, and inputs the semantic features into the hypernetwork 52 to adjust the weights of each parameter in the quality prediction network. The quality prediction network 53 determines the weights of each parameter and then evaluates the quality of the image.

[0137] In this way, the hyperIQA network can separate image quality prediction from image content understanding to simulate how humans perceive image quality and achieve image quality assessment of images input into the hyperIQA network.

[0138] In the embodiment of the present application, the IQA network a may be trained using images of various categories.

[0139] In this way, the accuracy of the IQA network a in evaluating the overall quality of image A can be improved, and the accuracy of the basic image quality score can be improved. In addition, the trained IQA network a has good generalization ability and can perform image quality detection on various categories of images.

[0140] In some embodiments, after the electronic device inputs image A into the IQA network a, the IQA network a outputs a value A, where the value A is a basic image quality score of image A.

[0141] In other embodiments, after the electronic device inputs image A into IQA network a, IQA network a outputs value A; the electronic device adjusts value A based on the maximum value that can be output by IQA network a to obtain value B, which is the basic image quality score of image A.

[0142] The value B satisfies: In this way, the basic image quality score of image A can be adjusted within a preset value range, which is convenient for subsequent comparison with the brightness score, detail score, and weighted calculation.

[0143] The preset value range may be 0 to 1, or 0 to 10, or any range, and the preset value range is not specifically limited here.

[0144] Exemplarily, taking the preset value range of 0 to 1 as an example, the value B may be the ratio of the value A to the maximum value that can be output by the IQA network a.

[0145] For example, taking the maximum value that can be output by IQA network a as 100 and the preset value range as 0 to 10, if value A is 80, then value B is 8.

[0146] It is understandable that the basic image quality score is a way of expressing the detection result, which is only an example, and the basic image quality score can also be replaced by other types of expressions, such as basic image quality rating (e.g., excellent, good, etc.). No specific limitation is made here. IQA network a is a way to obtain the basic image quality score, and the basic image quality score can also be obtained by any method, which is not specifically limited here.

[0147] For example, Figure 4 Taking the software structure shown in FIG. 1 as an example, the image quality assessment network module in the image processing plug-in includes: IQA network a. The image processing plug-in inputs image A into the IQA network a to obtain a basic image quality score.

[0148] S503: Perform facial key point detection on image A to confirm whether image A includes a face.

[0149] The key points of the face are the characteristic points of each part of the face, usually contour points and corner points. The key points of the face can include: 5 characteristic points, 68 characteristic points or 98 characteristic points of the face.

[0150] Facial key point detection refers to locating the key points of the face of an image, for example, locating the points of eyebrows, eyes, nose, mouth, and facial contour areas. When the electronic device detects the facial key points, it is confirmed that image A includes a face; when the electronic device does not detect the facial key points, it is confirmed that image A does not include a face.

[0151] For example, taking the facial key points corresponding to the 68 feature points of the face as an example, the electronic device can perform face detection on image A through the 68 feature points (68_face_landmarks) of the face. When the electronic device detects the 68 feature points, it is confirmed that image A includes a face; when the electronic device does not detect the 68 feature points, it is confirmed that image A does not include a face.

[0152] For example, Figure 4 Taking the software structure shown as an example, the feature extraction module in the image quality detection module detects key points of the face in image A to confirm whether image A includes a face.

[0153] If the electronic device does not detect a human face, the electronic device outputs the basic image quality score in S502. If the electronic device detects a human face, the electronic device executes S504-S518.

[0154] S504: When a face is detected, extract features of the face region in the image to obtain an image a corresponding to the face region.

[0155] The face region is the region corresponding to the face in image A.

[0156] In the embodiment of the present application, features can be extracted from the image by an image segmentation algorithm or any other method to obtain an image a corresponding to the face region. For example, the face region in image A can be segmented by an image segmentation algorithm implemented based on Pytorch (e.g., DeepLabv3 algorithm) to obtain an image a corresponding to the face region.

[0157] In some embodiments, a margin expansion strategy is used to segment the face region in image A. Margin expansion may include: expanding the edge of the face region outward. In this way, more face information can be retained, and the integrity of the face in image A can be improved.

[0158] For example, Figure 4 Taking the software structure shown as an example, the feature extraction module extracts features from the face area in image A to obtain image a corresponding to the face area.

[0159] S505 , perform brightness detection on image a to obtain a brightness score Sa, a brightness coefficient K, a deviation degree D1 and a deviation degree D2.

[0160] The brightness score is used to indicate the quality of image a, and can also be understood as the brightness score being used to indicate the quality of the face area, or the quality of the brightness indication of the face area, etc. The brightness coefficient K is used to indicate whether the brightness of image a needs to be adjusted. The degree of deviation D1 is used to indicate the degree of deviation between the overall brightness of image a and the preset brightness threshold, indicating the overall degree of deviation of the brightness of image a. The degree of deviation D2 is used to indicate the degree of deviation between the brightness corresponding to each pixel in image a and the overall brightness of image a, and can also be understood as the average degree of the weighted difference between each grayscale value and 128 and the overall deviation on the histogram corresponding to all pixels. The weight corresponding to each grayscale value can be the vertical coordinate of each grayscale value in the histogram.

[0161] The following describes how to calculate the brightness score Sa, the brightness coefficient K, the deviation degree D1, and the deviation degree D2.

[0162] In the embodiment of the present application, the brightness of image a can be detected by the brightness mean of image a, the median of the brightness values ​​of all pixels in image a, or any parameter related to brightness to obtain a brightness score. The brightness mean can reflect the average brightness of image a. Exemplarily, the brightness mean may include: the average value of the grayscale values ​​corresponding to the brightness of all pixels in image a.

[0163] In some embodiments, the brightness score Sa is obtained by using the brightness mean and the brightness threshold a. The brightness threshold a can be 127, 128, or any value, which is not specifically limited here. In this way, the brightness mean can reflect the average brightness of image a and the overall brightness of the face area. The brightness score obtained by the brightness mean can better reflect the quality of image a.

[0164] It is understandable that the brightness of image a can be represented by a grayscale value. The following takes the case where the brightness is represented by a grayscale value and the grayscale value corresponding to image A ranges from 0 to 255 as an example to illustrate the calculation of the brightness score.

[0165] In a possible implementation manner 1, the closer the brightness mean is to the brightness threshold a, the higher the brightness score is, and the higher the quality of the image indicated by the brightness score is.

[0166] For example, taking the brightness score value range as 0-10 and the brightness threshold a as 127, the brightness score Sa satisfies: in, is the mean brightness of image a.

[0167] In the second possible implementation, the closer the brightness mean is to the brightness threshold a, the lower the brightness score is, and the higher the quality of the image indicated by the brightness score is.

[0168] For example, taking the brightness score being in the range of 0-10 and the brightness threshold a including 0 and 255 as an example, the brightness score Sa satisfies: in, is the mean brightness of image a.

[0169] It is understandable that if the value range of the brightness score changes, the formula satisfied by the brightness score Sa may also change accordingly. No specific limitation is made here.

[0170] In some other embodiments, the median of the grayscale value can be obtained by counting the grayscale value of each pixel in the image a. The brightness score Sa is obtained by the median of the grayscale value and the brightness threshold a.

[0171] For example, taking the brightness score value range as 0-10 and the brightness threshold as 127, the brightness score Sa satisfies: Wherein, X is the median of the grayscale value of each pixel in image a.

[0172] In some embodiments, the value range of the brightness score Sa is the same as the preset value range. In this way, the value range of the brightness score is the same as the preset value range, which facilitates subsequent comparison with the detail score and the basic image quality score, as well as weighted calculation.

[0173] If the preset value range is changed, the formula satisfied by the brightness score Sa will be changed accordingly. The embodiment of the present application does not specifically limit the deformation of the formula.

[0174] In some embodiments, the overall brightness of image a can be calculated by the brightness mean. The degree of deviation D1 can be the degree of deviation between the brightness mean of image a and the brightness threshold b. The degree of deviation D2 can be the average of the difference between the grayscale value corresponding to all pixels in image a and the brightness threshold b and the degree of deviation D1 in the histogram. Or it can be understood that the degree of deviation D2 is the value obtained by weighted calculation of the difference between each grayscale value and 128 in the histogram of all pixels in image a and the difference between the grayscale value and the overall deviation. The corresponding weight of the difference between each grayscale value and 128 and the overall deviation is positively correlated with the vertical axis value corresponding to each grayscale value. The horizontal axis of the histogram is the grayscale value, and the vertical axis is positively correlated with the number of pixels corresponding to each grayscale value. In the histogram

[0175] The brightness threshold b can be 127, 128, or any value, and is not specifically limited here.

[0176] Exemplarily, taking the brightness threshold b as 128, D1 is the mean of the sum of the errors between the grayscale values ​​of all pixels in image a and 128. D2 represents the mean of the difference between the difference between all pixels and 128 and the overall deviation (deviation degree D1) on the overall histogram, or can be understood as the weighted average degree of the deviation between each grayscale value and 128 and the first deviation histogram. The overall histogram can be a histogram of all pixels in image a.

[0177] D1 meets the following requirements: Where S is the number of pixels, and xi is the grayscale value corresponding to each pixel. Then D2 satisfies: Hist[j] is the ordinate corresponding to each grayscale value in the histogram corresponding to all pixels in image a, j is the grayscale value corresponding to the histogram, and D1 is the deviation degree D1.

[0178] The brightness coefficient K satisfies:

[0179] In an embodiment of the present application, the brightness of image a can be detected by the brightness mean of image a, the median of the grayscale values ​​in image a, or any parameter related to brightness to obtain the brightness score Sa, brightness coefficient K, deviation degree D1 and deviation degree D2.

[0180] It is understandable that the brightness score Sa, the brightness coefficient K, the deviation degree D1 and the deviation degree D2 are all a way of expressing the test results, which are only examples, and the brightness score Sa, the brightness coefficient K, the deviation degree D1 and the deviation degree D2 can also be replaced by other types of expressions, such as brightness rating (e.g., excellent, good, etc.), etc. No specific limitation is made here.

[0181] S506: Determine whether the brightness score Sa, the brightness coefficient K, the deviation degree D1, the deviation degree D2 and the basic image quality score meet preset condition 1 or preset condition 2.

[0182] Preset condition 1 includes: k>1 and D1>0. Condition 2 includes: k>1, D1<0 and Sa <base。

[0183] For example, Figure 6 A schematic diagram of a brightness adjustment process provided by an embodiment of the present application. Figure 6 As shown,

[0184] After calculating the deviation degrees D1 and D2, the brightness coefficient can be obtained. When the brightness coefficient is less than or equal to 1, the brightness of the face area is judged to be normal, and no brightness adjustment is performed. When the brightness coefficient is greater than 1, the brightness of the face area is judged to be abnormal, and the brightness adjustment method is further determined according to the deviation degree D1. When the deviation degree D1 is greater than 0, the face area is judged to be overexposed, and the brightness of the face area is reduced; when the brightness score is less than the basic image quality score and the deviation degree D1 is less than 0, the face area is judged to be underexposed (insufficient exposure, insufficient brightness), and the brightness of the face area is increased.

[0185] When the preset condition 1 is met, S508 is executed. When the preset condition 2 is met, S508 is executed. When the preset condition 1 and the preset condition 2 are not met, no brightness adjustment is performed.

[0186] For example, Figure 4 Taking the software structure shown as an example, the image optimization module (or, brightness adjustment module) determines whether the brightness score Sa, brightness coefficient K, deviation degree D1, deviation degree D2 and basic image quality score meet preset condition 1 or preset condition 2.

[0187] S507: Adjust the brightness of the area in image A corresponding to image a.

[0188] Specifically, when preset condition 1 is met, the brightness of the area corresponding to image a in image A is reduced. When preset condition 2 is met, the brightness of the area corresponding to image a in image A is increased.

[0189] For example, Figure 4Taking the software structure shown as an example, the image optimization module (or brightness adjustment module) performs brightness adjustment on the area in image A corresponding to image a.

[0190] S508: Extract features from the detail area of ​​the face in image a to obtain image b corresponding to the detail area.

[0191] In the embodiment of the present application, the detail area may be an area corresponding to one or more parts of the face. Adaptively, the image b may correspond to one or more parts of the face. Exemplarily, the image b may include one part of the face, such as the eye; the image b may also include two parts: eyebrows and eyes. The embodiment of the present application does not specifically limit the number of images b, the parts included, etc.

[0192] It is understandable that the image b may be one or more images. When there are multiple images b, each image b corresponds to a different part of the face.

[0193] Exemplarily, taking the example that image b includes image b1, image b2 and image b3, image b1 may include eyes and eyebrows; image b2 may include hair; and image b3 may include a nose.

[0194] In the embodiment of the present application, the features of the face area in the image can be extracted by any method such as image segmentation algorithm, image segmentation model, etc., to obtain the image b corresponding to the detail area. For example, the detail area in image a can be segmented by DeepLabv3 to obtain the image b corresponding to the detail area.

[0195] In some embodiments, feature extraction may also be performed on the detail region of the face in the image A to obtain the image b corresponding to the detail region, which is not specifically limited here.

[0196] For example, Figure 4 Taking the software structure shown as an example, the feature extraction module extracts features from the detail area of ​​the face in image a to obtain image b corresponding to the detail area.

[0197] Exemplarily, taking the case where image b includes image b1, image b2 and image b3, the electronic device executes S509-S517.

[0198] S509 , input the image b1 into the IQA network b1 to obtain a detail score Sb1 .

[0199] The IQA network b1 is used to perform image quality detection on the image b1 and evaluate the quality of the image (the degree of image distortion). The detail score Sb1 is used to indicate the quality of the image b1, and it can also be understood that the detail score Sb1 is used to indicate the quality of the detail area corresponding to the image b1. The IQA network b1 can be the same as the IQA network a, or it can be different from the IQA network a. No specific limitation is made here.

[0200] In some embodiments, after the electronic device inputs the image b1 into the IQA network b1 , the IQA network b1 outputs a value C, where the value C is a detail score Sb1 of the image b1 .

[0201] In other embodiments, after the electronic device inputs the image b1 into the IQA network b1, the IQA network b1 outputs a value C; the electronic device adjusts the value C based on the maximum value that the IQA network b1 can output to obtain a value D, where the value D is the detail score Sb1 of the image b1.

[0202] The value D satisfies: In this way, the detail score Sb1 of the image b1 can be adjusted within a preset value range, which is convenient for subsequent comparison with the brightness score and the basic image quality score, weighted calculation, etc.

[0203] The preset value range can refer to the above corresponding description, which will not be repeated here.

[0204] Exemplarily, taking the preset value range of 0 to 1 as an example, the value D may be the ratio of the value C to the maximum value that can be output by the IQA network b1.

[0205] For example, taking the maximum value that can be output by the IQA network b1 as 100 and the preset value range as 0 to 10, if the value C is 80, the value D is 8.

[0206] In some embodiments, before inputting the image b1 into the IQA network b1, the image b1 is expanded, and the expanded image b1 is input into the IQA network b1 to obtain a detail score. The expansion may include: copying and arranging the image b1. In this way, the expanded image b1 can meet the image size requirements of the IQA network b1, and facilitate image quality detection (quality evaluation) of the image b1.

[0207] For example, taking the IQA network b1 requiring an image size of 24*24 as an example, if the size of the image b1 is 2*3, the image b1 can be copied and spliced ​​so that the size of the expanded image b1 is greater than or equal to 24*24.

[0208] In some embodiments, when performing image quality detection on image b1 through IQA network b1, IQA network b1 may perform downsampling processing on image b1 or expanded image b1 to meet the size requirement of IQA network b1. IQA network b1 may also perform image quality detection on image b1 or expanded image b1 by sampling a fixed central area. In this way, the phenomenon of inaccurate detail scoring caused by random sampling of image content can be reduced, and the accuracy of detail scoring can be improved.

[0209] For example, Figure 4 Taking the software structure shown in FIG. 1 as an example, the image quality assessment network module includes: an IQA network b1. The image processing plug-in inputs the image b1 into the IQA network b1 to obtain a detail score Sb1.

[0210] S510: Determine whether the detail score Sb1 is less than the basic image quality score.

[0211] When the detail score Sb1 is less than the basic image quality score, S511 is executed. The area corresponding to the image b1 in the image is optimized in a targeted manner, and the optimization efficiency is high. When the detail score Sb1 is greater than or equal to the basic image quality score, no processing is performed, and the area corresponding to the image b1 in the image may not be optimized, thereby shortening the time consumption of image optimization processing.

[0212] by Figure 4 Taking the software structure shown as an example, whether the detail score Sb1 of the image optimization module (or the brightness adjustment module) is less than the basic image quality score.

[0213] S511: Perform image enhancement processing on the area in image A corresponding to image a.

[0214] The image enhancement processing can refer to the above corresponding description, which will not be repeated here.

[0215] S512: Input the image b2 into the IQA network b2 to obtain a detail score Sb2.

[0216] The IQA network b2 is used to perform image quality detection on the image b2 and evaluate the quality of the image (the degree of image distortion). The detail score Sb2 is used to indicate the quality of the image b2, and it can also be understood that the detail score Sb2 is used to indicate the quality of the detail area corresponding to the image b2. The IQA network b2 can be the same as the IQA network a, or it can be different from the IQA network a. No specific limitation is made here.

[0217] The implementation principle of S512 is similar to the implementation principle of S509 above. For details, please refer to the corresponding description of S509 above, which will not be repeated here.

[0218] S513: Determine whether the detail score Sb2 is less than the basic image quality score.

[0219] When the detail score Sb2 is less than the basic image quality score, S514 is executed. When the detail score Sb2 is greater than or equal to the basic image quality score, no image enhancement processing is performed.

[0220] by Figure 4 Taking the software structure shown as an example, whether the detail score Sb2 of the image optimization module (or brightness adjustment module) is less than the basic image quality score.

[0221] S514: Perform image enhancement processing on the area in image A corresponding to image a.

[0222] The image enhancement processing can refer to the above corresponding description, which will not be repeated here.

[0223] S515: Input the image b3 into the IQA network b3 to obtain a detail score.

[0224] The IQA network b3 is used to perform image quality detection on the image b3 and evaluate the quality of the image (the degree of image distortion). The detail score Sb3 is used to indicate the quality of the image b3, and it can also be understood that the detail score Sb3 is used to indicate the quality of the detail area corresponding to the image b3. The IQA network b3 can be the same as the IQA network a, or it can be different from the IQA network a. No specific limitation is made here.

[0225] The implementation principle of S515 is similar to that of the above-mentioned S509. For details, please refer to the corresponding description of the above-mentioned S509, which will not be repeated here.

[0226] S516: Determine whether the detail score Sb3 is less than the basic image quality score.

[0227] When the detail score Sb3 is less than the basic image quality score, S517 is executed. When the detail score Sb3 is greater than or equal to the basic image quality score, no image enhancement processing is performed.

[0228] by Figure 4 Taking the software structure shown as an example, whether the detail score Sb3 of the image optimization module (or brightness adjustment module) is less than the basic image quality score.

[0229] S517: Perform image enhancement processing on the area in image A corresponding to image a.

[0230] The image enhancement processing can refer to the above corresponding description, which will not be repeated here.

[0231] S518: Output the optimized image A.

[0232] In summary, by comparing the quality of the entire image with the quality of the target object area in the image, when the image quality of the target object area is poor, the area is optimized. In this way, the image is optimized for the target object, the effect of the target object in the image is improved, and the optimization effect of the image is improved. In addition, the area related to the target object in the image is optimized, and some areas may not be optimized, which improves the optimization rate of the image and shortens the time consumed for image optimization processing.

[0233] It is understandable that in the above embodiment, the electronic device performs image processing judgment on the brightness of the target object and the detail area of ​​the target object. The electronic device may also not perform image processing judgment on the detail area of ​​the target object. Exemplarily, S508-S517 may not be executed, that is, the optimized image A is output after S507. The electronic device may also not perform image processing judgment on the brightness of the target object. Exemplarily, the electronic device may also not execute S505-S507, that is, the brightness is not adjusted, and image A is optimized according to the detail score.

[0234] Based on the above embodiment, the electronic device can also display brightness rating, basic image quality rating, detail rating, etc. In this way, the user can be prompted to confirm the editing of the image. For example, when the brightness rating is less than the basic image quality rating, the user can increase the brightness of the target object.

[0235] Based on the above embodiment, the electronic device can also perform a weighted calculation of the brightness score, the basic image quality score, and the detail score to obtain a comprehensive score, and display the comprehensive score. The comprehensive score is used to indicate the quality of the image that detects the target object and the image A as a whole. Exemplarily, the image processing plug-in transmits the comprehensive score of image A to the gallery application or the camera application. When displaying image A, the gallery application or the camera application displays the comprehensive score.

[0236] On the basis of the above embodiment, the electronic device may also display prompt information based on the basic image quality score, comprehensive score, brightness score, detail score, etc. The prompt information is used to guide the user to take photos. For example, the prompt information may guide the user to adjust the light for taking photos, the light and shadow of the face, the choice of background, and the composition of the photo. Exemplarily, when the brightness score is less than the basic image quality score, the prompt information may be "the brightness of the face is darker".

[0237] For example, Figure 7 The following is a flow chart of an image processing provided by an embodiment of the present application. Take an image captured in a high dynamic range imaging (HDR) mode as an example. Figure 7As shown, image A is an image captured in night scene mode. After the electronic device captures image A in HDR mode, it detects the quality of image A and obtains a basic image quality score base of 7.33; image A includes a face, and the electronic device extracts features from the face area to obtain image a. The electronic device detects the brightness of image a and obtains a brightness score Sa of 6.12, a brightness coefficient K of 1.02, and a deviation degree D1 of -1; the electronic device extracts features from the detail area of ​​the face to obtain images b1, b2, and b3, and detects the quality of images b1, b2, and b3, and obtains a detail score Sb1 of 8.11, a detail score Sb2 of 7.61, and a detail score Sb3 of 6.78.

[0238] Since the brightness score Sa is less than the basic image quality score base, the brightness coefficient k is greater than 1 and the deviation degree D1 is less than 0, the electronic device brightens the face in image A; since the detail score Sb3 is less than the basic image quality score base, the electronic device performs image enhancement processing on the nose area in image A.

[0239] from Figure 7 It can be seen that compared with image A1, the face brightness in the processed image is improved and the nose area is clearer.

[0240] The above embodiment is to optimize the images saved by the electronic device. In this way, the problem of time-consuming online real-time shooting algorithm is divided into two parts in an offline way to solve the contradiction between image quality and performance. Specifically, the basic image quality algorithm is embedded in the real-time shooting process, and the image quality further enhancement algorithm (for example, brightness adjustment, image enhancement algorithm, etc.) is embedded in the offline solution, so that the camera can quickly display the image. Image optimization processing is performed offline to improve the image quality.

[0241] In some embodiments, the electronic device may also detect the quality of an image (e.g., a raw image) in the photographing process to obtain a basic image quality score, a brightness score, a detail score, etc. The electronic device determines the algorithmic processing of the image captured by the camera based on the basic image quality score, the brightness score, and the detail score.

[0242] For example, taking portrait mode photography as an example, the electronic device may process the image captured by the camera using one or more of the following algorithms: portrait enhancement algorithm, filter algorithm (e.g., 3dlut), skin beautification algorithm, and face enhancement algorithm (e.g., face SR). The electronic device may select the portrait enhancement algorithm to process the image captured by the camera when the detail score is less than the basic image quality score. The electronic device may not select the portrait enhancement algorithm to process the image captured by the camera when the detail score is greater than the basic image quality score.

[0243] In this way, the algorithm used in image processing is targeted, which can improve the quality of images taken by electronic devices. In addition, the image processing algorithm is streamlined, the time for taking pictures is reduced, and the user experience is improved.

[0244] In some embodiments, the electronic device may further sort the brightness score and detail score, and select the algorithms corresponding to the M items with lower scores to process the images captured by the camera. In this way, by selecting M algorithms to process the images, the electronic device can display the images within a certain period of time, reducing the long shooting time and improving the user experience.

[0245] For example, Figure 8 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. Figure 8 As shown, the method includes:

[0246] S801. Acquire an image.

[0247] S801 may refer to the corresponding description of S501 above, which will not be repeated here.

[0248] S802: Detect image quality to obtain a first detection result.

[0249] The first detection result is used to indicate the quality of the entire image. The first detection result can be a score corresponding to the entire image (e.g., 7.58, etc.), or a grade corresponding to the entire image (e.g., good, etc.), or any form of detection result. No specific limitation is made here. Exemplarily, the first detection result can correspond to the basic image quality score mentioned above.

[0250] S803: When the image includes the target object, extract features of the target object in the image to obtain N first images, any one of which includes a preset feature of the target object, where N is an integer greater than 0.

[0251] The target object may be a living being such as a person or an animal, or an inanimate object such as a vehicle, or a part of a living being (e.g., a face, eyes, eyebrows, nose, etc.) or a part of an inanimate object. No specific limitation is made here.

[0252] The preset feature may be a feature corresponding to a part of the target object. For example, taking the target object as a face, the preset feature may be a feature corresponding to a detail area such as eyes and eyebrows.

[0253] In the embodiments of the present application, feature extraction can be performed through a convolutional network, a feature extraction model, or any other method, which is not specifically limited here.

[0254] S804: Detect the qualities of N first images to obtain N second detection results.

[0255] The second detection result is used to indicate the quality of a partial area of ​​the target object. The second detection result can be a score corresponding to the entire image (e.g., 7.58, etc.), or a grade corresponding to the entire image (e.g., good, etc.), or any form of detection result. No specific limitation is made here. Exemplarily, the second detection result can correspond to the detail score above.

[0256] In the embodiment of the present application, the quality of the first image can be detected by any method such as an image quality evaluation network, an algorithm for image quality evaluation, etc., which is not specifically limited here.

[0257] S805. For any second detection result among the N second detection results, when the quality of the target image indicated by any second detection result is lower than the quality of the image indicated by the first detection result, perform image enhancement processing on the target area in the image, the target image being the first image corresponding to any second detection result among the N first images, and the target area being the area corresponding to the target image.

[0258] The image enhancement processing can refer to the above corresponding description, which will not be repeated here.

[0259] In this way, the image quality of a partial area of ​​the target object is judged. When the image quality of a partial area of ​​the target object is low, image enhancement processing is performed on the area to improve the clarity or texture details of the area. When the image quality of a partial area of ​​the target object is high, image enhancement processing is not performed on the area, reducing the processing of the image, improving the processing speed of the entire image, and shortening the processing time of the image.

[0260] In a possible implementation, the first detection result includes: a first score; the image quality indicated by the first score is positively correlated with the first score; the second detection result includes: a second score; for any second detection result among the N second detection results, when the quality of the target image indicated by any second detection result is lower than the quality of the image indicated by the first detection result, image enhancement processing is performed on the target area in the image, including: for any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, image enhancement processing is performed on the target area in the image, and the target image is a first image corresponding to any second score among the N first images. .

[0261] The first score may correspond to the basic image quality score mentioned above, and the second score may correspond to the detail score mentioned above. Taking N as 3 as an example, the N second scores may correspond to the detail score b1, the detail score b2, and the detail score b3.

[0262] The preset threshold value may be 1 or any value, which is not specifically limited here. The preset threshold value is related to the value range of the first detection result and the value range of the second detection result.

[0263] Exemplarily, the preset threshold may be a ratio of a maximum value in a value range of the first detection result to a maximum value in a value range of the second detection result.

[0264] In this way, by quantifying the detection results into scores, it is convenient to compare the image qualities indicated by the detection results later.

[0265] In one possible implementation, the value range of the first score is the same as the value range of the second score; for any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, image enhancement processing is performed on the target area in the image, including: for any second score among the N second scores, when any second score is less than the first score, image enhancement processing is performed on the target area in the image.

[0266] In this way, the value range is the same, which facilitates subsequent score comparison and calculation and other subsequent processing.

[0267] In a possible implementation, detecting the quality of N first images to obtain N second detection results includes: detecting the quality of the N first images by using a pre-trained image quality assessment network to obtain N second detection results.

[0268] In this way, the image quality detection of the first image is realized through the image quality evaluation network, which is convenient, fast and easy to implement.

[0269] In a possible implementation, detecting the quality of N first images to obtain N second detection results includes: performing expansion processing on the N first images to obtain N expanded first images, wherein the sizes of the N expanded first images are all greater than or equal to a preset size; and detecting the quality of each expanded first image through a pre-trained image quality evaluation network to obtain N second detection results.

[0270] The preset size may be any size, for example, 224 mm*224 mm, 24 cm*24 cm, etc. No specific limitation is made here.

[0271] In this way, before quality detection is performed on the first image, expansion processing is performed so that the expanded first image can meet the size requirement of image quality detection, and then the image quality evaluation network can detect the first image.

[0272] In one possible implementation, when the image includes a target object, a brightness detection result of the target object is also obtained; the method also includes: when the image quality indicated by the brightness detection result is lower than the image quality of the image indicated by the first detection result, adjusting the brightness of the area corresponding to the target object in the image.

[0273] The brightness detection result can refer to the above corresponding description, which will not be repeated here. In the embodiment of the present application, the brightness detection of the target object can be achieved by any method such as algorithm, model, formula, etc. to obtain the brightness detection result, which will not be repeated here.

[0274] In this way, the brightness of the target object is detected, and when the image quality indicated by the brightness of the target object is low, the brightness of the area is adjusted to improve the brightness effect of the target object and the image quality. When the image quality indicated by the brightness of the target object is high, the brightness of the area is not adjusted, the image processing is reduced, the processing speed of the entire image is improved, and the image processing time is shortened.

[0275] In a possible implementation, before the brightness adjustment, the method further includes: when the image includes the target object, extracting features of the target object in the image by using an image segmentation algorithm to obtain a second image; and detecting the brightness of the second image to obtain a brightness detection result.

[0276] In this way, the target object is segmented from the image, making it easier to perform brightness detection on the target image.

[0277] In a possible implementation, extracting features of a target object in an image by using an image segmentation algorithm to obtain a second image includes: extracting features of a target object in an image by using an image segmentation algorithm and taking a margin expansion approach to obtain a second image.

[0278] In this way, when segmenting an image, the boundary filling method is used to retain more information of the target object, improve the integrity of the target object, and improve the detection effect of the target object.

[0279] In one possible implementation, the brightness detection result is related to the brightness mean, which is the average value of the brightness of each pixel in the second image; the image quality indicated by the brightness detection result is positively correlated with the first value, which is the absolute value of the difference between the brightness mean and the first brightness threshold.

[0280] The first brightness threshold may correspond to the brightness threshold a mentioned above. In this way, by performing brightness detection through the average brightness, the overall brightness of the target object can be better evaluated, thereby improving the accuracy of the brightness detection result.

[0281] In a possible implementation, the first brightness threshold is 127, and the brightness detection result includes: a third score, and the image quality indicated by the third score is positively correlated with the third score; In the case of Positive correlation: In the case of Positive correlation; is the mean brightness.

[0282] The third score may correspond to the brightness score mentioned above.

[0283] In this way, the closer the brightness mean value is to the brightness threshold, the higher the image quality indicated by the brightness detection result.

[0284] In a possible implementation, the brightness detection result also includes: a first deviation degree and a second deviation degree, the first deviation degree is the deviation degree between the brightness mean and the second brightness threshold, the second deviation degree is used to indicate the deviation degree between the brightness of all pixels corresponding to the target object and the brightness mean, and the brightness mean is the mean of the brightness of all pixels corresponding to the target object; when the image quality indicated by the brightness detection result is lower than the image quality indicated by the first detection result, the brightness of the area corresponding to the second image in the image is adjusted, including: when the ratio of the first deviation degree to the second deviation degree is less than the first threshold, and the third score is less than the first score, the target object is brightened; the absolute values ​​corresponding to the first threshold and the second threshold are the same, and the first threshold is less than the second threshold; when the ratio of the first deviation degree to the second deviation degree is greater than the second threshold, the target object is reduced in brightness.

[0285] The second brightness threshold may correspond to the brightness threshold b mentioned above.

[0286] In this way, the underexposure or overexposure of the target object can be judged through the two deviation degrees, thereby achieving brightening adjustment or lowering brightness adjustment of the target object, thereby improving the brightness adjustment effect.

[0287] In a possible implementation, the first deviation degree satisfies: S is the number of pixels, xi is the grayscale value corresponding to each pixel; the second deviation degree satisfies: Among them, Hist[j] is the ordinate corresponding to each grayscale value in the histogram corresponding to all pixels in the second image, and j is the grayscale value.

[0288] In this way, the method is simple and easy to implement.

[0289] In a possible implementation, acquiring the image includes: acquiring the image when the electronic device is in an idle scene.

[0290] The method for confirming the idle scene can refer to the corresponding instructions above and will not be repeated here.

[0291] In a possible implementation, before obtaining the first detection result, the method further includes: displaying an interface of a gallery application, the interface displaying an image and a first control; acquiring the image, including: acquiring the image in response to a trigger operation on the first control.

[0292] The first control may correspond to the image optimization 104 and the image optimization 206 mentioned above. Figure 1 The interface shown in c or Figure 2 In some embodiments, the electronic device triggers display of the first control when detecting that "more controls" are triggered. No specific limitation is made here.

[0293] In this way, image quality detection can be triggered in the gallery application, or image optimization can be performed based on the image quality detection.

[0294] In a possible implementation manner, the method further includes: performing weighted processing on the first detection result, the N second detection results, and the brightness detection result to obtain a detection result of the image; and displaying the detection result of the image.

[0295] In the embodiment of the present application, weighted processing of the first detection result and the N second detection results can be achieved in any manner, which is not specifically limited here.

[0296] The electronic device can display the detection result of the image to prompt and / or guide the user. The displayed detection result is used to indicate the quality of the image. The detection result can be a score corresponding to the image (for example, 7.58, etc.), or a grade corresponding to the image (for example, good, etc.), or any form of detection result. In some embodiments, the detection result can also include: prompt information. The prompt information is used to guide the user to take pictures. For example, the prompt information can guide the user to adjust the light for taking pictures, the light and shadow of the face, the choice of background, and the composition of the picture.

[0297] In this way, the quality of the area corresponding to the target object and the quality of the entire image can be weighted, thereby increasing the impact of the target object on the quality and the attention paid to the target object, so that the detection results after weighted processing meet user expectations.

[0298] It is understandable that the electronic device may also display the first detection result, the brightness detection result, etc. The present application embodiment does not specifically limit the content displayed on the interface. In some embodiments, the content displayed on the interface may include one or more of the following: comprehensive score, brightness score, detail score, etc.

[0299] In some embodiments, when the electronic device triggers image editing (for example, detecting a Figure 2 When the "Edit" control in the interface shown in e is triggered, the electronic device can obtain the first detection result, the brightness detection result, the image detection result, etc. through the image processing method provided in the embodiment of the present application. The electronic device can display one or more of the following: the first detection result, the brightness detection result, or the image detection result.

[0300] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0301] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0302] The image processing method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above method.

[0303] The image processing method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can refer to the above related descriptions, which will not be repeated here.

[0304] An embodiment of the present application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the above method.

[0305] The embodiment of the present application provides a chip or chip system. The chip or chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above-mentioned related embodiments, and will not be repeated here.

[0306] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions, and when the computer instructions are run on an electronic device, the electronic device executes the above method. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0307] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0308] An embodiment of the present application provides a computer program product, which includes a computer program code. When the computer program code is executed, the computer executes the above method.

[0309] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0310] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Get the image; Detecting the quality of the image to obtain a first detection result; In the case where the image includes a target object, extracting features of the target object in the image to obtain N first images, any one of the first images includes a preset feature of the target object, where N is an integer greater than 0; Detecting the qualities of the N first images to obtain N second detection results; For any second detection result among the N second detection results, when the quality of the target image indicated by any second detection result is lower than the quality of the image indicated by the first detection result, image enhancement processing is performed on the target area in the image, the target image is the first image corresponding to any second detection result among the N first images, and the target area is the area corresponding to the target image.

2. The method according to claim 1, characterized in that The first detection result includes: a first score; the image quality indicated by the first score is positively correlated with the first score; the second detection result includes: a second score; the image quality indicated by the second score is positively correlated with the second score; For any second detection result among the N second detection results, when the quality of the target image indicated by any second detection result is lower than the quality of the image indicated by the first detection result, performing image enhancement processing on the target area in the image, including: For any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, image enhancement processing is performed on the target area in the image, and the target image is the first image corresponding to any second score among the N first images.

3. The method according to claim 2, characterized in that The value range of the first score is the same as the value range of the second score; For any second score among the N second scores, when the ratio of any second score to the first score is less than a preset threshold, performing image enhancement processing on the target area in the image, including: For any second score among the N second scores, if any second score is smaller than the first score, image enhancement processing is performed on the target area in the image.

4. The method according to any one of claims 1 to 3, characterized in that: The detecting the qualities of the N first images to obtain N second detection results includes: The quality of the N first images is detected by a pre-trained image quality assessment network to obtain N second detection results.

5. The method according to any one of claims 1 to 3, characterized in that: The detecting the qualities of the N first images to obtain N second detection results includes: Performing expansion processing on the N first images to obtain N expanded first images, wherein sizes of the N expanded first images are all greater than or equal to a preset size; The quality of each of the expanded first images is detected by a pre-trained image quality assessment network to obtain the N second detection results.

6. The method according to any one of claims 1 to 5, characterized in that: In the case where the image includes a target object, a brightness detection result of the target object is also obtained; The method further comprises: In a case where the image quality indicated by the brightness detection result is lower than the image quality of the image indicated by the first detection result, brightness adjustment is performed on a region corresponding to the target object in the image.

7. The method according to claim 6, characterized in that Before adjusting the brightness, the method further includes: In the case where the image includes a target object, extracting features of the target object in the image by an image segmentation algorithm to obtain a second image; The brightness of the second image is detected to obtain the brightness detection result.

8. The method according to claim 7, characterized in that The step of extracting features of the target object in the image by using an image segmentation algorithm to obtain a second image includes: The second image is obtained by extracting features of the target object in the image through an image segmentation algorithm and adopting a margin expansion method.

9. The method according to any one of claims 6 to 8, characterized in that: The brightness detection result is related to a brightness mean, and the brightness mean is an average value of the brightness of each pixel in the second image; The image quality indicated by the brightness detection result is positively correlated with a first value, and the first value is an absolute value of a difference between the brightness mean value and a first brightness threshold value.

10. The method according to claim 9, characterized in that The first brightness threshold is 127, and the brightness detection result includes: a third score, and the image quality indicated by the third score is positively correlated with the third score; exist In the case of Positive correlation: In the case of Positive correlation is the brightness mean.

11. The method according to claim 10, characterized in that The brightness detection result further includes: a first deviation degree and a second deviation degree, the first deviation degree is the deviation degree between the brightness mean and the second brightness threshold, the second deviation degree is used to indicate the deviation degree between the brightness of all pixels corresponding to the target object and the brightness mean, the brightness mean is the mean of the brightness of all pixels corresponding to the target object; When the image quality indicated by the brightness detection result is lower than the quality of the image indicated by the first detection result, adjusting the brightness of the area corresponding to the second image in the image includes: When the ratio of the first deviation degree to the second deviation degree is less than a first threshold value, and the third score is less than the first score value, the target object is brightened; the absolute values ​​corresponding to the first threshold value and the second threshold value are the same, and the first threshold value is less than the second threshold value; When the ratio of the first deviation degree to the second deviation degree is greater than a second threshold, the target object is subjected to brightness reduction processing.

12. The method according to claim 11, characterized in that The second brightness threshold is 128, The first degree of deviation satisfies: D1 is the first deviation degree, S is the number of pixels in the second image, and xi is the grayscale value corresponding to the brightness of each pixel; The second degree of deviation satisfies: Among them, the Hist[j] is the vertical coordinate corresponding to each grayscale value in the histogram corresponding to all pixels in the second image, and j is the grayscale value.

13. The method according to any one of claims 1 to 12, characterized in that: Before obtaining the first detection result, the method further includes: Displaying an interface of a gallery application, wherein the interface displays the image and a first control; The acquiring of the image includes: acquiring the image in response to a triggering operation on the first control.

14. The method according to any one of claims 1 to 13, characterized in that: The method further comprises: Performing weighted processing on the first detection result, the N second detection results, and the brightness detection result to obtain a detection result of the image; The detection result of the image is displayed.

15. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the method according to any one of claims 1 to 14.

16. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 14.

18. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 14.

Images