Image processing method, electronic device and computer readable storage medium

By performing bright and dark image analysis, clustering and highlight component calculation of images, the image de-highlight processing process is simplified, the complex and unstable de-highlighting problem in the existing technology is solved, and convenient and stable image processing effect is achieved. .

CN114140356BActive Publication Date: 2024-11-12GUANGZHOU HUYA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111442856.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-12
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

The existing image de-highlighting processing method is complex and the effect is unstable, making it difficult to achieve convenient and stable de-highlighting processing in diverse scenarios.

Method used

By performing bright and dark map analysis on the images to be processed, the mask area is determined and clustered, the highlight component is calculated based on the brightness value of pixel points in the clustered area, and the highlighting processing is performed, which simplifies the highlighting processing process.

Benefits of technology

The convenience and stability of image de-highlighting processing are improved, complex calculations of separation of diffuse and specular components in the prior art are avoided, and processing efficiency and effect are significantly improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114140356B_ABST
    Figure CN114140356B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides an image processing method, an electronic device and a computer-readable storage medium, which relate to the field of image technology. The method comprises: processing an image to be processed to obtain a bright image, a dark image and a range image of the image to be processed; based on the brightness value of each pixel in the dark image, a mask area is determined from a region of interest set in the image to be processed; based on the brightness value of each pixel in the mask area in the bright image and the dark image, clustering is performed to obtain a clustering area; based on the brightness value of each pixel in the clustering area in the bright image and the brightness value in the range image, the highlight component of the corresponding pixel is analyzed; based on the highlight component, the mask area in the image to be processed is de-highlighted to obtain an output result image, thereby realizing image de-highlighting processing conveniently and stably.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular to an image processing method, an electronic device and a computer-readable storage medium. Background Art

[0002] Images are the basis of human vision, an objective reflection of natural scenery, and an important source for human beings to understand the world and themselves. "Image" is the distribution of light reflected or transmitted by an object, and "image" is the impression or cognition formed in the human brain by the image received by the human visual system.

[0003] As the scenarios of image generation and application become more and more diverse, image de-highlighting processing is required in many scenarios. However, current de-highlighting methods are relatively complex and the stability of the effect needs to be improved. Summary of the invention

[0004] One of the purposes of the present invention includes, for example, providing an image processing method, an electronic device, and a computer-readable storage medium to at least partially improve the convenience and stability of image de-highlighting processing.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, an embodiment of the present invention provides an image processing method, including:

[0007] Processing the image to be processed to obtain a bright image and a dark image of the image to be processed, and obtaining a range map of the image to be processed based on the bright image and the dark image;

[0008] Based on the brightness value of each pixel in the dark image, a mask area is determined from a region of interest set in the image to be processed;

[0009] Clustering is performed based on the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain a clustering area;

[0010] Analyze and obtain the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the clustering area in the bright image and the brightness value in the range image;

[0011] Based on the highlight component, a de-highlighting process is performed on the mask area in the image to be processed to obtain an output result image.

[0012] In an optional embodiment, the step of processing the image to be processed to obtain a bright image and a dark image of the image to be processed, and obtaining a range map of the image to be processed based on the bright image and the dark image analysis includes:

[0013] A bright image and a dark image of the image to be processed are obtained in the RGB domain, wherein the brightness value of each pixel in the bright image is the maximum value of the corresponding pixel in the image to be processed in the RGB domain, or the product of the maximum value and a first set coefficient; the brightness value of each pixel in the dark image is the minimum value of the corresponding pixel in the image to be processed in the RGB domain, or the product of the minimum value and a second set coefficient;

[0014] The brightness values ​​of the bright image and the dark image are subtracted to obtain a range image, wherein the brightness value of each pixel in the range image is the difference between the brightness value of each pixel in the image to be processed in the bright image and the brightness value in the dark image.

[0015] In an optional implementation, the step of determining a mask area from a region of interest set in the image to be processed based on the brightness value of each pixel in the dark image includes:

[0016] Calculate the average value of the brightness of each pixel in the dark image;

[0017] The area where each pixel point whose brightness value is greater than the mean value is located in the region of interest set in the image to be processed is used as the mask area.

[0018] In an optional implementation, the step of clustering the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain the clustering area includes:

[0019] Establishing a coordinate system by taking the brightness value of each pixel point in the mask area in the dark image as the abscissa and the brightness value of each pixel point in the bright image as the ordinate;

[0020] Convert each pixel point in the mask area into a coordinate point in the coordinate system, with the brightness value of the pixel point in the bright image as the ordinate value and the brightness value of the pixel point in the dark image as the abscissa value;

[0021] Clustering is performed on the coordinate points in the coordinate system to obtain a clustering area.

[0022] In an optional implementation, the step of clustering the coordinate points in the coordinate system to obtain clustering areas includes:

[0023] Obtaining the horizontal coordinate range and the vertical coordinate range of each coordinate point;

[0024] The minimum value in the horizontal axis range, the maximum value in the horizontal axis range and the maximum value in the vertical axis range are respectively used as the initial centers of clusters, and the k-means clustering algorithm is used to cluster the coordinate points to obtain three clustering areas.

[0025] In an optional implementation, the step of analyzing and obtaining the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the clustering area in the bright image and the brightness value in the range image includes:

[0026] For each of the coordinate points in each of the clustering areas, a ratio value of the pixel point is calculated according to a brightness value of the pixel point corresponding to the coordinate point in the bright image and a brightness value of the pixel point in the range image;

[0027] Calculate the target ratio value corresponding to each cluster area according to the ratio values ​​of the pixel points corresponding to all the coordinate points in each cluster area;

[0028] According to the brightness values ​​of the pixels corresponding to the coordinate points in the clustering area in the bright image, the brightness values ​​in the range image, and the corresponding target ratio values, the highlight components of the pixels corresponding to the coordinate points in each clustering area are calculated.

[0029] In an optional implementation manner, the ratio value of the pixel points is calculated using the following formula:

[0030] ratio = max / (range+eps)

[0031] The target ratio value corresponding to the cluster area is the median value of the ratio values ​​of each pixel point in the cluster area;

[0032] The highlight component of the pixel point corresponding to each coordinate point in each cluster area is calculated by the following formula:

[0033] Highlight component = max – target ratio * range;

[0034] Among them, ratio is the ratio value of the pixel; max is the brightness value of the pixel in the bright image; range is the brightness value of the pixel in the range image; eps is a non-zero positive value.

[0035] In an optional embodiment, the method further comprises:

[0036] Based on the image to be processed and the brightness value of each pixel in the mask area in the dark image, a pseudo-highlight-removed image of the mask area is obtained by analysis;

[0037] The pseudo de-highlighted image is set and processed to obtain an output preview image.

[0038] In a second aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image processing method when executing the program.

[0039] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the above-mentioned image processing method.

[0040] In a fourth aspect, an embodiment of the present invention provides an image processing device, including:

[0041] An image analysis module is used to process the image to be processed to obtain a bright image and a dark image of the image to be processed, and to obtain a range map of the image to be processed based on the bright image and the dark image; to determine a mask area from a region of interest set in the image to be processed based on the brightness value of each pixel in the dark image; to cluster the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain a clustering area;

[0042] The highlight processing module is used to analyze and obtain the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the cluster area in the bright image and the brightness value in the range image; based on the highlight component, the mask area in the image to be processed is de-highlighted to obtain an output result image.

[0043] The beneficial effects of the embodiments of the present invention include, for example: by processing the image to be processed, a bright image, a dark image, a range image and a mask area of ​​the image to be processed are obtained, clustering is performed based on the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain the clustering area, and according to the brightness values ​​of each pixel in the clustering area in the bright image and the brightness values ​​in the range image, the highlight components of the corresponding pixels are analyzed to obtain the highlight components, and then the mask area in the image to be processed is de-highlighted based on the highlight components to obtain an output result image, thereby realizing image de-highlighting processing conveniently and stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention is shown.

[0046] Figure 2 A schematic flow chart of an image processing method provided by an embodiment of the present invention is shown.

[0047] Figure 3 A schematic diagram of an image to be processed provided by an embodiment of the present invention is shown.

[0048] Figure 4 A schematic diagram of a bright image provided by an embodiment of the present invention is shown.

[0049] Figure 5 A schematic diagram of a dark image provided by an embodiment of the present invention is shown.

[0050] Figure 6 A schematic diagram of a range map provided by an embodiment of the present invention is shown.

[0051] Figure 7 A schematic diagram of a mask area provided by an embodiment of the present invention is shown.

[0052] Figure 8 One of the result graphs obtained after a cyclic clustering provided by an embodiment of the present invention is shown.

[0053] Fig. 9 The second diagram shows a result obtained after a cyclic clustering provided by an embodiment of the present invention.

[0054] Fig.10 A schematic diagram showing the ratio values ​​of each pixel point in a mask area provided by an embodiment of the present invention is shown.

[0055] Fig.11 A schematic diagram showing highlight components of each pixel point within a mask area provided by an embodiment of the present invention is shown.

[0056] Fig.12 A de-highlighting image provided by an embodiment of the present invention is shown.

[0057] Fig.13 An exemplary structural block diagram of an image processing device provided by an embodiment of the present invention is shown.

[0058] Icons: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication module; 140 - image processing device; 141 - image analysis module; 142 - highlight processing module. DETAILED DESCRIPTION

[0059] For some images, such as those with overexposed areas, highlight removal is required. Currently, the commonly used highlight removal method is to separate the overexposed areas into diffuse reflection components and specular reflection components through a series of algorithms, using the diffuse reflection components as the highlight removal result and the specular reflection components as the highlight elements. The corresponding processing method is relatively complex and the effect is unstable.

[0060] Based on the above research, an embodiment of the present invention provides an image processing solution, which implements de-highlighting processing through the relationship between light and dark, obtains an output result image, and improves the convenience and stability of de-highlighting processing.

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0065] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0066] Please refer to Figure 1 , is a block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 in this embodiment may be a server, a processing device, a processing platform, etc. capable of acquiring, analyzing, processing, and outputting images. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, the processor 120, and the communication module 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0067] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0068] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions.

[0069] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0070] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 The components shown in the figure can be implemented by hardware, software or a combination thereof. For example, in some scenarios, the electronic device 100 may also include a camera, a display screen, etc., to capture and display images.

[0071] Please refer to Figure 2 , which is a flow chart of an image processing method provided by an embodiment of the present invention, can be Figure 1 The electronic device 100 executes, for example, the processor 120 in the electronic device 100. The image processing method includes S110 to S150.

[0072] S110, processing the image to be processed to obtain a bright image and a dark image of the image to be processed, and obtaining a range map of the image to be processed based on analysis of the bright image and the dark image.

[0073] S120, determining a mask area from a region of interest set in the image to be processed based on the brightness value of each pixel in the dark image.

[0074] S130, clustering based on the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain a clustering area.

[0075] S140, analyzing and obtaining highlight components of corresponding pixels according to the brightness values ​​of each pixel in the clustering area in the bright image and the brightness values ​​in the range image.

[0076] S150, performing a de-highlighting process on the mask area in the image to be processed based on the highlight component to obtain an output result image.

[0077] Based on the above scheme, de-highlighting processing can be achieved directly through the light-dark relationship corresponding to the bright image and the dark image. Compared with the existing technology, there is no need to separate the diffuse reflection component and the mirror reflection component, and there is no need to perform complex calculations such as the direction of illumination to obtain the output result image of the de-highlighting processing, which significantly improves the convenience and stability of the de-highlighting processing.

[0078] In the embodiment of the present invention, the image to be processed may be an image in a variety of scenes. For example, it may be an image captured by a camera. For another example, it may be an image frame in a video. For another example, it may be an image input by a user, an image obtained from other devices, etc., which is not limited in the present embodiment. Figure 3 , is a schematic diagram of using a brush-stroke sketch image including a person as an image to be processed. In the case where the electronic device 100 has a camera, the image can be obtained by photographing the electronic device 100.

[0079] In S110, the bright image and the dark image can be obtained in a variety of ways. For example, in one implementation, the bright image and the dark image of the image to be processed can be obtained in the RGB domain, wherein the brightness value of each pixel in the bright image is the maximum value of the corresponding pixel in the image to be processed in the RGB domain. The brightness value of each pixel in the dark image is the minimum value of the corresponding pixel in the image to be processed in the RGB domain. Figure 4 , for Figure 3 The bright image is obtained by taking the maximum value of each pixel in the RGB domain. Figure 5 , for Figure 3 For each pixel in , take the minimum value in the RGB domain to get the dark image.

[0080] For another example, in other implementations, the brightness value of each pixel in the bright image may be the product of the maximum value of the corresponding pixel in the image to be processed in the RGB domain and the first set coefficient, and the brightness value of each pixel in the dark image may be the product of the minimum value of the corresponding pixel in the image to be processed in the RGB domain and the second set coefficient. Among them, the first set coefficient and the second set coefficient may be less than one, or greater than one, and may also be a certain set value range, and this embodiment does not limit this. For another example, the brightness value of each pixel in the bright image may be the product of the brightness value of the corresponding pixel in the image to be processed and a set coefficient greater than one, and the brightness value of each pixel in the dark image may be the product of the brightness value of the corresponding pixel in the image to be processed and a set coefficient less than one. This embodiment does not give examples one by one.

[0081] The range map can be obtained by: subtracting the brightness values ​​of the bright image from the dark image to obtain the range map, wherein the brightness value of each pixel in the range map is the difference between the brightness value of each pixel in the image to be processed in the bright image and the brightness value in the dark image. Figure 6 , for Figure 3 For each pixel in , the range map is obtained by subtracting the brightness value of the pixel in the bright image from the brightness value in the dark image.

[0082] In S120, the region of interest (ROI) can be flexibly set. For example, the entire region in the image to be processed can be used as the ROI. For another example, only a part of the image to be processed can be used as the ROI. For example, when the image to be processed is a person image, the face region can be used as the ROI.

[0083] Since the area that needs to be de-highlighted is the area with higher brightness in the region of interest, a mask area that needs to be de-highlighted can be determined from the region of interest based on the brightness. In one implementation, the mask area can be determined by calculating the average brightness value of each pixel in the dark image, and taking the area where each pixel with brightness greater than the average is located in the region of interest set in the image to be processed as the mask area. Figure 7 , in the case of taking the face area as the region of interest, Figure 3 The mask area determined in the image to be processed is shown.

[0084] It is understandable that the mask area may be determined in other ways. For example, the mask area may be determined based on the median brightness value of each pixel in the dark image, or based on the brightness value set by the user. This embodiment does not limit this.

[0085] After the mask area is determined, subsequent processing is performed on each pixel in the mask area, thereby reducing the processing amount while ensuring that the pixel points in the area of ​​interest are de-highlighted.

[0086] In S130, the clustering region can be obtained in a variety of ways. For example, each pixel point in the mask region can be converted into a coordinate point in a coordinate system to achieve clustering. In one implementation, a coordinate system can be established with the brightness value of each pixel point in the mask region in the dark image as the horizontal coordinate and the brightness value in the bright image as the vertical coordinate. Each pixel point in the mask region is converted into a coordinate point in the coordinate system with the brightness value of the pixel point in the bright image as the vertical coordinate value and the brightness value in the dark image as the horizontal coordinate value. Each of the coordinate points in the coordinate system is clustered to obtain a clustering region.

[0087] The clustering method in the coordinate system can be flexibly set. For example, in one implementation, the horizontal coordinate range and the vertical coordinate range of each coordinate point can be obtained, and the minimum value in the horizontal coordinate range, the maximum value in the horizontal coordinate range, and the maximum value in the vertical coordinate range are respectively used as the initial center of the cluster, and the k-means clustering algorithm is used to cluster each coordinate point to obtain three clustering areas.

[0088] In order to increase the robustness of the k-means clustering algorithm, before clustering the coordinate points, some "free" coordinate points far away from the initial center can be deleted.

[0089] For example, Figure 7 Each pixel in the mask area shown is Figure 4 The brightness value in the bright image shown is used as the vertical axis, and the Figure 5 The brightness value in the dark image shown is used as the horizontal axis to establish a coordinate system. Figure 7 Each pixel point in the mask area is converted into a coordinate point, so that each pixel point corresponds to each coordinate point. The ordinate value of each coordinate point is the value of the corresponding pixel point in Figure 4 The brightness value in the bright image shown, the horizontal axis value is the corresponding pixel point Figure 5 Brightness values ​​in the dark image shown.

[0090] After the pixels in the mask area are converted to coordinate points, they will roughly form a triangle shape in the coordinate system. In order to better achieve clustering, the horizontal and vertical coordinate ranges of each coordinate point can be calculated after removing impurities and smoothing the entire area. The minimum value in the horizontal coordinate range, the maximum value in the horizontal coordinate range, and the maximum value in the vertical coordinate range are used as the initial center of the cluster. The k-means clustering method is used to perform additional cyclic clustering for a set number of times, such as once, twice, or three times, to increase robustness. Please refer to Figure 8 , is the result after an additional round of clustering. See Fig. 9 , which is the result graph obtained after two additional cycles of clustering.

[0091] Accordingly, in S140, the highlight component of each cluster area can be obtained by partitioning. For example, in one implementation, for each coordinate point in each cluster area, the ratio value of the pixel point can be calculated according to the brightness value of the pixel point corresponding to the coordinate point in the bright image and the brightness value in the range image. The target ratio value corresponding to the cluster area is calculated according to the ratio value of the pixel points corresponding to all the coordinate points in each cluster area. The highlight component of the pixel point corresponding to each coordinate point in each cluster area is calculated according to the brightness value of the pixel point corresponding to each coordinate point in the cluster area in the bright image, the brightness value in the range image, and the corresponding target ratio value.

[0092] Among them, the ratio value of the pixel points corresponding to each coordinate point in each cluster area can be calculated by the following formula:

[0093] ratio = max / (range+eps)

[0094] Among them, ratio is the ratio value of the pixel; max is the brightness value of the pixel in the bright image; range is the brightness value of the pixel in the range image; eps is a non-zero positive value. Fig.10 , obtained by analysis Figure 7 Schematic diagram of the ratio values ​​of each pixel within the mask area shown.

[0095] In order to reduce the amount of calculation while ensuring the reliability of the highlight removal process, in this embodiment, the target ratio value can be calculated for each cluster area. For example, the median of the ratio values ​​of all pixels in each cluster area is used as the target ratio value corresponding to the cluster area. Then, the highlight component of the pixel point corresponding to each coordinate point in each cluster area is calculated based on the target ratio value. For example, the highlight component of each pixel point in each cluster area can be calculated by the following formula:

[0096] Highlight component = max – target ratio * range.

[0097] See also Fig.11 , obtained by analysis Figure 7 Schematic diagram of the highlight component of each pixel within the mask area shown.

[0098] In S150, the output result image is a de-highlighted image obtained by subtracting the highlight component from the image to be processed. Fig.12 ,for Figure 3 Subtract the image to be processed from Fig.11 After removing the highlight component shown, the de-highlight map is obtained.

[0099] In order to improve the imaging effect of the output result graph, you can set processing. For example, Fig.12 The de-highlighted image shown in FIG. 1 is expanded and smoothed to reduce the defects and artifacts of the output result image. Fig.11 The highlight component map shown is moderately blurred to make the final result smoother.

[0100] In order to further improve the image processing efficiency, the image to be processed may be reduced in size before executing the above processing flow to increase the processing speed.

[0101] In one implementation, a pseudo de-highlighted image can be obtained after simple processing to serve as an output preview image before the final output result image. Exemplarily, the pseudo de-highlighted image can be obtained in the following manner: based on the image to be processed and the brightness values ​​of each pixel in the mask area in the dark image, a pseudo de-highlighted image of the mask area is analyzed and obtained, and the pseudo de-highlighted image is set and processed to obtain an output preview image.

[0102] The average brightness of each pixel in the dark image can be calculated, and the brightness value of each pixel in the image to be processed is subtracted from the brightness value of the corresponding pixel in the dark image and then summed with the brightness average to serve as the brightness value of each pixel in the pseudo-highlight removal image.

[0103] When a pseudo de-highlighted image is obtained, for scenes with low precision requirements, the pseudo de-highlighted image can be globally brightened and used as the final output result image. For scenes with high precision requirements, after obtaining the pseudo de-highlighted image, the pseudo de-highlighted image can be processed and then displayed as an output preview image. When the output result image is obtained, the output result image can be displayed so that users can understand the approximate effect of the final output result image in advance.

[0104] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of an image processing device is given below. Fig.13 , Fig.13A functional module diagram of an image processing device 140 provided in an embodiment of the present invention, the image processing device 140 can be applied to Figure 1 The electronic device 100 is shown. It should be noted that the basic principle and technical effects of the image processing device 140 provided in this embodiment are the same as those of the above embodiment. For the sake of brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above embodiment. The image processing device 140 includes an image analysis module 141 and a highlight processing module 142.

[0105] Among them, the image analysis module 141 is used to process the image to be processed, obtain the bright image and the dark image of the image to be processed, and obtain the range map of the image to be processed based on the analysis of the bright image and the dark image; based on the brightness value of each pixel point in the dark image, determine the mask area from the area of ​​interest set in the image to be processed; based on the brightness value of each pixel point in the mask area in the bright image and the dark image, cluster the cluster area.

[0106] The highlight processing module 142 is used to analyze and obtain the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the clustering area in the bright image and the brightness value in the range image; based on the highlight component, the mask area in the image to be processed is de-highlighted to obtain an output result image.

[0107] Based on the above, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the above-mentioned image processing method.

[0108] By adopting the above scheme in the embodiment of the present invention, it is possible to conveniently and reliably implement highlight removal in a variety of scenarios. For example, in a photo-taking scenario, the image can be subjected to one-key highlight removal. For example, after an image is captured by an electronic device including a camera, if the user clicks the highlight removal button, the electronic device can execute the above image processing method on the image to obtain an output result graph. For another example, the electronic device can execute the above image processing method on a certain image in an offline state to obtain an output result graph. For another example, the electronic device can use the above image processing method to process each frame of the video frame by frame to obtain a highlight-removed video.

[0109] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0111] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Processing the image to be processed to obtain a bright image and a dark image of the image to be processed, and obtaining a range map of the image to be processed based on the bright image and the dark image, including: subtracting the brightness values ​​of the bright image from the dark image to obtain a range map, wherein the brightness value of each pixel in the range map is the difference between the brightness value of each pixel in the image to be processed in the bright image and the brightness value in the dark image; Based on the brightness value of each pixel in the dark image, a mask area is determined from a region of interest set in the image to be processed; Clustering is performed based on the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain a clustering area; Analyze and obtain the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the clustering area in the bright image and the brightness value in the range image; Based on the highlight component, a de-highlighting process is performed on the mask area in the image to be processed to obtain an output result image.

2. The image processing method according to claim 1, characterized in that: The step of processing the image to be processed to obtain a bright image and a dark image of the image to be processed, and obtaining a range map of the image to be processed based on the bright image and the dark image analysis includes: A bright image and a dark image of the image to be processed are obtained in the RGB domain, wherein the brightness value of each pixel in the bright image is the maximum value of the corresponding pixel in the image to be processed in the RGB domain, or the product of the maximum value and a first set coefficient; the brightness value of each pixel in the dark image is the minimum value of the corresponding pixel in the image to be processed in the RGB domain, or the product of the minimum value and a second set coefficient.

3. The image processing method according to claim 1, characterized in that: The step of determining a mask area from a region of interest set in the image to be processed based on the brightness value of each pixel in the dark image comprises: Calculate the average value of the brightness of each pixel in the dark image; The area where each pixel point whose brightness value is greater than the mean value is located in the region of interest set in the image to be processed is used as the mask area.

4. The image processing method according to claim 1, characterized in that: The step of clustering the brightness values ​​of each pixel in the mask area in the bright image and the dark image to obtain the clustering area comprises: Establishing a coordinate system by taking the brightness value of each pixel point in the mask area in the dark image as the abscissa and the brightness value of each pixel point in the bright image as the ordinate; Convert each pixel point in the mask area into a coordinate point in the coordinate system, with the brightness value of the pixel point in the bright image as the ordinate value and the brightness value of the pixel point in the dark image as the abscissa value; Clustering is performed on the coordinate points in the coordinate system to obtain a clustering area.

5. The image processing method according to claim 4, characterized in that: The step of clustering the coordinate points in the coordinate system to obtain a clustering area comprises: Obtaining the horizontal coordinate range and the vertical coordinate range of each coordinate point; The minimum value in the horizontal axis range, the maximum value in the horizontal axis range and the maximum value in the vertical axis range are respectively used as the initial centers of clusters, and the k-means clustering algorithm is used to cluster the coordinate points to obtain three clustering areas.

6. The image processing method according to claim 5, characterized in that: The step of analyzing and obtaining the highlight component of the corresponding pixel point according to the brightness value of each pixel point in the clustering area in the bright image and the brightness value in the range image comprises: For each of the coordinate points in each of the clustering areas, a ratio value of the pixel point is calculated according to a brightness value of the pixel point corresponding to the coordinate point in the bright image and a brightness value of the pixel point in the range image; Calculate the target ratio value corresponding to each cluster area according to the ratio values ​​of the pixel points corresponding to all the coordinate points in each cluster area; According to the brightness values ​​of the pixels corresponding to the coordinate points in the clustering area in the bright image, the brightness values ​​in the range image, and the corresponding target ratio values, the highlight components of the pixels corresponding to the coordinate points in each clustering area are calculated.

7. The image processing method according to claim 6, characterized in that: The ratio value of the pixel point is calculated by the following formula: ratio = max / (range+eps) The target ratio value corresponding to the cluster area is the median value of the ratio values ​​of each pixel point in the cluster area; The highlight component of the pixel point corresponding to each coordinate point in each cluster area is calculated by the following formula: Highlight component = max – target ratio * range; Among them, ratio is the ratio value of the pixel; max is the brightness value of the pixel in the bright image; range is the brightness value of the pixel in the range image; eps is a non-zero positive value.

8. The image processing method according to any one of claims 1 to 7, characterized in that: The method further comprises: Based on the image to be processed and the brightness value of each pixel in the mask area in the dark image, a pseudo-highlight-removed image of the mask area is obtained by analysis; The pseudo de-highlighted image is set and processed to obtain an output preview image.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method according to any one of claims 1 to 8 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 8.