Image processing method, computing device and storage medium

Abstract

The present invention discloses an image processing method, a computing device and a storage medium. The method comprises: processing an image to be processed according to a lighting environment of the image to be processed to obtain a first image; detecting whether the first image has color cast; if the first image has color cast, determining whether the first image needs color correction; if the first image needs color correction, performing a color reduction operation on the first image to obtain a second image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, a computing device and a storage medium. Background Art

[0002] With the continuous development of image processing technology, its application in image beautification and processing fields such as beauty, style transfer, and mapping has become more and more mature. On social media platforms such as short videos and photo walls, interesting and personalized beautified images are widely praised by users. However, the processing of images on the above platforms is mainly aimed at faces and human body areas, or introduces patterns such as mapping. Even for natural images, the focus is on making the image more beautiful rather than retaining the information of the image itself, so there is a possibility of image distortion.

[0003] For some information platforms, the main purpose of displaying images is to present as much detailed information as possible. If the images are over-colored or over-beautified, the image details will be lost, and the image quality will be difficult to guarantee, which will result in users not being able to obtain the information they need from the images, resulting in low user satisfaction. Intelligent processing is particularly important when adjusting large batches of images.

[0004] At present, there are two types of image processing methods commonly used in photo editing. One type uses mapping operations, that is, according to the mapping style selected by the user, the spatial geometric relationship is used to synthesize the image to be beautified and the target style with an asynchronous mapping algorithm. The user does not need to adjust the size and position of the map by himself, which improves the efficiency of image beautification and perfectly integrates the visual effects. It is convenient and practical, but no detailed image modification and modification are performed.

[0005] The other type is based on deep learning algorithms. For example, when an image needs to be corrected (such as color adjustment, brightness and contrast adjustment, etc.), first collect training samples and build a model, use the training samples to complete model training, and finally input the image to be processed into the trained model to obtain the output image, which retains more real color information and improves the authenticity of the display effect. However, this method has high requirements for the model and needs to collect specific training samples. It relies on manual assistance to a certain extent, and the quality of the output image is greatly affected by the sample.

[0006] Therefore, a new image processing method is needed to optimize the above processing process. Summary of the invention

[0007] To this end, the present invention provides an image processing solution to try to solve or at least alleviate the above problems.

[0008] According to one aspect of the present invention, there is provided an image processing method, the method comprising the following steps: first, processing an image to be processed according to the lighting environment of the image to be processed to obtain a first image; detecting whether the first image has color cast; if the first image has color cast, determining whether the first image requires color correction; if the first image requires color correction, performing a color reduction operation on the first image to obtain a second image.

[0009] Optionally, the image processing method according to the present invention also includes determining the lighting environment of the image to be processed, and the step of determining the lighting environment of the image to be processed includes: converting the image to be processed from the RGB color space to the Lab color space, and performing channel separation to obtain corresponding L channel image, a channel image and b channel image; calculating the average brightness and maximum brightness of the L channel image of the image to be processed; if the average brightness and the maximum brightness meet the first brightness condition, determining that the lighting environment of the image to be processed is a shadow environment; if the maximum brightness meets the second brightness condition, determining that the lighting environment of the image to be processed is a high light environment.

[0010] Optionally, in the image processing method according to the present invention, the first brightness condition and the second brightness condition are pre-generated in the following manner: collect images to obtain a first test image set and a second test image set, the first test image set including multiple images that need to adjust the brightness, and the second test image set including multiple images that do not need to adjust the brightness; calculate the average brightness and maximum brightness of each image in the first test image set and the second test image set under the L channel respectively, and generate a first average brightness set, a first maximum brightness set, a second average brightness set and a second maximum brightness set accordingly; perform cluster analysis on the first average brightness set, the first maximum brightness set, the second average brightness set and the second maximum brightness set to generate the first brightness condition and the second brightness condition.

[0011] Optionally, in the image processing method according to the present invention, the steps of respectively calculating the average brightness and maximum brightness of each image in the first test image set and the second test image set under the L channel, and correspondingly generating a first average brightness set, a first maximum brightness set, a second average brightness set, and a second maximum brightness set include: respectively obtaining the L channel image of each image in the first test image set and the second test image set under the Lab color space to generate a first L channel image set and a second L channel image set; calculating the average brightness and maximum brightness of each L channel image in the first L channel image set and the second L channel image set, and correspondingly generating a first average brightness set, a first maximum brightness set, a second average brightness set, and a second maximum brightness set.

[0012] Optionally, in the image processing method according to the present invention, the step of processing the image to be processed according to the lighting environment of the image to be processed to obtain a first image includes: if the lighting environment of the image to be processed is a shadow environment, enhancing the brightness of the image to be processed to obtain the first image; if the lighting environment of the image to be processed is a high-light environment, reducing the brightness of the high-brightness area of ​​the image to be processed, and enhancing the brightness of the low-brightness area of ​​the image to be processed to obtain the first image; if the lighting environment of the image to be processed is a normal environment, using the image to be processed as the first image.

[0013] Optionally, in the image processing method according to the present invention, the step of performing brightness enhancement on the image to be processed to obtain a first image includes: performing channel separation on the image to be processed in the RGB color space to obtain corresponding R channel image, G channel image and B channel image; performing brightness enhancement on the R channel image, G channel image and B channel image of the image to be processed respectively to obtain new R channel image, G channel image and B channel image, and synthesizing them to obtain the first image.

[0014] Optionally, in the image processing method according to the present invention, the step of reducing the brightness of the high-brightness area in the image to be processed and increasing the brightness of the low-brightness area in the image to be processed to obtain the first image includes: performing channel separation on the image to be processed in the Lab color space to obtain the corresponding L channel image, a channel image and b channel image; for each pixel point in the L channel image of the image to be processed, if the value of the pixel point is greater than the first brightness threshold, it is determined that the pixel point belongs to the high-brightness area, and the value of the pixel point is replaced with the first brightness threshold; if the value of the pixel point is less than the second brightness threshold, it is determined that the pixel point belongs to the low-brightness area, and the value of the pixel point is replaced with the second brightness threshold; and the L channel image after the replacement processing, the a channel image and the b channel image of the image to be processed are synthesized to obtain the first image.

[0015] Optionally, in the image processing method according to the present invention, the step of detecting whether the first image has color cast includes: performing channel separation on the first image in the Lab color space to obtain corresponding L channel image, a channel image and b channel image; respectively calculating the color means of the a channel image and the b channel image of the first image as the first color mean and the second color mean; determining the color cast value of the first image according to the first color mean and the second color mean; and judging whether the first image has color cast and determining the type of color cast based on the first color mean, the second color mean and the color cast value.

[0016] Optionally, in the image processing method according to the present invention, the step of judging whether the first image has color deviation and determining the type of color deviation based on the first color mean, the second color mean and the color deviation value includes: if the color deviation value is less than a preset color deviation threshold, determining that the first image has no color deviation; if the color deviation value is greater than the color deviation threshold, determining the color deviation type of the first image based on the size relationship between the first color mean, the second color mean and the preset color mean threshold, the color deviation types including reddish, greenish, bluish and yellowish.

[0017] Optionally, in the image processing method according to the present invention, the step of determining whether the first image requires color correction includes: performing channel separation on the first image in the RGB color space to obtain corresponding R channel image, G channel image and B channel image; respectively calculating the pixel mean of the R channel image, G channel image and B channel image of the first image as the first channel mean, the second channel mean and the third channel mean; and determining whether the first image needs to reduce the saturation of the color corresponding to its color cast type to correct the color through the first channel mean, the second channel mean and the third channel mean.

[0018] Optionally, in the image processing method according to the present invention, the step of performing a color subtraction operation on the first image to obtain a second image includes: converting the first image from an RGB color space to an HSV color space, and performing channel separation to obtain corresponding H channel images, S channel images, and V channel images; determining a channel component value domain of a color corresponding to the color cast type of the first image, the channel component value domain including an H channel component value range and an S channel component value range; for each pixel in the first image, if the value corresponding to the pixel in the H channel image is within the H channel component value range, and the value corresponding to the pixel in the S channel image is within the S channel component value range, then subtracting a preset pixel threshold from the value corresponding to the pixel in the H channel image to obtain the corresponding second image.

[0019] Optionally, the image processing method according to the present invention also includes: when the first image does not have color cast or does not require color correction, using the first image as the second image; sharpening the second image; and cropping the sharpened second image according to a preset size standard to obtain a final image.

[0020] According to another aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for execution by the at least one processor, and the program instructions include instructions for executing the image processing method as described above.

[0021] According to another aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the image processing method as described above.

[0022] According to the image processing scheme of the present invention, the image to be processed is processed based on the lighting environment of the image to be processed to obtain a first image, and it is detected whether the first image has color cast. If so, it is determined whether the first image needs color correction. If necessary, a color reduction operation is performed on the first image to obtain a second image, thereby realizing refined image retouching.

[0023] In the above technical solution, the L channel of the Lab color space is introduced, and the lighting environment of the image to be processed is determined through the pre-generated brightness conditions, and the brightness is adjusted accordingly according to the lighting environment to obtain the first image, eliminating the overexposure and dark effects caused by the environment. Among them, the brightness conditions determined by cluster analysis do not require excessive manual annotation and intervention in the generation process.

[0024] When detecting color cast, the color distribution and occupancy of the a channel image and the b channel image of the first image in the Lab color space are analyzed to determine whether there is color cast and the type of color cast. Considering the situation of false detection of color cast due to environmental factors, the color information of the R channel, G channel and B channel in the RGB color space is used to further determine whether color correction is required when the first image has color cast, distinguish the overall color of the image and the image with color cast, and subtract the color of the first image that needs color correction to obtain the second image, achieving the purpose of correcting color cast without changing the overall color matching and composition of the image. In addition, the second image can be sharpened and cropped to obtain the final image, thereby increasing clarity and displaying it at an appropriate size. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.

[0026] Figure 1 A block diagram of a computing device 100 according to an embodiment of the present invention is shown; and

[0027] Figure 2 FIG. 2 is a flowchart of an image processing method 200 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0029] Figure 1 FIG. 1 is a block diagram showing a structure of a computing device 100 according to an embodiment of the present invention.

[0030] like Figure 1 As shown, in a basic configuration 102, computing device 100 typically includes system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and system memory 106.

[0031] Depending on the desired configuration, the processor 104 may be any type of processing, including but not limited to: a microprocessor (UP), a microcontroller (UC), a digital information processor (DSP), or any combination thereof. The processor 104 may include one or more levels of cache, such as a primary cache 110 and a secondary cache 112, a processor core 114, and registers 116. An example processor core 114 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 may be used with the processor 104, or in some implementations, the memory controller 118 may be an internal part of the processor 104.

[0032] Depending on the desired configuration, system memory 106 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some implementations, application 122 may be arranged to execute instructions on the operating system by one or more processors 104 using program data 124.

[0033] Computing device 100 also includes storage 132 , which includes removable storage 136 and non-removable storage 138 .

[0034] The computing device 100 may also include a storage interface bus 134. The storage interface bus 134 enables communication from storage devices 132 (e.g., removable storage 136 and non-removable storage 138) to the basic configuration 102 via the bus / interface controller 130. At least a portion of the operating system 120, the application 122, and the program data 124 may be stored on the removable storage 136 and / or the non-removable storage 138, and when the computing device 100 is powered on or the application 122 is to be executed, it is loaded into the system memory 106 via the storage interface bus 134 and executed by one or more processors 104.

[0035] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.

[0036] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or its changes can be performed in the signal in a manner that encodes information. As non-limiting examples, communication media can include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable medium used herein can include both storage media and communication media.

[0037] The computing device 100 can be implemented as a personal computer including a desktop computer and a notebook computer configuration. Of course, the computing device 100 can also be implemented as a part of a small-sized portable (or mobile) electronic device, which can be a cellular phone, a digital camera, a personal digital assistant (PDA), a personal media player device, a wireless network browsing device, a personal head-mounted device, an application-specific device, or a hybrid device that can include any of the above functions. It can even be implemented as a server, such as a file server, a database server, an application server, and a WEB server. The embodiments of the present invention are not limited to this.

[0038] In an embodiment according to the present invention, the computing device 100 is configured to execute the image processing method 200 according to the present invention. The application 122 arranged on the operating system includes a plurality of program instructions for executing the image processing method 200 of the present invention, and these program instructions can instruct the processor 104 to execute the image processing method 200 of the present invention, so that the computing device 100 processes the image by executing the image processing method 200 of the present invention.

[0039] Figure 2 FIG. 2 is a flowchart of an image processing method 200 according to an embodiment of the present invention. The image processing method 200 may be executed in a computing device (eg, the aforementioned computing device 100).

[0040] like Figure 2 As shown, the method 200 starts at step S210. In step S210, the image to be processed is processed according to the lighting environment of the image to be processed to obtain a first image.

[0041] According to one embodiment of the present invention, before acquiring the first image, method 200 further includes determining the lighting environment of the image to be processed, and the lighting environment of the image to be processed can be determined in the following manner. In this embodiment, the image to be processed is first converted from the RGB color space to the Lab color space, and channel separation is performed to obtain corresponding L channel images, a channel images, and b channel images.

[0042] Here, the concept of color space is mentioned. For easy understanding, the following is a brief description of the RGB color space, Lab color space, XYZ color space, and HSV color space that are commonly used in image processing technology.

[0043] The RGB color space is based on the three basic colors R (Red), G (Green) and B (Blue). When an image is read in RGB form, the corresponding RGB three-channel image will be obtained, including the R channel image, the G channel image and the B channel image. The Lab color space is a color-opponent space, L represents brightness, a and b represent color-opponent dimensions, a represents the component from green to red, and b represents the component from blue to yellow.

[0044] If you need to get an image in Lab color space, you must first get the image in RGB color space, then convert it from RGB color space to XYZ color space, and finally convert it from XYZ color space to Lab color space. XYZ color space is a new colorimetric system based on RGB system, using three imaginary primary colors X, Y, and Z.

[0045] The HSV color space is a color space created based on the intuitive characteristics of color. It measures the color of an image based on H (Hue), S (Saturation), and V (Value). Images can be directly converted from the RGB color space to the HSV color space without the need for secondary conversion in other color spaces.

[0046] Of course, similar to the RGB color space, for images in the Lab color space, XYZ color space, and HSV color space, through channel separation processing, you can get the single-channel images in the corresponding color space. Each single-channel image has the same size and contains the same number of pixels, which generally corresponds to a two-dimensional array. The conversion methods and other details between the above-mentioned color spaces can be determined by referring to the conventional methods in image processing technology, and will not be repeated here.

[0047] In this embodiment, the image to be processed is first converted from the RGB color space to the XYZ color space, and then converted from the XYZ color space to the Lab color space, and channel separation is performed to obtain corresponding L channel images, a channel images, and b channel images.

[0048] Then, the average brightness and maximum brightness of the L channel image of the image to be processed are calculated. For the L channel image, the value of each pixel therein represents the brightness value at the position. Based on this, the average value and maximum value of all pixels in the L channel image of the image to be processed are obtained as the average brightness and maximum brightness of the L channel image.

[0049] If the average brightness and the maximum brightness meet the first brightness condition, the lighting environment of the image to be processed is determined to be a shadow environment; if the maximum brightness meets the second brightness condition, the lighting environment of the image to be processed is determined to be a high light environment; if the average brightness and the maximum brightness meet neither the first brightness condition nor the second brightness condition, the lighting environment of the image to be processed is determined to be a normal environment.

[0050] According to one embodiment of the present invention, the first brightness condition and the second brightness condition are pre-generated in the following manner: First, images are collected to obtain a first test image set and a second test image set, wherein the first test image set includes a plurality of images that need to adjust the brightness, and the second test image set includes a plurality of images that do not need to adjust the brightness.

[0051] D1 and D2 represent the first test image set and the second test image set respectively. D1 includes n images that need to adjust the brightness. 11 ,I 12 ,……,I 1n If D1={I 11 ,I 12 ,…,I 1n}, D2 includes n images that do not need to adjust brightness, with I 21 ,I 22 ,……,I 2n If we express it, we can get D2={I 21 ,I 22 ,…,I 2n}.

[0052] Then, the average brightness and maximum brightness of each image in the first test image set and the second test image set under the L channel are calculated respectively, and a first average brightness set, a first maximum brightness set, a second average brightness set, and a second maximum brightness set are generated accordingly.

[0053] In this embodiment, the L channel images of each image in the first test image set and the second test image set in the Lab color space are first obtained to generate the first L channel image set and the second L channel image set, and then the average brightness and maximum brightness of each L channel image in the first L channel image set and the second L channel image set are calculated, and the first average brightness set, the first maximum brightness set, the second average brightness set and the second maximum brightness set are generated accordingly.

[0054] LD1 and LD2 represent the first L-channel image set and the second L-channel image set respectively. LD1 includes image I 11 ,I 12 ,……,I 1n The L channel image l 11 , l 12 ,……,l 1n, LD2 includes image I 21 ,I 22 ,……,I 2n The L channel image l 21 , l 22 ,……,l 2n , then: LD1={l 11 ,l 12 ,…,l 1n}, LD2={l 21 ,l 22 ,…,l 2n}.

[0055] For each L channel image in LD1, the average brightness and maximum brightness are calculated respectively, so as to generate the first average brightness set Mean1 and the first maximum brightness set Max1 respectively. For each L channel image in LD2, the average brightness and maximum brightness are calculated respectively, so as to generate the second average brightness set Mean2 and the second maximum brightness set Max2 respectively. Then:

[0056]

[0057] Max1={M 11 ,M 12 ,…,M 1n} (2)

[0058]

[0059] Max2={M 21 ,M 22 ,…,M 2n} (4)

[0060] in, M 11 ~M 1n Respectively represent the L channel image l 11 ~l 1n The average brightness, maximum brightness, M 21 ~M 2n Respectively represent the L channel image l 21 ~l 2n The average brightness and maximum brightness.

[0061] The first average brightness set, the first maximum brightness set, the second average brightness set, and the second maximum brightness set are clustered to generate a first brightness condition and a second brightness condition. When clustering Mean1, Max1, Mean2, and Max2, a K-means clustering algorithm may be used, where "K" represents the number of cluster centers.

[0062] For example, with the average brightness as the horizontal coordinate and the maximum brightness as the vertical coordinate, an image can be mapped to a coordinate point. The horizontal coordinate of the coordinate point is the average brightness of the L channel image of the image, and the vertical coordinate is the maximum brightness of the L channel image of the image. Based on this, combined with formulas (1), (2), (3) and (4), the coordinate points of each image in D1 and D2 can be obtained. Since D1 corresponds to n coordinate points and D2 also corresponds to n coordinate points, the two groups of coordinate points are clustered using the K-means clustering algorithm, and the K value is 2 at this time.

[0063] Through cluster analysis, we finally get two cluster centers A1 and A2, where 90% of the n coordinate points corresponding to D1 are classified into the class where A1 is, and 90% of the n coordinate points corresponding to D2 are classified into the class where A2 is. In the class where A1 is, calculate the distance between the horizontal coordinate of each coordinate point belonging to D1 and the horizontal coordinate of the cluster center A1, and find the minimum distance and maximum distance Calculate the distance between the ordinate of each coordinate point belonging to D1 and the ordinate of the cluster center A1, and find the minimum distance M min and the maximum distance M max If the average brightness of the L channel image of an image is to The maximum brightness is within the range of M min To M max If the image is within the range of (whether the endpoint is included in the range depends on the situation), it can be determined that the lighting environment of the image is not a normal environment and brightness adjustment is required.

[0064] By collecting actual data and repeating the above clustering process many times, we can know that:

[0065] If the image is too dark, M min =0,M max =20, that is, when the average brightness is less than 80 and the maximum brightness is less than 20, it can be determined that the lighting environment of the image to be processed is a shadow environment, and the above condition is the first brightness condition;

[0066] If the image is overexposed, M min =255,M max =256, considering that the brightness value range is 0 to 255, that is, there is no need to consider the average brightness. When the maximum brightness is 255, it can be determined that the lighting environment of the image to be processed is a high light environment. The above condition is the second brightness condition.

[0067] According to an embodiment of the present invention, the first image may be acquired in the following manner: In this embodiment, if the lighting environment of the image to be processed is a normal environment, the image to be processed is used as the first image.

[0068] If the lighting environment of the image to be processed is a shadow environment, the image to be processed is brightness-enhanced to obtain a first image. Specifically, when the lighting environment of the image to be processed is a shadow environment, the image to be processed in the RGB color space is channel-separated to obtain corresponding R channel images, G channel images, and B channel images, and the R channel images, G channel images, and B channel images of the image to be processed are brightness-enhanced to obtain new R channel images, G channel images, and B channel images, and they are synthesized to obtain a first image.

[0069] For example, the two-dimensional arrays corresponding to the R channel image, the G channel image, and the B channel image of the image to be processed can be respectively substituted into the relevant formula of Gamma color correction for processing to obtain new channel values ​​after brightness enhancement, and then synthesized to obtain the first image. The formula is as follows:

[0070]

[0071] Among them, N in 、N out They represent the two-dimensional arrays of input and output respectively. min(·) and max(·) represent the minimum and maximum values ​​respectively. γ is the empirical threshold inferred based on the imaging principle, which is used to balance the display and the real color. The value is generally 2.2.

[0072] If the lighting environment of the image to be processed is a high-light environment, the brightness of the high-brightness area in the image to be processed is reduced, and the brightness of the low-brightness area in the image to be processed is increased to obtain the first image.

[0073] The highlights and shadows of the image are mainly reflected in the L channel. Therefore, when the lighting environment of the image to be processed is a high-light environment, the image to be processed in the Lab color space is firstly subjected to channel separation to obtain the corresponding L channel image, a channel image and b channel image. Then, for each pixel in the L channel image of the image to be processed, if the value of the pixel is greater than the first brightness threshold, it is determined that the pixel belongs to the high brightness area, and the value of the pixel is replaced with the first brightness threshold. If the value of the pixel is less than the second brightness threshold, it is determined that the pixel belongs to the low brightness area, and the value of the pixel is replaced with the second brightness threshold. Finally, the L channel image after the replacement process, the a channel image and the b channel image of the image to be processed are synthesized to obtain the first image.

[0074] Then, the process proceeds to step S220 to detect whether the first image has color cast.

[0075] According to one embodiment of the present invention, color cast detection can be performed on a first image in the following manner: First, the first image is channel-separated in a Lab color space to obtain a corresponding L channel image, an a channel image, and a b channel image, and then the color means of the a channel image and the b channel image of the first image are respectively calculated as the first color mean and the second color mean.

[0076] The a channel and b channel of the image reflect the overall color distribution information of the image. The a channel represents the change from green to red, and the b channel represents the change from blue to yellow. The value range of the pixels in the two channels is -128 to 127. For images with color cast, by statistically analyzing the histogram, it can be found that a certain color system presents an abrupt peak.

[0077] To obtain the color mean, first calculate the one-dimensional histograms corresponding to the a-channel image and the b-channel image of the first image. For the a-channel image and the b-channel image, count the number of times 0, 1, 2, ..., 255 appear for all the pixel values ​​(the value range is 0 to 255), and then generate their own one-dimensional histograms, denoted as hist a 、hist b .

[0078] The color mean is calculated based on the above one-dimensional histogram according to the following formula:

[0079]

[0080]

[0081] Among them, d a ,d b Respectively represent the first color mean and the second color mean, w a 、h a Respectively represent the width and height of the one-dimensional histogram of the a channel image, w b 、h b Respectively represent the width and height of the one-dimensional histogram of the b channel image, hist a [i], hist b [i] represents the number of times the pixel with value i appears in the a-channel image and the b-channel image respectively, i represents the chromaticity, that is, the distribution value of the color, i=0~255.

[0082] By a It can be known whether the overall color layout of the first image is red or green, as shown by d b It can be known whether the first image is blue or yellow as a whole. Next, the color deviation value of the first image is determined according to the first color mean value and the second color mean value.

[0083] Specifically, the difference between each single chromaticity and color mean in the one-dimensional histogram corresponding to the a channel image and the b channel image is calculated according to the following formula:

[0084] V ai =(id a -128)·hist a [i] (8)

[0085] V bi =(id b -128)·hist b [i] (9)

[0086] Among them, V a Represents the difference between the single chromaticity i and the first color mean in the one-dimensional histogram corresponding to the a channel image, V b Represents the difference between the single chromaticity i and the second color mean in the one-dimensional histogram corresponding to the b channel image.

[0087] By V a and V b , calculate the chromaticity mean difference of the whole image, and obtain the chromaticity mean difference of the a channel image and the b channel image of the first image respectively, as the first chromaticity mean difference and the second chromaticity mean difference The formula is as follows:

[0088]

[0089]

[0090] The overall color deviation value C of the first image is calculated by the first color mean, the second color mean, the first chromaticity mean difference and the second chromaticity mean difference, as follows:

[0091]

[0092] Finally, based on the first color mean, the second color mean and the color deviation value, it is determined whether the first image has color deviation and the type of color deviation is determined. According to one embodiment of the present invention, it is determined whether the first image has color deviation and the type of color deviation is determined in the following manner. In this embodiment, if the color deviation value is less than a preset color deviation threshold, it is determined that the first image does not have color deviation. If the color deviation value is greater than the color deviation threshold, the color deviation type of the first image is determined based on the size relationship between the first color mean, the second color mean and the preset color mean threshold. The color deviation types include red deviation, green deviation, blue deviation and yellow deviation.

[0093] The color deviation threshold is usually set to 1. If the color deviation value is less than 1, it can be determined that the first image does not have color deviation. If the color deviation value is greater than 1, it can be known that the first image has color deviation, but further judgment is required to determine the type of color deviation. The color mean threshold is usually set to 0. Based on this, the following logical judgment relationship can be obtained:

[0094] When the color deviation value C is greater than 1 and the first color mean d a When it is greater than 0, the first image is reddish;

[0095] When the color deviation value C is greater than 1 and the first color mean d a When it is less than 0, the first image is green;

[0096] When the color deviation value C is greater than 1 and the second color mean d b When it is greater than 0, the first image is yellowish;

[0097] When the color deviation value C is greater than 1 and the second color mean d b When it is less than 0, the first image is blue.

[0098] In step S230, if the first image has color cast, it is determined whether the first image needs color correction. Since the image may be detected to have color cast due to environmental factors, in order to avoid this unreasonable result, the mean value of each channel in the RGB color space can be introduced to assist in determining whether the image really has color cast and needs color correction.

[0099] According to one embodiment of the present invention, it can be determined whether the first image needs color correction in the following manner. First, the image of the first image in the RGB color space is channel-separated to obtain the corresponding R channel image, G channel image and B channel image, and then the pixel point means of the R channel image, G channel image and B channel image of the first image are respectively calculated as the first channel mean, the second channel mean and the third channel mean. Then, the first channel mean, the second channel mean and the third channel mean are used to determine whether the first image needs to reduce the saturation of the color corresponding to its color cast type to correct the color.

[0100] In this embodiment, respectively Indicates the mean of the first channel, the second channel, and the third channel. The following logical judgment relationship can be used to determine whether the saturation of the color corresponding to the color cast type of the first image needs to be reduced:

[0101] (1) When the color cast type of the first image is reddish, if Then reduce the saturation of red in the first image;

[0102] (2) When the color cast type of the first image is green, if Then reduce the saturation of green in the first image;

[0103] (3) When the color cast type of the first image is yellowish, if Then reduce the saturation of yellow in the first image;

[0104] (4) When the color cast type of the first image is blue, if The saturation of blue in the first image is reduced.

[0105] Here, |·| means finding the absolute value.

[0106] Finally, step S240 is executed. If the first image needs color correction, a color reduction operation is performed on the first image to obtain a second image.

[0107] According to an embodiment of the present invention, a color reduction operation may be performed on a first image in the following manner to obtain a second image. In this embodiment, the first image is first converted from an RGB color space to an HSV color space, and channel separation is performed to obtain corresponding H channel images, S channel images, and V channel images, and then the channel component value range of the color corresponding to the color cast type of the first image is determined, and the channel component value range includes an H channel component value range and an S channel component value range.

[0108] When the color cast type of the first image is reddish, the corresponding H and S channel component value ranges are:

[0109] 0 <H hsv <10 or 156 <H hsv <180,43 hsv <255;

[0110] When the color cast type of the first image is green, the corresponding H and S channel component value ranges are:

[0111] 35 <H hsv <77,43 hsv <255;

[0112] When the color cast type of the first image is yellowish, the corresponding H and S channel component value ranges are:

[0113] 26 <H hsv <34,43 hsv <255;

[0114] When the color cast type of the first image is blue, the corresponding H and S channel component value ranges are:

[0115] 100 <H hsv <124,43 hsv <255;

[0116] Among them, H hsv , S​​​​hsv Respectively represent the values ​​of pixels in the H channel image and the S channel image.

[0117] For each pixel in the first image, if the value corresponding to the pixel in the H channel image is within the value range of the H channel component, and the value corresponding to the pixel in the S channel image is within the value range of the S channel component, the value corresponding to the pixel in the H channel image is subtracted from the preset pixel threshold to obtain the corresponding second image. The pixel threshold is generally a value between 30 and 50.

[0118] For example, according to step S230, it is determined that the first image needs color correction, and its color cast type is blue. It is known that a color reduction operation needs to be performed on blue in the HSV color space, and then the corresponding values ​​in the H channel image are greater than 100 and less than 124, and the corresponding values ​​in the S channel image are greater than 43 and less than 255. The pixel points are searched, and the pixel threshold is subtracted from the corresponding values ​​of the found pixel points in the H channel image to obtain the corrected H channel image. After synthesizing the corrected H channel image with the S channel image and the V channel image of the first image into a new image, the new image is converted from the HSV color space to the RGB color space to obtain the second image.

[0119] According to yet another embodiment of the present invention, the method 200 further includes: when the first image does not have color cast or does not require color correction, using the first image as the second image.

[0120] Taking into account the requirements for image clarity and size, according to another embodiment of the present invention, method 200 further includes: sharpening the second image, and cropping the sharpened second image according to a preset size standard to obtain a final image.

[0121] The essence of image sharpening is to make the edges more prominent, visually achieving the effect of improving the overall clarity of the image. In this embodiment, the second image can be blurred by Gaussian filtering, and the original second image and the blurred second image are subtracted to obtain an image area containing only the edges, and then the original second image is added to the image area to obtain the sharpened second image.

[0122] Since the image to be processed is generally obtained by shooting, its size is mostly 3:2 (width: height), so the size of the obtained second image remains unchanged. However, the preset size standard when displaying the image is not necessarily 3:2, it may be 4:3 or other size ratios. In this case, the image should be cropped according to the size standard. For example, when the size standard is 4:3, compared with 3:2, it only needs to be cropped in the width direction, and no cropping is required in the height direction. In the specific processing, in order to keep the central area, the two sides of the sharpened second image can be cropped.

[0123] According to the image processing scheme of the embodiment of the present invention, the image to be processed is processed based on the lighting environment of the image to obtain a first image, and it is detected whether the first image has color cast. If so, it is determined whether the first image needs color correction. If so, a color reduction operation is performed on the first image to obtain a second image, thereby realizing refined image retouching.

[0124] In this technical solution, the L channel of the Lab color space is introduced, and the lighting environment of the image to be processed is determined through the pre-generated brightness conditions. The brightness is adjusted accordingly according to the lighting environment to obtain the first image, eliminating the overexposure and dark effects caused by the environment. Among them, the brightness conditions determined by cluster analysis do not require excessive manual annotation and intervention in the generation process.

[0125] When detecting color cast, the color distribution and occupancy of the a channel image and the b channel image of the first image in the Lab color space are analyzed to determine whether there is color cast and the type of color cast. Considering the situation of false color cast detection due to environmental factors, the color information of the R channel, G channel and B channel in the RGB color space is used to further determine whether color correction is required when the first image has color cast, distinguish the overall color of the image and the image with color cast, and subtract the color of the first image that needs color correction to obtain the second image, so as to achieve the purpose of correcting color cast without changing the overall color matching and composition of the image.

[0126] In addition, the second image can be sharpened and cropped to obtain the final image, thereby increasing clarity and displaying it at an appropriate size. In practical applications, when it is necessary to quickly process a large number of images taken in the studio or on-site, the above-mentioned refined image retouching technology is used to detect and correct them, eliminate the noise caused by external factors, and finally transmit the processed image to the application end, such as displaying the refined image on the web page, to enhance the user's viewing experience and reduce the workload of the staff, especially in the live broadcast scenario, which can accelerate the synchronization update of the website. The above solution is intelligent, time-saving, and low-cost, and has good commercial value and flexibility in the media industry.

[0127] A9 The method as described in any one of A1 to A8, further comprising:

[0128] When the first image does not have color cast or does not need color correction, using the first image as the second image;

[0129] performing a sharpening process on the second image;

[0130] The sharpened second image is cropped according to a preset size standard to obtain a final image.

[0131] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or parts of the method and apparatus of the present invention may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0132] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the image processing method of the present invention according to the instructions in the program code stored in the memory.

[0133] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0134] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0135] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0136] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0137] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0138] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0139] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0140] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0141] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0142] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and didactic purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. An image processing method, comprising: Processing the image to be processed according to the lighting environment of the image to be processed to obtain a first image; detecting whether the first image has color cast; If the first image has color cast, determining whether the first image needs color correction; If the first image needs color correction, a color reduction operation is performed on the first image to obtain a second image, The step of detecting whether the first image has color cast comprises: Perform channel separation on the image of the first image in the Lab color space to obtain corresponding L channel image, a channel image and b channel image; Based on a one-dimensional histogram of an a-channel image and a one-dimensional histogram of a b-channel image of the first image, respectively calculating a first color mean and a second color mean; Determining a color deviation value of the first image according to the first color mean value and the second color mean value; Based on the first color mean, the second color mean and the color deviation value, determining whether the first image has color deviation and determining the color deviation type, including: if the color deviation value is less than a preset color deviation threshold, determining that the first image has no color deviation; if the color deviation value is greater than the color deviation threshold, determining that the first image has color deviation, the color deviation threshold is 1, The color deviation value C is calculated as follows: in, is the first chromaticity mean difference, is the second chromaticity mean difference, d a and d b denote the first color mean and the second color mean respectively, and, where V ai =(i - d a - 128)·hist a [i], V bi =(i - d b - 128)·hist b [i], And, w a and h a Respectively represent the width and height of the one-dimensional histogram of the a channel image, w b and h b Respectively represent the width and height of the one-dimensional histogram of the b channel image, hist a [i] and hist b [i] represents the number of times the pixel with value i appears in the a-channel image and the b-channel image respectively, i represents the chromaticity, that is, the distribution value of the color, i=0~255.

2. The method according to claim 1, further comprising determining a lighting environment of the image to be processed, wherein the step of determining the lighting environment of the image to be processed comprises: The image to be processed is converted from the RGB color space to the Lab color space, and channel separation is performed to obtain corresponding L channel image, a channel image and b channel image; Calculate the average brightness and maximum brightness of the L channel image of the image to be processed; If the average brightness and the maximum brightness meet a first brightness condition, determining that the lighting environment of the image to be processed is a shadow environment; If the maximum brightness satisfies the second brightness condition, it is determined that the lighting environment of the image to be processed is a high-light environment.

3. The method of claim 2, wherein: The first brightness condition and the second brightness condition are pre-generated in the following manner: Collecting images to obtain a first test image set and a second test image set, wherein the first test image set includes a plurality of images that need to adjust brightness, and the second test image set includes a plurality of images that do not need to adjust brightness; Calculate the average brightness and maximum brightness of each image in the first test image set and the second test image set under the L channel respectively, and generate a first average brightness set, a first maximum brightness set, a second average brightness set, and a second maximum brightness set accordingly; A cluster analysis is performed on the first average brightness set, the first maximum brightness set, the second average brightness set, and the second maximum brightness set to generate a first brightness condition and a second brightness condition.

4. The method according to any one of claims 1 to 3, wherein: The step of processing the image to be processed according to the lighting environment of the image to be processed to obtain the first image includes: If the lighting environment of the image to be processed is a shadow environment, enhancing the brightness of the image to be processed to obtain a first image; If the lighting environment of the image to be processed is a high-light environment, reducing the brightness of the high-brightness area in the image to be processed, and increasing the brightness of the low-brightness area in the image to be processed, so as to obtain a first image; If the lighting environment of the image to be processed is a normal environment, the image to be processed is used as the first image.

5. The method of claim 1, wherein: The step of judging whether the first image has color cast and determining the type of color cast based on the first color mean value, the second color mean value and the color cast value comprises: If the color deviation value is greater than the color deviation threshold, the color deviation type of the first image is determined according to the size relationship between the first color mean, the second color mean and a preset color mean threshold, and the color deviation type includes red, green, blue and yellow.

6. The method of claim 1, wherein: The step of determining whether the first image needs color correction comprises: Performing channel separation on the first image in the RGB color space to obtain corresponding R channel image, G channel image and B channel image; Calculating pixel point means of the R channel image, the G channel image, and the B channel image of the first image respectively as the first channel mean, the second channel mean, and the third channel mean; It is determined whether the first image needs to reduce the saturation of the color corresponding to the color cast type to correct the color by using the first channel mean, the second channel mean and the third channel mean.

7. The method of claim 1, wherein: The step of performing a color reduction operation on the first image to obtain a second image comprises: Convert the first image from the RGB color space to the HSV color space, and perform channel separation to obtain a corresponding H channel image, an S channel image, and a V channel image; Determine a channel component value range of a color corresponding to the color cast type of the first image, wherein the channel component value range includes an H channel component value range and an S channel component value range; For each pixel in the first image, if the value corresponding to the pixel in the H channel image is within the value range of the H channel component, and the corresponding value in the S channel image is within the value range of the S channel component, then the value corresponding to the pixel in the H channel image is subtracted from the preset pixel threshold to obtain the corresponding second image.

8. The method of claim 1, further comprising: When the first image does not have color cast or does not need color correction, using the first image as the second image; performing a sharpening process on the second image; The sharpened second image is cropped according to a preset size standard to obtain a final image.

9. A computing device comprising: at least one processor; as well as A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1 to 8.

10. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the method according to any one of claims 1 to 8.

Images