Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a to-be-processed second white light image; obtaining a first pixel mean value of all pixel points on the second white light image in the R channel, a second pixel mean value of the B channel and a third pixel mean value of the G channel; on the basis that the ratio of the first pixel mean value to the second pixel mean value is larger than a first threshold value and the ratio of the first pixel mean value to the third pixel mean value is larger than a second threshold value, saturation values and brightness values of all pixel points, not larger than the red hue threshold value, on the second white light image are adjusted, and a target white light image is obtained; obtaining a plurality of second detail layer images with the same size based on the target white light image; performing weighted fusion on pixel values of all pixel points on each second detail layer image to obtain a target detail layer image; and performing weighted fusion on the target detail layer image and the second white light image to obtain the target image, thereby improving the enhancement effect of the target image.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] Electronic imaging technology is a relatively important core technology in modern medical endoscopes. It can further improve the resolution and real-time performance of endoscopic images. By inserting the lens system of the medical electronic endoscope from the patient's natural cavity or the cavity formed by minimally invasive surgery, the illumination light, i.e., white light, is focused and guided to the target area, i.e., the lesion area, using the light source system, and the reflected light enters the image sensor, i.e., the CMOS sensor, through the lens. The photodiode in the CMOS sensor converts the light signal into a charge signal, which is then amplified by an amplifier and converted into a digital signal by an analog-to-digital converter (ADC). The digital signal is then processed by de-mosaicing, color correction, and noise reduction to generate a high-definition color image, which can be transmitted to the display in real time for the doctor to observe and operate, so that the doctor can observe the lesions in the patient's body more clearly and accurately. At the same time, the host system of the electronic endoscope can also store the high-definition color image for subsequent analysis and recording of the patient's lesion.

[0003] Existing electronic imaging technology is used in electronic endoscopes to obtain images inside the patient's body. Image enhancement technology is usually used to post-process the images to improve image quality (such as enhancing the sharpness and contrast of the image to improve the clarity of human tissues (blood vessels, organ edges, etc.) in the abdominal cavity) so that doctors can intuitively and accurately obtain the pathological conditions in the patient's body. However, since there are fewer red light components in white light, the original image collected by the image sensor is generally green. At present, a white balance algorithm is generally used to process the original image. The pixel values ​​on the R channel of all pixels on the original image are gain-processed with the pixel values ​​on the B channel to achieve an image similar to the red-green-blue balance observable by the human eye. However, in the use scenario of medical endoscopes, if a large area of ​​red objects appears in the collected original image, such as heavy bleeding during surgery, the pixel value of the red part of the blood in the original image will overflow due to its high saturation, that is, the pixel value of the R channel in the original image will be larger as a whole, resulting in a larger red coverage in the original image. As a result, the original image will lose the details of the original red part, and will not be able to display details such as mucosal blood vessels covered by the oversaturated red, resulting in unsatisfactory image display effects, thereby increasing the risk of doctors making misjudgments during surgery due to the inability to observe the details of the mucosal blood vessels. Summary of the invention

[0004] (1) Technical issues to be solved

[0005] The present invention provides an image processing method, device, electronic device and storage medium, which aim to restore the original detail content of mucosal blood vessels covered by supersaturated red by reducing the saturation value and brightness value of a large area covered by supersaturated red on a second white light image to be processed, and improve the enhancement effect of the original detail content.

[0006] (2) Technical solution

[0007] In a first aspect, an embodiment of the present invention provides an image processing method, comprising:

[0008] Acquiring a second white light image to be processed;

[0009] Obtaining a first pixel mean value in the R channel, a second pixel mean value in the B channel, and a third pixel mean value in the G channel of all pixel points on the second white light image;

[0010] Based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold, adjusting the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold to acquire a target white light image;

[0011] Based on the target white light image, acquiring a target detail layer image;

[0012] The target detail layer image and the second white light image are weightedly fused to obtain a target image.

[0013] In a possible implementation, adjusting the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold to obtain a target white light image includes:

[0014] Performing image conversion on the second white light image to obtain a first HSV image;

[0015] Determine a pixel point to be processed whose red hue value on the first HSV image is less than or equal to a red hue threshold;

[0016] Adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image;

[0017] Perform image conversion on the second HSV image to obtain the target white light image.

[0018] In a possible implementation manner, acquiring a target detail layer image based on the target white light image includes:

[0019] Performing denoising processing on the target white light image to obtain a first denoised image;

[0020] Based on the first denoised image, acquiring a plurality of second detail layer images of the same size;

[0021] The pixel values ​​of all pixels on each of the second detail layer images are weightedly fused to obtain a target detail layer image.

[0022] In a possible implementation, acquiring a plurality of second detail layer images of the same size based on the first denoised image includes:

[0023] Performing down-sampling processing on the first denoised image to obtain a plurality of down-sampled images of different sizes;

[0024] Based on the multiple down-sampled images of different sizes, obtaining multiple first detail layer images of different sizes;

[0025] Upsampling the plurality of first detail layer images of different sizes respectively to obtain the corresponding plurality of second detail layer images of the same size;

[0026] The size of the second detail layer image is the same as the size of the second white light image.

[0027] In a possible implementation, the obtaining of a plurality of first detail layer images of different sizes based on the plurality of downsampled images of different sizes includes:

[0028] Performing upsampling processing on the downsampled images of different sizes to obtain corresponding upsampled images of multiple different sizes;

[0029] Pixel differences of all corresponding pixels on each of the down-sampled images and the corresponding up-sampled image at the same size are obtained to obtain the first detail layer images of the plurality of different sizes.

[0030] In a possible implementation manner, acquiring a target detail layer image based on the target white light image includes:

[0031] Performing denoising on the target white light image to obtain a second denoised image;

[0032] performing detail enhancement processing on the second denoised image to obtain a third detail layer image;

[0033] The pixel values ​​of all pixels on the RGB channels of the third detail layer image are weightedly fused to obtain a target detail layer image.

[0034] In a possible implementation manner, obtaining a second white light image to be processed includes:

[0035] Get the original image;

[0036] Based on the original image, acquiring a first white light image having RGB features;

[0037] The first white light image is processed using a white balance algorithm to obtain the second white light image.

[0038] In a possible implementation, the first denoised image is downsampled to obtain a plurality of downsampled images of different sizes, including:

[0039] Step S1: down-sampling the first denoised image to obtain a first down-sampled image;

[0040] Step S2: performing denoising processing on the first down-sampled image to obtain a denoised first down-sampled image;

[0041] Step S3: performing downsampling processing on the first downsampled image after denoising to obtain a second downsampled image, wherein the size of the first downsampled image is different from the size of the second downsampled image;

[0042] Step S4: Repeat steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes.

[0043] In a second aspect, an embodiment of the present invention provides an image processing device, comprising:

[0044] An image acquisition module, used for acquiring a second white light image;

[0045] A pixel mean acquisition module, used to acquire a first pixel mean of all pixels on the second white light image in the R channel, a second pixel mean of the B channel, and a third pixel mean of the G channel;

[0046] A pixel comparison module, used to obtain a comparison result between a ratio of the first pixel mean to the second pixel mean and a first threshold, and a comparison result between a ratio of the first pixel mean to a third pixel mean of the G channel and a second threshold;

[0047] an image determination module, configured to adjust the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold, so as to obtain a target white light image;

[0048] An image enhancement module, used for acquiring a target detail layer image based on the target white light image;

[0049] The image fusion module is used to perform weighted fusion on the target detail layer image and the second white light image to obtain a target image.

[0050] In a possible implementation, the image acquisition module includes

[0051] An image acquisition unit, used for acquiring an original image with RAW features;

[0052] A first image processing unit is used to process the original image with RAW characteristics to obtain a first white light image with RGB characteristics.

[0053] The second image processing unit is used to perform white balance processing on the first white light image to obtain the second white light image.

[0054] In a possible implementation, the image determination module includes

[0055] A first image conversion unit, used for converting the second white light image having RGB characteristics into a first HSV image;

[0056] A pixel point determination unit, used to determine a pixel point to be processed whose red hue value on the first HSV image is less than or equal to a red hue threshold;

[0057] A pixel adjustment unit, used for adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image;

[0058] The second image format conversion module is used to convert the second HSV image into the target white light image with RGB features.

[0059] In a possible implementation, the image enhancement module includes

[0060] A first image denoising unit, configured to denoise the target white light image to obtain a first denoised image;

[0061] A detail image determining unit, configured to obtain a plurality of second detail layer images of the same size based on the first denoised image;

[0062] The detail image processing unit is used to perform weighted fusion on the pixel values ​​of all pixels on each of the second detail layer images to obtain a target detail layer image.

[0063] In a possible implementation manner, the detail image determination unit includes

[0064] An image downsampling processing module, used for performing downsampling processing on the first denoised image to obtain a plurality of downsampled images of different sizes;

[0065] A first detail layer image determination module, configured to obtain a plurality of first detail layer images of different sizes based on the plurality of down-sampled images of different sizes;

[0066] A second detail layer image determination module, configured to perform upsampling processing on the plurality of first detail layer images of different sizes respectively, to obtain the corresponding plurality of second detail layer images of the same size;

[0067] Wherein, the size of the second detail layer image is the same as the size of the second white light image;

[0068] The image downsampling processing module is used to perform downsampling processing on the first denoised image to obtain a plurality of downsampled images of different sizes, and is configured as follows:

[0069] Step S1, down-sampling the first denoised image to obtain a first down-sampled image; Step S2, denoising the first down-sampled image to obtain a denoised first down-sampled image; Step S3, down-sampling the denoised first down-sampled image to obtain a second down-sampled image, wherein the size of the first down-sampled image is different from the size of the second down-sampled image; Step S4, repeating steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes;

[0070] The first detail layer image determination module is used to obtain a plurality of first detail layer images of different sizes based on the plurality of down-sampled images of different sizes, and is configured as follows:

[0071] The downsampled images of each different size are all upsampled to obtain a corresponding plurality of upsampled images of different sizes; pixel differences of all corresponding pixel points on each downsampled image of the same size and the corresponding upsampled image are obtained to obtain the first detail layer images of the plurality of different sizes.

[0072] In a possible implementation, the image enhancement module includes

[0073] A second image denoising unit, used to eliminate noise points of the target white light image to obtain a second denoised image;

[0074] A third detail layer determining unit, configured to perform detail enhancement processing on the second denoised image to obtain the third detail layer image;

[0075] The third detail image processing unit is used to weightedly fuse the pixel values ​​of all pixels on the RGB channels of the third detail layer image to obtain the target detail layer image.

[0076] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0077] In a fourth aspect, an embodiment of the present invention provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor.

[0078] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code, wherein when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0079] (3) Beneficial effects

[0080] In summary, by reducing the saturation value and brightness value of a larger area covered by the oversaturated red on the second white light image to be processed, the original details of the mucosal blood vessels covered by the oversaturated red are restored, and the enhancement effect of the original details is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0082] Figure 1 It is a flow chart of an image processing method of the present invention.

[0083] Figure 2 is another flow chart of an image processing method of the present invention.

[0084] Figure 3 is another flow chart of an image processing method of the present invention.

[0085] Figure 4 is another flow chart of an image processing method of the present invention.

[0086] Figure 5 is another flow chart of an image processing method of the present invention.

[0087] Figure 6 is another flow chart of an image processing method of the present invention.

[0088] Figure 7 is another flow chart of an image processing method of the present invention.

[0089] Figure 8 is another flow chart of an image processing method of the present invention.

[0090] Fig. 9 It is a structural schematic diagram of an invented image processing device.

[0091] Fig.10 It is a display effect diagram of the original image of the present invention.

[0092] Fig.11 It is a display effect diagram of the target image of the present invention.

[0093] In the figure:

[0094] 10-image acquisition module; 20-pixel mean acquisition module; 30-pixel comparison module; 40-image determination module; 50-image enhancement module; 60-image fusion module. DETAILED DESCRIPTION

[0095] The following is a further detailed description of the embodiments of the present invention in conjunction with the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the described embodiments, and covers any modification, replacement and improvement of parts, components and connection modes without departing from the spirit of the present invention.

[0096] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0097] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0098] The image processing method of the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster of multiple server groups. Any electronic device can implement the image processing method of the embodiment of the present disclosure by a processor calling a computer-readable instruction stored in a memory.

[0099] Figure 1 A flow chart of an image processing method of the present invention is shown as follows: Figure 1 As shown, the image processing method of the disclosed embodiment of the present invention may include the following steps S10-S50.

[0100] Step S10: Acquire a second white light image to be processed.

[0101] In a possible implementation, the second white light image to be processed may be an original image directly acquired by controlling a white light camera device of a medical endoscope through an electronic device to acquire an image of a target object, or may be an original image processed by an image processing device. For example, the second white light image may be an original image after image processing after a real-time image data stream acquired by a camera of a medical endoscope, wherein the target object may be a mucosal blood vessel with a large area covered by blood, i.e., a mucosal blood vessel with the original red detail texture lost. The mucosal blood vessel may be a gastric antral mucosal blood vessel, an oral mucosal blood vessel, a rectal mucosal blood vessel, or a nasal mucosal blood vessel, etc. Therefore, the second white light image is an image to be processed with RGB features that is covered with a large area of ​​oversaturated red and cannot clearly display the original red detail content of the mucosal blood vessel (e.g., Fig.10 As shown), of course, due to the different parameters of the medical endoscope camera, the data format of the original image directly obtained will also be different. For example, the camera device of some existing medical endoscopes is based on the COMS image sensor to collect images of the target object, resulting in the image data of the original image obtained by the COMS image sensor being a BAYER array image, i.e., a RAW image, which cannot be directly visualized on the terminal display. Therefore, the format of the RAW image needs to be processed accordingly to obtain a second white light image with RGB characteristics, thereby meeting the visualization requirements.

[0102] like Figure 2 As shown, the method for acquiring the second white-light image to be processed according to the embodiment disclosed in the present invention includes the following steps S110 - S130 .

[0103] S110: Acquire an original image.

[0104] In a possible implementation, the CMOS image sensor in the camera of the medical endoscope is used to capture the image of the target object to obtain an original image. The image data of the original image is a BAYER array image, that is, a RAW image, as shown in the following table:

[0105]

[0106] The letter in each box in the table represents a pixel on the RAW image, where B represents the pixel value of the blue channel, Gr represents the pixel value of the green channel in odd rows, Gb represents the pixel value of the green channel in even rows, and R represents the pixel value of the red channel. Therefore, the pixels on the RAW image only have the pixel value of one of the R, G, and B channels, that is, there is no pixel that has the pixel value of the RGB channels at the same time, that is, the RGB feature.

[0107] S120. Based on the original image, obtain a first white-light image having RGB features.

[0108] In a possible implementation, since the original image cannot be directly displayed due to its own format and cannot meet the needs of subsequent image processing, it is necessary to use a color interpolation algorithm to process the pixel values ​​of all pixels on the R, G, and B channels of the original image to obtain a first white light image with RGB characteristics, that is, all pixels on the first white light image have pixel values ​​of RGB channels at the same time. Specifically, the color interpolation algorithm includes a bilinear interpolation method or a neighboring pixel average method. Exemplarily, this embodiment uses the neighboring pixel average method to process the pixel values ​​of all pixels on the original image, and arranges the pixel values ​​according to the distribution of the pixel values ​​in the above table, and uses the corresponding calculation formula to respectively process the pixels with R characteristics, the pixels with Gr characteristics, and the pixels with G characteristics. The pixel points with Gb features and the pixel points with B features are processed so that the pixel points have RGB features. Specifically, except for the pixels in the 1st row and the 1st column and the K+1th row and the K+1th column, the pixels in other rows or columns have 8 adjacent pixels, wherein the adjacent pixel average method is similarly processed for the pixel points with R features and the pixel points with B features. For example, the pixel point with R features in the 3rd row and the 3rd column is represented as R(3,3), and the pixel value of the R channel of the pixel point at this position is the pixel value of R(3,3), and the pixel value of the G channel of the pixel point is the average of the values ​​of the adjacent 4 pixels with G channel features, namely G(3,3)=1 / 4*[ =Gr(3,2)+Gr(3,4)+Gb(2,3)+Gb(4,3)]. Similarly, the pixel value of the B channel of this pixel is the average of the values ​​of its four neighboring pixels with B channel characteristics, that is, B(3,3)=1 / 4*[B(2,2)+B(2,4)+B(4,2)+B(4,4)]. Similarly, taking the pixel with B characteristics in the 4th row and 4th column as an example, it is represented as B(4,4). The pixel value of the B channel of the pixel at this position is the pixel value of B(4,4), and the pixel value of the R channel of this pixel is the average of the values ​​of its four neighboring pixels with R channel characteristics, R(4,4)=1 / 4*[R(3,3)+R(3,5)+R(5,3)+R(5 ,5)], the pixel value of the G channel of the pixel is the average of the values ​​of the four adjacent pixels with G channel characteristics, that is, G(4,4)=1 / 4*[Gb(4,3)+Gb(4,5)+Gr(3,4)+Gr(5,4)], and the adjacent pixel average method is similar for the pixel points with Gb characteristics and the pixel points with Gr characteristics. For example, the pixel point with Gr characteristics in the 3rd row and 4th column is represented as Gr(3,4), and the pixel value of the G channel of the pixel at this position is the pixel value of Gr(3,4), and the pixel value of the R channel of the pixel is the average of the values ​​of the two adjacent pixels with R channel characteristics, that is, R(3,4)=1 / 2*[R(3,3)+R(3,5)], the pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(3,4)=1 / 2*[B(2,4)+B(4,4)]. Taking the pixel with Gb characteristics at the 4th row and the 3rd column as an example, it is characterized as Gb(4,3). The pixel value of the G channel of the pixel at this position is the pixel value of Gb(4,3), and the pixel value of the R channel of the pixel is the average of the values ​​of the two adjacent pixels with R channel characteristics, that is, R(4,3)=1 / 2*[R(3,3)+R(5,3)]. The pixel value of the B channel of the pixel is the average of the values ​​of the two adjacent pixels with B channel characteristics, that is, B(4,3)=1 / 2*[B(4,2)+B(4,4)]. Therefore, the pixel values ​​of all pixels on the first white light image have corresponding RGB characteristics. ,

[0109] S130: Process the first white light image using a white balance algorithm to obtain the second white light image.

[0110] In a possible implementation, since the first white light image with RGB features obtained after being processed by the color interpolation algorithm has the problem of white misalignment, that is, the overall image display effect is greenish, in order to meet the subsequent display effect of the target image, it is necessary to use a white balance algorithm to perform gain processing on the R channel pixel values, G channel pixel values, and B channel pixel values ​​of all pixels on the first white light image to obtain the second white light image. Similarly, the second white light image also has RGB features. For example, the corresponding formulas Rwb=R*Rgain, Gwb=G*Ggain, Bwb=B*Bgain, wherein Rgain, Ggain, and Bgain are respectively the red channel gain coefficient, the green channel gain coefficient, and the blue channel gain coefficient, and the value range is usually 0-10. Rwb, Gwb, and Bwb are respectively the pixel values ​​corresponding to the R channel, the G channel, and the B channel on the second white light image. Since the pixel values ​​of the image are usually integers ranging from 0 to 255, the value ranges of Rwb, Gwb, and Bwb should be adjusted to 0 to 1023 to prevent the loss of pixel value data on the second white light image.

[0111] S20, obtaining a first pixel mean value in the R channel, a second pixel mean value in the B channel, and a third pixel mean value in the G channel of all pixel points on the second white light image.

[0112] In a possible implementation, in order to determine whether the saturation value and the brightness value of all pixels on the second white light image need to be adjusted, it is necessary to judge whether the pixel value on the R channel of the second white light image is oversaturated. In this embodiment, a mean algorithm is used to perform mean processing on the pixel values ​​of all pixels on the second white light image on the R channel, the B channel, and the G channel to obtain the corresponding first pixel mean, second pixel mean, and third pixel mean. The use of the mean algorithm can reduce the calculation complexity. For example, the corresponding mean formula is used. Calculate the corresponding pixel means on the R channel, B channel, and G channel respectively, where R mean , G mean , B mean are the corresponding first pixel mean, second pixel mean and third pixel mean respectively, N is the total number of all pixels on the second white light image, R wb , G wb , B wb are the pixel values ​​on the R channel, G channel, and B channel corresponding to all pixels on the second white light image.

[0113] S30. Based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold, the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold are adjusted to obtain a target white light image.

[0114] In a possible implementation, for example, the oversaturation judgment condition for the pixel value of the R channel on the second white light image is that the R mean With G mean The ratio is greater than the first threshold A1. And R mean With B mean The ratio is greater than the second threshold value A2, that is Among them, R mean Much larger than G mean With B mean , and the value range of A1 and A2 is 1-255. Specifically, the value of A1 is 1.2 and the value of A2 is 1.2. Conversely, if or , it is determined that the pixel values ​​of the R channel on the second white light image are not oversaturated, and the second white light image is directly output as the target image. Based on the judgment result that the ratio of the first pixel mean to the second pixel mean is greater than the first threshold value and the ratio of the first pixel mean to the third pixel mean is greater than the second threshold value, it is identified that the pixel values ​​of the R channel on the second white light image are oversaturated, so it is necessary to adjust the saturation value and brightness value of the oversaturated red covered area on the second white light image to reduce the saturation value and brightness value of all pixels on the oversaturated red covered area, so as to improve the enhancement effect of the original red detail content on the second white light image, and the original detail content refers to the detail layer image on the second white light image.

[0115] like Figure 3 As shown, the method of adjusting the saturation value and brightness value of all pixels on the second white light image that are not greater than the red hue threshold to obtain a target white light image includes the following steps S310-S340.

[0116] S310: Perform image conversion on the second white light image to obtain a first HSV image.

[0117] In a possible implementation, in order to restore the original red details on the second white light image with a large area covered by oversaturated red, it is necessary to convert the second white light image, i.e., the image to be processed with RGB features, into a first HSV image, so that the red hue value of the first HSV image can be screened later. Specifically, first, the formula r=R wb / 255, g=G wb / 255, b=3 wb / 255 Normalize the pixel values ​​of all pixels on the second white light image on the corresponding RGB channels to obtain the values ​​of the intermediate variables r, g, and b, where R wb , G wb With B wb They are respectively represented as the pixel values ​​of the corresponding R channel, G channel and B channel on the second white light image. Then, formula (1), formula (2) and formula (3) are used in turn: v = max (r, g, b); formula (1)

[0118]

[0119] Get the corresponding intermediate variables v, s, and h respectively. Since the pixel values ​​of the image are integers of 0-255, and the intermediate variables v and s are floating-point values, i.e., non-integer values ​​with decimal points, it is necessary to use anti-normalization processing on the v and s values, i.e., use the formula S=s*255 and V=v*255 to calculate the corresponding S value, i.e., the saturation value, and the V value, i.e., the brightness value, respectively. Since the value range of the intermediate variable h is 0-360, use the formula The H value, i.e., the red hue value, is calculated to ensure that the red hue value, saturation value, and brightness value of all pixels of the first HSV image are within the integer range of 0-255 for storage and screening.

[0120] S320: Determine the pixel points to be processed whose red hue values ​​on the first HSV image are less than or equal to the red hue threshold.

[0121] In one possible implementation, in order to facilitate the adjustment of the red hue value of the second white light image so as to better restore the original red details of the mucosal blood vessels on the second white light image with a large area covered by oversaturated red, it is necessary to compare the red hue values ​​of all pixels on the first HSV image with the red hue threshold, and screen out all pixels whose red hue values, i.e., H values, are less than or equal to the red hue threshold. Specifically, the red hue threshold is 0-20, and exemplarily, it is 15.

[0122] S330 , adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image.

[0123] In a possible implementation, the formula S′=S*G is used. S Perform gain processing on the S value, where Gs is the gain coefficient of all pixels on the first HSV image on the S channel, ranging from 0 to 10, using the formula Normalize the V values ​​of all pixels on the first HSV image, where G V is an intermediate variable for normalizing the V value on the first HSV image, and then using the formula V t =G r *255 performs denormalization on the normalized V value, thereby adjusting the V value of all pixels on the first HSV image to obtain a second HSV image, where S′ and V′ represent the saturation value and brightness value of the second HSV image respectively.

[0124] S340: Perform image conversion on the second HSV image to obtain the target white light image.

[0125] In a possible implementation, in order to obtain the subsequent target detail layer image, the second HSV image is format converted to obtain a target white light image with RGB features to meet the needs of subsequent image processing. Exemplarily, the formulas H″=H′*2, S″=S′ / 255, and V″=V′ / 255 are used to normalize all pixels on the second HSV image to obtain the values ​​of intermediate variables H″, S″ and V″, where H′, S′ and V′ are the red hue value, saturation value and brightness value of all pixels on the second HSV image, respectively. H′ is a red hue value less than or equal to 15. Then, formulas (4), (5), (6), (7) and (8) are used respectively.

[0126] p=V″*(1-S″); Formula (4)

[0127] q=V″*(1-f*S″); Formula (5)

[0128] t=V″*(1-(1-f)*S″); Formula (6)

[0129]

[0130] Get the corresponding intermediate variables h respectively i , f, p, q and t values, and based on the above intermediate variable h i , f, p, q and t values ​​according to formula (9)

[0131]

[0132] The corresponding intermediate variables r1, g1, and b1 values ​​are obtained respectively, and then based on the intermediate variables r1, g1, and b1 values, the corresponding formulas R1=r1*255, G1=g1*255, and B1=b1*255 are used to perform inverse normalization processing to obtain the corresponding R1, G1, and B1 values, thereby obtaining the target white light image with RGB features.

[0133] By adjusting the saturation value and the brightness value at the same time, not only can the clarity of the oversaturated red coverage area on the entire second white light image be improved by reducing the saturation of the oversaturated red coverage area on the entire second white light image, but also the brightness of the oversaturated red coverage area on the entire second white light image can be reduced, thereby avoiding red overflow due to excessively high brightness values, thereby affecting the enhancement effect of the second white light image.

[0134] S40: Acquire a target detail layer image based on the target white light image.

[0135] In one possible implementation, in order to improve the restoration effect of the original detail content of the mucosal blood vessels in the area covered by oversaturated red, it is necessary to extract the detail layer image of the target white light image and enhance the lost red detail content. Exemplarily, a pyramid filtering fusion algorithm or a color channel fusion algorithm can be used to extract and enhance the detail layer image of the target white light image.

[0136] like Figure 4 As shown, based on the target white light image, a method for extracting and enhancing the detail layer image of the target white light image by using a pyramid filtering fusion algorithm to obtain the target detail layer image includes the following steps S410-S430.

[0137] S410: Perform denoising processing on the target white light image to obtain a first denoised image.

[0138] In a possible implementation, since the pyramid filter fusion algorithm can extract relatively rich detail content on the target white light image to be processed, and thus can be used to enhance the pixel value of the detail content to restore the lost red detail content covered by the oversaturated red, it is necessary to remove the noise on the target white light image that affects the subsequent rich detail information of the red detail content, so as to prevent the noise from adversely affecting the enhancement effect of the subsequent red detail content. Specifically, before processing the noise points on the target white light image, first determine the filtering algorithm for processing the noise points and construct the filtering kernel of the filtering algorithm. The filtering kernel can be the filtering kernel of the Gaussian filtering algorithm. The Gaussian filtering algorithm is used to filter the pixel values ​​of the RGB channels of all pixels on the target white light image to obtain the first denoised image, that is, the corresponding formula is used. Convolution operations are performed on the R channel pixel values, the G channel pixel values, and the B channel pixel values ​​on the target white light image to obtain the first denoised image, where CoreGaussian is the selected filter kernel. is a convolution operator, R1, G1, and B1 are the pixel values ​​on the corresponding RGB channels of the target white light image, and R′, G′, and B′ are the pixel values ​​on the corresponding RGB channels of the first denoised image.

[0139] S420: Based on the first denoised image, obtain a plurality of second detail layer images of the same size.

[0140] In one possible implementation, in order to improve the restoration effect of the original detail content of the mucosal blood vessels in the area covered by the oversaturated red color, it is necessary to extract and enhance the original detail content of the oversaturated red color covered area on the first denoised image, that is, the detail layer image that can reflect the original detail content. Because the original detail content features expressed by detail layer images of different sizes are not exactly the same, before obtaining multiple detail layer images of the same size, it is necessary to first obtain multiple detail layer images of different sizes, so that the different detail contents of the original detail content of the oversaturated red color covered area on the first denoised image can be retained, thereby improving the restoration effect of the original detail content.

[0141] like Figure 5 As shown, based on the first denoised image, the method for obtaining a plurality of second detail layer images of the same size includes steps S4210-S4230.

[0142] S4210: Down-sample the first denoised image to obtain a plurality of down-sampled images of different sizes.

[0143] In a possible implementation, before obtaining a plurality of second detail layer images of the same size, the size of the first denoised image needs to be reduced multiple times to obtain a plurality of detail layer images of different sizes. The size reduction of the first denoised image may be processed by a multi-scale algorithm or other existing algorithms such as a downsampling algorithm. For example, the size of the first denoised image is reduced by 1 / 4 downsampling processing, that is, half of the pixel value points on all row data and half of the pixel points on all column data on the first denoised image are removed to reduce the size of the first denoised image. Specifically, the pixel points of the first denoised image are shown in the following table.

[0144]

[0145] Remove the 2nd row, 4th row, 6th row, 8th row until the last even row and the 2nd column, 4th column, 6th column, 8th column until the last even column. Of course, you can also remove the 1st row, 3rd row, 5th row, 7th row until the last odd row and the 1st column, 3rd column, 5th column, 7th column until the last odd column. Exemplarily, the 1 / 4 downsampling processing algorithm is used to remove the even rows and even columns of the first denoised image Id1 to reduce the size of the image, and obtain the first down-sampled image Id2. The pixels of the first down-sampled image Id2 are shown in the following table

[0146]

[0147] Wherein, M=2N-1, N is the number of rows or columns, M is the odd-numbered row or column where the pixel points of the first denoised image are located, and then the first down-sampled image Id2 is denoised by using the same filter core in the Gaussian filter algorithm as that for denoising the target white light image to obtain the corresponding second denoised image, and then the second denoised image is again down-sampled by using a 1 / 4 down-sampling method to reduce the size of the second denoised image to obtain a second down-sampled image Id3. Similarly, the above steps are repeated three times in sequence to finally obtain a third down-sampled image Id4, a fourth down-sampled image Id5 and a fifth down-sampled image Id6. It should be noted that the number of down-sampling processes is n times, the number of denoising processes on the down-sampled images is n-1 times, and n is a positive integer, and its value range can be set according to actual needs. In this embodiment, the value of n is 6.

[0148] like Figure 6 As shown, the method of downsampling the first denoised image to obtain a plurality of downsampled images of different sizes includes steps S1-S4

[0149] S1. Down-sample the first denoised image to obtain a first down-sampled image.

[0150] S2. Perform denoising on the first down-sampled image to obtain a denoised first down-sampled image.

[0151] S3. Perform down-sampling processing on the first down-sampled image after denoising to obtain a second down-sampled image, wherein the size of the first down-sampled image is different from the size of the second down-sampled image.

[0152] S4. Repeat steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes.

[0153] In a possible implementation, the first denoised image is recorded as Id1, and the first denoised image is downsampled to obtain a first downsampled image after size reduction, which is marked as Id2. In order to further eliminate the noise on the rich detail information on the target white light image that affects the subsequent red detail content, the first downsampled image Id2 is further denoised to obtain the first denoised downsampled image Id2. In order to further obtain the detail content on the first denoised downsampled image Id2, the first denoised downsampled image Id2 is downsampled again to obtain a second downsampled image Id3 with a further size reduction. The second downsampled image Id3 is denoised and downsampled in turn, and the process is repeated multiple times to obtain multiple downsampled images of different sizes, such as the third downsampled image Id4, the fourth downsampled image Id5, and the fifth downsampled image Id6.

[0154] S4220. Based on the multiple down-sampled images of different sizes, obtain multiple first detail layer images of different sizes.

[0155] In one possible implementation, in order to ensure that all detail contents on the oversaturated red covered area on the target white light image can be fully acquired, the detail contents on the downsampled images of different sizes should be processed accordingly, so as to acquire the detail features on each downsampled image of different sizes and obtain the corresponding first detail layer images of different sizes.

[0156] like Figure 7 As shown, based on the multiple down-sampled images of different sizes, the method for obtaining multiple first detail layer images of different sizes includes steps S4221-S4222

[0157] S4221. Perform upsampling processing on the downsampled images of different sizes to obtain corresponding upsampled images of multiple different sizes.

[0158] In a possible implementation, in order to obtain detail features on downsampled images of different sizes, the sizes of the downsampled images of different sizes should be enlarged to obtain corresponding images of corresponding sizes after size enlargement, and then the detail features on the downsampled images of different sizes can be obtained. Exemplarily, a 4-fold upsampling processing algorithm is used to enlarge the size of each downsampled image to obtain a corresponding plurality of upsampled images. Specifically, the corresponding pixel value can be inserted between the pixel values ​​on all row data and the pixel values ​​on all column data on each upsampled image, as shown in the following table.

[0159]

[0160] Corresponding pixel values ​​are inserted between the pixel values ​​on all column data and the pixel values ​​on all row data on each upsampled image, wherein the corresponding pixel value can be obtained by using the adjacent pixel method. Specifically, taking the blank pixel point inserted in the first column of the second row in the above table as an example, the average value of the pixel value of the first column of the first row and the pixel value of the first column of the second row is taken as the pixel value of the blank pixel point, that is, Taking the blank pixel inserted in the second column of the second row in the above table as an example, the average of the pixel values ​​of the first column of the first row, the second column of the first row, the first column of the second row, and the second column of the second row is taken as the pixel value of the blank pixel. Take the blank pixel inserted in the second column of the first row in the above table as an example, the average of the pixel value of the first column of the first row and the pixel value of the first column of the second row is taken as the pixel value of the blank pixel. Where I is the downsampled image, I(x, y) is the pixel value on the xth row and yth column of the downsampled image, I′ is the upsampled image, I′(m, n) is the pixel value on the mth row and nth column of the upsampled image, and after upsampling the first downsampled image Id2, the second downsampled image Id3, the third downsampled image Id4, the fourth downsampled image Id5, and the fifth downsampled image Id6, the corresponding first upsampled image, the second upsampled image, the third upsampled image, the fourth upsampled image, and the fifth upsampled image are obtained, which are respectively denoted as Iu1, Iu2, Iu3, Iu4, Iu5, that is, the first up-sampled image Iu1 is obtained by up-sampling the first down-sampled image Id2, wherein the first denoised image has the same size as the first up-sampled image Iu1, the first down-sampled image Id2 has the same size as the second up-sampled image Iu2, the second down-sampled image Id3 has the same size as the third up-sampled image Iu3, the third down-sampled image Id4 has the same size as the fourth up-sampled image Iu4, and the fourth down-sampled image Id5 has the same size as the fifth up-sampled image Iu4.

[0161] S4222. Obtain pixel differences of all corresponding pixels on each of the down-sampled images of the same size and the corresponding up-sampled image to obtain the first detail layer images of the multiple different sizes.

[0162] In a possible implementation, in order to obtain detailed features on downsampled images of different sizes, the pixel values ​​of all pixels at the same position on the downsampled image and the upsampled image of the same size should be subjected to difference processing. For example, the formula Ir is used. i =Id i -Iu i To obtain the pixel difference between the downsampled image and the upsampled image obtained by upsampling it, i is the serial number of the downsampled image, Ir i is the i-th first detail layer image, that is, when i=1, the pixel difference between the first denoised image Id1 and the first upsampled image Iu1 can be obtained to obtain the first first detail layer image Ir 1 , and so on. When i=5, the pixel difference between the fourth down-sampled image Id5 and the fifth up-sampled image Iu5 can be obtained to obtain the fifth first detail layer image Ir 5 .

[0163] S4230, performing up-sampling processing on the plurality of first detail layer images of different sizes respectively to obtain the corresponding plurality of second detail layer images of the same size.

[0164] In a possible implementation, in order to ensure that the detail features on the first detail layer images of different sizes can all be retained on the target detail layer image, the sizes of the first detail layer images of different sizes should be correspondingly enlarged so that the sizes of all the first detail layer images of different sizes are converted into the second detail layer images of the same size, so that image fusion processing can be performed to preserve the detail features. Specifically, for the first first detail layer image Ir 1 Perform an upsampling process on the second first detail layer image Ir 2 Perform upsampling twice, and so on, for the fifth first detail layer image Ir 5 The upsampling process is performed five times to ensure that the sizes of all corresponding second detail layer images are the same, wherein the size of the second detail layer image is the same as the size of the second white light image.

[0165] S430: Perform weighted fusion on the pixel values ​​of all pixels on each of the second detail layer images to obtain a target detail layer image.

[0166] In a possible implementation, in order to obtain the target detail layer image, the second detail layer images of the same size should be subjected to image fusion processing to ensure that the detail features on all the second detail layer images are preserved to improve the enhancement effect of the detail content. The formula D=k is used. 1 *Ir 1 ′+k 2 *Ir′ 2 +k 3 *Ir′ 3 +k 4 *Ir′ 4 +k 5 *Ir′ 5 For multiple second detail layer images Ir i ′ is weightedly fused to obtain the target detail layer image D, where k1 to k5 are weight coefficients of the second detail layer images of different sizes, and the value range is -100 to 100. Because the details displayed on images of different sizes are different, such as smaller images have finer texture layers, and larger images have coarser texture layers, by fusing images of different sizes, more finer texture layers can be retained, thereby improving the restoration effect of the original detail content of the oversaturated red coverage area on the target white light image. The k1 to k5 can be obtained using a machine learning method or can be obtained using manually annotated empirical values. Exemplarily, the method of using machine learning to obtain weight coefficients in this embodiment includes the following steps:

[0167] Step 1: acquiring a data set of multiple pairs of image data, wherein the pair of image data includes red oversaturated image data and red normal image data acquired based on photographing the same object, wherein the same object refers to the area where the lesion of the same patient is located, such as the stomach, liver, gallbladder and pancreas, muscle tissue, etc., and the red oversaturated image data refers to image data acquired from the stomach wall in a state of heavy bleeding, and the red normal image data refers to image data acquired from the stomach wall in a normal state;

[0168] Step 2: Divide multiple pairs of data sets into training sets and test sets in a ratio of 8:2;

[0169] Step 3: Use pyramid decomposition to obtain the detailed layer after the fusion of each image in the training set;

[0170] Step 4: Determine the loss function used by the model, which may be one or more of the mean absolute error (MAE), mean square error (MSE), and cross entropy loss function;

[0171] Step 5: Use the model to train the detail layers of the training set images with different numbers of layers, and use the back propagation algorithm to update the model parameters. The model can be Mamba, Transformer, support vector machine, random forest, linear regression and other models;

[0172] Step 6: Apply the weight coefficient obtained from training to the test set for verification and use the same loss function for evaluation. If the difference between the loss function result obtained on the test set and the training set is within ±10%, stop training and obtain the weight coefficient; otherwise, repeat steps 2 to 6 until the difference between the loss function result obtained on the test set and the training set is within ±10%;

[0173] like Figure 8 As shown, based on the target white light image, a method for extracting and enhancing the detail layer image of the target white light image by using a color channel fusion algorithm to obtain a target detail layer image includes the following steps S440-S460.

[0174] S440: Perform denoising processing on the target white light image to obtain a second denoised image.

[0175] In a possible implementation, since the color channel fusion algorithm can extract relatively rich detail content on the target white light image to be processed, and thus can be used to enhance the pixel value of the detail content to restore the lost red detail content covered by the oversaturated red, it is necessary to remove the noise on the target white light image that affects the subsequent rich detail information of the red detail content, so as to prevent the adverse effect of the noise on the subsequent enhancement effect of the red detail content. For example, all pixel points on the RGB channels of the target white light image are denoised using a low-pass filtering algorithm to obtain a second denoised image, that is, the corresponding formula is used. All pixels on the RGB channels of the target white light image are filtered, where R1, G1, and B1 represent the corresponding pixel values ​​on the RGB channels of the target white light image, R", G", and B" represent the corresponding pixel values ​​on the RGB channels of the second denoised image, and CoreLow r 、CoreLow g With CoreLow b They are respectively the low-pass filter kernels of the RGB channels of the target white light image. The low-pass filter kernel can be a Gaussian filter kernel, a mean filter kernel, a median filter kernel, a bilateral filter kernel, or a guided filter kernel. In this embodiment, the low-pass filter kernel is preferably a mean filter kernel.

[0176] S450: Perform detail enhancement processing on the second denoised image to obtain a third detail layer image.

[0177] In a possible implementation, in order to highlight the details on the second denoised image, the pixel values ​​of the details should be gain processed. For example, in order to highlight the original red details on the target white light image, so as to achieve a better restoration of the red details lost due to the oversaturated red coverage, a high-pass filtering algorithm is used to gain all pixel values ​​on the R″, G″, and B″ channels on the second denoised image to obtain a third detail layer image that can highlight the red details. Specifically, the formula is used. The pixel values ​​of all pixels on the RGB channels of the second denoised image are subjected to gain processing, where CoreHigh Y 、CoreHigh g 、CoreHigh b is the high-pass filter kernel, Gain r 、Gain g With Cain b It is the gain coefficient corresponding to the R″, G″, and B″ channels, and its value range is -100 to 100.

[0178] S460: Perform weighted fusion on the pixel values ​​of all pixels on the RGB channels of the third detail layer image to obtain a target detail layer image.

[0179] In a possible implementation, in order to obtain the target detail layer image, the pixel values ​​of all pixels on the third detail layer image after the detail content is processed by gain should be subjected to weighted fusion processing. For example, the formula D′=k 11 *R″′+k 12 *G″′+k 13 *B″′ performs weighted fusion on the pixel values ​​of all pixels on the RGB channels of the third detail layer image to obtain the target detail layer image (such as Fig.11 As shown), k11-k13 are weight coefficients, and the value range is -100 to 100. The acquisition method of k11-k13 is the same as the acquisition method of k1 to k5, which will not be repeated here.

[0180] S50: Perform weighted fusion on the target detail layer image and the second white light image to obtain a target image.

[0181] In a possible implementation, in order to obtain the target image of the original red detail content of the mucosal blood vessels in the area covered by the oversaturated red, the target detail layer image after detail enhancement processing should be fused with the second white light image to restore the original detail content of the mucosal blood vessels covered by the oversaturated red and improve the enhancement effect of the target image. For example, the formula R2=k 21 *D′+k 22 *R wb , G2=k 23 *D′+k 24 *G wb , B2=k 25 *D′+k 26 *B wb The pixel values ​​of all pixels on the RGB channels of the target detail layer image and the pixel values ​​of all pixels on the RGB channels of the second white light image are weightedly fused to obtain the target image, where R2, G2 and B2 represent the pixel values ​​of all pixels on the RGB channels of the target image, respectively, and k21-k26 are corresponding weight coefficients ranging from -100 to 100. The acquisition method of k21-k26 is the same as that of k11, K12 and k13, so it will not be repeated.

[0182] Fig. 9 FIG. 4 shows a schematic diagram of the structure of an image processing device of the present invention. Fig. 9 As shown, an image processing device according to a disclosed embodiment of the present invention includes

[0183] An image acquisition module 10, used to acquire a second white light image;

[0184] A pixel mean acquisition module 20, used to acquire a first pixel mean of all pixels on the second white light image in the R channel, a second pixel mean of the B channel, and a third pixel mean of the G channel;

[0185] A pixel comparison module 30, used to obtain a comparison result between a ratio of the first pixel mean to the second pixel mean and a first threshold, and a comparison result between a ratio of the first pixel mean to a third pixel mean of the G channel and a second threshold;

[0186] an image determination module 40, configured to adjust the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold value based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold value and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold value, so as to obtain a target white light image;

[0187] An image enhancement module 50, configured to obtain a target detail layer image based on the target white light image;

[0188] The image fusion module 60 is used to perform weighted fusion on the target detail layer image and the second white light image to obtain a target image.

[0189] In a possible implementation, the image acquisition module 10 further includes:

[0190] An image acquisition unit, used for acquiring an original image with RAW features;

[0191] A first image processing unit, used for processing the original image with RAW characteristics to obtain a first white light image with RGB characteristics;

[0192] The second image processing unit is used to perform white balance processing on the first white light image to obtain the second white light image.

[0193] In a possible implementation, the image determination module 40 further includes:

[0194] A first image conversion unit, used for converting the second white light image having RGB characteristics into a first HSV image;

[0195] A pixel point determination unit, used to determine a pixel point to be processed whose red hue value on the first HSV image is less than or equal to a red hue threshold;

[0196] A pixel adjustment unit, used for adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image;

[0197] The second image format conversion module is used to convert the second HSV image into the target white light image with RGB features.

[0198] In a possible implementation, the image enhancement module 50 further includes:

[0199] A first image denoising unit, configured to denoise the target white light image to obtain a first denoised image;

[0200] A detail image determining unit, configured to obtain a plurality of second detail layer images of the same size based on the first denoised image;

[0201] The detail image processing unit is used to perform weighted fusion on the pixel values ​​of all pixels on each of the second detail layer images to obtain a target detail layer image.

[0202] In a possible implementation manner, the detail image determining unit further includes:

[0203] An image downsampling processing module, used for performing downsampling processing on the first denoised image to obtain a plurality of downsampled images of different sizes;

[0204] A first detail layer image determination module, configured to obtain a plurality of first detail layer images of different sizes based on the plurality of down-sampled images of different sizes;

[0205] The second detail layer image determination module is used to upsample the multiple first detail layer images of different sizes respectively to obtain the corresponding multiple second detail layer images of the same size; wherein the size of the second detail layer image is the same as the size of the second white light image.

[0206] In a possible implementation, the image downsampling processing module is further used to:

[0207] Step S1, performing downsampling processing on the first denoised image to obtain a first downsampled image;

[0208] Step S2, performing denoising processing on the first down-sampled image to obtain a denoised first down-sampled image;

[0209] Step S3, performing downsampling processing on the first downsampled image after denoising to obtain a second downsampled image, wherein the size of the first downsampled image is different from the size of the second downsampled image;

[0210] Step S4, repeating steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes.

[0211] In a possible implementation manner, the first detail layer image determination module is further used to:

[0212] Performing upsampling processing on the downsampled images of different sizes to obtain corresponding upsampled images of multiple different sizes;

[0213] Pixel differences of all corresponding pixels on each of the down-sampled images and the corresponding up-sampled image at the same size are obtained to obtain the first detail layer images of the plurality of different sizes.

[0214] In a possible implementation, the image enhancement module 50 further includes:

[0215] A second image denoising unit, used to eliminate noise points of the target white light image to obtain a second denoised image;

[0216] A third detail layer determining unit, configured to perform detail enhancement processing on the second denoised image to obtain the third detail layer image;

[0217] The third detail image processing unit is used to weightedly fuse the pixel values ​​of all pixels on the RGB channels of the third detail layer image to obtain the target detail layer image.

[0218] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation thereof can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0219] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0220] The embodiment of the present disclosure also proposes an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory. The electronic device can be provided as a server or a terminal device. For example, the electronic device includes a processing component, which further includes one or more processors, and a memory resource represented by a memory, which is used to store instructions executable by the processing component, such as an application. The application stored in the memory may include one or more modules, each corresponding to a set of instructions. In addition, the processing component is configured to execute instructions to perform the above method, and the electronic device may also include a power supply component configured to perform power management of the electronic device, a wired or wireless network interface configured to connect the electronic device to a network, and an input / output interface (I / O interface). The electronic device can operate based on an operating system stored in the memory, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0221] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0222] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0223] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0224] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0225] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0226] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0227] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0228] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0229] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0230] It should be clear that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present invention is not limited to the specific steps and structures described above and shown in the figures. In addition, for the sake of brevity, a detailed description of known methods and technologies is omitted here.

[0231] The above is only an embodiment of the present application and is not limited to the present application. For those skilled in the art, the present application may have various changes and variations without departing from the scope of the present invention. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image processing method, characterized in that: include Acquiring a second white light image to be processed; Obtaining a first pixel mean value in the R channel, a second pixel mean value in the B channel, and a third pixel mean value in the G channel of all pixels on the second white light image; Based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold, adjusting the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold to acquire a target white light image; Based on the target white light image, acquiring a target detail layer image; The target detail layer image and the second white light image are weightedly fused to obtain a target image.

2. The image processing method according to claim 1, characterized in that: The step of adjusting the saturation value and the brightness value of all pixels on the second white light image that are not greater than the red hue threshold to obtain a target white light image includes: Performing image conversion on the second white light image to obtain a first HSV image; Determine a pixel point to be processed whose red hue value on the first HSV image is less than or equal to a red hue threshold; Adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image; Perform image conversion on the second HSV image to obtain the target white light image.

3. The image processing method according to claim 1, characterized in that: The step of acquiring a target detail layer image based on the target white light image comprises: Performing denoising on the target white light image to obtain a first denoised image; Based on the first denoised image, acquiring a plurality of second detail layer images of the same size; The pixel values ​​of all pixels on each of the second detail layer images are weightedly fused to obtain a target detail layer image.

4. The image processing method according to claim 3, characterized in that: Based on the first denoised image, a plurality of second detail layer images of the same size are obtained, including Performing down-sampling processing on the first denoised image to obtain a plurality of down-sampled images of different sizes; Based on the multiple down-sampled images of different sizes, obtaining multiple first detail layer images of different sizes; Upsampling the plurality of first detail layer images of different sizes respectively to obtain the corresponding plurality of second detail layer images of the same size; The size of the second detail layer image is the same as the size of the second white light image.

5. The image processing method according to claim 4, characterized in that: Based on the multiple down-sampled images of different sizes, a plurality of first detail layer images of different sizes are obtained, including Performing upsampling processing on the downsampled images of different sizes to obtain corresponding upsampled images of multiple different sizes; Pixel differences of all corresponding pixels on each of the down-sampled images and the corresponding up-sampled image at the same size are obtained to obtain the first detail layer images of the plurality of different sizes.

6. The image processing method according to claim 1, characterized in that: The step of acquiring a target detail layer image based on the target white light image comprises: Performing denoising on the target white light image to obtain a second denoised image; performing detail enhancement processing on the second denoised image to obtain a third detail layer image; The pixel values ​​of all pixels on the RGB channels of the third detail layer image are weightedly fused to obtain a target detail layer image.

7. The image processing method according to claim 1, characterized in that: The step of acquiring a second white light image to be processed comprises: Get the original image; Based on the original image, acquiring a first white light image having RGB features; The first white light image is processed using a white balance algorithm to obtain the second white light image.

8. The image processing method according to claim 4, characterized in that: The first denoised image is downsampled to obtain a plurality of downsampled images of different sizes, including Step S1: down-sampling the first denoised image to obtain a first down-sampled image; Step S2: performing denoising processing on the first down-sampled image to obtain a denoised first down-sampled image; Step S3: performing downsampling processing on the first downsampled image after denoising to obtain a second downsampled image, wherein the size of the first downsampled image is different from the size of the second downsampled image; Step S4: Repeat steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes.

9. An image processing device, characterized in that: include An image acquisition module, used for acquiring a second white light image; A pixel mean acquisition module, used to acquire a first pixel mean of all pixels on the second white light image in the R channel, a second pixel mean of the B channel, and a third pixel mean of the G channel; A pixel comparison module, used to obtain a comparison result between a ratio of the first pixel mean to the second pixel mean and a first threshold, and a comparison result between a ratio of the first pixel mean to a third pixel mean of the G channel and a second threshold; an image determination module, configured to adjust the saturation values ​​and brightness values ​​of all pixels on the second white light image that are not greater than a red hue threshold based on the fact that a ratio of the first pixel mean to the second pixel mean is greater than a first threshold and a ratio of the first pixel mean to the third pixel mean is greater than a second threshold, so as to obtain a target white light image; An image enhancement module, used for acquiring a target detail layer image based on the target white light image; The image fusion module is used to perform weighted fusion on the target detail layer image and the second white light image to obtain a target image.

10. The image processing device according to claim 8, characterized in that: The image acquisition module comprises An image acquisition unit, used for acquiring an original image with RAW features; A first image processing unit, used for processing the original image with RAW characteristics to obtain a first white light image with RGB characteristics; The second image processing unit is used to perform white balance processing on the first white light image to obtain the second white light image.

11. The image processing device according to claim 8, characterized in that: The image determination module includes A first image conversion unit, used for converting the second white light image having RGB characteristics into a first HSV image; A pixel point determination unit, used to determine a pixel point to be processed whose red hue value on the first HSV image is less than or equal to a red hue threshold; A pixel adjustment unit, used for adjusting the saturation value and the brightness value of the pixel to be processed to obtain a second HSV image; The second image format conversion module is used to convert the second HSV image into the target white light image with RGB features.

12. The image processing device according to claim 8, characterized in that: The image enhancement module includes A first image denoising unit, configured to denoise the target white light image to obtain a first denoised image; A detail image determining unit, configured to obtain a plurality of second detail layer images of the same size based on the first denoised image; The detail image processing unit is used to perform weighted fusion on the pixel values ​​of all pixels on each of the second detail layer images to obtain a target detail layer image.

13. The image processing device according to claim 12, characterized in that: The detail image determination unit comprises An image downsampling processing module, used for performing downsampling processing on the first denoised image to obtain a plurality of downsampled images of different sizes; A first detail layer image determination module, configured to obtain a plurality of first detail layer images of different sizes based on the plurality of down-sampled images of different sizes; A second detail layer image determination module, configured to perform upsampling processing on the plurality of first detail layer images of different sizes respectively, to obtain the corresponding plurality of second detail layer images of the same size; Wherein, the size of the second detail layer image is the same as the size of the second white light image; The image downsampling processing module is used to perform downsampling processing on the first denoised image to obtain a plurality of downsampled images of different sizes, and is configured as follows: Step S1, down-sampling the first denoised image to obtain a first down-sampled image; Step S2, denoising the first down-sampled image to obtain a denoised first down-sampled image; Step S3, down-sampling the denoised first down-sampled image to obtain a second down-sampled image, wherein the size of the first down-sampled image is different from the size of the second down-sampled image; Step S4, repeating steps S2 and S3 in sequence to obtain the multiple down-sampled images of different sizes; The first detail layer image determination module is used to obtain a plurality of first detail layer images of different sizes based on the plurality of down-sampled images of different sizes, and is configured as follows: The downsampled images of each different size are all upsampled to obtain a corresponding plurality of upsampled images of different sizes; pixel differences of all corresponding pixel points on each downsampled image of the same size and the corresponding upsampled image are obtained to obtain the first detail layer images of the plurality of different sizes.

14. The image processing device according to claim 8, characterized in that: The image enhancement module includes A second image denoising unit, used to eliminate noise points of the target white light image to obtain a second denoised image; A third detail layer determining unit, configured to perform detail enhancement processing on the second denoised image to obtain the third detail layer image; The third detail image processing unit is used to weightedly fuse the pixel values ​​of all pixels on the RGB channels of the third detail layer image to obtain the target detail layer image.

15. An electronic device, characterized in that: comprising a processor; and a memory for storing processor executable instructions, wherein the processor is configured to implement the method of any one of claims 1 to 8 when executing the instructions stored in the memory.

16. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images