A method and system for enhancing dark part of choledochoscope image

Abstract

The present invention provides a method and system for enhancing the dark part of a choledochoscope image. The image taken by the choledochoscope is segmented, and the dark channel image and the mean channel image of each image block are calculated with a window H×W; the first parameter and the second parameter of the image block are obtained according to the dark channel image and the mean channel image; the first parameter and the second parameter are embedded in the image block to obtain an embedding vector of the image block, and a target image block is obtained according to the embedding vector; the new pixel value of each channel of the target image block is calculated according to the average brightness value of all non-target image blocks and the average gray value of each channel of the non-target image block within a preset range centered on the target image block, and the non-target image block and the obtained new target image block are spliced ​​to obtain an enhanced choledochoscope image. The present invention can automatically judge the dark part in the choledochoscope image, and can identify the dark part even if the entire image is dark, and then enhance the dark part.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement, and in particular to a method and system for enhancing dark parts of a choledochoscopic image. Background Art

[0002] Choledochoscope is a diagnostic and treatment method that uses a biliary endoscope to examine bile duct and pancreatic duct lesions. It is mainly used to examine bile duct diseases, including cholelithiasis, bile duct cancer, pancreatitis, pancreatic duct diseases, etc. During the operation, the doctor passes the endoscope through the patient's mouth, esophagus, stomach, and finally to the duodenum, and then enters the bile duct and pancreatic duct through a special guide wire and catheter to examine and treat these parts. Compared with other endoscopes, choledochoscopes have a long penetration distance and are more susceptible to noise.

[0003] Image enhancement is an important part of digital image processing, and its purpose is to improve the visual effect of the image. By adjusting parameters such as contrast, brightness, and saturation, the visual effect of the image can be improved, making the image clearer and richer in details. In addition, the brightness and color distribution of the image can be made more in line with the observation habits of the human eye, improving the comfort of observation. On the other hand, it can reduce noise and interference in the image, such as blur, particle noise, motion blur, etc., making the image clearer and more stable. Image enhancement of choledochoscope images helps to improve the detection rate and diagnostic accuracy of doctors for lesions, and can help doctors see the microstructure and vascular distribution of tissues more clearly, as well as the boundary between lesions and surrounding normal tissues. Existing image enhancement methods are mostly used to reduce noise or increase contrast. However, in the process of using choledochoscopes, due to the long path of the guide wire and catheter, and the human body is a relatively complex organism, dark areas often appear. This not only appears in the process of penetrating into the bile duct and pancreatic duct, but also appears when observing lesions. Most of the enhancement methods in the existing technology enhance the whole, and cannot enhance the dark areas. How to enhance the dark areas for medical staff to observe is an urgent problem to be solved in this field. Summary of the invention

[0004] In order to solve the above problems, in a first aspect, the present invention provides a method for enhancing dark parts of a choledochoscope image, the method comprising the following steps:

[0005] The image captured by the choledochoscope is segmented to obtain a plurality of image blocks, and a dark channel image and a mean channel image of each image block are calculated with a window H×W, wherein the sizes of the dark channel image and the mean channel image are the same as the size of the image block; a first parameter and a second parameter of the image block are obtained according to the dark channel image and the mean channel image; the first parameter and the second parameter are embedded in the image block to obtain an embedding vector of the image block, and a target image block is obtained according to the embedding vector;

[0006] The new pixel value of each channel of the target image block is calculated according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, and the non-target image blocks and the obtained new target image blocks are spliced ​​to obtain the enhanced choledochal endoscopy image.

[0007] Preferably, the dark channel image and the mean channel image of each image block are calculated using the window H×W, specifically:

[0008] The dark channel image is a grayscale image obtained by calculating the minimum value of each pixel and all pixels in its surrounding area. The specific calculation method of the dark channel image is:

[0009] The mean channel image is a grayscale image obtained by calculating the average value of all channels of each pixel and its surrounding area. The specific calculation method of the mean channel image is:

[0010] Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

[0011] Preferably, the first parameter and the second parameter of the image block are obtained according to the dark channel image and the mean channel image, specifically:

[0012] Taking the average value of the dark channel image corresponding to the image block as the first parameter;

[0013] The average value of the mean channel image corresponding to the image block is used as the second parameter.

[0014] Preferably, embedding the first parameter and the second parameter into an image block to obtain an embedding vector of the image block, and obtaining the target image block according to the embedding vector, specifically comprises:

[0015] A mapping matrix is ​​obtained by pre-training, wherein the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; an element with coordinates (first parameter, second parameter) in the mapping matrix is ​​searched, and the searched element is used as an embedding vector; wherein M is an integer between 1 and 255;

[0016] The attention of all image blocks is calculated, the attention is input into the neural network, and the output of the softmax layer is used to determine whether it is a target image block; the softmax layer is connected after the neural network.

[0017] Preferably, the new pixel value of each channel of the target image block is calculated according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, specifically:

[0018] Calculate the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks centered on the target image block, and if it is less than 1, then add D+2 until the ratio is not less than 1; wherein D is an odd number with an initial value of 3;

[0019] Calculate the average grayscale value g1 of each channel of non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increase or decrease the corresponding channel of the target image block by the ratio;

[0020] The average brightness value of all non-target image blocks and the average brightness value of the target image block increased or reduced by the ratio are calculated by the linear weighted average method. If the deviation between the two is greater than the threshold, the target value of each channel is queried based on the deviation, and each channel of the target image block is corrected based on the target value.

[0021] In another aspect, the present invention provides a choledochoscope image dark part enhancement system, the system comprising the following modules:

[0022] A dark area determination module is used to segment the image taken by the choledochoscope to obtain multiple image blocks, calculate the dark channel image and the mean channel image of each image block with a window H×W, and the size of the dark channel image and the mean channel image is the same as the size of the image block; obtain the first parameter and the second parameter of the image block according to the dark channel image and the mean channel image; embed the first parameter and the second parameter into the image block to obtain an embedding vector of the image block, and obtain the target image block according to the embedding vector;

[0023] The dark part enhancement module is used to calculate the new pixel value of each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, and splice the non-target image blocks and the obtained new target image blocks to obtain the enhanced choledochus endoscope image.

[0024] Preferably, the dark channel image and the mean channel image of each image block are calculated using the window H×W, specifically:

[0025] The dark channel image is a grayscale image obtained by calculating the minimum value of each pixel and all pixels in its surrounding area. The specific calculation method of the dark channel image is:

[0026] The mean channel image is a grayscale image obtained by calculating the average value of all channels of each pixel and its surrounding area. The specific calculation method of the mean channel image is:

[0027] Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

[0028] Preferably, the first parameter and the second parameter of the image block are obtained according to the dark channel image and the mean channel image, specifically:

[0029] Taking the average value of the dark channel image corresponding to the image block as the first parameter;

[0030] The average value of the mean channel image corresponding to the image block is used as the second parameter.

[0031] Preferably, embedding the first parameter and the second parameter into an image block to obtain an embedding vector of the image block, and obtaining the target image block according to the embedding vector, specifically comprises:

[0032] A mapping matrix is ​​obtained by pre-training, wherein the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; an element with coordinates (first parameter, second parameter) in the mapping matrix is ​​searched, and the searched element is used as an embedding vector;

[0033] The attention of all image blocks is calculated, the attention is input into the neural network, and the output of the softmax layer is used to determine whether it is a target image block; the softmax layer is connected after the neural network.

[0034] Preferably, the new pixel value of each channel of the target image block is calculated according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, specifically:

[0035] Calculate the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks centered on the target image block, and if it is less than 1, then add D+2 until the ratio is not less than 1; wherein D is an odd number with an initial value of 3;

[0036] Calculate the average grayscale value g1 of each channel of non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increase or decrease the corresponding channel of the target image block by the ratio;

[0037] The average brightness value of all non-target image blocks and the average brightness value of the target image block increased or reduced by the ratio are calculated by the linear weighted average method. If the deviation between the two is greater than the threshold, the target value of each channel is queried based on the deviation, and each channel of the target image block is corrected based on the target value.

[0038] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described above when executed by a processor.

[0039] In view of the problem that the prior art cannot enhance dark areas separately, the present invention uses image segmentation and dark channel images and mean channel images of image blocks, combined with embedded vectors, to determine the target image block, that is, the area that needs to be enhanced in the dark area, and then enhances the target image block using the overall brightness value of the image and information around the target image block. The present invention can enhance the dark area separately without changing the information of other areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a flow chart of Embodiment 1;

[0042] Figure 2 This is a flow chart of S1 in Embodiment 1;

[0043] Figure 3 is a schematic diagram of the mapping matrix;

[0044] Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0045] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Embodiment 1

[0048] The present invention provides a method for enhancing dark parts of choledochoscope images. Figure 1 As shown, the method comprises the following steps:

[0049] S1, segmenting the image taken by the choledochoscope to obtain multiple image blocks, calculating the dark channel image and the mean channel image of each image block with a window H×W, wherein the sizes of the dark channel image and the mean channel image are the same as the size of the image block; obtaining a first parameter and a second parameter of the image block according to the dark channel image and the mean channel image; embedding the first parameter and the second parameter into the image block to obtain an embedding vector of the image block, and obtaining a target image block according to the embedding vector;

[0050] The image captured by the choledochoscope can be from a picture stored on a disk or a captured video, where each frame of the video is an image. The dark part refers to the darker part of the image, so that the human eye cannot observe the details. After obtaining the captured image, the image is segmented into multiple image blocks (patches) of the same size, and then for each pixel in the image block, a dark channel image and an average channel image are obtained with a window H×W, where the dark channel (DarkChannel) refers to the minimum value of all channels in a pixel and its surrounding area, and the average channel (Average Channel) refers to the average value of all channels in a pixel and its surrounding area. The dark channel image is a grayscale image composed of dark channels of multiple pixels, and the average channel image is a grayscale image composed of average channels of multiple pixels.

[0051] In a specific embodiment, the dark channel image and the mean channel image of each image block are calculated using the window H×W, specifically:

[0052] The dark channel image is a grayscale image obtained by calculating the minimum value of each pixel and all pixels in its surrounding area. The specific calculation method of the dark channel image is:

[0053] That is, first find the minimum value of the three RGB channels, and then calculate the area w k The minimum value of the minimum values ​​of all RGB channels is obtained to obtain the dark channel pixel value of pixel k.

[0054] The mean channel image is a grayscale image obtained by calculating the average value of all channels of each pixel and its surrounding area. The specific calculation method of the mean channel image is:

[0055] That is, first find the average value of the three RGB channels, and then calculate the area w k The average value of the average values ​​of all three RGB channels in the pixel k is obtained. In another expression, Where n represents the area w k The number of pixels in the image.

[0056] Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

[0057] Among them, the dark channel image reflects the lowest pixel situation of the image block, and the mean channel image reflects the average brightness of the image block or whether it is a dark part. However, since an image may be dark as a whole, if conventional methods such as threshold method are used to determine whether it is a dark part, misjudgment may occur or most image blocks will be considered as dark parts. This method is inaccurate. Based on this, in one embodiment of the present invention, the first parameter and the second parameter of the image block are further obtained according to the dark channel image and the mean channel image. Specifically, each image block corresponds to a dark channel image and a mean channel image, and the average value of the dark channel image corresponding to the image block is calculated as the first parameter, and the average value of the mean channel image corresponding to the image block is calculated as the second parameter.

[0058] The first parameter and the second parameter reflect the overall darkness of the image block. However, due to the lack of comparison with other image blocks, especially non-dark image blocks, it is impossible to accurately determine whether an image block is a dark area based on the first parameter and the second parameter alone. In a specific embodiment, an attention mechanism is used. The attention mechanism can help determine the relationship between image blocks. In the present invention, this relationship is mainly whether it is a dark area, that is, the attention mechanism can determine whether an image block is a dark area in the entire image. The attention mechanism involves two aspects. One is the embedding of image blocks, that is, patch embedding. Since the first parameter and the second parameter reflect the overall darkness of the image block, in this embodiment, the first parameter and the second parameter are used to determine the target image block, such as Figure 2 As shown, the specific steps include:

[0059] S11, obtaining a mapping matrix through pre-training, wherein the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; searching for an element with coordinates (first parameter, second parameter) in the mapping matrix, and using the searched element as an embedding vector;

[0060] S12, calculating the attention of all image blocks, inputting the attention into the neural network, and judging whether the output of the softmax layer is the target image block; the softmax layer is connected after the neural network.

[0061] In a specific embodiment, the mapping matrix is ​​a 255×255 matrix, each element of the mapping matrix is ​​a one-dimensional vector, and the first parameter and the second parameter are used as coordinates to find the corresponding vector from the mapping matrix. In another alternative embodiment, the mapping matrix is ​​an M×M matrix, each matrix element is an interval, for example, M=15, then it is a 15×15 matrix, such as Figure 3As shown in the figure, if the first parameter is 15 and the second parameter is 105, it will fall into the V5th interval, that is, the corresponding matrix element is V5. This method can reduce the amount of data in the mapping matrix and reduce the difficulty of training during training, but the accuracy may decrease. Where M is an integer between 1 and 255.

[0062] After obtaining the vector corresponding to the image block, the attention result of the image block can be obtained through the Q matrix, K matrix and V matrix in the attention mechanism, where the result has the same dimension as the corresponding vector found from the mapping matrix. Then, the neural network and softmax layer can be used to determine whether the image block is the target image block, which refers to the image block belonging to the dark part.

[0063] It should be noted that before performing steps S11 and S12, it is necessary to construct a training set to train the mapping matrix and the Q, K, V matrices in the attention mechanism and the neural network. The training adopts the gradient descent method. After the mapping matrix, Q matrix, K matrix, V matrix and neural network are obtained through training, steps S11 and S12 are basically operations on the matrix, and the processing speed can meet the requirements.

[0064] S2, calculating a new pixel value for each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, and splicing the non-target image blocks and the obtained new target image blocks to obtain an enhanced choledochus endoscope image.

[0065] When enhancing the dark area, information from the entire image and information around the dark area is needed. In a specific embodiment, the new pixel value of each channel of the target image block is calculated based on the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, specifically:

[0066] Calculate the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks centered on the target image block, and if it is less than 1, then add D+2 until the ratio is not less than 1; wherein D is an odd number with an initial value of 3;

[0067] Since the target image block needs to be enhanced based on the surrounding pixels, if the surrounding area is all dark, the reference is not very meaningful, so it is necessary to further expand the range of the surrounding area. For example, when D=3, that is, the ratio is not less than 1, then from a total of 9 areas, there are 5 that are not dark parts, and the target area can be enhanced based on these 5 areas. Among them, the D areas of the target image block refer to areas of the size of D image blocks, and the size of the image block is the same as the size of the image block divided in S1.

[0068] Calculate the average grayscale value g1 of each channel of non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increase or decrease the corresponding channel of the target image block by the ratio;

[0069] When performing image enhancement, the non-target image blocks around the target image block are first enhanced, and each channel is enhanced separately by using a per-channel enhancement method. Specifically, according to the ratio of g1 and g2, the channel corresponding to the target image block is increased or reduced according to the ratio. For example, if g1=10 and g2=12 are calculated for the R channel, the pixel value of the R channel in the target image block becomes 5 / 6 of the original value.

[0070] The average brightness value of all non-target image blocks and the average brightness value of the target image block increased or reduced by the ratio are calculated by the linear weighted average method. If the deviation between the two is greater than the threshold, the target value of each channel is queried based on the deviation, and each channel of the target image block is corrected based on the target value.

[0071] After the surrounding pixels are corrected, corrections need to be made according to the overall situation of the image. The brightness value calculated by the linear weighted average method is more in line with the visual effect of the human eye. In this process, the weighted average method is used to calculate the average brightness value of all non-target image blocks and the average brightness value of the target image block obtained in the previous step. If the deviation between the two is greater than the threshold, the threshold refers to the difference between the two that the human eye can perceive. Then the target value of each channel is determined according to the deviation. For example, if the deviation is 5, the corresponding target value of the R channel is 2, then 2 is added to the R channel in the target image block to complete the correction. That is, the target value of each channel is queried based on the deviation, and each channel of the target image block is corrected based on the target value, which means that the target value corresponding to each channel is found according to the deviation value, and the channel corresponding to the target image block is added with the target value corresponding to the channel. Among them, the weighted average method refers to the brightness value calculated by 0.299*R+0.587*G+0.114*B.

[0072] In another embodiment, after executing S1 and S2, S3 is further included, wherein S3 is used to adjust the brightness of the result obtained by S2, and the brightness adjustment includes but is not limited to contrast enhancement, histogram equalization, Gamma adjustment, etc.

[0073] Embodiment 2

[0074] The present invention provides a choledochoscope image dark part enhancement system 200, such as Figure 4 As shown, the system includes the following modules:

[0075] The dark area determination module 201 is used to segment the image taken by the choledochoscope to obtain multiple image blocks, calculate the dark channel image and the mean channel image of each image block with a window H×W, and the size of the dark channel image and the mean channel image is the same as the size of the image block; obtain the first parameter and the second parameter of the image block according to the dark channel image and the mean channel image; embed the first parameter and the second parameter into the image block to obtain an embedding vector of the image block, and obtain the target image block according to the embedding vector;

[0076] The dark part enhancement module 202 is used to calculate the new pixel value of each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, and to splice the non-target image blocks and the obtained new target image blocks to obtain an enhanced choledochal endoscope image.

[0077] Preferably, the dark channel image and the mean channel image of each image block are calculated using the window H×W, specifically:

[0078] The dark channel image is a grayscale image obtained by calculating the minimum value of each pixel and all pixels in its surrounding area. The specific calculation method of the dark channel image is:

[0079] The mean channel image is a grayscale image obtained by calculating the average value of all channels of each pixel and its surrounding area. The specific calculation method of the mean channel image is:

[0080] Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

[0081] Preferably, the first parameter and the second parameter of the image block are obtained according to the dark channel image and the mean channel image, specifically:

[0082] Taking the average value of the dark channel image corresponding to the image block as the first parameter;

[0083] The average value of the mean channel image corresponding to the image block is used as the second parameter.

[0084] Preferably, embedding the first parameter and the second parameter into an image block to obtain an embedding vector of the image block, and obtaining the target image block according to the embedding vector, specifically comprises:

[0085] A mapping matrix is ​​obtained by pre-training, wherein the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; an element with coordinates (first parameter, second parameter) in the mapping matrix is ​​searched, and the searched element is used as an embedding vector;

[0086] The attention of all image blocks is calculated, the attention is input into the neural network, and the output of the softmax layer is used to determine whether it is a target image block; the softmax layer is connected after the neural network.

[0087] Preferably, the new pixel value of each channel of the target image block is calculated according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, specifically:

[0088] Calculate the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks centered on the target image block, and if it is less than 1, then add D+2 until the ratio is not less than 1; wherein D is an odd number with an initial value of 3;

[0089] Calculate the average grayscale value g1 of each channel of non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increase or decrease the corresponding channel of the target image block by the ratio;

[0090] The average brightness value of all non-target image blocks and the average brightness value of the target image block increased or reduced by the ratio are calculated by the linear weighted average method. If the deviation between the two is greater than the threshold, the target value of each channel is queried based on the deviation, and each channel of the target image block is corrected based on the target value.

[0091] Embodiment 3

[0092] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0093] Embodiment 4

[0094] The present invention further provides a computer device, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the method described in the first embodiment is implemented.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing dark areas of choledochoscope images, characterized in that: The method comprises the following steps: The image captured by the choledochoscope is segmented to obtain a plurality of image blocks, and a dark channel image and a mean channel image of each image block are calculated with a window H×W, wherein the sizes of the dark channel image and the mean channel image are the same as the size of the image block; a first parameter and a second parameter of the image block are obtained according to the dark channel image and the mean channel image; the first parameter and the second parameter are embedded in the image block to obtain an embedding vector of the image block, and a target image block is obtained according to the embedding vector; Calculate a new pixel value for each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range centered on the target image block, and splice the non-target image blocks and the obtained new target image blocks to obtain an enhanced choledochoscope image; The obtaining of the first parameter and the second parameter of the image block according to the dark channel image and the mean channel image is specifically: taking the average value of the dark channel image corresponding to the image block as the first parameter; taking the average value of the mean channel image corresponding to the image block as the second parameter; The first parameter and the second parameter are embedded in the image block to obtain the embedding vector of the image block, and the target image block is obtained according to the embedding vector, specifically: a mapping matrix is ​​obtained by pre-training, the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; the element with coordinates (first parameter, second parameter) in the mapping matrix is ​​searched, and the searched element is used as the embedding vector; wherein M is an integer between 1 and 255; the attention of all image blocks is calculated, the attention is input into the neural network, and the output of the softmax layer is judged whether it is the target image block; the softmax layer is connected after the neural network; The method of calculating a new pixel value of each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range with the target image block as the center is specifically as follows: calculating the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks with the target image block as the center, if less than 1, then D+2, until the ratio is not less than 1; wherein D is an odd number with an initial value of 3; calculating the average grayscale value g1 of each channel of the non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increasing or decreasing the corresponding channel of the target image block by the ratio; calculating the average brightness value of all non-target image blocks and the average brightness value of the target image block increased or decreased by the ratio by a linear weighted average method, and if the deviation between the two is greater than a threshold, querying the target value of each channel based on the deviation, and correcting each channel of the target image block based on the target value.

2. The method according to claim 1, characterized in that The dark channel image and mean channel image of each image block are calculated using the window H×W, specifically: The dark channel image is a grayscale image obtained by calculating the minimum channel value of the area around each pixel. The specific calculation method of the dark channel image is: The mean channel image is a grayscale image obtained by calculating the average values ​​of the three RGB channels in the area around each pixel. The specific calculation method of the mean channel image is: Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

3. A choledochoscope image dark part enhancement system, characterized in that: The system includes the following modules: A dark area determination module is used to segment the image taken by the choledochoscope to obtain multiple image blocks, calculate the dark channel image and the mean channel image of each image block with a window H×W, and the size of the dark channel image and the mean channel image is the same as the size of the image block; obtain the first parameter and the second parameter of the image block according to the dark channel image and the mean channel image; embed the first parameter and the second parameter into the image block to obtain an embedding vector of the image block, and obtain the target image block according to the embedding vector; A dark part enhancement module is used to calculate a new pixel value for each channel of a target image block according to an average brightness value of all non-target image blocks and an average grayscale value of each channel of non-target image blocks within a preset range centered on the target image block, and to splice the non-target image blocks and the obtained new target image blocks to obtain an enhanced choledochoscope image; The obtaining of the first parameter and the second parameter of the image block according to the dark channel image and the mean channel image is specifically: taking the average value of the dark channel image corresponding to the image block as the first parameter; taking the average value of the mean channel image corresponding to the image block as the second parameter; The first parameter and the second parameter are embedded in the image block to obtain an embedding vector of the image block, and the target image block is obtained according to the embedding vector, specifically: a mapping matrix is ​​obtained by pre-training, the size of the mapping matrix is ​​M×M, and the element of the matrix is ​​a vector; the element with coordinates of (first parameter, second parameter) in the mapping matrix is ​​searched, and the searched element is used as the embedding vector; wherein M is an integer between 1 and 255; the attention of all image blocks is calculated, the attention is input into the neural network, and the output of the softmax layer is used to determine whether it is the target image block; The softmax layer is connected after the neural network; The method of calculating a new pixel value of each channel of the target image block according to the average brightness value of all non-target image blocks and the average grayscale value of each channel of the non-target image blocks within a preset range with the target image block as the center is specifically as follows: calculating the ratio of the number of non-target image blocks to the number of target image blocks in an area of ​​D image blocks with the target image block as the center, if less than 1, then D+2, until the ratio is not less than 1; wherein D is an odd number with an initial value of 3; calculating the average grayscale value g1 of each channel of the non-target image blocks in the area of ​​D image blocks with the target image block as the center, and the average grayscale value g2 of each channel of the target image block, and according to the ratio of g1 to g2, increasing or decreasing the corresponding channel of the target image block by the ratio; calculating the average brightness value of all non-target image blocks and the average brightness value of the target image block increased or decreased by the ratio by a linear weighted average method, and if the deviation between the two is greater than a threshold, querying the target value of each channel based on the deviation, and correcting each channel of the target image block based on the target value.

4. The system according to claim 3, characterized in that The dark channel image and mean channel image of each image block are calculated using the window H×W, specifically: The dark channel image is a grayscale image obtained by calculating the minimum channel value of the area around each pixel. The specific calculation method of the dark channel image is: The mean channel image is a grayscale image obtained by calculating the average values ​​of the three RGB channels in the area around each pixel. The specific calculation method of the mean channel image is: Among them, J c (x) represents the value of pixel x on the c channel, where the c channel is one of the r channel, g channel, and b channel. min() represents the minimum value, avg() represents the average value, and w k Represents the kth window, and the window size is H×W.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 2.

Images