A choledochoscope image processing system for minimally invasive gallstone extraction

Abstract

The present application relates to the field of image enhancement technology, in particular to a choledochoscope image processing system for gallstone minimally invasive extraction. The system comprises a region division module, a first analysis module, a second analysis module, a coefficient determination module and an image enhancement module. Firstly, according to the morphological features of the color difference region, the confidence degree of the color difference region belonging to the stone region is determined; combined with the confidence degree, according to the dynamic change of each color difference region in the continuous multiple frames of choledochoscope images, the abnormality degree of the color difference region is determined; according to the channel value of each color difference region in different color channels, the saliency of the color difference region is determined; combined with the abnormality degree and the saliency, the enhancement coefficient of the color difference region is obtained, and the gamma transformation algorithm is used to perform adaptive image enhancement on the color difference region in the choledochoscope image, and the choledochoscope enhanced image is obtained. The present application realizes adaptive enhancement of the choledochoscope image according to the light change and the region morphological features.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image enhancement, in particular to a choledochoscope image processing system for gallstone minimally invasive extraction. BACKGROUND

[0002] Gallstone is a common biliary system disease, and traditional open surgery has large trauma and slow recovery. With the rapid development of minimally invasive surgery technology, percutaneous transhepatic choledochoscopic lithotomy and other minimally invasive surgeries have become the mainstream treatment. The core of this operation depends on the guidance of the choledochoscope. The operator observes the internal environment of the bile duct in real time through the endoscopic image to locate and remove the stones. Current clinical practice relies heavily on the naked eye of the operator to directly observe the original choledochoscope image without processing, which has significant limitations. First, the bile duct environment is complex, often with bile, mucus, blood and other interference, resulting in blurred images, low contrast and unclear details. Second, the stones are similar in composition, color and bile duct wall or inflammatory tissue, especially when they are small or embedded, which makes them easy to miss. Therefore, it is necessary to process the choledochoscope image.

[0003] In the minimally invasive gallstone extraction surgery, the current clinical system usually only has basic adjustment functions such as brightness, contrast and color manual adjustment, and cannot adaptively adjust according to different situations, resulting in poor imaging effect. SUMMARY

[0004] In order to solve the technical problem of poor imaging effect of the choledochoscope image, the purpose of the present application is to provide a choledochoscope image processing system for gallstone minimally invasive extraction, and the technical solution adopted is as follows:

[0005] In the first aspect, the present application provides a choledochoscope image processing system for gallstone minimally invasive extraction, which comprises:

[0006] The region division module is used for acquiring the choledochoscope image; the single frame choledochoscope image is regionally divided to obtain a plurality of color difference regions;

[0007] The first analysis module is used for determining the confidence degree of the color difference region belonging to the stone region according to the morphological characteristics of the color difference region;

[0008] The second analysis module is used for combining the confidence degree of each color difference region, and determining the abnormality degree of the color difference region according to the dynamic change of each color difference region in the continuous multiple frames of choledochoscope images;

[0009] The coefficient determination module is used for determining the saliency degree of the color difference region according to the channel value of each color difference region in different color channels; and combining the abnormality degree and the saliency degree to obtain the enhancement coefficient of each color difference region;

[0010] The image enhancement module is used to adaptively enhance each chromatic difference region in the cholangioscopic image based on the enhancement coefficient of each chromatic difference region and using the gamma transform algorithm to obtain the enhanced cholangioscopic image.

[0011] Furthermore, the process of dividing a single frame of cholangioscopic image into regions yields multiple chromatic aberration regions, including:

[0012] For any frame of cholangioscopy image, perform an iterative search on the undivided pixels in the cholangioscopy image, and divide adjacent pixels with a gray level difference less than a set threshold into the same color difference region, until all pixels in the image are traversed, resulting in multiple color difference regions.

[0013] Further, determining the confidence level that the color difference region belongs to the stone region based on the morphological characteristics of the color difference region includes:

[0014] Determine the circumcircle of each color difference region;

[0015] The confidence level of a color difference region belonging to a stone region is determined by combining the grayscale difference between each pixel in the color difference region and its four neighboring pixels, and the area of ​​the circumscribed circle relative to the color difference region. Among these factors, both the grayscale difference and the area of ​​the circumscribed circle are negatively correlated with the confidence level.

[0016] Furthermore, the area of ​​the circumscribed circle relative to the color difference region is the area difference between the circumscribed circle and the color difference region.

[0017] Furthermore, the determination of the anomalousness of the chromatic aberration region based on the confidence level of each chromatic aberration region and the dynamic changes of each chromatic aberration region in multiple consecutive frames of cholangioscopic images includes:

[0018] By using optical flow, the chromatic aberration region is tracked to obtain the same chromatic aberration region in cholangioscopic images of different frames;

[0019] Compare the area of ​​the same chromatic aberration region in adjacent frames of cholangioscopy images to determine the degree of dynamic change in area;

[0020] Compare the color difference in the same color difference region in adjacent frames of cholangioscopic images to determine the dynamic variation of brightness;

[0021] The anomaly degree of the color difference region is determined by combining the confidence level of each color difference region, the dynamic change degree of the area, and the dynamic change degree of the brightness.

[0022] Furthermore, determining the anomaly degree of the color difference region by combining the confidence level of each color difference region, the dynamic change degree of the area, and the dynamic change degree of the brightness includes:

[0023] The average value of the dynamic change in area of ​​the same chromatic aberration region in all adjacent frames of choledochoscope images is determined as the area change characteristic;

[0024] The average value of the dynamic change in brightness and darkness in the same chromatic difference region in all adjacent frames of choledochoscope images is determined as the brightness and darkness change feature;

[0025] The sum of the area change characteristics and the light and dark change characteristics is negatively correlated and normalized to obtain the fit of the color difference area as the stone area.

[0026] The anomaly of the color difference region is determined by combining the confidence level and fit of the color difference region; both the confidence level and fit are positively correlated with the anomaly of the color difference region.

[0027] Furthermore, the method for obtaining the salience of the color difference region is as follows:

[0028] The channel values ​​include: R channel values, G channel values, and B channel values;

[0029] The average R channel value of all pixels in the color difference region is used as the region's R channel value; and the region's G channel value and B channel value are determined.

[0030] For any color difference region, the sum of the region's R channel value and the region's G channel value is used as the numerator, and the sum of the region's R channel value, the region's G channel value, and the region's B channel value is used as the denominator. The ratio of the numerator and the denominator is used as the saliency of the color difference region.

[0031] Furthermore, the enhancement coefficient for each color difference region is obtained by combining the anomaly and the saliency, including:

[0032] Analyze the anomaly degree of all color difference regions to obtain the average anomaly degree of the regions;

[0033] Using any color difference region as the target color difference region, compare the anomaly degree of the target color difference region with the average anomaly degree of the region to determine the degree of anomalous change.

[0034] The enhancement coefficient of the target color difference region is obtained by combining the anomalous change degree and salience of the target color difference region.

[0035] Furthermore, the adaptive image enhancement of each chromatic difference region in the cholangioscopic image, based on the enhancement coefficient of each chromatic difference region and utilizing a gamma transform algorithm, includes:

[0036] The enhancement coefficient of each chromatic difference region is used as the gamma value. Using the gamma transform algorithm, adaptive image enhancement is performed on each chromatic difference region in the cholangioscopic image to obtain the enhanced chromatic difference region.

[0037] Furthermore, the method for obtaining the enhanced cholangioscopic image is as follows: the enhanced chromatic aberration region is replaced by the chromatic aberration region before enhancement to obtain the enhanced cholangioscopic image.

[0038] Secondly, a cholangioscopic image processing method for minimally invasive stone removal of bile duct stones is provided, the method comprising the following steps:

[0039] Acquire cholangioscopic images; divide a single frame of cholangioscopic image into regions to obtain multiple chromatic difference regions;

[0040] Based on the morphological characteristics of the color difference region, determine the confidence level that the color difference region belongs to the stone region;

[0041] By combining the confidence level of each color difference region, the abnormality of each color difference region is determined based on the dynamic changes of each color difference region in multiple consecutive frames of cholangioscopic images.

[0042] Based on the channel values ​​of each color difference region in different color channels, the salience of the color difference region is determined; combining the anomaly and the salience, the enhancement coefficient of each color difference region is obtained.

[0043] Based on the enhancement coefficient of each chromatic difference region, the gamma transform algorithm is used to adaptively enhance each chromatic difference region in the cholangioscopic image to obtain the enhanced cholangioscopic image.

[0044] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the various possible implementations of the first aspect.

[0045] Fourthly, embodiments of the present invention provide a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0046] Fifthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the various possible implementations of the first aspect.

[0047] The embodiments of the present invention have at least the following beneficial effects:

[0048] This invention analyzes the influences present during actual acquisition of cholangioscopic images. First, considering the relatively smooth surface of gallstones compared to the non-smooth, wrinkled interior of the bile duct, the morphological characteristics of the defined color difference regions are analyzed to determine the confidence level of each region belonging to a gallstone region. Next, considering that the size of the gallstone remains constant while the wrinkles in the inner wall change due to bile duct peristalsis, and that intestinal wrinkles obstruct the illumination from the cholangioscopic endoscope, the light intensity changes at the same location as the endoscope moves. Because of their smooth surface, gallstone regions exhibit strong light reflection, resulting in similar gallstone regions across different frames. Therefore, the dynamic changes of each color difference region in multiple consecutive frames of cholangioscopic images are analyzed to determine the anomaly level. Finally, the color characteristics of the actual gallstone region are analyzed, and the salience of each color difference region is determined based on its channel values ​​in different color channels. Combining anomaly and salience, the enhancement coefficient of each color difference region is obtained. Based on the enhancement coefficient of each chromatic difference region, a gamma transform algorithm is used to adaptively enhance each chromatic difference region in the cholangioscopic image, resulting in an enhanced cholangioscopic image. This invention adaptively enhances the cholangioscopic image according to changes in light and the morphological characteristics of the region, thus enabling clearer identification of stone areas. Attached Figure Description

[0049] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A system block diagram of a choledochoscope image processing system for minimally invasive stone removal of bile duct stones, provided as an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of a choledochoscope image processing method for minimally invasive stone removal of bile duct stones, provided as an embodiment of the present invention. Detailed Implementation

[0052] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a cholangioscopic image processing system for minimally invasive stone removal of bile duct stones proposed according to the present invention.

[0053] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0054] In the description of the embodiments of the present invention, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" means two or more.

[0055] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0057] The embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0058] This invention provides a specific implementation method for a cholangioscopic image processing system for minimally invasive stone removal of bile duct stones. This method is applicable to cholangioscopic image processing scenarios. In this scenario, cholangioscopic images are first acquired.

[0059] Hardware device connection and setup in this scenario:

[0060] Cholangiopancreatography (CPC) devices primarily use the endoscope itself as the imaging device, and there are two main types: fiber optic CPCs and electronic CPCs. Fiber optic CPCs transmit images through tens of thousands of extremely fine optical fibers, with an eyepiece at the end of the lens that requires a camera to capture the image. In contrast, electronic CPCs integrate a miniature CMOS image sensor at the end of the lens, directly outputting electrical signals, resulting in superior image quality and making them the modern mainstream.

[0061] For fiberoptic cholangioscopy, a dedicated camera is required to capture images through its eyepiece. For electronic cholangioscopy, the built-in sensor signals are directly connected to the main unit via a cable.

[0062] The cold light source illuminates the cholangioscope through optical fibers, ensuring a bright view inside the bile duct without burning tissue.

[0063] The image processing unit is the core device for connecting the cholangioscope. It is responsible for powering the light source and sensors, receiving raw image signals, performing preliminary processing such as color correction and noise reduction, and then outputting standard video signals such as HDMI and SDI.

[0064] Video signals are digitized using image acquisition equipment. A professional acquisition card is installed inside the computer's PCIe expansion card or in an external USB acquisition box. It receives the video signal output from the host and converts it into a digital data stream that the computer can recognize and process. This is a professional choice to ensure high-quality, low-latency acquisition. Many modern cholangioscopy systems integrate recording and storage functions, such as built-in hard drives, which can directly save video and still images.

[0065] Display devices are used to display the acquired images in real time for the surgical team to observe.

[0066] The computer runs specialized image acquisition and management software to control the operation of the acquisition card and compress and store the incoming digital video stream to the hard drive.

[0067] When acquiring cholangioscopic images, start all equipment, run the acquisition software, and adjust the light source brightness and white balance to ensure clear images and true colors. After recording stops, properly save the video files and, in accordance with medical record management regulations, link and archive them with the patient's other electronic medical records (EMR). Annotate key segments of the cholangioscopic video promptly, such as marking time points, to facilitate subsequent research and analysis.

[0068] The cholangioscopic video is processed by frame segmentation to obtain multiple consecutive frames of cholangioscopic images.

[0069] The original cholangioscopic images suffer from significant defects due to the physical and physiological factors within the complex bile duct environment. Optical interference also exists, with uneven projection of a cold light source within the narrow bile duct creating a light-dark boundary zone. Strong contrast exists between the stone shadow area and the mucosal reflective area, leading to localized overexposure or loss of detail. Furthermore, the acquired cholangioscopic images are affected by the complexity of the target object; pigment stones (black / brown) and cholesterol stones (yellow / white) differ significantly in morphology and reflectivity, and are easily confused with inflammatory proliferative tissue, bile duct wall folds, or blood clots. Dynamic manipulation also contributes to the image stability; motion blur caused by the lens touching the duct wall and turbulence artifacts from the flow of irrigation fluid further reduce image stability.

[0070] The combined effect of these factors severely hinders the accuracy of artificial vision in identifying stones: low contrast obscures the outline of small stones (<3mm), stones under the mucus membrane form "visual camouflage" due to similar refractive indices, and blood adsorption can further fabricate dark red clump shadows that resemble stones.

[0071] Therefore, the acquired real-time cholangioscopy images need to be enhanced to make each region more distinct and to better distinguish between the stone area and the normal area.

[0072] The following description, in conjunction with the accompanying drawings, details a specific solution for a choledochoscope image processing system for minimally invasive removal of bile duct stones provided by the present invention.

[0073] Please see Figure 1 The diagram illustrates a system block diagram of a cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to an embodiment of the present invention. The system includes the following modules:

[0074] The region segmentation module 10 is used to acquire cholangioscopic images; it divides a single frame of cholangioscopic image into regions to obtain multiple color difference regions.

[0075] When enhancing the acquired cholangioscopic images, the first step is to divide each single frame of the cholangioscopic image into multiple chromatic aberration regions. It should be noted that each frame of the cholangioscopic image corresponds to multiple chromatic aberration regions.

[0076] For any frame of cholangioscopic image, an iterative search is performed on the undivided pixels in the cholangioscopic image. Adjacent pixels with a grayscale difference less than a set threshold are assigned to the same color difference region until all pixels in the image are traversed, resulting in multiple color difference regions. In this embodiment of the invention, the threshold value is set to 5. In other embodiments, the implementer may adjust this value according to the actual situation.

[0077] More specifically: Obtain any pixel in the cholangioscopic image and designate it as a reference pixel. Using the reference pixel as the center, perform a pixel search, grouping pixels with a grayscale difference less than a set threshold within their eight neighboring regions into the same category as the reference pixel. Then, use these pixels as new reference pixels for the pixel search, continuing until the grayscale difference between the reference pixel and its eight neighboring pixels no longer meets the set threshold, at which point further iteration is impossible. The current category of pixels is then classified, forming a region defined as a color difference region. Next, select an unclassified pixel and iterate again until all pixels in the complete frame of the cholangioscopic image have been traversed, completing the region division in the cholangioscopic image and obtaining multiple color difference regions.

[0078] The first analysis module 20 is used to determine the confidence level of the color difference region belonging to the stone region based on the morphological characteristics of the color difference region.

[0079] Because the inside of the bile duct is not smooth and uniform, but has many folds, and the inner wall tissue and stones may merge, it is difficult to highlight the stone area if the cholangioscopic image is enhanced as a whole, making it impossible to distinguish some small stones from the folds of the inner wall; therefore, it is necessary to separate the stone area from the folded area of ​​the bile duct inner wall and then enhance it to different degrees.

[0080] Because the folds of the bile duct wall are relatively continuous and adjacent to the bile duct wall, their grayscale differences are small; while stones are independent individuals. When multiple stones are piled together, a relatively obvious boundary line will be formed between the stones, and the surface of the stones is relatively smooth; therefore, the confidence level of different color difference regions as stone regions can be obtained based on the color difference regions obtained above.

[0081] For each color difference region, determine the circumcircle of each color difference region, which is the largest outer circle of the color difference region.

[0082] To obtain the area of ​​the circumcircle of the color difference region relative to the outer edge of the color difference region, more specifically: the area difference between the circumcircle and the color difference region is taken as the outer edge area of ​​the circumcircle relative to the color difference region.

[0083] The confidence level of a color difference region belonging to a stone region is determined by combining the grayscale difference between each pixel in the color difference region and its four neighboring pixels, and the area of ​​the circumscribed circle relative to the color difference region. Among these factors, both the grayscale difference and the area of ​​the circumscribed circle are negatively correlated with the confidence level.

[0084] In this embodiment of the invention, taking the a-th color difference region as an example, the confidence level that the a-th color difference region belongs to the stone region is... The calculation formula is:

[0085] ;

[0086] in, Indicates the first The first color difference region The grayscale value of each pixel Indicates the first The first color difference region The average gray level of the pixels in the four neighboring regions of a given pixel. Indicates the first The outer circle of the color difference region and the first The area difference of each color difference region, that is, the area difference of the circumcircle relative to the first color difference region. The outer area of ​​each color difference region Indicates the first The number of pixels in each color difference region.

[0087] In the formula for calculating confidence level, This indicates the continuity of gray levels near the i-th pixel. The smaller the value, the more similar the grayscale of the i-th pixel is to its neighboring pixels. Since the surface of a stone is smoother, the grayscale difference near pixels belonging to the stone region is smaller. This indicates the smoothness of the area's surface. The smaller the value, the smoother the area, and the greater the probability that the corresponding color difference area belongs to the end region. Indicates the first The regularity of the color difference regions is as follows: the stone region is a relatively regular stone, while the folded region has various and irregular shapes. Therefore, the difference between the maximum outer circle of the color difference region and the area of ​​the color difference region is calculated. The larger the area difference, the more irregular the color difference region is; conversely, the smaller the area difference, the more regular the color difference region is, and the greater the possibility that it is a stone.

[0088] The second analysis module 30 is used to combine the confidence level of each color difference region and determine the abnormality of each color difference region based on the dynamic changes of each color difference region in multiple consecutive frames of cholangioscopic images.

[0089] It should be noted that the first analysis module 20 analyzes the morphological characteristics of the stones and the folded areas of the inner wall in a single frame of choledochoscopy image. However, because the bile duct is filled with mucus, which adheres to the stones and the inner wall of the bile duct, and the bubbles formed by the mucus are quite similar in shape to the stones, it is easy to make a misjudgment and identification error if only the morphological characteristics are used for judgment. Therefore, the morphological changes of different color difference areas during the movement of the choledochoscope in the bile duct are analyzed to determine whether it is really a stone area.

[0090] Acquire multiple consecutive frames of cholangioscopy images and obtain the chromatic aberration region in each frame. Use optical flow to track the chromatic aberration region to obtain the same chromatic aberration region in different frames of cholangioscopy images. In other words, use optical flow to track the same chromatic aberration region to ensure that the same chromatic aberration region can be analyzed in different frames of cholangioscopy images.

[0091] Because the size of the gallstone does not change, while the folds of the inner wall change due to the peristalsis of the bile duct, and the folds of the intestine block the illumination light from the cholangioscope, the light at the same location will change as the cholangioscope moves; and because the surface of the gallstone area is smooth, light shining on the surface of the gallstone will create relatively strong reflected light, and the gallstone area seen in different frames of cholangioscopic images is quite similar; therefore, by comparing the changes in the same chromatic aberration area in adjacent frames of cholangioscopic images, the abnormality of the chromatic aberration area can be obtained.

[0092] The area of ​​the same chromatic aberration region in adjacent frames of cholangioscopy images is compared to determine the dynamic change in area. Taking the a-th chromatic aberration region in the t-th frame of cholangioscopy image as an example, the corresponding dynamic change in area is... for: ;in, Let be the area of ​​the a-th chromatic aberration region in the (t-1)-th frame of the cholangioscopy image. It is the area of ​​the a-th color difference region in the t-th frame of the cholangioscopic image that is the same as the a-th color difference region in the (t-1)-th frame of the cholangioscopic image. It can also be understood as the area of ​​the a-th color difference region in the t-th frame of the cholangioscopic image. The sign is for absolute values. In this embodiment of the invention, the area of ​​the region is the number of pixels within the region.

[0093] Compare the chromatic aberrations in the same chromatic aberration region in adjacent frames of cholangioscopy images to determine the dynamic range of brightness; taking the a-th chromatic aberration region in the t-th frame of cholangioscopy image as an example, its corresponding dynamic range of brightness is... for: ;in, Let be the average gray value of the a-th chromatic difference region in the (t-1)-th frame of the cholangioscopy image. The gray value of the a-th color difference region in the t-th frame of the cholangioscopic image is the same as that in the (t-1)-th frame of the cholangioscopic image. It can also be understood as the gray value of the a-th color difference region in the t-th frame of the cholangioscopic image. The sign is used for absolute values. In this embodiment of the invention, the average gray level of a region is the average gray level of all pixels within that region. It should be noted that the dynamic changes in area and brightness of the chromatic aberration region are not calculated for the first frame of the cholangioscopic image.

[0094] By combining the confidence level, the dynamic change in area, and the dynamic change in brightness of each color difference region, the anomaly of the color difference region is determined. More specifically: the average value of the dynamic change in area of ​​the same color difference region in all adjacent frames of cholangioscopic images is determined as the area change feature; the average value of the dynamic change in brightness of the same color difference region in all adjacent frames of cholangioscopic images is determined as the brightness change feature; the sum of the area change feature and the brightness change feature is negatively correlated and normalized to obtain the fit of the color difference region to the stone region; the anomaly of the color difference region is determined by combining the confidence level and the fit of the color difference region; wherein, both the confidence level and the fit are positively correlated with the anomaly of the color difference region.

[0095] In this embodiment of the invention, taking the a-th color difference region as an example, its corresponding anomaly degree... The calculation formula is:

[0096] ;

[0097] in, Indicates the first The confidence level that each color difference area belongs to the stone area. Indicates the first The first frame of cholangioscopy images Dynamic change in the area of ​​each color difference region Indicates the first The first frame of cholangioscopy images The dynamic variation of brightness and darkness in each color difference area This indicates the frame number of the cholangioscopic image; alternatively, a subset of consecutive frames of cholangioscopic images can be selected for comparison. For example, for the latest cholangioscopic image, the first 10 frames can be selected for analysis and comparison. In this case, the value of m is 10, and the 10th frame of the cholangioscopic image is the latest cholangioscopic image. Represents an exponential function with the natural constant as its base;

[0098] In adjacent frames of cholangioscopic images, the less dynamic the area change of the same chromatic aberration region, the greater the likelihood that it is a stone region; and the less dynamic the change in brightness, the greater the likelihood that it is a stone region; thus, the anomalousness of different chromatic aberration regions can be obtained.

[0099] The coefficient determination module 40 is used to determine the salience of each color difference region based on the channel values ​​of each color difference region in different color channels; and to obtain the enhancement coefficient of each color difference region by combining the anomaly and the salience.

[0100] The purpose of enhancing cholangioscopic images is to highlight the stone area in the cholangioscopic image and make the stone area more obvious. Therefore, different degrees of enhancement are needed for different color difference areas.

[0101] Since the original acquired cholangioscopic images are in color, the images to be enhanced are also in color. However, the aforementioned region segmentation module 10 to the second analysis module 30 analyzes the cholangioscopic images based on grayscale images. Therefore, it is also necessary to determine the degree of enhancement for different color difference regions based on the RGB images. It should be noted that the cholangioscopic images mentioned in region segmentation module 10 to the second analysis module 30 are essentially grayscale images of cholangioscopic images.

[0102] In this embodiment of the invention, the different color channels include: R channel, G channel, and B channel. The corresponding channel values ​​include: R channel value, G channel value, and B channel value.

[0103] Since the area around the gallstone is yellow and the bile duct is red, the salience value of the current color difference area is obtained based on the RGB values ​​of different color difference areas.

[0104] The average R channel value of all pixels in the color difference region is used as the region's R channel value; the average G channel value of all pixels in the color difference region is used as the region's G channel value; and the average B channel value of all pixels in the color difference region is used as the region's B channel value.

[0105] For any color difference region, the sum of the region's R channel value and the region's G channel value is used as the numerator, and the sum of the region's R channel value, the region's G channel value, and the region's B channel value is used as the denominator. The ratio of the numerator and the denominator is used as the saliency of the color difference region.

[0106] Since yellow is composed of red and green as primary colors, the larger the R channel value and G channel value of the color difference region, the greater the probability that the color difference region being analyzed is a stone region, and therefore the higher the significance.

[0107] Furthermore, by combining the anomaly degree and the saliency, the enhancement coefficient of each color difference region is obtained: the anomaly degree of all color difference regions is analyzed to obtain the average anomaly degree of the region; taking any color difference region as the target color difference region, the anomaly degree of the target color difference region and the average anomaly degree of the region are compared to determine the anomaly mutation degree; by combining the anomaly mutation degree and saliency of the target color difference region, the enhancement coefficient of the target color difference region is obtained; wherein, both the anomaly mutation degree and the saliency degree are negatively correlated with the enhancement coefficient.

[0108] In some embodiments, the regional average anomaly is the average of the anomalies in all chromatic aberration regions in the cholangioscopic image.

[0109] In some embodiments, the anomalous mutation degree is the absolute value of the difference between the anomalous degree of the target color difference region and the average anomalous degree of the region.

[0110] In some embodiments, taking the a-th color difference region as the target color difference region as an example, the enhancement coefficient of the a-th color difference region is... The calculation formula is:

[0111] ;

[0112] in, Indicates the first Abnormality of each color difference area This represents the average regional anomaly of all chromatic aberration regions in a single frame of cholangioscopy image. Indicates the first The salience of each color difference region Indicates the degree of aberration in the target color difference region. This represents the hyperbolic tangent function.

[0113] The greater the abnormality of a color difference area, the more likely it is to be a stone area; conversely, if a certain color difference area has a high abnormality but low significance, it indicates that the image was interfered with during the cholangioscopy, and therefore a greater degree of enhancement is required.

[0114] The image enhancement module 50 is used to adaptively enhance each chromatic difference region in the cholangioscopic image based on the enhancement coefficient of each chromatic difference region and using the gamma transform algorithm to obtain the enhanced cholangioscopic image.

[0115] The enhancement coefficient of each chromatic difference region is used as the gamma value. Using the gamma transform algorithm, adaptive image enhancement is performed on each chromatic difference region in the cholangioscopic image to obtain the enhanced chromatic difference region.

[0116] Taking the v-th pixel in the a-th color difference region as an example, the expression for the gamma transform is:

[0117]

[0118] in, Indicates the first The enhanced pixel value of the v-th pixel in each color difference region. This represents a constant, which is taken as an empirical value of 2 in this invention. Indicates the first The enhancement coefficient for each color difference region Indicates the first The pixel value of the vth pixel in the color difference region before enhancement.

[0119] This leads to the enhanced chromatic aberration region; the enhanced chromatic aberration region is then replaced by the original chromatic aberration region to obtain the enhanced cholangioscopic image.

[0120] Please see Figure 2 , Figure 2 This invention provides a flowchart of a cholangioscopic image processing method for minimally invasive stone removal of bile duct stones, comprising the following steps:

[0121] Acquire cholangioscopic images; divide a single frame of cholangioscopic image into regions to obtain multiple chromatic difference regions;

[0122] Based on the morphological characteristics of the color difference region, determine the confidence level that the color difference region belongs to the stone region;

[0123] By combining the confidence level of each color difference region, the abnormality of each color difference region is determined based on the dynamic changes of each color difference region in multiple consecutive frames of cholangioscopic images.

[0124] Based on the channel values ​​of each color difference region in different color channels, the salience of the color difference region is determined; combining the anomaly and the salience, the enhancement coefficient of each color difference region is obtained.

[0125] Based on the enhancement coefficient of each chromatic difference region, the gamma transform algorithm is used to adaptively enhance each chromatic difference region in the cholangioscopic image to obtain the enhanced cholangioscopic image.

[0126] Alternatively, the transmission medium may be a wired link, such as, but not limited to, coaxial cable, fiber optic cable and digital subscriber line, or a wireless link, such as, but not limited to, wireless Fidelity (WIFI), Bluetooth and mobile device networks.

[0127] It should be noted that the device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above.

[0128] This invention provides a computer device. Exemplarily, the computer device includes: a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the computer device can execute any of the aforementioned cholangioscopic image processing systems for minimally invasive bile duct stone removal.

[0129] Furthermore, embodiments of the present invention also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute the cholangioscopic image processing system for minimally invasive stone removal of bile duct stones provided in the embodiments of the present invention.

[0130] In this embodiment of the invention, the device can be divided into functional modules according to the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and is only a logical functional division. In actual implementation, there may be other division methods.

[0131] When each module is divided according to its function, the device may also include a signal uploading module, a determination module, and an adjustment module. It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0132] It should be understood that the device provided in this embodiment of the invention is used to execute the above-described cholangioscopic image processing system for minimally invasive removal of bile duct stones, and therefore can achieve the same effect as the above-described implementation method.

[0133] When using integrated units, the device may include a processing module and a storage module. When applied to a device, the processing module can be used to control and manage the device's operations. The storage module can be used to support the device in executing program code, etc. The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as described in this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of Digital Signal Processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0134] In addition, the device provided in the embodiments of the present invention may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the cholangioscopic image processing system for minimally invasive stone removal of bile duct stones provided in the above embodiments.

[0135] This invention also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned method steps to implement the cholangioscopic image processing system for minimally invasive stone removal of bile duct stones provided in the above embodiments.

[0136] This invention also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the cholangioscopic image processing system for minimally invasive removal of bile duct stones provided in the above embodiments.

[0137] In this invention, the apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments are all used to execute the corresponding methods described above. Therefore, the beneficial effects they achieve can be referred to the beneficial effects in the corresponding methods described above, and will not be repeated here. Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways.

[0138] The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0139] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0140] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0141] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0142] The above content is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A cholangioscopic image processing system for minimally invasive stone removal of bile duct stones, characterized in that, The system includes the following modules: The region segmentation module is used to acquire cholangioscopic images; it divides a single frame of cholangioscopic image into regions to obtain multiple color difference regions. The first analysis module is used to determine the confidence level of a color difference region belonging to a stone region based on the morphological characteristics of the color difference region. The confidence level is obtained by: determining the circumcircle of each color difference region; and combining the grayscale difference between each pixel in the color difference region and its four neighboring pixels, and the area of ​​the circumcircle relative to the outer edge of the color difference region, to determine the confidence level of the color difference region belonging to a stone region. The grayscale difference and the outer edge area are both negatively correlated with the confidence level. The second analysis module is used to track chromatic aberration regions using optical flow to obtain the same chromatic aberration region in cholangioscopic images of different frames; compare the area of ​​the same chromatic aberration region in adjacent cholangioscopic images to determine the dynamic change degree of area; compare the chromatic aberration of the same chromatic aberration region in adjacent cholangioscopic images to determine the dynamic change degree of brightness; and combine the confidence level of each chromatic aberration region, the dynamic change degree of area, and the dynamic change degree of brightness to determine the anomaly degree of the chromatic aberration region. The anomaly degree is obtained by: determining the average value of the dynamic change degree of area of ​​the same chromatic aberration region in all adjacent cholangioscopic images as the area change feature; determining the average value of the dynamic change degree of brightness of the same chromatic aberration region in all adjacent cholangioscopic images as the brightness change feature; performing negative correlation normalization on the sum of the area change feature and the brightness change feature to obtain the fit degree of the chromatic aberration region as the stone region; and combining the confidence level and fit degree of the chromatic aberration region to determine the anomaly degree of the chromatic aberration region. Both the confidence level and the fit degree are positively correlated with the anomaly degree of the chromatic aberration region. The coefficient determination module is used to determine the saliency of each color difference region based on the channel values ​​of each color difference region in different color channels; and to obtain the enhancement coefficient of each color difference region by combining the anomaly and the saliency; wherein the enhancement coefficient is obtained by: analyzing the anomaly of all color difference regions to obtain the average anomaly of the regions; taking any color difference region as the target color difference region, comparing the anomaly of the target color difference region with the average anomaly of the regions to determine the anomalous change degree; and combining the anomalous change degree and the saliency of the target color difference region to obtain the enhancement coefficient of the target color difference region; the formula for calculating the enhancement coefficient is: ;in, This represents the enhancement coefficient for the a-th color difference region; Indicates the first Abnormality of each color difference area This represents the average regional anomaly of all chromatic aberration regions in a single frame of cholangioscopy image. Indicates the first The salience of each color difference region Indicates the degree of aberration in the target color difference region. Represents the hyperbolic tangent function; The image enhancement module is used to adaptively enhance each chromatic difference region in the cholangioscopic image based on the enhancement coefficient of each chromatic difference region and using the gamma transform algorithm to obtain the enhanced cholangioscopic image.

2. The cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to claim 1, characterized in that, The process of dividing a single frame of cholangioscopy image into regions yields multiple chromatic aberration regions, including: For any frame of cholangioscopy image, perform an iterative search on the undivided pixels in the cholangioscopy image, and divide adjacent pixels with a gray level difference less than a set threshold into the same color difference region, until all pixels in the image are traversed, resulting in multiple color difference regions.

3. The cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to claim 1, characterized in that, The area of ​​the circumscribed circle relative to the color difference region is the area difference between the circumscribed circle and the color difference region.

4. The cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to claim 1, characterized in that, The method for obtaining the salience of the color difference region is as follows: The channel values ​​include: R channel values, G channel values, and B channel values; The average R channel value of all pixels in the color difference region is used as the region's R channel value; and the region's G channel value and B channel value are determined. For any color difference region, the sum of the region's R channel value and the region's G channel value is used as the numerator, and the sum of the region's R channel value, the region's G channel value, and the region's B channel value is used as the denominator. The ratio of the numerator and the denominator is used as the saliency of the color difference region.

5. The cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to claim 1, characterized in that, The method of adaptive image enhancement for each color difference region in the cholangioscopic image, based on the enhancement coefficient of each color difference region and using a gamma transform algorithm, includes: The enhancement coefficient of each chromatic difference region is used as the gamma value. Using the gamma transform algorithm, adaptive image enhancement is performed on each chromatic difference region in the cholangioscopic image to obtain the enhanced chromatic difference region.

6. The cholangioscopic image processing system for minimally invasive stone removal of bile duct stones according to claim 1, characterized in that, The method for obtaining the enhanced cholangioscopic image is as follows: the enhanced chromatic aberration area is replaced by the chromatic aberration area before enhancement to obtain the enhanced cholangioscopic image.

Images