Image recognition method and system for urinary system calculus

Abstract

The invention relates to the technical field of medical image processing and intelligent diagnosis. The image recognition method for the urinary system calculus comprises the steps that a dynamic video stream of a calculus area is obtained through an endoscope image collection module, and a current frame image is extracted; performing illumination compensation processing on the image, eliminating vascular interference and light reflection, and enhancing the boundary of a calculus main body; based on multispectral feature fusion, separating yellow-like spectrum features of uric acid stones and grey-white-like spectrum features of calcium stones, and calculating stone surface color distribution dispersion; performing multi-angle texture energy detection by adopting a direction sensitive convolution kernel to generate a calcification crystallization density coefficient; inputting the color distribution dispersion and the calcification crystallization density coefficient into a decision tree model, and outputting a calculus type identifier; and according to the stone type identifier and the three-dimensional projection area of the stone contour, frequency-energy combination parameters of the laser stone crusher are generated, and the laser pulse emission time sequence is controlled in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and intelligent diagnosis, and more specifically, to an image recognition method and system for urinary system stones. Background Art

[0002] In the field of diagnosis and treatment of urinary stones, traditional stone composition analysis methods mainly rely on laboratory tests after the stone is discharged, such as chemical analysis, infrared spectroscopy analysis, etc. These methods are not only time-consuming and unable to provide real-time stone composition information during surgery, but also unable to directly judge the composition of stones that have not been discharged. In the treatment of stones, especially in the application of laser lithotripsy, the frequency and energy parameters of the laser are currently mainly set based on the doctor's experience, and there is a lack of precise matching for different stone compositions and sizes, which may lead to poor lithotripsy or cause unnecessary damage to surrounding tissues. In addition, in terms of endoscopic image processing, the existing technology lacks effective real-time processing methods for problems such as vascular interference and reflection in the stone area, which affects the accuracy and reliability of stone identification.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the inability to accurately identify the composition of stones in real time during surgery, the lack of precise laser lithotripsy parameter optimization for stone composition and size, and insufficient processing of interference factors in endoscopic image processing. These problems limit the accuracy and safety of diagnosis and treatment of urinary stones. Summary of the invention

[0004] The present invention provides an image recognition method for urinary system stones, comprising:

[0005] S1. Obtain a dynamic video stream of the urinary system stone area through an endoscope image acquisition module, and extract a current frame image containing a stone outline;

[0006] S2. Performing vascular interference elimination and reflection suppression processing on the current frame image through the illumination compensation module to generate an enhanced main image of the stone;

[0007] S3. Based on the multispectral feature fusion module, the yellow-like spectrum features of uric acid stones and the gray-white spectrum features of calcium stones are separated from the main image of the stone, and the discreteness of the color distribution on the stone surface is calculated;

[0008] S4. Based on the adaptive texture analysis module, a direction-sensitive convolution kernel is used to perform multi-angle texture energy detection on the stone surface to generate a calcification crystal density coefficient;

[0009] S5. Inputting the color distribution dispersion and the calcification crystal density coefficient into a stone composition decision tree model, and outputting a stone type identifier, wherein the identifier includes uric acid stones, calcium oxalate stones, and calcium phosphate stones;

[0010] S6. Based on the stone type identifier and the three-dimensional projection area of ​​the stone contour, the frequency-energy combination parameters of the laser lithotripsy are generated through a dynamic energy matching module, and the laser pulse emission timing is controlled in real time.

[0011] Furthermore, the step S2 comprises:

[0012] S21. A hemoglobin absorption spectrum filter is used to eliminate red interference in the blood vessel area in the image, and the filtering wavelength range is 580-620nm;

[0013] S22. Perform polarized light reflection suppression processing on the filtered image, and dynamically adjust the compensation matrix weight by calculating the pixel saturation difference value of the mirror reflection area;

[0014] S23. Based on the difference in edge sharpness between the stone and surrounding tissues, an adaptive contrast stretching algorithm is used to enhance the main boundary of the stone.

[0015] Furthermore, the calculation method of the color distribution dispersion in step S3 is:

[0016] S31. extracting a* channel (red-green axis) and b* channel (yellow-blue axis) data of the stone area in CIELab color space;

[0017] S32. Based on the yellow feature of uric acid stones, calculate the coefficient of variation of the pixel value of the b* channel in the stone area:

[0018] ;

[0019] in, is the b* channel standard deviation, is the b* channel mean;

[0020] S33. According to the gray-white characteristics of calcium stones, the joint discreteness of the a* channel and the b* channel is calculated:

[0021] ;

[0022] in, is the a* channel standard deviation, is the a* channel mean;

[0023] S34. and The weighted sum is used as the color distribution dispersion.

[0024] Furthermore, the process of generating the calcification crystal density coefficient in step S4 includes:

[0025] S41. Construct directional Gabor filter banks with calcification features along four directions of 0°, 45°, 90°, and 135° respectively;

[0026] S42. Perform multi-directional filtering on the stone surface image and extract the energy peak of the filtering response in each direction:

[0027] ;

[0028] in, for The Gabor kernel function of the direction, It is the surface image of the stone;

[0029] S43. Calculate the ratio of the variance to the mean of the energy peak as the calcification crystal density coefficient.

[0030] Furthermore, the stone composition decision tree model is constructed as follows:

[0031] S51. The first-level decision node determines whether the color distribution dispersion exceeds the calcium stone threshold, and if not, it is classified as uric acid stone;

[0032] S52. The second-level decision node divides the calcification crystal density coefficient into intervals. When the coefficient is greater than 0.7, it is classified as calcium phosphate stone; otherwise, it is classified as calcium oxalate stone.

[0033] S53. Dynamically adjust the decision tree node threshold according to the stone disintegration rate feedback collected in real time during the operation.

[0034] Furthermore, the generation logic of the frequency-energy combination parameter in step S6 includes:

[0035] S61. Establish a mapping table of stone types and basic energies, wherein uric acid stones correspond to a low-frequency and low-energy mode, calcium oxalate stones correspond to a medium-frequency and medium-energy mode, and calcium phosphate stones correspond to a high-frequency and high-energy mode;

[0036] S62. Calculate the energy compensation coefficient based on the projection area of ​​the stone contour in the three-dimensional reconstruction model:

[0037] ;

[0038] Among them, A is the current stone projection area, is the base area (5mm²), is the empirical coefficient;

[0039] S63. Multiply the basic energy value by the compensation coefficient to generate the final laser output energy.

[0040] Furthermore, the dynamic threshold adjustment method of step S53 includes:

[0041] S71. During the laser action, the maximum splash velocity of the stone fragments is captured by the high-speed camera module ;

[0042] S72. When When it is lower than the expected value, the classification threshold of the calcification crystal density coefficient is increased proportionally;

[0043] S73. When the fragmentation speed fails to meet the standard for three consecutive times, the retraining process of the decision tree model is triggered.

[0044] Furthermore, the weighted summation formula of step S34 is:

[0045] ;

[0046] in, for The weight coefficient of for The weight coefficient of , Determined based on the differences in color characteristic distribution of uric acid / calcium stones in the urinary stone sample library.

[0047] Furthermore, the construction formula of the directional Gabor kernel function is:

[0048] ;

[0049] in, and is the coordinate value after coordinate transformation, is the ellipticity, is the standard deviation, is the wavelength, is the phase shift.

[0050] Furthermore, it also includes an intraoperative safety monitoring module:

[0051] S101. Calculate the minimum distance between the laser action area and the ureter wall in real time, and trigger laser occlusion when the distance is less than 2 mm;

[0052] S102. Analyze the amount of melanin generated in the image through the tissue carbonization detection submodule and dynamically reduce the energy output;

[0053] S103. When it is detected that the blood turbulence interference exceeds the safety threshold, the pulse interval extension protection mechanism is activated.

[0054] The above-mentioned embodiments of the present invention have at least the following beneficial effects: the present invention can accurately separate the characteristics of uric acid stones and calcium stones through multi-spectral feature fusion and adaptive texture analysis technology, accurately calculate the discreteness of the color distribution on the stone surface and the calcification crystal density coefficient, and thus realize rapid and accurate identification of the stone type. Combined with the three-dimensional projection area of ​​the stone contour, accurate frequency-energy combination parameters can be generated for the laser lithotripsy, and the laser pulse emission timing can be controlled in real time, thereby improving the effect of laser lithotripsy, reducing damage to surrounding tissues, and improving the safety and effectiveness of the operation.

[0055] In addition, the present invention also has good intraoperative adaptability and safety. Through the dynamic energy matching module, the laser parameters can be adjusted in real time according to the type and size of the stone to meet the treatment needs of different stones. At the same time, the intraoperative safety monitoring module can monitor the distance between the laser action area and the ureter wall in real time, analyze the blood turbulence interference and tissue carbonization, take protective measures in time, further reduce the surgical risk, and ensure patient safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0057] Figure 1 A schematic flow chart of an image recognition method for urinary system stones provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0058] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0059] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0060] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0061] Reference below Figure 1 , Figure 1The following is a flow chart of an image recognition method for urinary system stones provided by an embodiment of the present invention. Figure 1 As shown, a method 100 for image recognition of urinary system stones includes:

[0062] S1. Obtain a dynamic video stream of the urinary system stone area through an endoscope image acquisition module, and extract a current frame image containing a stone outline;

[0063] S2. Performing vascular interference elimination and reflection suppression processing on the current frame image through the illumination compensation module to generate an enhanced main image of the stone;

[0064] S3. Based on the multispectral feature fusion module, the yellow-like spectrum features of uric acid stones and the gray-white spectrum features of calcium stones are separated from the main image of the stone, and the discreteness of the color distribution on the stone surface is calculated;

[0065] S4. Based on the adaptive texture analysis module, a direction-sensitive convolution kernel is used to perform multi-angle texture energy detection on the stone surface to generate a calcification crystal density coefficient;

[0066] S5. Inputting the color distribution dispersion and the calcification crystal density coefficient into a stone composition decision tree model, and outputting a stone type identifier, wherein the identifier includes uric acid stones, calcium oxalate stones, and calcium phosphate stones;

[0067] S6. Based on the stone type identifier and the three-dimensional projection area of ​​the stone contour, the frequency-energy combination parameters of the laser lithotripsy are generated through a dynamic energy matching module, and the laser pulse emission timing is controlled in real time.

[0068] It should be noted that, in the present invention, firstly, a dynamic video stream of the urinary system stone area is obtained through an endoscopic image acquisition module, and a current frame image containing the stone outline is extracted. The endoscopic image acquisition module here refers to a device specifically used to obtain internal images of the urinary system, which can capture the dynamic changes of the stone area in real time. This module usually includes a high-definition camera, a light source, and an image transmission system, which can transmit the image of the stone area to the processing system in the form of a video stream. The current frame image refers to an image extracted from the dynamic video stream at a specific moment, which contains the outline information of the stone and is the basic data for subsequent processing.

[0069] Specifically, the resolution of the endoscopic image acquisition module should be no less than 1920×1080 pixels to ensure that the detailed features of the stones can be clearly captured. The light source usually uses a cold light source with a wavelength range of 400-700 nanometers to ensure the brightness and contrast of the image. When extracting the current frame image, the system will identify the outline of the stone through an image analysis algorithm, which can accurately extract the boundary of the stone based on edge detection technology, such as the Canny edge detection algorithm. In addition, the frame rate of the dynamic video stream should be no less than 30 frames per second to ensure that the changes of the stones in different positions and angles can be captured.

[0070] Preferably, in order to improve the accuracy of stone contour extraction, the endoscope lens can be automatically calibrated before image acquisition to eliminate lens distortion. At the same time, multimodal image fusion technology can be used to combine endoscopic images with ultrasound images to further enhance the visualization of stones. When extracting the current frame image, in addition to the Canny edge detection algorithm, the convolutional neural network (CNN) in deep learning can also be used for contour extraction to improve the accuracy and robustness of the extraction. In addition, in order to adapt to different surgical environments, the endoscopic image acquisition module can also be equipped with adjustable light source intensity and color temperature to optimize image quality.

[0071] In some embodiments, step S2 includes:

[0072] S21. A hemoglobin absorption spectrum filter is used to eliminate red interference in the blood vessel area in the image, and the filtering wavelength range is 580-620nm;

[0073] S22. Perform polarized light reflection suppression processing on the filtered image, and dynamically adjust the compensation matrix weight by calculating the pixel saturation difference value of the mirror reflection area;

[0074] S23. Based on the difference in edge sharpness between the stone and surrounding tissues, an adaptive contrast stretching algorithm is used to enhance the main boundary of the stone.

[0075] It should be noted that in step S2, the illumination compensation module processes the current frame image in order to eliminate vascular interference and reflections, while enhancing the main boundary of the stone. The illumination compensation module here refers to an image processing unit that optimizes the image through a specific algorithm to improve the visibility of the stone area. Specifically, the module includes three sub-steps: first, a hemoglobin absorption spectrum filter is used to eliminate red interference in the vascular area; second, the filtered image is subjected to polarized light reflection suppression processing; and finally, the main boundary of the stone is enhanced through an adaptive contrast stretching algorithm. These processing steps work together to make the stone clearer in the image, which is convenient for subsequent analysis.

[0076] Specifically, the function of the hemoglobin absorption spectrum filter is to remove red interference in the blood vessel area in the image. Its filtering wavelength range is 580-620 nanometers. This is because hemoglobin has a strong absorption effect on light in this wavelength range. The polarized light reflection suppression processing calculates the pixel saturation difference value of the mirror reflection area and dynamically adjusts the compensation matrix weight to reduce the impact of reflection on the image. The adaptive contrast stretching algorithm enhances the main boundary of the stone according to the difference in edge sharpness between the stone and the surrounding tissue, making the stone more prominent in the image. The setting of these parameters and the application of the algorithm are all aimed at improving the quality of the image and providing a better basis for the identification and analysis of stones.

[0077] Preferably, when performing hemoglobin absorption spectrum filtering, the filtering wavelength range can be adjusted according to different surgical environments and stone types to achieve the best filtering effect. For example, in some cases, the wavelength range can be extended to 570-630 nanometers to more comprehensively cover the absorption peak of hemoglobin. In the polarized light reflection suppression process, multi-angle polarized light detection technology can be used to capture reflected light from different angles, so as to more accurately calculate the pixel saturation difference value. In addition, the adaptive contrast stretching algorithm can dynamically adjust the stretching parameters according to the size and shape of the stone to better highlight the stone boundary. These refined operating steps and alternatives can further improve the effect of image processing and enhance the accuracy of stone identification.

[0078] In some embodiments, the color distribution dispersion in step S3 is calculated as follows:

[0079] S31. extracting a* channel (red-green axis) and b* channel (yellow-blue axis) data of the stone area in CIELab color space;

[0080] S32. Based on the yellow feature of uric acid stones, calculate the coefficient of variation of the pixel value of the b* channel in the stone area:

[0081] ;

[0082] in, is the b* channel standard deviation, is the b* channel mean;

[0083] S33. According to the gray-white characteristics of calcium stones, the joint discreteness of the a* channel and the b* channel is calculated:

[0084] ;

[0085] in, is the a* channel standard deviation, is the a* channel mean;

[0086] S34. and The weighted sum is used as the color distribution dispersion.

[0087] It should be noted that in step S3, the calculation of the color distribution discreteness is based on the color characteristics of the stones, with the purpose of distinguishing uric acid stones from calcium stones. The CIELab color space here is a color space defined by the International Commission on Illumination (CIE), which can accurately describe the three-dimensional characteristics of color, in which the a* channel represents the red-green axis and the b* channel represents the yellow-blue axis. By extracting the a* and b* channel data of the stone area in the CIELab color space, the color characteristics of the stones can be analyzed. For the yellow characteristics of uric acid stones, the coefficient of variation of the b* channel is calculated to evaluate the discreteness of the yellow distribution; for the gray-white characteristics of calcium stones, the joint discreteness of the a* channel and the b* channel is calculated to comprehensively evaluate the discreteness of the gray-white distribution. Finally, the two discretenesses are weighted and summed to obtain the comprehensive color distribution discreteness for subsequent stone type judgment.

[0088] Specifically, the data extraction of a* and b* channels in CIELab color space is completed by image processing algorithms, which can convert RGB images into CIELab color space. For the yellow characteristics of uric acid stones, the coefficient of variation is calculated by dividing the standard deviation of the b* channel by its mean and then multiplying it by 100%, which reflects the relative discreteness of the yellow pixel value. For the gray-white characteristics of calcium stones, the joint discreteness is calculated by weighted summing the ratio of the standard deviation of the a* channel and the b* channel to the mean, which comprehensively considers the changes in red, green, and yellow and blue. In practical applications, the mean and standard deviation of the a* and b* channels can be calculated by image analysis software or custom algorithms, and the setting and calculation methods of these parameters are conventional techniques in the field of image processing.

[0089] Preferably, when calculating the color distribution discreteness, the weight coefficient can be dynamically adjusted according to the data in the stone sample library. For example, if the yellow feature of uric acid stones in the sample library is more significant, the weight of the b* channel coefficient of variation can be appropriately increased. In addition, in order to improve the extraction accuracy of color features, a noise filtering step can be added in the image preprocessing stage, and Gaussian filtering or median filtering can be used to remove random noise in the image. In practical applications, other color features, such as brightness information, can also be combined to further enhance the ability to distinguish stone types. These refined operating steps and alternatives can improve the accuracy and robustness of the calculation of color distribution discreteness, thereby better supporting the identification of stone types.

[0090] In some embodiments, the process of generating the calcification crystal density coefficient in step S4 includes:

[0091] S41. Construct directional Gabor filter banks with calcification features along four directions of 0°, 45°, 90°, and 135° respectively;

[0092] S42. Perform multi-directional filtering on the stone surface image and extract the energy peak of the filtering response in each direction:

[0093] ;

[0094] in, for The Gabor kernel function of the direction, It is the surface image of the stone;

[0095] S43. Calculate the ratio of the variance to the mean of the energy peak as the calcification crystal density coefficient.

[0096] It should be noted that in step S4, the generation process of the calcification crystal density coefficient is achieved through multi-directional texture energy detection. The direction-sensitive convolution kernel here refers to a mathematical tool that can detect texture features in a specific direction in an image, which is usually used in the field of image analysis and processing. By constructing a directional Gabor filter group with calcification characteristics, the stone surface image can be multi-directionally filtered to extract the texture energy peaks in each direction. These energy peaks reflect the distribution of calcification crystals on the stone surface, and then by calculating the ratio of the variance to the mean of the energy peak, the calcification crystal density coefficient is obtained for subsequent stone type judgment.

[0097] Specifically, the directional Gabor filter group is constructed along four directions of 0°, 45°, 90°, and 135°, which can fully cover the texture features of the stone surface. The Gabor filter is a linear filter whose kernel function has directionality and frequency selectivity, and can effectively extract local texture information in the image. When applying the Gabor filter, it is first necessary to filter the stone surface image and extract the filter response in each direction. Then, by calculating the energy peak of these responses, the texture intensity of the stone surface is evaluated. The energy peak is calculated by summing the square sum of each pixel value in the filter response image, which reflects the significance of the texture in a specific direction. Finally, by calculating the ratio of the variance to the mean of these energy peaks, the calcification crystal density coefficient is obtained, which can quantify the distribution density of calcification crystals on the stone surface.

[0098] Preferably, when constructing a directional Gabor filter, the filter parameters, such as wavelength, standard deviation, and ellipticity, can be adjusted according to the size and texture characteristics of the stone. For example, for smaller stones, a shorter wavelength and a smaller standard deviation can be used to improve the detection ability of detailed textures. In addition, when calculating the energy peak, a multi-scale Gabor filter group can be used to extract texture energies at different scales, and then these energies are fused to more comprehensively reflect the texture characteristics of the stone surface. In practical applications, other texture analysis methods, such as local binary pattern (LBP) features, can also be combined to further enhance the description of the stone surface texture. These refined operating steps and alternatives can improve the accuracy and robustness of the calculation of the calcification crystal density coefficient, thereby better supporting the identification of stone types.

[0099] In some embodiments, the stone composition decision tree model is constructed as follows:

[0100] S51. The first-level decision node determines whether the color distribution dispersion exceeds the calcium stone threshold, and if not, it is classified as uric acid stone;

[0101] S52. The second-level decision node divides the calcification crystal density coefficient into intervals. When the coefficient is greater than 0.7, it is classified as calcium phosphate stone; otherwise, it is classified as calcium oxalate stone.

[0102] S53. Dynamically adjust the decision tree node threshold according to the stone disintegration rate feedback collected in real time during the operation.

[0103] It should be noted that the construction of the stone composition decision tree model is based on the two key features of color distribution discreteness and calcification crystal density coefficient. Decision tree is a common classification model, which classifies data through a series of decision nodes. In the present invention, the decision tree model determines the type of stone through two main decision nodes. The first-level decision node uses color distribution discreteness to distinguish uric acid stones from calcium stones, because uric acid stones usually have a lower color distribution discreteness, while calcium stones are relatively high. The second-level decision node further subdivides calcium stones according to the calcification crystal density coefficient, dividing them into calcium oxalate stones and calcium phosphate stones. In addition, the decision tree model can also dynamically adjust the decision tree node threshold according to the stone disintegration rate feedback collected in real time during the operation to adapt to different surgical conditions and stone characteristics.

[0104] Specifically, the first-level decision node of the decision tree model sets a threshold for the color distribution dispersion. When the color distribution dispersion is lower than the threshold, the stone is classified as a uric acid stone; otherwise, it is classified as a calcium stone. Further classification of calcium stones is completed through the second-level decision node, which is based on the interval division of the calcification crystal density coefficient. When the calcification crystal density coefficient is greater than 0.7, the stone is classified as a calcium phosphate stone; otherwise, it is classified as a calcium oxalate stone. These thresholds and interval divisions are based on a large amount of experimental data and clinical experience, and can effectively distinguish different types of stones. In addition, the dynamic adjustment mechanism of the decision tree model is achieved by monitoring the disintegration rate of the stone under the action of laser. If the disintegration rate is lower than the expected value, it means that the current classification may be inaccurate, and the threshold of the decision tree node needs to be adjusted to improve the accuracy of the classification.

[0105] Preferably, in order to improve the accuracy and adaptability of the decision tree model, more features and sample data can be introduced in the model training stage. For example, in addition to the color distribution discreteness and the calcification crystal density coefficient, the shape, size and surface texture of the stone can also be considered to build a more comprehensive decision tree model. In the dynamic adjustment mechanism, in addition to monitoring the stone disintegration rate, other feedback information, such as the consumption of laser energy and the degree of stone fragmentation, can also be combined to comprehensively judge whether it is necessary to adjust the threshold of the decision tree node. In addition, the training of the decision tree model can adopt machine learning algorithms, such as ID3, C4.5 or CART, which can automatically optimize the structure and threshold setting of the decision tree according to the training data. In practical applications, the decision tree model can also be regularly updated and optimized to adapt to the changing clinical needs and stone characteristics.

[0106] In some embodiments, the generation logic of the frequency-energy combination parameter in step S6 includes:

[0107] S61. Establish a mapping table between stone types and basic energy: uric acid stones correspond to low-frequency and low-energy mode (8Hz, 0.8J), calcium oxalate stones correspond to medium-frequency and medium-energy mode (12Hz, 1.2J), and calcium phosphate stones correspond to high-frequency and high-energy mode (15Hz, 1.5J);

[0108] S62. Calculate the energy compensation coefficient based on the projection area of ​​the stone contour in the three-dimensional reconstruction model:

[0109] ;

[0110] Among them, A is the current stone projection area, is the base area (5mm²), is the empirical coefficient;

[0111] S63. Multiply the basic energy value by the compensation coefficient to generate the final laser output energy.

[0112] It should be noted that the generation logic of the frequency-energy combination parameter in step S6 is to optimize the working parameters of the laser lithotripsy based on the stone type and the three-dimensional projection area of ​​the stone contour. The frequency-energy combination parameter here refers to the combination of the emission frequency of the laser pulse and the energy of each pulse during the operation of the laser lithotripsy. By establishing a stone type and basic energy mapping table, different basic energy modes can be set according to different stone types. At the same time, the energy compensation coefficient is calculated according to the three-dimensional projection area of ​​the stone contour to adjust the laser output energy, thereby realizing accurate laser lithotripsy for stones of different sizes and types.

[0113] Specifically, the stone type and basic energy mapping table is a preset table that matches the stone type with the corresponding laser frequency and energy parameters. For example, uric acid stones correspond to low-frequency and low-energy mode (8Hz, 0.8J), calcium oxalate stones correspond to medium-frequency and medium-energy mode (12Hz, 1.2J), and calcium phosphate stones correspond to high-frequency and high-energy mode (15Hz, 1.5J). These parameters are set according to the physical characteristics of the stones and clinical experience, and can effectively break up the corresponding types of stones. When calculating the energy compensation coefficient, it is first necessary to determine the projected area of ​​the stone outline in the three-dimensional reconstructed model, and then calculate the compensation coefficient according to the formula. The reference area (5mm²) and empirical coefficient (0.2) in the formula are derived from experimental data and clinical experience, and are used to adjust the laser energy to adapt to stones of different sizes.

[0114] Preferably, in order to further optimize the effect of laser lithotripsy, the parameters in the basic energy mapping table can be fine-tuned according to the specific location and shape of the stone. For example, for stones close to the ureteral wall, the laser energy can be appropriately reduced to reduce damage to surrounding tissues. When calculating the energy compensation coefficient, more factors can be introduced, such as the hardness of the stone and the individual differences of the patient, to more accurately adjust the laser energy. In addition, the frequency and energy of the laser can be dynamically adjusted in combination with real-time image feedback and the doctor's operating experience to achieve the best lithotripsy effect. In actual applications, the laser lithotripsy machine can be equipped with an intelligent control system to automatically adjust the laser parameters according to the type and size of the stone, thereby improving the safety and efficiency of the operation.

[0115] In some embodiments, the dynamic threshold adjustment method of step S53 includes:

[0116] S71. During the laser action, the maximum splash velocity of the stone fragments is captured by the high-speed camera module ;

[0117] S72. When When it is lower than the expected value, the classification threshold of the calcification crystal density coefficient is increased proportionally;

[0118] S73. When the fragmentation speed fails to meet the standard for three consecutive times, the retraining process of the decision tree model is triggered.

[0119] It should be noted that the dynamic threshold adjustment method of step S53 is to optimize the classification threshold of the decision tree model based on the maximum splash speed of the stone fragments. The high-speed camera module here is a device that can capture images at a high frame rate, and is used to monitor the splash of the stone fragments under the action of the laser in real time. By analyzing the maximum splash speed of the fragments, it can be determined whether the classification of the current decision tree model is accurate. If the splash speed is lower than the expected value, it means that the current classification may be inaccurate, and the classification threshold of the calcification crystal density coefficient needs to be adjusted. When the fragment speed fails to meet the standard for multiple consecutive times, the retraining process of the decision tree model is triggered to adapt to different stone characteristics and surgical conditions.

[0120] Specifically, the frame rate of the high-speed camera module should be no less than 1000 frames per second to ensure that the splashing process of the broken fragments can be accurately captured. The maximum splashing speed of the broken fragments is calculated by analyzing the image sequence captured by the high-speed camera module. An image recognition algorithm is usually used to track the movement trajectory of the broken fragments and calculate their speed. The classification threshold of the calcification crystal density coefficient is set based on a large amount of experimental data and clinical experience, and the initial value can be adjusted according to the stone type and surgical conditions. For example, if the maximum splashing speed of the broken fragments is lower than the expected value, it means that the current calcification crystal density coefficient may be too high, and the classification threshold needs to be appropriately lowered. The specific number of consecutive times that the speed of the broken fragments does not meet the standard can be set according to actual conditions, such as 3 times or 5 times, to ensure that the adjustment of the decision tree model is sufficiently reliable.

[0121] Preferably, in order to improve the accuracy and adaptability of dynamic threshold adjustment, multi-view shooting technology can be introduced in the high-speed camera module to capture the splash of broken pieces from different angles to reduce the error caused by perspective deviation. When adjusting the classification threshold of the calcification crystal density coefficient, other feedback information, such as the consumption of laser energy and the degree of stone fragmentation, can be combined to make a comprehensive judgment. In addition, the retraining process of the decision tree model can use machine learning algorithms, such as random forests or gradient boosting trees, which can automatically optimize the structure and threshold settings of the decision tree based on new data. In practical applications, the decision tree model can also be regularly updated and optimized to adapt to changing clinical needs and stone characteristics.

[0122] In some embodiments, the weighted summation formula of step S34 is:

[0123] ;

[0124] in, for The weight coefficient of for The weight coefficient of , Determined based on the differences in color characteristic distribution of uric acid / calcium stones in the urinary stone sample library.

[0125] It should be noted that the weighted summation formula in step S34 is used to combine the yellow characteristics of uric acid stones and the gray-white characteristics of calcium stones to obtain an index that comprehensively reflects the dispersion of stone color distribution. The weighted summation here is a mathematical operation method that combines multiple feature values ​​into a comprehensive value by assigning different weights to different features. The weight coefficient ( and ) reflects the importance of different features in the comprehensive index, and their values ​​are determined based on the difference in color feature distribution of uric acid stones and calcium stones in the urinary stone sample library. This weighted summation method can more accurately reflect the color characteristics of stones, thereby improving the accuracy of stone type identification.

[0126] Specifically, the color distribution dispersion is obtained by weighted summing the yellow feature of uric acid stones (coefficient of variation of the b* channel) and the gray feature of calcium stones (joint dispersion of the a* channel and the b* channel). and The value of can be adjusted according to the statistical data in the stone sample library. For example, if the yellow feature of uric acid stones in the sample library is more prominent, the value of On the contrary, if the gray-white characteristics of calcium stones are more important, you can increase The weight coefficients can be automatically optimized based on sample data using machine learning algorithms, such as linear regression or support vector machines. In addition, the weighted summation result can be input as a feature value into a subsequent decision tree model for stone type classification.

[0127] Preferably, in order to further improve the accuracy and robustness of the calculation of the color distribution discreteness, more color features can be introduced into the weighted summation formula. For example, in addition to the a* and b* channels, the characteristics of the L* channel (brightness channel) can also be considered to more comprehensively describe the color information of the stone. When determining the weight coefficient, a cross-validation method can be used to adjust the weight coefficient through multiple experiments to find the optimal combination. In addition, the characteristics of other color spaces (such as RGB or HSV) can be combined to further enhance the ability to describe the color of the stone. In practical applications, the weight coefficient can be dynamically adjusted according to different stone types and surgical conditions to meet different recognition needs.

[0128] In some embodiments, the construction formula of the directional Gabor kernel function is:

[0129] ;

[0130] in, and is the coordinate value after coordinate transformation, is the ellipticity, is the standard deviation, is the wavelength, is the phase shift.

[0131] It should be noted that the formula for constructing the directional Gabor kernel function is used to generate filters of specific directions and frequencies to extract the texture features of the stone surface. The Gabor kernel function here is a linear filter, whose design is inspired by the human visual system and can effectively capture the local texture information in the image. The parameters in the formula include the coordinate values ​​after coordinate transformation ( and ), ellipticity, standard deviation, wavelength and phase shift, these parameters together determine the shape and characteristics of the Gabor filter. By adjusting these parameters, Gabor filters suitable for different directions and frequencies can be generated for multi-directional and multi-scale texture analysis of stone surface images.

[0132] Specifically, the construction process of the Gabor kernel function includes the following steps: First, determine the center frequency and direction of the filter. These parameters determine the sensitivity of the filter to specific frequency and directional textures. Then, select the appropriate standard deviation and ellipticity to control the spatial extension and shape of the filter. The standard deviation determines the bandwidth of the filter, and the ellipticity affects the directionality of the filter. Finally, set the phase offset to adjust the response characteristics of the filter. In practical applications, according to the texture characteristics of the stone surface, a suitable combination of parameters can be selected to generate the optimal Gabor filter group. For example, for fine and dense textures, a smaller standard deviation and a higher center frequency can be selected; for coarse and sparse textures, a larger standard deviation and a lower center frequency can be selected.

[0133] Preferably, in order to improve the extraction effect of Gabor filter on the texture features of the stone surface, a multi-scale and multi-directional filter group can be used. Specifically, a group of Gabor filters can be generated at different scales, and each group of filters covers different directions and frequency ranges to fully capture the texture information of the stone surface. In addition, other texture analysis methods, such as local binary patterns (LBP) or gray-level co-occurrence matrix (GLCM), can be combined to further enhance the description of the texture features of the stone surface. In practical applications, the parameters of the Gabor filter can be dynamically adjusted according to the specific type of stone and surgical requirements to achieve the best texture feature extraction effect. These refined operating steps and alternatives can improve the accuracy and robustness of stone identification, thereby better supporting subsequent diagnosis and treatment.

[0134] In some embodiments, an intraoperative safety monitoring module is also included:

[0135] S101. Calculate the minimum distance between the laser action area and the ureter wall in real time, and trigger laser occlusion when the distance is less than 2 mm;

[0136] S102. Analyze the amount of melanin generated in the image through the tissue carbonization detection submodule and dynamically reduce the energy output;

[0137] S103. When it is detected that the blood turbulence interference exceeds the safety threshold, the pulse interval extension protection mechanism is activated.

[0138] It should be noted that the intraoperative safety monitoring module is set up to monitor the surgical environment in real time during the laser lithotripsy process to ensure the safety of the operation. The intraoperative safety monitoring module here is an integrated monitoring system that uses a variety of sensors and image analysis technologies to detect key parameters such as the distance between the laser action area and the ureter wall, tissue carbonization, and blood turbulence interference in real time. When these parameters exceed the safety threshold, the system will automatically take corresponding protective measures, such as laser lockout, energy reduction, or pulse interval extension, to prevent damage to surrounding tissues.

[0139] Specifically, the intraoperative safety monitoring module includes the following three submodules: First, the minimum distance between the laser action area and the ureteral wall is calculated in real time. This process is achieved through an image analysis algorithm. When the distance is less than 2 mm, the system triggers the laser lockout mechanism to prevent the laser from damaging the ureteral wall. Secondly, the tissue carbonization detection submodule dynamically adjusts the laser energy output by analyzing the amount of melanin generated in the image. When signs of tissue carbonization are detected, the system automatically reduces the laser energy to reduce thermal damage. Finally, the blood turbulence interference detection submodule monitors blood flow. When the blood turbulence interference exceeds the safety threshold, the system activates the pulse interval extension protection mechanism to ensure the safety of the surgical process. The setting and monitoring mechanism of these parameters are based on a large amount of experimental data and clinical experience, and can effectively ensure the safety and effectiveness of the operation.

[0140] Preferably, in order to further improve the accuracy and reliability of the intraoperative safety monitoring module, three-dimensional reconstruction technology can be introduced in the real-time distance calculation to more accurately measure the distance between the laser action area and the ureter wall. In addition, the tissue carbonization detection submodule can be combined with spectral analysis technology to more accurately judge the degree of tissue carbonization by detecting changes in light absorption at specific wavelengths. For blood turbulence interference detection, Doppler ultrasound technology can be introduced to monitor the blood flow speed and direction in real time, so as to more accurately assess the blood turbulence. In practical applications, these monitoring modules can also be integrated with intelligent control systems to achieve automated surgical safety monitoring and protection, further reduce surgical risks, and ensure patient safety.

[0141] The above-mentioned embodiments of the present invention have the following beneficial effects: The present invention obtains the dynamic video stream of the urinary system stone area in real time through the endoscopic image acquisition module, and extracts the current frame image containing the stone outline to provide accurate data for subsequent processing. The illumination compensation module eliminates vascular interference and reflection, enhances the main boundary of the stone, and improves the image quality. The multi-spectral feature fusion module separates the yellow-like spectrum features of uric acid stones and the gray-white spectrum features of calcium stones, calculates the color distribution discreteness, and provides color feature basis for stone type identification. The adaptive texture analysis module uses a direction-sensitive convolution kernel to perform multi-angle texture energy detection on the stone surface, generates a calcification crystal density coefficient, further enriches the stone feature information, and improves the recognition accuracy. The stone composition decision tree model outputs a stone type identifier based on the color distribution discreteness and the calcification crystal density coefficient to achieve rapid and accurate classification of stone types. The dynamic energy matching module generates the frequency-energy combination parameters of the laser lithotripsy according to the stone type identifier and the three-dimensional projection area of ​​the stone contour, and controls the laser pulse emission timing in real time to achieve accurate matching and real-time adjustment of the laser lithotripsy parameters, improve the lithotripsy effect, reduce damage to surrounding tissues, and enhance the safety and effectiveness of the operation. At the same time, the intraoperative safety monitoring module monitors key parameters such as the distance between the laser action area and the ureteral wall, tissue carbonization, and blood turbulence interference in real time. When these parameters exceed the safety threshold, the system automatically takes corresponding protective measures, such as laser lockout, energy reduction, or pulse interval extension, to effectively ensure the safety of the operation and reduce the risk of damage to surrounding tissues.

[0142] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0143] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A method for image recognition of urinary system stones, characterized in that: The following steps are involved: The dynamic video stream of the urinary system stone area is obtained through the endoscope image acquisition module, and the current frame image containing the stone outline is extracted; Performing blood vessel interference elimination and reflection suppression processing on the current frame image through an illumination compensation module to generate an enhanced stone main body image; Extracting the yellow-like spectrum characteristics of uric acid stones and the gray-white spectrum characteristics of calcium stones from the enhanced stone main body image, and then calculating the color distribution dispersion of the stone main body surface; A direction-sensitive convolution kernel was used to detect the multi-angle texture energy of the stone body surface to generate the calcification crystal density coefficient; Inputting the color distribution dispersion and the calcification crystal density coefficient into a stone composition decision tree model, and outputting a stone type identifier, wherein the stone types include uric acid stones, calcium oxalate stones, and calcium phosphate stones; According to the stone type identifier and the three-dimensional projection area of ​​the stone contour, the frequency-energy combination parameters of the laser lithotripsy are generated through a dynamic energy matching module, and the laser pulse emission timing is controlled in real time.

2. The method according to claim 1, characterized in that The process of performing blood vessel interference elimination and reflection suppression processing on the current frame image by using the illumination compensation module to generate an enhanced stone main body image comprises the following steps: Based on the current frame image, a hemoglobin absorption spectrum filter is used to eliminate the red interference in the blood vessel area in the image to obtain a filtered image, wherein the filtering wavelength range is 580-620nm; The filtered image is subjected to polarized light reflection suppression processing, and the pixel saturation difference value of the specular reflection area is calculated, thereby dynamically adjusting the compensation matrix weight; According to the pixel saturation difference between the stone and the surrounding tissue, an adaptive contrast stretching algorithm was used to enhance the main boundary of the stone.

3. The method according to claim 1, characterized in that The color distribution dispersion is obtained by: Extract the a* channel data and b* channel data of the stone area in the CIELab color space; wherein the a* channel data is the red-green axis data, and the b* channel yellow-blue axis data is the yellow-blue axis data; According to the yellow feature of uric acid stones, the coefficient of variation of the pixel value of the b* channel in the stone area is calculated: ； in, is the b* channel standard deviation, is the b* channel mean, is the coefficient of variation of the pixel value of the b* channel in the stone area; Based on the gray-white characteristics of calcium stones, the joint discreteness of the a* channel and the b* channel is calculated: ； in, is the a* channel standard deviation, is the a* channel mean, is the joint discreteness of a* channel and b* channel; The coefficient of variation of the pixel value of the b* channel in the stone area The joint discreteness of the a* channel and the b* channel The weighted sum is used as the color distribution dispersion.

4. The method according to claim 1, characterized in that: The calcification crystal density coefficient is generated by the following steps: Directional Gabor filter banks with calcification features are constructed along the four directions of 0°, 45°, 90°, and 135°; Perform multi-directional filtering on the stone surface image and extract the energy peak of the filtering response in each direction, as shown in the following formula: ； in, for The Gabor kernel function of the direction, For the stone surface image, for Energy peak in direction; The ratio of the variance to the mean of the energy peak is calculated, and the ratio is used as the calcification crystal density coefficient.

5. The method according to claim 1, characterized in that The stone composition decision tree model is optimized by the following steps: The first-level decision node determines whether the color distribution dispersion exceeds the calcium stone threshold, and if not, it is classified as uric acid stone; The second-level decision node divides the calcification crystal density coefficient into intervals. When the coefficient is > 0.7, it is classified as calcium phosphate stone, otherwise it is classified as calcium oxalate stone; The decision tree node thresholds were dynamically adjusted based on the stone disintegration rate feedback collected in real time during surgery.

6. The method according to claim 1, characterized in that The frequency-energy combination parameter, i.e., the combination of pulse frequency and energy parameter, is obtained in the following manner: Establish a stone type and basic energy mapping table, in which uric acid stones correspond to a low-frequency, low-energy mode, calcium oxalate stones correspond to a medium-frequency, medium-energy mode, and calcium phosphate stones correspond to a high-frequency, high-energy mode; The energy compensation coefficient is calculated based on the projection area of ​​the stone contour in the three-dimensional reconstruction model: ； Among them, A is the current stone projection area, is the base area (5mm²), is the empirical coefficient, is the energy compensation coefficient; The base energy value is multiplied by the compensation factor to generate the final laser output energy.

7. The method according to claim 5, characterized in that The decision tree node threshold is dynamically adjusted according to the stone disintegration rate feedback collected in real time during the operation. The following steps are involved: During the laser action, the maximum splash velocity of the stone fragments was captured by a high-speed camera module; When the maximum splash velocity is lower than the expected value, the classification threshold of the calcification crystal density coefficient is increased proportionally; When the fragmentation speed fails to meet the standard for three consecutive times, the retraining process of the decision tree model is triggered.

8. The method according to claim 3, characterized in that The weighted sum is expressed as follows: ； in, for The weight coefficient of for The weight coefficient of , Determined based on the color characteristic distribution differences of uric acid / calcium stones in the urinary stone sample library, Represents the weighted summation result.

9. The method according to claim 4, characterized in that The directional Gabor kernel function is expressed as: ； in, and is the coordinate value after coordinate transformation, is the ellipticity, is the standard deviation, is the wavelength, is the phase shift.

10. An image recognition system for urinary stones, characterized in that: include: The data acquisition module acquires the dynamic video stream of the urinary system stone area through the endoscope image acquisition module, and extracts the current frame image containing the stone outline; An illumination compensation module performs vascular interference elimination and reflection suppression processing on the current frame image to generate an enhanced stone main body image; A spectral feature fusion module extracts the yellow-like spectrum features of uric acid stones and the gray-white spectrum features of calcium stones from the enhanced stone main body image, and then calculates the color distribution dispersion of the stone main body surface; The adaptive texture analysis module uses a direction-sensitive convolution kernel to perform multi-angle texture energy detection on the surface of the stone body to generate a calcification crystal density coefficient; an identification module, inputting the color distribution dispersion and the calcification crystal density coefficient into a stone composition decision tree model, and outputting a stone type identifier, wherein the stone types include uric acid stones, calcium oxalate stones, and calcium phosphate stones; According to the stone type identifier and the three-dimensional projection area of ​​the stone contour, the frequency-energy combination parameters of the laser lithotripsy are generated through a dynamic energy matching module, and the laser pulse emission timing is controlled in real time.

Images