Method for removing residual pieces of kidney stones under fluorescence guidance

Abstract

The application discloses a method for removing residual pieces of kidney stones under fluorescence guidance, and belongs to the technical field of medical devices. The method first establishes a database containing standard fluorescence spectrum curves of different types of stones. During the operation, a multispectral imaging system integrated in an endoscope emits excitation light of at least three wavelengths and collects a time sequence of multispectral fluorescence images of the operation area. The background and target spectrum are separated by a spectral decomposition algorithm, and the target spectrum is matched and identified with the database to locate the residual pieces of stones. Finally, the identification result is superimposed in real time on the endoscope white light image as a highlighted signal to generate an augmented reality guide image, which guides the doctor to accurately remove the residual pieces of stones and finally confirms the removal through systematic scanning and data backtracking. The application solves the problem of low identification rate and high missed detection rate of small and hidden residual pieces of stones in traditional operations, realizes high-specificity identification and intuitive spatial guidance, and significantly improves the completeness of removal and the operation efficiency.

Description

Technical Field

[0001] This invention relates to the field of medical devices and medical imaging technology, and in particular to a method for removing kidney stone fragments under fluorescence guidance. Background Technology

[0002] Percutaneous nephrolithotomy (PCNL) and flexible ureteroscopic lithotripsy are the mainstream minimally invasive surgical methods for treating kidney stones. The core goal of the surgery is to completely remove all fragments from the patient's body while breaking up the stones, because any residual stone fragments may become the core of stone recurrence, leading to repeated surgeries in a short period of time, increasing the patient's pain and financial burden.

[0003] Currently, intraoperative stone localization and removal primarily rely on endoscopic white light visual feedback. Surgeons use the endoscopic camera system to observe anatomical structures such as the renal pelvis and calyces on a monitor, visually identifying and removing visible stone fragments using forceps or negative pressure suction. However, this traditional method has several inherent and serious technical drawbacks: First, the internal structure of the kidney is complex, with many blind spots, such as the narrow neck of the renal calyces and the posterior calyces, which are difficult to fully cover under direct white light. Second, after the stone is fragmented by laser or ultrasound, it produces numerous tiny fragments, which may be less than 1 mm in size and similar in color to the surrounding renal pelvic mucosa or blood clots, resulting in extremely low contrast under white light and making them easily missed. Third, the procedure generates minor bleeding, tissue edema, and "dust" from the stone fragments, all of which further contaminate the surgical field and reduce visual recognition rates. Statistics show that the incidence of residual fragments is high after stone removal relying on white light vision, a key issue affecting the long-term efficacy of the surgery.

[0004] To address this issue, fluorescence-guided techniques have been introduced in existing technologies. For example, some techniques involve injecting specific fluorescent dyes into the patient, hoping that the dyes will selectively adhere to the surface of the stones and then fluoresce under specific excitation light. However, this method has significant drawbacks: the metabolism and adhesion of fluorescent dyes in the kidneys are uncontrollable; they may adhere not only to stones but also extensively to normal urothelium, resulting in excessively strong background fluorescence, low signal-to-noise ratio, and an inability to clearly highlight the stone boundaries. Furthermore, introducing exogenous dyes increases the patient's risk of allergies and metabolic burden. Another approach is to utilize the autofluorescence properties of certain components of the stone. However, existing methods typically use a single wavelength of excitation light to excite the stone and a wideband filter to receive the fluorescence signal. This method acquires a two-dimensional intensity image, and its core problem is the inability to effectively distinguish the fluorescence of the target stone from the inherent autofluorescence of background tissues (such as mucosa or blood clots). Because both may exhibit similar fluorescence intensities over a wide wavelength range, stone fragments are often confused with the background in the final image, making it difficult for doctors to accurately identify and locate tiny residual fragments, especially those embedded in the mucosa.

[0005] Therefore, there is an urgent need to invent a method for removing kidney stone fragments from surrounding biological tissue with high specificity and sensitivity under high background noise, and to achieve real-time, intuitive intraoperative guidance, so as to completely solve the clinical problem of residual stones after surgery. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides a method for removing renal stone fragments under fluorescence guidance, comprising the following steps:

[0007] Step 1: Establish a database of kidney stone fluorescence spectral characteristics containing standard fluorescence spectral curves of kidney stone samples with multiple chemical composition types;

[0008] Step 2: Integrate the multispectral fluorescence imaging system into the endoscope system. The multispectral fluorescence imaging system includes a multi-band laser emission unit, a high-speed spectral scanning imaging unit, and an image processing computer.

[0009] Step 3: During the stone removal stage, the multi-band laser emission unit is controlled to cyclically emit at least three narrowband excitation beams with different center wavelengths to the surgical area, and the corresponding time-series multispectral fluorescence image sequence is acquired by the high-speed spectral scanning imaging unit. The time-series multispectral fluorescence image sequence contains image data with spectral dimensions under different excitation wavelengths.

[0010] Step 4: The image processing computer performs real-time analysis on the time-series multispectral fluorescence image sequence, separates the background spectrum and target spectrum from the mixed spectrum through spectral decomposition, and matches the target spectrum with the kidney stone fluorescence spectral feature database to identify the stone fragment pixels.

[0011] Step 5: Mark the identified stone fragment pixels as a highlight signal, and superimpose and fuse the highlight signal into the white light video stream of the endoscope in real time to generate and display an augmented reality guided image;

[0012] Step 6: Based on visual guidance from augmented reality guidance images, remove the highlighted stone fragments and confirm the removal effect based on the real-time updated augmented reality guidance images during the removal process.

[0013] Preferably, step 1, establishing the kidney stone fluorescence spectral characteristic database, specifically includes:

[0014] Ex vivo human kidney stone samples were collected to represent various types of kidney stones, including calcium oxalate monohydrate stones, calcium oxalate dihydrate stones, magnesium ammonium phosphate stones, uric acid stones, and cystine stones.

[0015] Fluorescence spectroscopy was performed on multiple different locations of each type of stone sample using a spectral scanning device. The fluorescence spectroscopy was performed using narrow-band excitation light with a wavelength range of 380 nm to 420 nm, and the diameter of the irradiated spot at each location was less than 100 μm.

[0016] The fluorescence signal emitted at each point, with a wavelength range of 420 nm to 750 nm, is received by a spectrometer, the spectrometer having a spectral resolution better than 5 nm.

[0017] The spectral signals from multiple locations of each type of stone sample were averaged and denoised to generate a standard fluorescence spectrum curve representing the fluorescence characteristics of that type of stone.

[0018] The standard fluorescence spectrum curves of each type of kidney stone are associated with and stored with the corresponding chemical composition type labels to form a database of kidney stone fluorescence spectrum characteristics.

[0019] Preferably, step 2, configuring and calibrating the multispectral fluorescence imaging system, specifically includes:

[0020] The multi-band laser emitting unit is capable of emitting at least three narrowband excitation beams with different center wavelengths. The center wavelengths include a main excitation wavelength and at least two auxiliary excitation wavelengths. The main excitation wavelength is selected as 395 nanometers, and the auxiliary excitation wavelengths are selected as 415 nanometers and 450 nanometers.

[0021] The output fiber of the multi-band laser emitting unit is coupled to the fiber optic illumination channel of the endoscope, so that the narrowband excitation beam can be transmitted to the surgical area through the endoscope.

[0022] The high-speed spectral scanning imaging unit is connected to the camera channel of the endoscope through the endoscope camera interface. The high-speed spectral scanning imaging unit includes a tunable acousto-optic tunable filter or a liquid crystal tunable filter for spectral dispersion of the received fluorescence.

[0023] Before the operation begins, the multispectral fluorescence imaging system is calibrated in a dark environment using a standard fluorescence target plate. The calibration includes calibration of the uniformity of illumination at the end of the endoscope and calibration of the spectral response of the high-speed spectral scanning imaging unit. A brightness correction coefficient matrix and a spectral response correction curve are generated and stored in an image processing computer.

[0024] Preferably, step 3, which involves acquiring a time-series multispectral fluorescence image sequence, specifically includes:

[0025] During the clearance phase, the endoscope tip is placed in the renal calyx or renal pelvis area, and the multispectral fluorescence imaging system is activated;

[0026] The image processing computer controls the multi-band laser emitting unit to cyclically emit narrowband excitation beams of the main excitation wavelength and the auxiliary excitation wavelength at a switching frequency higher than 50 Hz;

[0027] For each frame of fluorescence image excited by a specific excitation wavelength, the high-speed spectral scanning imaging unit controls the tunable acousto-optic tunable filter or liquid crystal tunable filter inside it to scan in 10-nanometer increments within the wavelength range of 420 nm to 750 nm, and acquires fluorescence intensity images at each step wavelength of the narrow band.

[0028] By cyclically switching the excitation wavelength and repeating the spectral scan, a three-dimensional data cube containing two-dimensional spatial information and one-dimensional spectral information is generated for each excitation wavelength. The time-series multispectral fluorescence image sequence is a four-dimensional data stream composed of these three-dimensional data cubes.

[0029] Preferably, step 4, which involves real-time identification and location of stone fragments, specifically includes the following sub-steps:

[0030] Step 4.1: For the latest three-dimensional data cube currently acquired, the image processing computer selects the region with uniform fluorescence intensity distribution in the image as the background region of interest, extracts the complete fluorescence spectrum of all pixels in the background region of interest under multiple excitation wavelengths, and fits the autofluorescence background spectrum model representing the mixture of renal pelvis mucosa, blood and tissue fluid under the current surgical background using the principal component analysis method.

[0031] Step 4.2: Perform linear unmixing calculation on the original mixed fluorescence spectrum collected from each pixel in the current field of view and the autofluorescence background spectrum model obtained in Step 4.1. Use a non-negative matrix factorization algorithm to subtract the background spectral components from the mixed spectrum to separate the suspected target spectral components of each pixel.

[0032] Step 4.3: Match the suspected target spectral components of each pixel obtained in Step 4.2 with all standard fluorescence spectral curves in the kidney stone fluorescence spectral feature database. The matching calculation uses the normalized correlation coefficient method to calculate the similarity between the suspected target spectral components and each standard fluorescence spectral curve in the 420 nm to 650 nm band.

[0033] Step 4.4: When the normalized correlation coefficient between the suspected target spectral component of a pixel and any standard fluorescence spectrum curve in the database exceeds a preset identification threshold, it is determined that there is a kidney stone fragment at the corresponding spatial location of the pixel. The identification threshold is determined by in vitro experimental statistics and ranges from 0.75 to 0.85.

[0034] Preferably, step 5, generating and displaying the augmented reality guide image, specifically includes:

[0035] The image processing computer receives a synchronized white light video stream from the endoscope camera;

[0036] The image processing computer marks all the pixels identified in step 4 as stone fragments as a bright display signal with a specific color, which is a bright green or magenta that forms a high contrast with the natural color of human tissue.

[0037] The image processing computer integrates the highlighted display signal into the corresponding white light video frame in real time and accurately according to its spatial coordinate position in a semi-transparent superposition manner to generate the augmented reality guidance image. In the augmented reality guidance image, the stone fragments are clearly marked with highlighted color outlines, while the normal anatomical structure retains the original white light color.

[0038] The generated augmented reality guidance images are transmitted in real time and displayed on the main monitor in the operating room to provide doctors with operational guidance.

[0039] Preferably, step 6, which involves performing precise removal based on augmented reality-guided images, specifically includes:

[0040] The doctor observes the augmented reality-guided images displayed on the main monitor to identify the areas of stone fragments that are highlighted.

[0041] The doctor uses forceps or a negative pressure suction device to move the instrument tip to the corresponding spatial position of the highlighted area in the augmented reality guided image;

[0042] Depending on the aggregation state and size of the stone fragments, the method of removal is selected by using stone pliers or negative pressure suction.

[0043] During the removal process, the multispectral fluorescence imaging system continues to work, repeating steps 3 to 5 to perform real-time imaging, identification, and image updates of the surgical area.

[0044] Once a fragment of a kidney stone is successfully removed, its corresponding highlighted marker on the augmented reality guided image immediately disappears, and the doctor confirms the removal effect in real time based on the disappearance of the highlighted marker.

[0045] Following the above procedure, the doctor removed all the highlighted stone fragments one by one until there were no highlighted signals in the current endoscopic field of view.

[0046] Preferably, after step 6, step 7 is also included: system-assisted confirmation that the debris has been completely removed;

[0047] Step 7 specifically includes: the doctor operating the endoscope to systematically scan the entire target renal pelvis and calyces region;

[0048] During the systematic scanning process, the multispectral fluorescence imaging system works continuously, and the image processing computer records and stores all time-series multispectral fluorescence image sequences and corresponding analysis results throughout the scanning process.

[0049] After the scan is completed, the image processing computer performs a retrospective analysis on all the stored historical data to check whether there is a matching signal at any historical moment and any spatial location that was identified as a stone fragment in step 4 but was not removed in subsequent operations.

[0050] If retrospective analysis reveals a matching signal of an uncleaned stone fragment, the image processing computer will re-mark the historical spatial location of the matching signal on the current augmented reality guided image using a preset flashing pattern and send a review prompt to the doctor.

[0051] The removal of stone fragments is confirmed only when the systematic scan is completed and the retrospective analysis finds no matching signals for any remaining stone fragments.

[0052] Preferably, the operation of the multi-band laser emitting unit and the high-speed spectral scanning imaging unit is synchronously controlled by an image processing computer;

[0053] The synchronization control specifically involves the image processing computer generating a synchronization trigger signal, which includes an excitation wavelength switching command and a spectral scan start command.

[0054] The synchronous trigger signal is first sent to the multi-band laser emitting unit to control it to emit a narrowband excitation beam with a specified center wavelength;

[0055] After the narrowband excitation beam emission reaches stability, the synchronous trigger signal is delayed by a preset stabilization time before being sent to the high-speed spectral scanning imaging unit to control it to start spectral scanning imaging of the fluorescence at the excitation wavelength.

[0056] The stabilization time is determined experimentally based on the laser's response time and optical path transmission time to ensure that the spectral scanning imaging unit acquires fluorescence signals in a stable excitation state.

[0057] Preferably, the matching calculation process in step 4.3 incorporates spectral shape weighting factor and intensity weighting factor for optimization;

[0058] The spectral shape weighting factor is used to differentiate the spectral similarity contribution of different wavelength ranges in the calculation of normalized correlation coefficient, and to give higher weight to the specific wavelength range where the characteristic peak of stone fluorescence is located.

[0059] The intensity weighting factor is used to dynamically adjust the effective range of the identification threshold based on the overall fluorescence intensity of the suspected target spectral component. For signals with weak fluorescence intensity, the identification threshold is appropriately relaxed to avoid missed detections, while for non-specific signals with excessive fluorescence intensity, the identification threshold is increased to avoid false detections.

[0060] The image processing computer uses machine learning algorithms to train and optimize the spectral shape weighting factor and intensity weighting factor using confirmed spectral data of stone fragments and background tissue data, in order to adapt to the changes in imaging conditions for different patients and different surgical stages.

[0061] The beneficial effects of this invention are:

[0062] 1. This invention establishes a standard fluorescence spectrum database based on the chemical composition of stones in advance and employs multi-band excitation and high-spectral-resolution imaging techniques during surgery to acquire rich temporal multispectral fluorescence image sequences of the surgical area. Through real-time spectral decomposition algorithms, it effectively removes autofluorescence interference from background tissues (such as mucosa and blood) and accurately identifies stones based on their unique "spectral fingerprint" characteristics. This method fundamentally overcomes the shortcomings of traditional white light vision, which relies on low color and morphological contrast, and traditional broadband fluorescence imaging, which suffers from high background noise and poor specificity. It enables the reliable detection of sub-millimeter-sized, transparent, or mucosal stone fragments, significantly reducing the intraoperative false negative rate.

[0063] 2. This invention converts the identified location information of stone fragments into a highlighted signal in real time and superimposes it onto the endoscopic white light image to generate an augmented reality guided image. This image allows doctors to clearly and intuitively see the spatial location and outline of all targets to be removed while retaining familiar anatomical background, realizing a shift from "searching based on experience" to "following the map." Doctors can directly target the highlighted areas and receive immediate feedback through the real-time disappearance of the marked signals during the operation. This guidance mechanism significantly shortens the time for finding and confirming the location of fragments, improves the success rate of a single operation, thereby shortening the overall surgical time of the fragment removal stage and ensuring the thoroughness of the removal;

[0064] 3. This invention not only provides real-time intraoperative guidance but also designs a systematic postoperative scanning and data retrospective analysis confirmation process. After the surgeon completes subjective removal, the system guides a comprehensive scan of the surgical area and records all data. Subsequently, retrospective analysis checks for any unremoved positive signals in the historical data. This design constitutes an objective technical verification line, capable of identifying and highlighting residual fragments that may be overlooked due to blind spots or momentary obstruction, effectively compensating for potential oversights in manual exploration. This closed-loop process elevates surgical outcomes from reliance on the surgeon's personal experience and subjective judgment to a new level of verification aided by an objective technical system, providing a reliable guarantee for minimizing the incidence of postoperative residual stones. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0067] Figure 2 This is a flowchart of the step 4 of the method of the present invention, which involves real-time identification and positioning of stone fragments. Detailed Implementation

[0068] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0069] Please see Figures 1-2 This invention provides a fluorescence-guided method for removing kidney stone fragments. The method begins with establishing a database of kidney stone fluorescence spectral characteristics. The specific steps are as follows: Collect ex vivo human kidney stone samples of calcium oxalate monohydrate, calcium oxalate dihydrate, magnesium ammonium phosphate, uric acid, and cystine stones. Using a spectral scanning device, irradiate at least ten different locations on each type of stone sample with narrow-band excitation light in the wavelength range of 380 nm to 420 nm, with each irradiated spot having a diameter of less than 100 micrometers. Receive fluorescence signals in the wavelength range of 420 nm to 750 nm using a spectrometer with a spectral resolution better than 5 nm.

[0070] The spectral signals from all locations for each type of sample were averaged and noise was eliminated to generate a standard fluorescence spectrum curve. This curve was then associated with and stored as a label representing the corresponding stone chemical composition type, forming an initial database. This database provides a precise reference standard for subsequent intraoperative spectral matching, making the identification process type-specific and surpassing traditional methods that rely on a single intensity signal.

[0071] In one possible implementation, the configuration and calibration of the multispectral fluorescence imaging system is a key preparatory step for carrying out the method. This system is integrated into a standard percutaneous nephroscope. The multi-band laser emitting unit is configured to emit narrow-band excitation beams with three center wavelengths of 395 nm, 415 nm, and 450 nm. The output fiber of the multi-band laser emitting unit is coupled to the fiber optic illumination channel of the nephroscope.

[0072] The high-speed spectral scanning imaging unit includes a liquid crystal tunable filter and connects to the camera channel of the nephroscope via an interface. Preoperative calibration is performed in a dark-field environment: the end of the nephroscope is aligned with a standard fluorescent target plate, and excitation light of three wavelengths is emitted sequentially. A brightness correction coefficient matrix is ​​generated by analyzing the uniformity of the target plate image, and a system spectral response correction curve is generated by measuring the known spectral response of the target plate. This calibration process effectively eliminates the influence of illumination inhomogeneity and inherent system spectral response bias on the measurement data, ensuring the accuracy and reliability of subsequently acquired time-series multispectral fluorescence image sequences.

[0073] In one possible implementation, the process of acquiring a time-series multispectral fluorescence image sequence in real time during the procedure is as follows: After the kidney stone is crushed, a nephroscope is placed inside the target renal calyx. The system is activated, and the image processing computer controls the multi-band laser emission unit to cyclically emit excitation beams of 395 nm, 415 nm, and 450 nm at frequencies higher than 50 Hz.

[0074] For each frame of an image excited at a specific wavelength, the high-speed spectral scanning imaging unit controls a liquid crystal tunable filter to perform step scans from 420 nm to 750 nm in 10-nanometer intervals, acquiring a narrowband fluorescence intensity image at each step wavelength. A three-dimensional data cube containing two-dimensional spatial information and one-dimensional spectral information can be obtained at each excitation wavelength.

[0075] By cyclically switching excitation wavelengths and repeatedly scanning, a four-dimensional data stream, including a time dimension, is generated—a sequence of real-time multispectral fluorescence images. This acquisition method simultaneously obtains information from multiple excitation and emission wavelengths, providing rich spectral data for subsequent differentiation of stones from background tissue.

[0076] In one possible implementation, firstly, for the most recently acquired data, a uniform region in the image is selected to extract the background spectrum, and principal component analysis is used to fit an autofluorescence background spectrum model representing the current mixing of renal pelvic mucosa and tissue fluid. Secondly, the original mixed spectrum of each pixel is linearly unmixed with the background model, and a non-negative matrix factorization algorithm is used to separate the suspected target spectral components.

[0077] Next, the separated spectra are matched with standard curves in the database. The matching calculation uses the normalized correlation coefficient method to calculate the similarity in the 420 nm to 650 nm wavelength range. The pre-set identification threshold is determined through a large number of in vitro experiments and is fixed between 0.75 and 0.85. When the matching similarity of a pixel exceeds this threshold, it is determined that there is a stone fragment at that location. The core of this series of spectral processing steps is to specifically extract the spectral features of the stone from the complex intraoperative background, achieving highly sensitive detection of small and hidden fragments.

[0078] In one possible implementation, the step of generating and displaying the augmented reality guidance image specifically includes: an image processing computer synchronously receiving a white light video stream from a nephroscope camera; marking all identified stone fragment pixels as a bright magenta signal; and precisely fusing this bright signal into the corresponding white light video frame according to its spatial coordinates in a semi-transparent overlay to generate the augmented reality guidance image. The generated image is transmitted in real time to the main monitor in the operating room.

[0079] On the monitor screen, normal kidney anatomy retains its natural color, while all identified stone fragments, regardless of their original color, size, or whether they are embedded in the mucosa, are clearly outlined by a uniform magenta halo. This step transforms abstract spectral identification results into intuitive visual spatial guidance, greatly reducing the cognitive load on doctors and enabling targeted removal procedures.

[0080] In one possible implementation, the process of performing precise removal based on augmented reality-guided images is as follows: The physician observes the guided image marked in magenta on a monitor, operates a negative pressure suction sheath or stone forceps, and moves the instrument tip to the corresponding spatial position in the marked area. Depending on the size and degree of aggregation of the debris, the physician selects suction or forceps for removal. Simultaneously, a multispectral fluorescence imaging system continuously operates, updating the image in real time.

[0081] Once a fragment is successfully removed, its magenta marker on the guiding image immediately disappears in the next frame. The surgeon uses the disappearance of the marker to confirm the removal effect in real time. Following this procedure, the surgeon removes all highlighted areas one by one until there is no marker signal left in the current endoscopic field of view. This process achieves a real-time closed loop between surgical operation and imaging feedback, making the removal operation precise and verifiable.

[0082] In one possible implementation, the system-assisted confirmation step of complete debris removal is performed after the main debris removal is completed. The physician systematically moves the nephroscope to scan the entire target renal pelvis and calyces region. During the scan, the system continuously records all imaging data and analysis results. After the scan is completed, the image processing computer initiates retrospective analysis, reviewing all historical data frames to check for any residual signals that were previously identified but may have been overlooked due to changes in viewing angle.

[0083] If such a signal is detected, the system will mark its historical location on the current augmented reality guided image with a flashing circle and issue a prompt. The doctor will then relocate and remove the fragment based on the flashing prompt. Only when a systematic scan is completed and retrospective analysis reveals no further suspicious signals can the removal of the stone fragment be definitively confirmed. This step constitutes a dual verification mechanism for the removal effect, further reducing the risk of residual stones after the procedure.

[0084] In one possible implementation, the operation of the multi-band laser emitting unit and the high-speed spectral scanning imaging unit is synchronized and controlled by an image processing computer. The image processing computer generates a synchronization trigger signal containing excitation and acquisition commands. This signal is first sent to the multi-band laser emitting unit to trigger the activation of the laser at the specified wavelength. After the laser output has stabilized for a preset short period of time, the signal is then sent to the high-speed spectral scanning imaging unit to trigger the start of spectral scanning.

[0085] This stabilization time was determined through preliminary experiments to ensure that the laser power was stable at the time of acquisition. This synchronous control mechanism ensures that each spectral data cube is acquired under stable illumination by a specific wavelength of laser, avoiding signal distortion caused by transients or delays during laser switching, and ensuring the temporal consistency of data in the time-series multispectral fluorescence image sequence.

[0086] In one possible implementation, the spectral feature matching calculation process incorporates spectral shape weighting factors and intensity weighting factors for optimization. The spectral shape weighting factor is used to assign different weights to different wavelength ranges when calculating the normalized correlation coefficient. For example, for calcium oxalate monohydrate stones, a higher weight is assigned to the wavelength range containing their known characteristic peaks to enhance the importance of the characteristic peak shape in the matching process.

[0087] The intensity weighting factor dynamically adjusts the effective identification threshold based on the overall fluorescence intensity of the suspected target spectrum: for very weak signals, the threshold is appropriately relaxed to improve detection sensitivity; for non-specific signals with excessively high intensity, the threshold is increased to ensure specificity. The image processing computer uses machine learning algorithms to train and optimize these weighting factors using known stone and background spectral data, enabling them to adapt to individual patient differences and changes in the optical environment at different stages of surgery, thereby improving the robustness and accuracy of the identification algorithm in various complex scenarios.

[0088] Example

[0089] This embodiment was applied to a 52-year-old male patient diagnosed with a complex staghorn calculus in his left kidney. Preoperative computed tomography (CT) scan showed the calculus filling the lower calyx and renal pelvis of the left kidney, with a maximum diameter of approximately 3.5 cm and a CT value of approximately 1200 HU, indicating a high-density stone. The patient underwent standard percutaneous nephrolithotomy. After establishing a percutaneous renal access and using ultrasound combined with pneumatic lithotripsy to fragment the main stone and remove most of the fragments, the surgeon faced the challenge of clearing any remaining tiny fragments deep within the lower calyx and the neck of the calyx. At this point, the fluorescence-guided renal stone fragment removal method of this invention was applied.

[0090] Step 1: Establish a database of fluorescence spectral characteristics of kidney stones;

[0091] Prior to this surgery, the hospital laboratory had established a database of fluorescence spectral characteristics for kidney stones. Laboratory technicians collected over 100 excised human kidney stone samples from five different types of stones, including calcium oxalate monohydrate, calcium oxalate dihydrate, magnesium ammonium phosphate, uric acid, and cystine stones. Measurements were performed using a high-precision microscopic fluorescence spectroscopy scanning device. This device emits a narrow-band excitation light with continuously tunable wavelengths in the range of 380 nm to 420 nm, which is focused through a microscope objective to form an excitation spot with a diameter of approximately 80 μm on the surface of the stone sample. The excitation spot automatically selects at least 10 different locations on the sample surface for illumination.

[0092] The fluorescence signal generated at each site after excitation is received by a fiber-coupled high-sensitivity spectrometer with an effective detection wavelength range of 420 nm to 750 nm. Its internal grating and detector configuration achieves a spectral resolution of 3 nm. For a typical calcium oxalate monohydrate stone sample, under 395 nm excitation light, its fluorescence spectrum exhibits two characteristic peaks at approximately 550 nm and 650 nm. Technicians first performed dark noise subtraction on the raw spectral data from the 10 sites of this sample, then removed abnormal spectral curves caused by surface contamination or measurement instability. Finally, they performed an arithmetic average of the remaining effective spectral curves to obtain a smooth, standard fluorescence spectrum curve representing the characteristics of this sample. ,in Indicates the emission wavelength.

[0093] Through the above process, the database ultimately stores standard fluorescence spectral curves for five types of stones. Each curve uses the emission wavelength λ as the independent variable and the normalized fluorescence intensity value as the dependent variable. The database is stored in structured file format, with each spectral curve associated with its stone composition type tag.

[0094] Step 2: System integration and preoperative calibration;

[0095] The standard percutaneous nephrolithotomy system in the operating room (including a rigid nephroscope, white light source, camera system, and monitor) is integrated with the multispectral fluorescence imaging system of this invention. The multi-band laser emission unit contains three independent semiconductor laser modules with center wavelengths set to 395nm (main excitation wavelength), 415nm, and 450nm (auxiliary excitation wavelength), respectively. The laser output from each laser is filtered by a narrow-band filter and coupled into a single illumination fiber through an optical fiber combiner. This illumination fiber is then connected in parallel with the original white light illumination fiber bundle of the nephroscope through a specially designed optical fiber coupling interface, allowing the laser excitation light and white light to be transmitted to the end of the nephroscope independently or simultaneously.

[0096] The core of the high-speed spectral scanning imaging unit is a liquid crystal tunable filter, mounted in front of the image sensor of the nephroscope camera. This filter can be electrically controlled to rapidly adjust its passband center wavelength, thereby achieving spectral scanning. This unit is connected to an image processing computer via a video cable, and the white light camera signal from the nephroscope is also connected to the same computer. Before the surgery, after the nephroscope has been sterilized, on-site calibration is performed. The end of the nephroscope is aligned with a standard fluorescent target plate containing a known fluorescent substance (such as Rhodamine B) with a uniform concentration distribution. The system controls the laser to sequentially emit excitation light of three wavelengths, and the high-speed spectral scanning imaging unit acquires images of the target plate.

[0097] First, for each excitation light, the system analyzes the brightness uniformity of the target image. Due to illumination and optical system attenuation, the brightness at the image edges is typically lower than that at the center. The computer then calculates a brightness correction coefficient for each pixel. This ensures uniform brightness in the corrected image. Secondly, the system controls the liquid crystal tunable filter to perform step scanning, measuring the system response of the standard fluorescent target plate under a known standard spectrum, thus obtaining a system spectral response curve. All correction data and All data were saved. In this embodiment, brightness correction increased the brightness of the edge of the field of view by approximately 30% to achieve the same level as the central area.

[0098] Step 3: Intraoperative multispectral fluorescence image acquisition;

[0099] After the main stone has been fragmented and most of it removed, the surgery enters the stage of fine debris removal. The surgeon inserts a nephroscope through the percutaneous renal tract into the lower calyx of the left kidney. At this point, the strong white light source is turned off, leaving only about 10% of the background white light intensity for basic visual observation. The surgeon clicks the "Start Fluorescence Guidance" button on the image processing computer's software interface.

[0100] The system begins operation. The image processing computer sends a synchronization trigger signal according to a preset timing sequence. First, it controls the multi-band laser emission unit to turn on the laser with a center wavelength of 395nm and turn off the other lasers. After the laser output stabilizes for approximately 2ms (this stabilization time has been determined through pre-experimentation to ensure that the laser power reaches more than 99% of the preset value), the synchronization trigger signal triggers the high-speed spectral scanning imaging unit to begin acquisition. The passband center wavelength of the liquid crystal tunable filter starts at 420nm, incrementing in 10nm steps up to 750nm. At each step, for example, at a center wavelength of 500nm, the filter only allows fluorescence in a very narrow band (bandwidth approximately 10nm) near 500nm to pass through, and the image sensor acquires a frame of a two-dimensional fluorescence intensity image within this narrow band. After completing a scan from 420nm to 750nm, a stack of spectral images under 395nm excitation is obtained, forming a three-dimensional data cube. .

[0101] Subsequently, the synchronous trigger signal controls the laser to rapidly switch to a 415nm wavelength, and the above spectral scanning process is repeated to obtain a data cube. Then switch to 450nm to obtain... One cycle is defined as the complete cycle of the three excitation wavelengths. In this embodiment, the system controls the cycle frequency to be 10 Hz, meaning that 10 cycles of data acquisition are completed per second. By repeating this cycle, a continuous temporal multispectral fluorescence image sequence is formed.

[0102] Step 4: Real-time spectral recognition and augmented reality image generation;

[0103] Image processing computers perform pipelined processing on real-time incoming data cubes. Taking one processing cycle as an example, the computer retrieves the latest set of data. , and Perform the analysis.

[0104] Spectral Decomposition and Background Modeling: The software algorithm first automatically selects a seemingly uniform region without obvious fragmentation as the background region of interest (ROI) on the current 395nm excited fluorescence image (e.g., a 550nm band image). Complete spectral data of all pixels within this region at the three excitation wavelengths are extracted. Assuming the background mainly consists of renal pelvic mucosa and a small amount of bleeding, its spectral characteristics are characterized by a broad band that slowly attenuates with increasing wavelength. The algorithm uses principal component analysis to reduce the dimensionality of these background spectra, extracting the first two principal components as the background spectral basis vectors. and The linear combination of the two is sufficient to characterize more than 99% of the background spectral variation under the current field of view, thereby establishing a background spectral model.

[0105] Target spectral separation: for each pixel within the field of view The observed mixed spectrum Modeled as target spectrum Linear superposition with the background spectrum:

[0106]

[0107] in These are the weighting coefficients for the background contribution. They are solved using a nonnegative matrix factorization algorithm, under constraints. Minimize the reconstruction error under the condition that all components are non-negative. After solving, the spectrum of the separated suspected target can be obtained. .

[0108] Spectral feature matching and optimized identification: Compared with five standard spectral curves in the database (Type represents the type of kidney stone) is used for matching. The matching is calculated using the weighted normalized correlation coefficient (NCC):

[0109]

[0110] in, and It is the mean. This is the spectral shape weighting factor. In this embodiment, based on database knowledge, calcium oxalate monohydrate stones have characteristic peaks at 550 nm and 650 nm, therefore, it is set... The value is 2.0 in the 540-560nm and 640-660nm ranges, and 1.0 in other ranges, to enhance the importance of the characteristic peak in the matching.

[0111] After matching, the largest NCC value is taken as the matching score for that pixel. Identification is not simply about... The system compares the value to a fixed threshold (e.g., 0.8). An intensity weighting factor is introduced. It depends on the overall intensity of the target spectrum. The final dynamic discrimination threshold Determined by the following formula:

[0112]

[0113] in, This is a reference intensity value. and These are adjustment parameters (set to 0.05 and 0.01 in this example). The physical meaning of this formula is: for strength For very weak signals (possibly deep or tiny fragments), appropriately lower the threshold requirement (down to 0.75) to improve detection sensitivity and prevent false negatives; for intensity... Excessively high signals may originate from nonspecific reflectance or strongly fluorescent contaminants, thus the threshold tends to be 0.80 or higher to ensure specificity. When At that time, the pixel was determined to be a fragment of a kidney stone.

[0114] Augmented reality image synthesis: All pixels identified as stone fragments are marked with a bright magenta signal. Simultaneously, the computer receives a white light video stream from the nephroscope. The image processing algorithm precisely overlays and fuses the magenta highlight signal with 50% transparency onto the corresponding white light video frame. The synthesized augmented reality guided image is displayed in real time on the main monitor. The doctor can see that several tiny yellowish-white dots (estimated diameter 0.2-0.5mm), which were originally indistinguishable from the pale red mucosa under white light, as well as a small aggregate in the shadow of the renal calyx neck, are clearly marked with a magenta halo, while the normal renal pelvic mucosa and vascular patterns retain their natural color.

[0115] Step 5: Guided clearing and real-time feedback;

[0116] The surgeon, observing the augmented reality-guided image, first aligned the opening of the suction sheath with an area marked by a magenta halo at the bottom of the lower calyx of the kidney. During suction, the image on the monitor updated in real time. Once the fragment was successfully suctioned out, the corresponding magenta halo disappeared immediately in the next frame (approximately 0.1 seconds later). The surgeon then moved the nephroscope and suction sheath to the next marked area. For a slightly larger fragment, about 1mm in diameter, attached to the calyx neck mucosa, suction was ineffective. The surgeon switched to fine-tipped stone retrieval forceps, easily clamping and removing the fragment under the precise guidance of the magenta halo, after which the halo disappeared. Throughout the process, the surgeon no longer needed to painstakingly search for the target against a bloody background; simply following the guidance of the magenta halo significantly improved the removal efficiency.

[0117] Step 6: Systematic scanning and final confirmation;

[0118] After clearing all the magenta markings from the current field of view, the surgeon did not immediately end the procedure. He operated the nephroscope, slowly and systematically scanning from the lower calyx of the left kidney to the renal pelvis, and then to the other calyces, ensuring no blind spots. During this time, the fluorescence-guided system continuously ran and recorded all data. After completing the entire scan, the surgeon clicked the "Final Confirmation" button on the software. The image processing computer initiated a retrospective analysis program, retrieving all time-series data recorded from the start to the end of the scan, and quickly re-running the recognition algorithm. The analysis results showed that in a certain historical frame, a weak signal point (matching score 0.78) in a corner of the middle calyx was briefly detected, but was not captured again in subsequent frames due to changes in perspective. The system immediately marked this historical location on the current screen with a flashing magenta circle. The surgeon readjusted the nephroscope to that angle and carefully examined the location indicated by the flashing circle, indeed finding a tiny stone fragment less than 0.3 mm in diameter embedded in a mucosal fold, which was then removed. Subsequently, the system's retrospective analysis found no further positive findings, confirming that the fragment had been completely removed. The surgery was completed.

[0119] To objectively evaluate the effectiveness of the method of this invention, our research team selected 20 patients with complex kidney stones similar to those in this embodiment for a prospective controlled study. All patients underwent PCNL surgery performed by the same surgical team. They were randomly divided into two groups: the control group (n=10) underwent debris removal using conventional white light endoscopy; the experimental group (n=10) underwent debris removal using the fluorescence-guided method described in this embodiment. Within 24 hours postoperatively, all patients underwent low-dose CT scans, and a radiologist unaware of their group assignments assessed for the presence of residual stones larger than 2 mm.

[0120] Comparison indicators Control group (traditional white light therapy) Experimental group (fluorescence-guided method of this invention) Intraoperative assessment of the number of micro-fragments (<2mm) Based on experience, doctors estimate that there are an average of 5-10 lesions reported, but they cannot pinpoint the exact location. The system detects and labels in real time, and on average, it can accurately locate 15-25 tiny fragments per kidney. Average removal time per fragment Approximately 45 seconds. The location needs to be repeatedly checked and confirmed. Approximately 15 seconds. The target location is continuously highlighted, making the operation straightforward. Total debris removal stage time Average time: 25.3 minutes. Average time: 11.7 minutes. Number of cases with residual stones >2mm detected by postoperative CT 4 cases (40% residual stone rate). One case (10% residual stone rate) had a residual stone located in the renal calyx diverticulum, which is a blind spot for this method. Average maximum diameter of postoperative residual stones 3.1mm. 1.5mm (only 1 case).

[0121] Explanation of the comparative method:

[0122] Control group (traditional white light method): After the main stone was fragmented, the doctor relied entirely on the white light camera image from the nephroscope to observe it on a monitor. The doctor adjusted the angle of the nephroscope and the flushing water flow to look for suspicious stone fragments, mainly relying on the differences in color (white / yellow vs. red), reflectivity, and shape between the fragments and the mucosa. After finding a suspicious target, an attempt was made to remove it by suction or forceps, and repeated flushing and observation were performed to confirm that it had been completely removed. This continued until the doctor subjectively believed that no stone fragments were visible in the field of view.

[0123] Experimental group (fluorescence-guided method of this invention): The steps are as described in detail in this embodiment. From system calibration to starting the fluorescence-guided mode, the doctor performs targeted removal based on augmented reality images, and finally performs a system scan and system retrospective confirmation.

[0124] Results Analysis: As shown in the table above, compared with the traditional white light method, the fluorescence-guided method provided by this invention can detect more tiny fragments that are difficult to detect with the naked eye and achieve precise positioning, thereby significantly shortening the time for clearing fragments at a single site and the total time spent in the clearing stage. More importantly, it reduces the rate of clinically significant residual stones (>2mm) after surgery from 40% to 10%, proving that it can more thoroughly remove stone fragments and potentially reduce the risk of stone recurrence. This invention, by combining specific spectral recognition with augmented reality navigation, provides an efficient and reliable technical means to solve the clinical problem of residual stones after kidney stone surgery.

[0125] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0126] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for removing renal stone fragments under fluorescence guidance, characterized in that, Includes the following steps: Step 1: Establish a database of kidney stone fluorescence spectral characteristics containing standard fluorescence spectral curves of kidney stone samples with multiple chemical composition types; Step 2: Integrate the multispectral fluorescence imaging system into the endoscope system. The multispectral fluorescence imaging system includes a multi-band laser emission unit, a high-speed spectral scanning imaging unit, and an image processing computer. Step 3: During the stone removal stage, the multi-band laser emission unit is controlled to cyclically emit at least three narrowband excitation beams with different center wavelengths to the surgical area, and the corresponding time-series multispectral fluorescence image sequence is acquired by the high-speed spectral scanning imaging unit. The time-series multispectral fluorescence image sequence contains image data with spectral dimensions under different excitation wavelengths. Step 4: The image processing computer performs real-time analysis on the time-series multispectral fluorescence image sequence, separates the background spectrum and target spectrum from the mixed spectrum through spectral decomposition, and matches the target spectrum with the kidney stone fluorescence spectral feature database to identify the stone fragment pixels. Step 5: Mark the identified stone fragment pixels as a highlight signal, and superimpose and fuse the highlight signal into the white light video stream of the endoscope in real time to generate and display an augmented reality guided image; Step 6: Based on visual guidance from augmented reality guidance images, remove the highlighted stone fragments and confirm the removal effect based on the real-time updated augmented reality guidance images during the removal process.

2. The method for removing renal stone fragments under fluorescence guidance according to claim 1, characterized in that, Step 1, establishing a database of fluorescence spectral characteristics for kidney stones, specifically includes: Ex vivo human kidney stone samples were collected to represent various types of kidney stones, including calcium oxalate monohydrate stones, calcium oxalate dihydrate stones, magnesium ammonium phosphate stones, uric acid stones, and cystine stones. Fluorescence spectroscopy was performed on multiple different locations of each type of stone sample using a spectral scanning device. The fluorescence spectroscopy was performed using narrow-band excitation light with a wavelength range of 380 nm to 420 nm, and the diameter of the irradiated spot at each location was less than 100 μm. The fluorescence signal emitted at each point, with a wavelength range of 420 nm to 750 nm, is received by a spectrometer, the spectrometer having a spectral resolution better than 5 nm. The spectral signals from multiple locations of each type of stone sample were averaged and denoised to generate a standard fluorescence spectrum curve representing the fluorescence characteristics of that type of stone. The standard fluorescence spectrum curves of each type of kidney stone are associated with and stored with the corresponding chemical composition type labels to form a database of kidney stone fluorescence spectrum characteristics.

3. The method for removing renal stone fragments under fluorescence guidance according to claim 1, characterized in that, Step 2, configuring and calibrating the multispectral fluorescence imaging system, specifically includes: The multi-band laser emitting unit is capable of emitting at least three narrowband excitation beams with different center wavelengths. The center wavelengths include a main excitation wavelength and at least two auxiliary excitation wavelengths. The main excitation wavelength is selected as 395 nanometers, and the auxiliary excitation wavelengths are selected as 415 nanometers and 450 nanometers. The output fiber of the multi-band laser emitting unit is coupled to the fiber optic illumination channel of the endoscope, so that the narrowband excitation beam can be transmitted to the surgical area through the endoscope. The high-speed spectral scanning imaging unit is connected to the camera channel of the endoscope through the endoscope camera interface. The high-speed spectral scanning imaging unit includes a tunable acousto-optic tunable filter or a liquid crystal tunable filter for spectral dispersion of the received fluorescence. Before the operation begins, the multispectral fluorescence imaging system is calibrated in a dark environment using a standard fluorescence target plate. The calibration includes calibration of the uniformity of illumination at the end of the endoscope and calibration of the spectral response of the high-speed spectral scanning imaging unit. A brightness correction coefficient matrix and a spectral response correction curve are generated and stored in an image processing computer.

4. The method for removing renal stone fragments under fluorescence guidance according to claim 3, characterized in that, Step 3, which involves acquiring time-series multispectral fluorescence image sequences, specifically includes: During the clearance phase, the endoscope tip is placed in the renal calyx or renal pelvis area, and the multispectral fluorescence imaging system is activated; The image processing computer controls the multi-band laser emitting unit to cyclically emit narrowband excitation beams of the main excitation wavelength and the auxiliary excitation wavelength at a switching frequency higher than 50 Hz; For each frame of fluorescence image excited by a specific excitation wavelength, the high-speed spectral scanning imaging unit controls the tunable acousto-optic tunable filter or liquid crystal tunable filter inside it to scan in 10-nanometer increments within the wavelength range of 420 nm to 750 nm, and acquires fluorescence intensity images at each step wavelength of the narrow band. By cyclically switching the excitation wavelength and repeating the spectral scan, a three-dimensional data cube containing two-dimensional spatial information and one-dimensional spectral information is generated for each excitation wavelength. The time-series multispectral fluorescence image sequence is a four-dimensional data stream composed of these three-dimensional data cubes.

5. The method for removing renal stone fragments under fluorescence guidance according to claim 4, characterized in that, Step 4, which involves real-time identification and location of stone fragments, specifically includes the following sub-steps: Step 4.1: For the latest three-dimensional data cube currently acquired, the image processing computer selects the region with uniform fluorescence intensity distribution in the image as the background region of interest, extracts the complete fluorescence spectrum of all pixels in the background region of interest under multiple excitation wavelengths, and fits the autofluorescence background spectrum model representing the mixture of renal pelvis mucosa, blood and tissue fluid under the current surgical background using the principal component analysis method. Step 4.2: Perform linear unmixing calculation on the original mixed fluorescence spectrum collected from each pixel in the current field of view and the autofluorescence background spectrum model obtained in Step 4.

1. Use a non-negative matrix factorization algorithm to subtract the background spectral components from the mixed spectrum to separate the suspected target spectral components of each pixel. Step 4.3: Match the suspected target spectral components of each pixel obtained in Step 4.2 with all standard fluorescence spectral curves in the kidney stone fluorescence spectral feature database. The matching calculation uses the normalized correlation coefficient method to calculate the similarity between the suspected target spectral components and each standard fluorescence spectral curve in the 420 nm to 650 nm band. Step 4.4: When the normalized correlation coefficient between the suspected target spectral component of a pixel and any standard fluorescence spectrum curve in the database exceeds a preset identification threshold, it is determined that there is a kidney stone fragment at the corresponding spatial location of the pixel. The identification threshold is determined by in vitro experimental statistics and ranges from 0.75 to 0.

85.

6. The method for removing renal stone fragments under fluorescence guidance according to claim 1, characterized in that, Step 5, generating and displaying the augmented reality guide image, specifically includes: The image processing computer receives a synchronized white light video stream from the endoscope camera; The image processing computer marks all the pixels identified in step 4 as stone fragments as a bright display signal with a specific color, which is a bright green or magenta that forms a high contrast with the natural color of human tissue. The image processing computer integrates the highlighted display signal into the corresponding white light video frame in real time and accurately according to its spatial coordinate position in a semi-transparent superposition manner to generate the augmented reality guidance image. In the augmented reality guidance image, the stone fragments are clearly marked with highlighted color outlines, while the normal anatomical structure retains the original white light color. The generated augmented reality guidance images are transmitted in real time and displayed on the main monitor in the operating room to provide doctors with operational guidance.

7. The method for removing renal stone fragments under fluorescence guidance according to claim 1, characterized in that, Step 6, which involves performing precise image removal based on augmented reality guidance, specifically includes: The doctor observes the augmented reality-guided images displayed on the main monitor to identify the areas of stone fragments that are highlighted. The doctor uses forceps or a negative pressure suction device to move the instrument tip to the corresponding spatial position of the highlighted area in the augmented reality guided image; Depending on the aggregation state and size of the stone fragments, the method of removal is selected by using stone pliers or negative pressure suction. During the removal process, the multispectral fluorescence imaging system continues to work, repeating steps 3 to 5 to perform real-time imaging, identification, and image updates of the surgical area. Once a fragment of a kidney stone is successfully removed, its corresponding highlighted marker on the augmented reality guided image immediately disappears, and the doctor confirms the removal effect in real time based on the disappearance of the highlighted marker. Following the above procedure, the doctor removed all the highlighted stone fragments one by one until there were no highlighted signals in the current endoscopic field of view.

8. The method for removing renal stone fragments under fluorescence guidance according to claim 1, characterized in that, Following step 6, step 7 is also included: system-assisted confirmation that the debris has been completely removed; Step 7 specifically includes: the doctor operating the endoscope to systematically scan the entire target renal pelvis and calyces region; During the systematic scanning process, the multispectral fluorescence imaging system works continuously, and the image processing computer records and stores all time-series multispectral fluorescence image sequences and corresponding analysis results throughout the scanning process. After the scan is completed, the image processing computer performs a retrospective analysis on all the stored historical data to check whether there is a matching signal at any historical moment and any spatial location that was identified as a stone fragment in step 4 but was not removed in subsequent operations. If retrospective analysis reveals a matching signal of an uncleaned stone fragment, the image processing computer will re-mark the historical spatial location of the matching signal on the current augmented reality guided image using a preset flashing pattern and send a review prompt to the doctor. The removal of stone fragments is confirmed only when the systematic scan is completed and the retrospective analysis finds no matching signals for any remaining stone fragments.

9. The method for removing renal stone fragments under fluorescence guidance according to claim 3, characterized in that, The operation of the multi-band laser emitting unit and the high-speed spectral scanning imaging unit is synchronously controlled by an image processing computer. The synchronization control specifically involves the image processing computer generating a synchronization trigger signal, which includes an excitation wavelength switching command and a spectral scan start command. The synchronous trigger signal is first sent to the multi-band laser emitting unit to control it to emit a narrowband excitation beam with a specified center wavelength; After the narrowband excitation beam emission reaches stability, the synchronous trigger signal is delayed by a preset stabilization time before being sent to the high-speed spectral scanning imaging unit to control it to start spectral scanning imaging of the fluorescence at the excitation wavelength. The stabilization time is determined experimentally based on the laser's response time and optical path transmission time to ensure that the spectral scanning imaging unit acquires fluorescence signals in a stable excitation state.

10. The method for removing renal stone fragments under fluorescence guidance according to claim 5, characterized in that, The matching calculation process in step 4.3 incorporates spectral shape weighting factors and intensity weighting factors for optimization; The spectral shape weighting factor is used to differentiate the spectral similarity contribution of different wavelength ranges in the calculation of normalized correlation coefficient, and to give higher weight to the specific wavelength range where the characteristic peak of stone fluorescence is located. The intensity weighting factor is used to dynamically adjust the effective range of the identification threshold based on the overall fluorescence intensity of the suspected target spectral component. For signals with weak fluorescence intensity, the identification threshold is appropriately relaxed to avoid missed detections, while for non-specific signals with excessive fluorescence intensity, the identification threshold is increased to avoid false detections. The image processing computer uses machine learning algorithms to train and optimize the spectral shape weighting factor and intensity weighting factor using confirmed spectral data of stone fragments and background tissue data, in order to adapt to the changes in imaging conditions for different patients and different surgical stages.

Images