Intelligent image recognition system and method in fluorescence state

Abstract

The invention discloses an intelligent image recognition method in a fluorescence state, and relates to the technical field of medical equipment, and the method comprises the following steps: collecting a fluorescence image and a white light image at the same time point through 4K fluorescence endoscope camera equipment, and carrying out the image preprocessing, including illumination correction, fuzzy correction, filtering and denoising, and edge enhancement; based on a Mask RCNN algorithm, performing intelligent identification on a tumor boundary in the fluorescence image, and identifying the tumor boundary in the fluorescence image; performing spatial alignment on the fluorescent image and the white light image through an image registration technology; based on a GAN algorithm, fusing the identified tumor boundary with the white light image to generate a high-precision fused image; automatically marking a safe cutting edge in the white light image according to a preset safe cutting edge distance; and finally, displaying the marked white light image in real time to assist a doctor in tumor excision. According to the invention, the operation burden of a doctor in image switching is reduced, and the accuracy of an operation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to an intelligent image recognition system and method under a fluorescent state. Background Art

[0002] Fluorescence laparoscopy is widely used in malignant tumor resection surgery. This technique uses the injection of fluorescent dyes and exploits the differences in their metabolic response between tumor cells and normal cells, causing tumor tissue to display a distinct fluorescent signal under the microscope, thereby helping doctors identify tumor boundaries. The advantage of fluorescence laparoscopy lies in its ability to clearly distinguish tumor tissue from normal tissue, providing real-time tumor boundary information, particularly in complex surgical environments.

[0003] Through fluorescence imaging, doctors can accurately identify tumor boundaries in real time, improving surgical success rates and patient survival rates. However, existing fluorescence laparoscopic systems typically use split-screen displays for fluorescence and white light images, requiring doctors to frequently switch between the two screens, increasing the complexity of surgical procedures and the risk of errors. Furthermore, the fusion of fluorescence and white light images often results in loss of image detail, making it difficult to provide high-precision tumor boundary information. Doctors rely on visual memory to match the tumor extent in fluorescence and white light images, which can result in significant subjective errors and affect surgical accuracy. Summary of the Invention

[0004] The present invention aims to address the shortcomings of existing technologies by proposing an intelligent image recognition system and method under fluorescence conditions. Its advantages include real-time annotation of tumor boundaries and safe resection margins, reducing the surgeon's operational burden during image switching and improving surgical precision.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An intelligent image recognition method under a fluorescent state comprises the following steps:

[0007] Step 1: Use a 4K fluorescence laparoscope camera to capture fluorescence images and white light images at the same time point, and perform image preprocessing, including illumination correction, blur correction, filtering denoising, and edge enhancement;

[0008] Step 2: Based on the Mask RCNN algorithm, the tumor boundary in the fluorescence image is intelligently identified. The tumor boundary in the fluorescence image is identified; through image registration technology, the fluorescence image and the white light image are spatially aligned;

[0009] Step 3: Based on the GAN algorithm, the identified tumor boundary is fused with the white light image to generate a high-precision fused image; based on the preset safe resection margin distance, the safe resection margin is automatically marked in the white light image; finally, the marked white light image is displayed in real time to assist doctors in tumor resection.

[0010] The present invention is further configured such that the image preprocessing adopts an adaptive filtering algorithm, and the adaptive filtering algorithm can dynamically adjust filtering parameters according to specific conditions of the image to retain image details.

[0011] The present invention is further configured such that the Mask RCNN algorithm extracts image features through a convolutional neural network, generates candidate regions in combination with a region proposal network, and finally generates a pixel-level tumor boundary mask through a fully convolutional network.

[0012] The present invention is further configured such that the Mask RCNN algorithm can generate pixel-level tumor boundary masks, provide more accurate tumor boundary information, and simultaneously perform target detection, bounding box regression, and pixel-level segmentation, completing multiple tasks in one network.

[0013] The present invention is further configured such that the white light image annotation of the safe resection margin adopts an adaptive resection margin algorithm, which can dynamically adjust the resection margin distance according to the size and position of the tumor.

[0014] The present invention is further configured such that the GAN algorithm ensures that the fused image does not lose details and is highly consistent with the real image through adversarial training of the generator and the discriminator.

[0015] The present invention is further configured such that the generator is responsible for generating a fused image, and the discriminator is responsible for judging the authenticity of the generated image. The two continuously improve the quality of the fused image through adversarial training. During the image fusion process, the details of the fluorescence image and the white light image can be retained, ensuring that the fused image has high precision in marking the tumor boundary and the safe resection margin. The GAN algorithm can adaptively adjust the fusion parameters according to the specific situation of the image, ensuring that high-quality fused images can be generated in different surgical environments.

[0016] An intelligent image recognition system under fluorescence state, which applies the above-mentioned image recognition method, includes an image acquisition module, an image preprocessing module, a tumor boundary recognition module, an image registration and fusion module, a safe resection margin annotation module and a display and interaction module.

[0017] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0018] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0019] The beneficial effects of the present invention are:

[0020] 1. Real-time identification of tumor boundaries in fluorescent images and annotation of them in white light images assists doctors in accurately removing tumor tissue. Through intelligent image recognition technology, tumor boundaries and safe resection margins are annotated in real time, reducing the doctor's operational burden during image switching and improving surgical accuracy.

[0021] 2. The Mask RCNN algorithm accurately identifies tumor boundaries in fluorescence images, ensuring accurate and real-time recognition. The Mask RCNN algorithm is unique in that it can generate pixel-level tumor boundary masks, ensuring accurate identification of tumor boundaries.

[0022] 3. Using the GAN algorithm, the fluorescence image and the white light image are fused with high quality, ensuring that the fused image retains detail. The GAN algorithm is unique in that it uses adversarial training between the generator and the discriminator to ensure that the fused image is highly consistent with the real image. This ensures that detail is not lost during the fusion of the fluorescence and white light images, providing highly accurate tumor boundary information.

[0023] 4. Based on the preset safe margin distance and the identified tumor boundary, the module automatically marks the safe margin in the white light image, assisting the surgeon in performing precise resection. This module utilizes an adaptive margin algorithm that dynamically adjusts the margin distance based on the size and location of the tumor, reducing subjective errors caused by the surgeon's reliance on visual memory and ensuring surgical safety.

[0024] 5. Real-time display of annotated white-light images and human-computer interaction allow surgeons to adjust safe margin distances based on surgical needs. This module utilizes a high-resolution display to ensure image clarity and detail visibility. Real-time display of annotated white-light images reduces the number of steps required by surgeons during surgery, improving surgical efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the recognition process of an intelligent image recognition system in a fluorescent state proposed by the present invention;

[0026] Figure 2 This is a schematic diagram of fluorescence recognition preprocessing of an intelligent image recognition system under fluorescence state proposed by the present invention;

[0027] Figure 3 This is a schematic diagram of white light tracking and precise recognition of an intelligent image recognition system in a fluorescent state proposed by the present invention;

[0028] Figure 4 This is a schematic diagram of the practical combined application of the Mask RCNN algorithm and the GAN algorithm in an intelligent image recognition system under fluorescent state proposed by the present invention. DETAILED DESCRIPTION

[0029] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0030] The following describes in detail embodiments of the present invention, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0031] In the description of this patent, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings. They are only for the convenience of describing this patent and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on this patent.

[0032] In the description of this patent, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," "connected," and "set" should be understood in a broad sense. For example, they can refer to fixed connection or set, detachable connection or set, or integral connection or set. Those skilled in the art will understand the specific meanings of the above terms in this patent based on the specific circumstances.

[0033] Reference Figure 1-4 , an intelligent image recognition method under a fluorescent state, comprising the following steps:

[0034] Step 1: Use a 4K fluorescence laparoscope camera to capture fluorescence images and white light images at the same time point, ensuring temporal and spatial consistency between the two images. Image preprocessing, including illumination correction, blur correction, filtering and denoising, and edge enhancement, is then performed to improve image quality.

[0035] Step 2: Based on the Mask RCNN algorithm, intelligently identify the tumor boundary in the fluorescence image to ensure the accuracy of the recognition; through image registration technology, spatially align the fluorescence image and the white light image to ensure the consistency of the two images in time and space;

[0036] Step 3: Based on the GAN algorithm, the identified tumor boundary is fused with the white light image to generate a high-precision fused image; the safe resection margin is automatically marked in the white light image according to the preset safe resection margin distance; finally, the marked white light image is displayed in real time to assist doctors in tumor resection, using a high-resolution display to ensure image clarity and detail visibility.

[0037] In this embodiment, the image preprocessing adopts an adaptive filtering algorithm. The adaptive filtering algorithm can dynamically adjust filtering parameters according to the specific conditions of the image to retain the details of the image.

[0038] Furthermore, the Mask RCNN algorithm extracts image features through a convolutional neural network, combines it with a region proposal network to generate candidate regions, and finally generates a pixel-level tumor boundary mask through a fully convolutional network; the Mask RCNN algorithm can generate pixel-level tumor boundary masks, provide more accurate tumor boundary information, and simultaneously perform target detection, bounding box regression, and pixel-level segmentation, completing multiple tasks in one network.

[0039] The white light image annotation safety margin uses an adaptive margin algorithm that can dynamically adjust the margin distance according to the size and location of the tumor.

[0040] It is worth mentioning that the GAN algorithm ensures that the fused image does not lose details and is highly consistent with the real image through adversarial training of the generator and the discriminator; the generator is responsible for generating the fused image, and the discriminator is responsible for judging the authenticity of the generated image. The two continuously improve the quality of the fused image through adversarial training. During the image fusion process, the details of the fluorescence image and the white light image can be retained, ensuring that the fused image has high precision in the annotation of tumor boundaries and safe resection margins. The GAN algorithm can adaptively adjust the fusion parameters according to the specific situation of the image, ensuring that high-quality fused images can be generated in different surgical environments.

[0041] An intelligent image recognition system under fluorescence state, which applies the above-mentioned image recognition method, includes an image acquisition module, an image preprocessing module, a tumor boundary recognition module, an image registration and fusion module, a safe resection margin annotation module and a display and interaction module.

[0042] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0043] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0044] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. An intelligent image recognition method under fluorescent state, characterized in that: The following steps are involved: Step 1: Use a 4K fluorescence laparoscope camera to capture fluorescence images and white light images at the same time point, and perform image preprocessing, including illumination correction, blur correction, filtering denoising, and edge enhancement; Step 2: Based on the Mask RCNN algorithm, the tumor boundary in the fluorescence image is intelligently identified. The tumor boundary in the fluorescence image is identified; through image registration technology, the fluorescence image and the white light image are spatially aligned; Step 3: Based on the GAN algorithm, the identified tumor boundary is fused with the white light image to generate a high-precision fused image; based on the preset safe resection margin distance, the safe resection margin is automatically marked in the white light image; finally, the marked white light image is displayed in real time to assist doctors in tumor resection.

2. The intelligent image recognition method under fluorescence state according to claim 1, characterized in that: The image preprocessing adopts an adaptive filtering algorithm, which can dynamically adjust filtering parameters according to the specific conditions of the image to retain the details of the image.

3. The intelligent image recognition method under fluorescence state according to claim 1, characterized in that: The MaskRCNN algorithm extracts image features through a convolutional neural network, generates candidate regions in combination with a region proposal network, and finally generates a pixel-level tumor boundary mask through a fully convolutional network.

4. The intelligent image recognition method under fluorescence state according to claim 3, characterized in that: The MaskRCNN algorithm can generate pixel-level tumor boundary masks, provide more accurate tumor boundary information, and simultaneously perform target detection, bounding box regression, and pixel-level segmentation, completing multiple tasks in one network.

5. The intelligent image recognition method under fluorescence state according to claim 1, characterized in that: The white light image marking safety margin adopts an adaptive margin algorithm, which can dynamically adjust the margin distance according to the size and position of the tumor.

6. The intelligent image recognition method under fluorescence state according to claim 1, characterized in that: The GAN algorithm ensures that the fused image does not lose details and is highly consistent with the real image through adversarial training of the generator and the discriminator.

7. The intelligent image recognition method under fluorescence state according to claim 6, characterized in that: The generator is responsible for generating the fused image, and the discriminator is responsible for judging the authenticity of the generated image. The two continuously improve the quality of the fused image through adversarial training. During the image fusion process, the details of the fluorescence image and the white light image can be retained, ensuring that the fused image has high precision in the annotation of the tumor boundary and the safe resection margin. The GAN algorithm can adaptively adjust the fusion parameters according to the specific situation of the image, ensuring that high-quality fused images can be generated in different surgical environments.

8. An intelligent image recognition system in a fluorescent state, characterized in that: The image recognition system applies the image recognition method according to any one of claims 1 to 7, and the image recognition system includes an image acquisition module, an image preprocessing module, a tumor boundary recognition module, an image registration and fusion module, a safe resection margin annotation module, and a display and interaction module.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images