Real-time image-guided surgery system

By segmenting the patient's historical DICOM image data and establishing virtual anatomical structure model, combined with real-time surgical image data, the problem that traditional navigation systems cannot provide accurate anatomical navigation information is solved, and more accurate and individualized surgical guidance is achieved , improves the safety and efficiency of the surgery.

CN118453115BActive Publication Date: 2024-11-01NANTONG INFECTIOUS DISEASE PREVENTION & CONTROL INST
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410625510.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-20
Publication Date
2024-11-01
Estimated Expiration
2044-05-20

AI Technical Summary

Technical Problem

Traditional anatomical navigation systems are unable to accurately analyze the patient's internal anatomy, resulting in the inability to provide sufficiently accurate anatomical navigation information and the inability to update surgical guidance information based on the performed anatomical path.

Method used

By obtaining the patient's historical DICOM image data, a virtual anatomical structure model is established, and surgically assisted guidance path planning and real-time data analysis are provided to provide intelligent surgical guidance.

Benefits of technology

Accurate analysis and real-time guidance of the patient's anatomy structure are achieved, improving the positioning accuracy and safety of the operation, shortening the operation time, and improving the postoperative rehabilitation situation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118453115B_ABST
    Figure CN118453115B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of surgical image processing technology, and in particular to a real-time image guidance system based on surgical operations. The modules of the system include the following functions: establishing a three-dimensional model of a virtual anatomical structure based on historical DICOM image data to generate a virtual anatomical structure model; designing a surgical model based on real-time surgical image data and a virtual anatomical structure model to obtain a surgical model; planning and intelligently adjusting a surgical auxiliary guidance path based on the surgical model to generate intelligent surgical auxiliary guidance path data; analyzing and processing abnormal depth of surgery based on real-time surgical instrument depth information to generate abnormal depth analysis data; transmitting the abnormal depth analysis data and the intelligent surgical auxiliary guidance path data to a terminal device to perform a visual guidance feedback operation. The present invention provides more intelligent surgical guidance auxiliary information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of surgical image processing, and in particular to a real-time image guidance system based on surgical operations. Background Art

[0002] The real-time image guidance system for surgical operations is based on advanced medical imaging technology and navigation systems, and is designed to assist doctors in improving the accuracy and safety of operations. The system uses high-resolution imaging equipment such as CT scans and MRI to obtain detailed imaging information of the patient's internal structures in real time. Through professional surgical planning software and navigation systems, real-time images can be combined with the patient's actual anatomical structure during surgery to assist in locating the target area, planning the operation steps, and providing precise guidance. This technology is used in fields such as neurosurgery and cardiac surgery, and helps minimize interference with surrounding healthy tissues. However, traditional anatomical navigation systems cannot accurately analyze the patient's internal anatomical structure, making it impossible to provide sufficiently accurate anatomical navigation information, and cannot update surgical guidance information based on the executed anatomical path. Summary of the invention

[0003] Based on this, the present invention provides a real-time image guidance system based on surgical operations to solve at least one of the above technical problems.

[0004] To achieve the above object, a real-time image guidance system based on surgical operation includes:

[0005] The historical image segmentation module is used to obtain the historical DICOM image data of patients in the medical database; perform image internal structure segmentation processing based on the historical DICOM image data to generate historical segmented DICOM image data;

[0006] A virtual anatomical structure model building module is used to build a three-dimensional model of a virtual anatomical structure based on historical segmented DICOM image data to generate a virtual anatomical structure model;

[0007] A surgical model design module is used to obtain real-time surgical imaging data of patients; a surgical model is designed based on the real-time surgical imaging data and a virtual anatomical structure model to obtain a surgical model;

[0008] A surgical assisted guidance path planning module is used to perform surgical assisted guidance path planning based on a surgical operation model and generate surgical assisted guidance path data;

[0009] A real-time surgical auxiliary information analysis module is used to collect real-time surgical data based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information; perform abnormal depth analysis processing of the surgery based on the real-time surgical instrument depth information to generate abnormal depth analysis data; perform path intelligent adjustment processing on surgical auxiliary guidance path data based on the real-time surgical execution path information to generate intelligent surgical auxiliary guidance path data;

[0010] The visual guidance feedback module is used to transmit abnormal depth analysis data and intelligent surgery assisted guidance path data to the terminal device to perform visual guidance feedback operations.

[0011] The present invention obtains the patient's historical DICOM image data from the medical database, and then accurately segments these historical DICOM images through advanced image internal structure segmentation processing technology to generate historical segmented DICOM image data, so that the system can accurately extract and present the internal information of the patient's anatomical structure, and provide a reliable basis for subsequent virtual anatomical structure modeling and individualized guidance path planning. Using the historical segmented DICOM image data, the advanced three-dimensional modeling technology is used to construct the model of the virtual anatomical structure, and the accurate segmentation data of the patient's anatomical structure, the virtual anatomical structure model building module can create a highly realistic three-dimensional anatomical structure model, finely restore the patient's physiological characteristics, provide individualized anatomical structure information for surgical operations, and provide a more intuitive and comprehensive patient anatomical structure reference. Based on the surgical model, accurate guidance path planning and processing are performed. By comprehensively considering the patient's individualized anatomical structure information and real-time surgical image data, the module generates accurate surgical auxiliary guidance path data, provides intuitive and accurate guidance information for surgeons, enables medical professionals to perform operations in a targeted manner during the operation, improves the objectivity of the operation, provides a real-time and individualized guidance path for the surgery, and improves the positioning accuracy of the operation, which helps to shorten the operation time and improve the patient's postoperative recovery. While providing effective guidance through intelligent path planning, it also brings a safer and more efficient execution process for surgical operations. Through real-time surgical data collection of real-time surgical imaging data, these real-time surgical data are used for multi-faceted analysis and processing, including real-time surgical instrument depth information for abnormal depth analysis, and real-time surgical execution path information for path intelligent adjustment processing. Abnormal depth analysis data provides surgeons with accurate information on instrument depth during real-time surgery, helping to identify potential surgical risks and abnormal conditions. Intelligent path adjustment processing uses real-time surgical execution path information to intelligently adjust the surgical auxiliary guidance path, providing a more flexible and real-time guidance path to adapt to changes during surgery. The abnormal depth analysis data and intelligent surgical auxiliary guidance path data are transmitted to the terminal device to achieve visual guidance feedback. This module presents real-time abnormal depth analysis information and intelligent guidance paths to surgeons by using intuitive data forms or other visualization methods, providing intuitive and clear surgical auxiliary information, helping doctors to more intuitively understand the patient's physiological condition and surgical guidance information during surgery, so as to make decisions and adjust surgical strategies more effectively.

[0012] Preferably, the historical image segmentation module includes the following functions:

[0013] Obtain historical DICOM image data of patients in the medical database;

[0014] Perform noise reduction processing on historical DICOM image data to generate historical noise-reduced DICOM image data; perform image standardization processing on historical noise-reduced DICOM image data to generate historical standard DICOM image data;

[0015] Mark the characteristic nodes of the historical standard DICOM image data and generate characteristic node data;

[0016] Perform spatial calibration and matching processing of multi-modal images on historical standard DICOM image data according to feature node data to generate historical calibrated DICOM image data;

[0017] The historical calibrated DICOM image data is segmented according to the internal structure of the image to generate historical segmented DICOM image data.

[0018] The present invention obtains the historical DICOM image data of patients in the medical database, and can understand the image data collected by multiple angles and different devices of the patients, including CT images, ultrasonic images, etc., and comprehensively understand the individual anatomical structure of the patients, providing a rich data basis for subsequent analysis and guidance. The historical DICOM image data is processed to remove noise and interference, and clear and high-quality historical noise-reduced DICOM image data is generated, which helps to improve the accuracy and stability of subsequent processing. The historical noise-reduced DICOM image data is standardized to ensure that the images are consistent in terms of scale, contrast and brightness, and generate historical standard DICOM image data. The standardized images help to perform subsequent processing and analysis more accurately. Marking key feature nodes in the historical standard DICOM image data, such as the edges of organs, important anatomical structures, etc., generates data containing these feature node information, helps to extract more biologically meaningful features, and provides more refined positioning information for subsequent segmentation and matching. Combined with the feature node data, the historical standard DICOM image data is subjected to spatial calibration and matching processing of multi-modal images to ensure the spatial consistency of images of different modalities, generate historical calibration DICOM image data, and provide a reliable basis for subsequent accurate internal structure segmentation. Utilizing historical calibrated DICOM image data and adopting advanced edge detection and threshold segmentation technology, the patient's internal structure is accurately segmented to generate historical segmented DICOM image data, providing high-quality, individualized anatomical structure information for subsequent virtual modeling and surgical guidance.

[0019] Preferably, the performing image internal structure segmentation processing on the historical calibration DICOM image data comprises:

[0020] The edge detection algorithm is used to analyze the edge of the internal structure of the historical calibration DICOM image data to generate the internal structure edge data;

[0021] Based on the threshold segmentation technology and the internal structure edge data, the historical calibrated DICOM image data is segmented to generate the historical segmented DICOM image data.

[0022] The present invention uses edge detection algorithm to process historical calibration DICOM image data, performs edge analysis of the internal structure of the image, identifies edge information in the image, helps to extract edge features of the internal structure, and provides a basis for subsequent segmentation. Based on threshold segmentation technology and combined with internal structure edge data, the historical calibration DICOM image data is segmented to perform image internal structure segmentation processing, and through reasonable threshold selection and edge information guidance, accurate segmentation of the internal structure is achieved to generate historical segmented DICOM image data.

[0023] Preferably, the virtual anatomical structure model building module includes the following functions:

[0024] Perform slice data analysis and processing of DICOM images based on historical segmented DICOM image data to generate DICOM image slice data;

[0025] According to the DICOM image slice data, the image space coordinate data is collected for the historical segmented DICOM image data to generate the image space coordinate data;

[0026] A three-dimensional model of a virtual anatomical structure is established based on the historical segmentation DICOM image data and the corresponding image space coordinate data to generate a virtual anatomical structure model.

[0027] The present invention performs slice data analysis and processing based on historical segmentation of DICOM image data, involving cutting three-dimensional DICOM image data into multiple two-dimensional slices to better analyze and process the anatomical structure information of each slice. By using DICOM image slice data to collect image spatial coordinate data, the system can obtain the precise position information of each slice in three-dimensional space, and provide necessary coordinate data for subsequent three-dimensional modeling. A three-dimensional model of a virtual anatomical structure is established based on historical segmentation of DICOM image data and corresponding image space coordinate data. By integrating information from multiple slices, a patient-individualized, high-precision virtual anatomical structure model is generated to provide accurate anatomical structure information for surgical guidance.

[0028] Preferably, the three-dimensional modeling of the virtual anatomical structure based on the historical segmentation DICOM image data and the corresponding image space coordinate data comprises:

[0029] Establishing a three-dimensional model of the initial virtual anatomical structure according to the historical segmentation DICOM image data and the corresponding image space coordinate data, and generating an initial virtual anatomical structure model;

[0030] The initial virtual anatomical structure model is filled and adjusted by using the interpolation method to generate an adjusted virtual anatomical structure model;

[0031] The adjusted virtual anatomical structure model is subjected to model smoothing optimization processing to generate a virtual anatomical structure model.

[0032] The present invention combines segmented DICOM image data with accurate image space coordinate data to obtain coordinate data of each two-dimensional image in three-dimensional space, arranges the two-dimensional images in the three-dimensional space coordinate data to establish the architecture information of the virtual anatomical structure model, and fills and adjusts the architecture information of the virtual anatomical structure model through subsequent interpolation technology to obtain a more continuous and refined virtual anatomical structure model. The interpolation method helps to eliminate existing spatial gaps, improve the integrity and realism of the model, perform model smoothing optimization processing on the adjusted virtual anatomical structure model, smooth out the defects caused by the residual image in the three-dimensional model and the edge of the image, reduce irregular geometric shapes, and thus generate a virtual anatomical structure model.

[0033] Preferably, the surgical model design module includes the following functions:

[0034] Acquire real-time surgical imaging data of patients;

[0035] Perform feature node matching processing on real-time surgical image data according to feature node data of historical segmented DICOM image data to generate feature node matching data;

[0036] Based on the feature node matching data, real-time surgical image data is mapped to the virtual anatomical structure model for surgical model design to generate a surgical model.

[0037] The present invention obtains the surgical image data of the patient in real time to ensure that the model design can be carried out based on the actual surgical situation of the current patient. The feature node data of the historical segmented DICOM image data is used to perform feature node matching processing on the real-time surgical image data, identify key feature nodes in the real-time surgical image, such as the edge of the organ or other important anatomical structures, and match them with the historical data to establish a corresponding relationship between the real-time image and the historical segmented data, and generate feature node matching data according to the feature node matching processing. These data record the corresponding positions of the key feature nodes in the real-time surgical image in the historical segmented data, and provide key positioning information for subsequent model design. Based on the feature node matching data, the real-time surgical image data is mapped to the virtual anatomical structure model, and the spatial correspondence between the real-time image and the virtual anatomical structure model is achieved through the matching feature node information. The surgical model is designed on the basis of the virtual anatomical structure model, and the real-time surgical image data is combined with the virtual anatomical structure model to generate a surgical model that matches the patient's current anatomical structure, so as to better understand the patient's actual anatomical situation and improve the individualization and accuracy of surgical planning.

[0038] Preferably, the surgical assisted guidance path planning module includes the following functions:

[0039] Based on the preset U-Net model, the abnormal area of ​​the surgical model is analyzed and processed to generate the abnormal area data of the model;

[0040] The surgical assisted guidance path is designed based on the abnormal area data of the model to generate surgical assisted guidance path data.

[0041] The present invention uses a preset U-Net model to perform abnormal area analysis and processing on a surgical model. The U-Net model is a deep learning architecture, which is particularly suitable for image segmentation tasks. The pre-designed U-Net model is used to identify abnormal areas in a surgical model, such as lesions or risk areas, to generate model abnormal area data. These data represent the location and range of abnormal or critical areas identified in the surgical model, providing key information for subsequent guidance path design. According to the model abnormal area data, a surgical auxiliary guidance path design is performed, and an optimized guidance path is provided, such as a guidance path for the skin surface and internal anatomical structures, taking into account the location and characteristics of the abnormal area, to ensure that the operation can be more accurate and safe when performed in the abnormal area. Based on the surgical auxiliary guidance path design, the final surgical auxiliary guidance path data is generated. These data include geometric information, direction and corresponding guidance instructions of the guidance path, providing real-time and personalized surgical guidance for surgeons.

[0042] Preferably, the real-time surgical auxiliary information analysis module includes the following functions:

[0043] Performing real-time surgical data acquisition based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information;

[0044] Mapping real-time surgical instrument depth information to a surgical model to perform a reference depth conversion of the surgical instrument, thereby generating reference depth data of the surgical instrument;

[0045] Perform effective area marking processing on the surgical model to generate effective area data of the model;

[0046] Perform abnormal depth analysis and processing of the surgical instrument based on the surgical instrument baseline depth data and the model effective area data to generate abnormal depth analysis data;

[0047] Mapping the real-time surgical execution path information to the surgical operation model to perform a reference distance conversion of the surgical execution path, and generating surgical execution path reference distance data;

[0048] The surgical auxiliary guidance path data is intelligently adjusted according to the surgical execution path benchmark distance data to generate intelligent surgical auxiliary guidance path data.

[0049] The present invention maps real-time surgical instrument depth information to a surgical model, performs a reference depth conversion of the surgical instrument, generates reference depth data of the surgical instrument, helps to align the depth information of the real-time surgical instrument with the model, and provides an accurate reference for subsequent abnormal depth analysis. The surgical model is marked with an effective area, and model effective area data is generated, which is used to mark the effective anatomical area in the model to ensure the accuracy of depth analysis and path adjustment. Based on the surgical instrument reference depth data and the model effective area data, the abnormal depth analysis of the surgical instrument is performed to generate abnormal depth analysis data, which helps to detect whether the surgical instrument deviates from the normal depth range and provide real-time abnormal depth information. The real-time surgical execution path information is mapped to the surgical model to perform a reference distance conversion of the surgical execution path, and the surgical execution path reference distance data is generated. By aligning the real-time execution path with the model, accurate distance conversion is achieved, and an accurate reference is provided for path intelligent adjustment. Based on the surgical execution path reference distance data, the surgical auxiliary guidance path data is intelligently adjusted, and intelligent surgical auxiliary guidance path data is generated, which helps to intelligently adjust the guidance path according to the actual surgical execution situation, and ensure that the guidance path is consistent with the actual surgery.

[0050] Preferably, the real-time analysis and processing of abnormal depth of surgery based on surgical instrument reference depth data and model effective area data includes:

[0051] Perform real-time calculation and processing of the shortest edge distance of the effective area of ​​the surgical instrument benchmark depth data and the model effective area data to generate real-time shortest edge distance data;

[0052] When the real-time shortest edge distance data is less than a preset effective area edge threshold, the real-time surgical instrument depth information corresponding to the real-time shortest edge distance data is marked as abnormal depth data;

[0053] The abnormal depth data is analyzed and processed for abnormal depth of surgical instruments to generate abnormal depth analysis data.

[0054] The present invention performs calculation and processing of the shortest edge distance of the effective area in real-time operation for surgical instrument reference depth data and model effective area data, considers the position of the surgical instrument in the effective area, calculates the shortest distance between the instrument and the edge of the effective area, and the direction of the shortest distance, thereby generating real-time shortest edge distance data, which are used to record the distance of the surgical instrument relative to the edge of the effective area and provide real-time depth information. When the real-time shortest edge distance data is less than the preset effective area edge threshold, the corresponding real-time surgical instrument depth information is marked as abnormal depth data, and according to the pre-set threshold, it is determined whether the surgical instrument deviates from the expected effective area range, so as to achieve real-time marking of abnormal depth. The marked abnormal depth data is subjected to abnormal depth analysis processing of the surgical instrument, including further analysis of the specific situation of the abnormal depth, including specific information such as the degree and direction of the abnormal depth, so as to provide more detailed abnormal depth analysis data.

[0055] Preferably, the intelligent path adjustment processing of the surgical auxiliary guidance path data according to the surgical execution path reference distance data includes:

[0056] The surgical auxiliary guidance path data is updated according to the surgical execution path benchmark distance data, and the updated surgical auxiliary guidance path data is intelligently adjusted for the shortest path using the Dijkstra algorithm to generate intelligent surgical auxiliary guidance path data.

[0057] The present invention updates the surgical auxiliary guidance path data according to the surgical execution path benchmark distance data. The system considers the actual situation of the surgical execution path and dynamically adjusts the guidance path to adapt to the real-time surgical execution situation. The updated surgical auxiliary guidance path data is intelligently adjusted by the Dijkstra algorithm. The Dijkstra algorithm is a classic algorithm for calculating the shortest path between nodes in a graph. The guidance path is optimized by effectively selecting the path, making it more reasonable and efficient in actual surgical execution. Finally, intelligent surgical auxiliary guidance path data is generated, including the adjusted guidance path, to provide surgeons with more intelligent and real-time surgical guidance.

[0058] The beneficial effects of the present application are that the real-time image guidance system based on surgical operations of the present invention establishes a more detailed three-dimensional anatomical structure model by analyzing the multimodal images of DICOM, and can accurately analyze the patient's internal anatomical structure, thereby providing a sufficiently accurate anatomical guidance route, and can update the surgical guidance information in real time according to the executed anatomical path, as well as feedback abnormal information on the depth distance of the instrument during the surgical operation, so as to better assist medical personnel in performing subsequent surgical operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a real-time image guidance system based on surgical operations according to the present invention;

[0060] Figure 2 for Figure 1 Schematic diagram of the functional flow of the historical image segmentation module;

[0061] Figure 3 for Figure 1 Schematic diagram of the functional flow of the real-time surgical auxiliary information analysis module;

[0062] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0064] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0065] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0066] To achieve this, please refer to Figures 1 to 3 The present invention provides a real-time image guidance system based on surgical operation, comprising:

[0067] The historical image segmentation module is used to obtain the historical DICOM image data of patients in the medical database; perform image internal structure segmentation processing based on the historical DICOM image data to generate historical segmented DICOM image data;

[0068] A virtual anatomical structure model building module is used to build a three-dimensional model of a virtual anatomical structure based on historical segmented DICOM image data to generate a virtual anatomical structure model;

[0069] A surgical model design module is used to obtain real-time surgical imaging data of patients; a surgical model is designed based on the real-time surgical imaging data and a virtual anatomical structure model to obtain a surgical model;

[0070] A surgical assisted guidance path planning module is used to perform surgical assisted guidance path planning based on a surgical operation model and generate surgical assisted guidance path data;

[0071] A real-time surgical auxiliary information analysis module is used to collect real-time surgical data based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information; perform abnormal depth analysis processing of the surgery based on the real-time surgical instrument depth information to generate abnormal depth analysis data; perform path intelligent adjustment processing on surgical auxiliary guidance path data based on the real-time surgical execution path information to generate intelligent surgical auxiliary guidance path data;

[0072] The visual guidance feedback module is used to transmit abnormal depth analysis data and intelligent surgery assisted guidance path data to the terminal device to perform visual guidance feedback operations.

[0073] The present invention obtains the patient's historical DICOM image data from the medical database, and then accurately segments these historical DICOM images through advanced image internal structure segmentation processing technology to generate historical segmented DICOM image data, so that the system can accurately extract and present the internal information of the patient's anatomical structure, and provide a reliable basis for subsequent virtual anatomical structure modeling and individualized guidance path planning. Using the historical segmented DICOM image data, the advanced three-dimensional modeling technology is used to construct the model of the virtual anatomical structure, and the accurate segmentation data of the patient's anatomical structure, the virtual anatomical structure model building module can create a highly realistic three-dimensional anatomical structure model, finely restore the patient's physiological characteristics, provide individualized anatomical structure information for surgical operations, and provide a more intuitive and comprehensive patient anatomical structure reference. Based on the surgical model, accurate guidance path planning and processing are performed. By comprehensively considering the patient's individualized anatomical structure information and real-time surgical image data, the module generates accurate surgical auxiliary guidance path data, provides intuitive and accurate guidance information for surgeons, enables medical professionals to perform operations in a targeted manner during the operation, improves the objectivity of the operation, provides a real-time and individualized guidance path for the surgery, and improves the positioning accuracy of the operation, which helps to shorten the operation time and improve the patient's postoperative recovery. While providing effective guidance through intelligent path planning, it also brings a safer and more efficient execution process for surgical operations. Through real-time surgical data collection of real-time surgical imaging data, these real-time surgical data are used for multi-faceted analysis and processing, including real-time surgical instrument depth information for abnormal depth analysis, and real-time surgical execution path information for path intelligent adjustment processing. Abnormal depth analysis data provides surgeons with accurate information on instrument depth during real-time surgery, helping to identify potential surgical risks and abnormal conditions. Intelligent path adjustment processing uses real-time surgical execution path information to intelligently adjust the surgical auxiliary guidance path, providing a more flexible and real-time guidance path to adapt to changes during surgery. The abnormal depth analysis data and intelligent surgical auxiliary guidance path data are transmitted to the terminal device to achieve visual guidance feedback. This module presents real-time abnormal depth analysis information and intelligent guidance paths to surgeons by using intuitive data forms or other visualization methods, providing intuitive and clear surgical auxiliary information, helping doctors to more intuitively understand the patient's physiological condition and surgical guidance information during surgery, so as to make decisions and adjust surgical strategies more effectively.

[0074] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of a real-time image guidance system based on surgical operations of the present invention. In this embodiment, the real-time image guidance system based on surgical operations includes:

[0075] The historical image segmentation module is used to obtain the historical DICOM image data of patients in the medical database; perform image internal structure segmentation processing based on the historical DICOM image data to generate historical segmented DICOM image data;

[0076] In an embodiment of the present invention, firstly, by connecting to a medical database, historical DICOM images of patients undergoing surgical operations, such as CT scans, X-rays, and magnetic resonance images, are retrieved from the medical database. Subsequently, these historical DICOM images are analyzed and processed through image internal structure segmentation processing to generate accurate historical segmented DICOM image data. For example, by segmenting DICOM multimodal images using edge detection and threshold segmentation technology, the structures of different tissues and organs, such as lungs, heart, and bones, can be extracted and segmented, forming targeted historical segmented DICOM images, which provide high-quality basic data for the establishment of subsequent virtual anatomical structure models.

[0077] A virtual anatomical structure model building module is used to build a three-dimensional model of a virtual anatomical structure based on historical segmented DICOM image data to generate a virtual anatomical structure model;

[0078] In an embodiment of the present invention, the processed structural segmentation information is obtained from the historical segmentation DICOM image data. These segmentation data may include the position and boundary of anatomical structures such as lungs, heart, and blood vessels. The slice data analysis of the historical segmentation DICOM image data involves extracting the structural segmentation information of each slice, including the contour and region of each anatomical structure. The image space coordinate information of the historical segmentation DICOM image data is collected by using the slice data, and the boundary points and corresponding three-dimensional coordinates of each segmentation region are associated. According to the collected image space coordinate data, a three-dimensional model of the virtual anatomical structure is established, and the segmented two-dimensional structure information can be mapped to the three-dimensional space by using volume rendering or surface reconstruction technology to generate an accurate virtual anatomical structure model. On the basis of the established three-dimensional model, optimization and adjustment are performed, including steps such as removing unnecessary noise, smoothing the surface, and adjusting the proportion of the structure, to ensure that the virtual anatomical structure model is closer to the actual patient's anatomical structure. The generated virtual anatomical structure model is verified, and the virtual model is ensured to be consistent with the real anatomical structure in shape and position by comparing it with the image of the actual patient.

[0079] A surgical model design module is used to obtain real-time surgical imaging data of patients; a surgical model is designed based on the real-time surgical imaging data and a virtual anatomical structure model to obtain a surgical model;

[0080] In an embodiment of the present invention, the patient's image data is acquired in real time during the surgical operation, such as the patient's body surface irradiated by monitoring equipment and the image data of medical staff performing surgical operations, to reflect the current surgical status. The feature node data extracted from the historical segmented DICOM image data is matched with the corresponding feature nodes in the real-time surgical image data, which helps to align the virtual anatomical structure model with the actual surgical scene. Based on the matching results of the feature nodes, the real-time surgical image data is matched and aligned with the virtual anatomical structure model, and the real-time surgical image data is mapped to the surface structure of the virtual anatomical structure model to generate a real-time surgical model, including patient-specific anatomical structure, surgical instrument position and other information, and the surgical model is continuously updated in real time to adapt to changes in the surgical scene, which is achieved through the continuous input of real-time image data and the iteration of feature node matching.

[0081] A surgical assisted guidance path planning module is used to perform surgical assisted guidance path planning based on a surgical operation model and generate surgical assisted guidance path data;

[0082] In an embodiment of the present invention, a preset U-Net model is used to perform abnormal area analysis on a surgical model, which involves detecting abnormal structures, lesions or other special situations within the surgical area. The preset U-Net model is a model trained by a training set based on abnormal areas of anatomical structures. Based on the analysis results of the U-Net model, model abnormal area data is generated to indicate areas that require special attention in the surgical scene, providing important information for subsequent guidance path planning. Based on the model abnormal area data, a surgical auxiliary guidance path is designed, including the shortest guidance path information of the surface skin area, the guidance path information of the abnormal area inside the anatomical structure, etc. The designed surgical auxiliary guidance path is converted into specific surgical auxiliary guidance path data, including the coordinate points and direction of the path, so as to perform real-time guidance during the operation.

[0083] A real-time surgical auxiliary information analysis module is used to collect real-time surgical data based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information; perform abnormal depth analysis processing of the surgery based on the real-time surgical instrument depth information to generate abnormal depth analysis data; perform path intelligent adjustment processing on surgical auxiliary guidance path data based on the real-time surgical execution path information to generate intelligent surgical auxiliary guidance path data;

[0084] In an embodiment of the present invention, real-time surgical data is collected through real-time surgical imaging data, including extracting the real-time position and depth information of surgical instruments, and recording the path of surgical execution. Abnormal depth analysis is performed based on the real-time surgical instrument depth information collected, and the shortest edge distance calculation method may be used to detect whether the surgical instrument deviates from the predetermined operation area, and the relevant information is marked as abnormal depth data. Based on the analysis results of the real-time surgical instrument depth information, abnormal depth analysis data is generated and provided to surgeons to help them monitor the position of surgical instruments, prevent misoperation and make adjustments in time. According to the real-time surgical execution path information, the surgical auxiliary guidance path data is intelligently adjusted, and the Dijkstra algorithm and other algorithms are used to adjust the guidance path in real time according to the actual surgical situation to ensure the rationality of the path during the operation. Combined with the analysis results of the real-time surgical execution path information, intelligent surgical auxiliary guidance path data is generated, which will reflect the guidance path in the actual surgical process and provide surgeons with more intelligent guidance support.

[0085] The visual guidance feedback module is used to transmit abnormal depth analysis data and intelligent surgery assisted guidance path data to the terminal device to perform visual guidance feedback operations.

[0086] In an embodiment of the present invention, the abnormal depth analysis data and the intelligent surgical assisted guidance path data are transmitted to the terminal device to perform a visual guidance feedback operation, which can be completed through network transmission or other communication means. The visual guidance feedback operation is performed on the terminal device, which may include displaying the abnormal depth analysis results on the real-time surgical image, marking the abnormal depth on the instrument position, and marking the intelligently adjusted path information on the guidance path, and updating the visual parameter information of the abnormal depth analysis data and the intelligent surgical assisted guidance path data in real time, so as to provide the latest surgical guidance assistance information at any time.

[0087] Preferably, the historical image segmentation module includes the following functions:

[0088] Obtain historical DICOM image data of patients in the medical database;

[0089] Perform noise reduction processing on historical DICOM image data to generate historical noise-reduced DICOM image data; perform image standardization processing on historical noise-reduced DICOM image data to generate historical standard DICOM image data;

[0090] Mark the characteristic nodes of the historical standard DICOM image data and generate characteristic node data;

[0091] Perform spatial calibration and matching processing of multi-modal images on historical standard DICOM image data according to feature node data to generate historical calibrated DICOM image data;

[0092] The historical calibrated DICOM image data is segmented according to the internal structure of the image to generate historical segmented DICOM image data.

[0093] The present invention obtains the historical DICOM image data of patients in the medical database, and can understand the image data collected by multiple angles and different devices of the patients, including CT images, ultrasonic images, etc., and comprehensively understand the individual anatomical structure of the patients, providing a rich data basis for subsequent analysis and guidance. The historical DICOM image data is processed to remove noise and interference, and clear and high-quality historical noise-reduced DICOM image data is generated, which helps to improve the accuracy and stability of subsequent processing. The historical noise-reduced DICOM image data is standardized to ensure that the images are consistent in terms of scale, contrast and brightness, and generate historical standard DICOM image data. The standardized images help to perform subsequent processing and analysis more accurately. Marking key feature nodes in the historical standard DICOM image data, such as the edges of organs, important anatomical structures, etc., generates data containing these feature node information, helps to extract more biologically meaningful features, and provides more refined positioning information for subsequent segmentation and matching. Combined with the feature node data, the historical standard DICOM image data is subjected to spatial calibration and matching processing of multi-modal images to ensure the spatial consistency of images of different modalities, generate historical calibration DICOM image data, and provide a reliable basis for subsequent accurate internal structure segmentation. Utilizing historical calibrated DICOM image data and adopting advanced edge detection and threshold segmentation technology, the patient's internal structure is accurately segmented to generate historical segmented DICOM image data, providing high-quality, individualized anatomical structure information for subsequent virtual modeling and surgical guidance.

[0094] As an example of the present invention, refer to Figure 2 As shown, Figure 1 The function flow diagram of the historical image segmentation module in this example includes:

[0095] S201: Acquire historical DICOM image data of patients in the medical database;

[0096] In the embodiment of the present invention, firstly, by connecting to a medical database, historical DICOM images of patients undergoing surgical operations, such as CT scans, X-ray films, and magnetic resonance images, are retrieved from the medical database.

[0097] S202: performing noise reduction processing on the historical DICOM image data to generate historical noise-reduced DICOM image data; performing image standardization processing on the historical noise-reduced DICOM image data to generate historical standard DICOM image data;

[0098] In the embodiment of the present invention, a Gaussian filter algorithm is used to perform noise reduction processing on historical DICOM image data. Gaussian filtering is a commonly used linear filtering algorithm that can effectively remove noise in an image. An appropriate Gaussian kernel size and standard deviation parameter are set, and appropriate values ​​are selected according to the specific situation to balance the denoising effect and image detail retention. Gaussian filtering is applied to each DICOM image to generate historical noise reduction DICOM image data. The historical noise reduction DICOM image data is standardized using a grayscale standardization algorithm. According to the grayscale distribution of the image, an appropriate standardization method, such as linear transformation or histogram equalization, is selected to perform grayscale value standardization processing on the historical noise reduction DICOM image data to generate historical standard DICOM image data.

[0099] S203: marking characteristic nodes of historical standard DICOM image data to generate characteristic node data;

[0100] In an embodiment of the present invention, in the historical standard DICOM image data, a key structure or lesion to be marked is selected by a pre-set feature node marking method, which may be a specific region of an organ, etc. For the selected key structure or lesion, a feature node is created by marking a specific point or region on the image, and these feature nodes may be pixel points in the image, center points or edge points of the region, etc. The position information of the marked feature node is saved into a data structure to generate feature node data, which generally includes coordinate information, associated anatomical structure edge information, etc. For multiple historical standard DICOM image data, the marking consistency of the feature nodes is ensured, that is, the feature node marks of the same structure or lesion in different images should have similar positions.

[0101] S204: performing spatial calibration and matching processing of multimodal images on the historical standard DICOM image data according to the characteristic node data to generate historical calibrated DICOM image data;

[0102] In an embodiment of the present invention, one or more common reference structures are selected from the feature node data. These structures should be accurately marked in different modal images. The selected reference structures are used to spatially match the historical standard DICOM image data of different modalities using a suitable image registration algorithm, including a rigid or non-rigid registration method, so that the feature nodes are aligned between different modalities. The historical standard DICOM image data is transformed accordingly through the obtained registration transformation parameters to ensure the consistency of the common reference structure between different modalities. For the matched images, quality verification is performed to ensure that the calibration results meet expectations, especially the spatial consistency of the reference structure. The historical standard DICOM image data after spatial calibration and matching is saved to generate historical calibrated DICOM image data.

[0103] S205: Perform image internal structure segmentation processing on the historical calibration DICOM image data to generate historical segmentation DICOM image data.

[0104] In an embodiment of the present invention, a target structure to be segmented, such as an organ, tissue, etc., is selected from historical calibrated DICOM image data. According to the selected target structure, a suitable image segmentation algorithm is selected, including threshold segmentation, edge detection, etc. According to the characteristics of the actual image data, the parameters of the segmentation algorithm are adjusted to obtain the best segmentation result. For example, the edge features of the DICOM image are first distinguished using edge detection technology, and a suitable threshold is selected for image segmentation using threshold segmentation technology to segment the image into the target structure and the background, ensuring that the segmentation result maintains the accuracy and integrity of the anatomical structure, and post-processing steps such as removing small areas and filling holes can be applied to generate historical segmented DICOM image data.

[0105] Preferably, the performing image internal structure segmentation processing on the historical calibration DICOM image data comprises:

[0106] The edge detection algorithm is used to analyze the edge of the internal structure of the historical calibration DICOM image data to generate the internal structure edge data;

[0107] Based on the threshold segmentation technology and the internal structure edge data, the historical calibrated DICOM image data is segmented to generate the historical segmented DICOM image data.

[0108] The present invention uses edge detection algorithm to process historical calibration DICOM image data, performs edge analysis of the internal structure of the image, identifies edge information in the image, helps to extract edge features of the internal structure, and provides a basis for subsequent segmentation. Based on threshold segmentation technology and combined with internal structure edge data, the historical calibration DICOM image data is segmented to perform image internal structure segmentation processing, and through reasonable threshold selection and edge information guidance, accurate segmentation of the internal structure is achieved to generate historical segmented DICOM image data.

[0109] In an embodiment of the present invention, an edge detection algorithm, such as a Sobel operator or a Canny edge detection algorithm, is used to perform edge analysis processing on the historical calibration DICOM image data, emphasizing the edge information of different structures in the image to provide auxiliary information for subsequent segmentation. The result generated by the edge detection algorithm will be the edge information of each structure in the image, such as the edge contour of an organ, tissue or other anatomical structure in the image, to generate internal structure edge data. Based on a threshold segmentation technique, such as Otsu's method, the internal structure edge data is binarized to obtain a binary image containing a target structure and a background, the binarized internal structure edge image is applied to the historical calibration DICOM image data, the threshold parameters of the Otsu's method are designed according to the internal structure edge data, the image is segmented into a target structure and a background according to the threshold parameters, and the anatomical structure information of each target structure is marked to generate historical segmented DICOM image data.

[0110] Preferably, the virtual anatomical structure model building module includes the following functions:

[0111] Perform slice data analysis and processing of DICOM images based on historical segmented DICOM image data to generate DICOM image slice data;

[0112] According to the DICOM image slice data, the image space coordinate data is collected for the historical segmented DICOM image data to generate the image space coordinate data;

[0113] A three-dimensional model of a virtual anatomical structure is established based on the historical segmentation DICOM image data and the corresponding image space coordinate data to generate a virtual anatomical structure model.

[0114] The present invention performs slice data analysis and processing based on historical segmentation of DICOM image data, involving cutting three-dimensional DICOM image data into multiple two-dimensional slices to better analyze and process the anatomical structure information of each slice. By using DICOM image slice data to collect image spatial coordinate data, the system can obtain the precise position information of each slice in three-dimensional space, and provide necessary coordinate data for subsequent three-dimensional modeling. A three-dimensional model of a virtual anatomical structure is established based on historical segmentation of DICOM image data and corresponding image space coordinate data. By integrating information from multiple slices, a patient-individualized, high-precision virtual anatomical structure model is generated to provide accurate anatomical structure information for surgical guidance.

[0115] In an embodiment of the present invention, slice data analysis and processing of DICOM images is performed based on historical segmentation DICOM image data, and slice information of parameters such as CT images and magnetic resonance images in DICOM images is analyzed, which involves selecting the axial direction, angle and thickness of the slice, and analyzing the slice data of the image in the historical segmentation DICOM image data based on the axial direction, angle and thickness of the slice to generate DICOM image slice data. According to the DICOM image slice data, spatial coordinate data of the image is collected for the historical segmentation DICOM image data, and the relative coordinates of the pixel points of each image in the historical segmentation DICOM image data are analyzed based on the slice position, slice direction, pixel size, etc. of the DICOM image slice data, and the extracted slice coordinate information is converted into spatial coordinates. The coordinate space can be a three-dimensional space designed by selecting an origin coordinate to collect the spatial coordinate data corresponding to each DICOM image, including three-dimensional coordinates (x, y, z), indicating the position of each pixel in the coordinate system, so as to form a complete spatial coordinate set of historical segmentation DICOM image data, and generate image spatial coordinate data. The corresponding image spatial coordinate data is integrated with the historical segmentation DICOM image data to ensure that each pixel point has corresponding three-dimensional coordinate information. An initial virtual anatomical structure model is created in 3D modeling software using the integrated imaging data. Each segmented area is 3D voxelized or surface modeled to reflect the shape of the anatomical structure. The initial virtual anatomical structure model is filled and adjusted using interpolation to ensure the smoothness of the model and the integrity of the anatomical structure, including interpolation of segmented edges and image gaps, filling of holes in the model, etc. The generated virtual anatomical structure model is optimized, including smoothing operations, removal of unnecessary details, and adjustment of model proportions, etc., to make it more consistent with the actual anatomical structure, thereby generating a virtual anatomical structure model.

[0116] Preferably, the three-dimensional modeling of the virtual anatomical structure based on the historical segmentation DICOM image data and the corresponding image space coordinate data comprises:

[0117] Establishing a three-dimensional model of the initial virtual anatomical structure according to the historical segmentation DICOM image data and the corresponding image space coordinate data, and generating an initial virtual anatomical structure model;

[0118] The initial virtual anatomical structure model is filled and adjusted by using the interpolation method to generate an adjusted virtual anatomical structure model;

[0119] The adjusted virtual anatomical structure model is subjected to model smoothing optimization processing to generate a virtual anatomical structure model.

[0120] The present invention combines segmented DICOM image data with accurate image space coordinate data to obtain coordinate data of each two-dimensional image in three-dimensional space, arranges the two-dimensional images in the three-dimensional space coordinate data to establish the architecture information of the virtual anatomical structure model, and fills and adjusts the architecture information of the virtual anatomical structure model through subsequent interpolation technology to obtain a more continuous and refined virtual anatomical structure model. The interpolation method helps to eliminate existing spatial gaps, improve the integrity and realism of the model, perform model smoothing optimization processing on the adjusted virtual anatomical structure model, smooth out the defects caused by the residual image in the three-dimensional model and the edge of the image, reduce irregular geometric shapes, and thus generate a virtual anatomical structure model.

[0121] In an embodiment of the present invention, the corresponding image space coordinate data, such as the three-dimensional coordinates (x, y, z) of the pixel points of each historical segmentation DICOM image data, are integrated with the historical segmentation DICOM image data. The integrated image data is used to create an initial virtual anatomical structure model in a three-dimensional modeling software, and each segmented area is subjected to three-dimensional voxelization or surface modeling to reflect the shape of the anatomical structure. The initial virtual anatomical structure model is filled and adjusted using an interpolation method to ensure the smoothness of the model and the integrity of the anatomical structure, including interpolation of segmentation edges and image gaps, filling of holes inside the model, etc. The generated virtual anatomical structure model is optimized, including smoothing operations, removal of unnecessary details, adjustment of model proportions, etc., to make it more consistent with the actual anatomical structure, thereby generating a virtual anatomical structure model.

[0122] Preferably, the surgical model design module includes the following functions:

[0123] Acquire real-time surgical imaging data of patients;

[0124] Perform feature node matching processing on real-time surgical image data according to feature node data of historical segmented DICOM image data to generate feature node matching data;

[0125] Based on the feature node matching data, real-time surgical image data is mapped to the virtual anatomical structure model for surgical model design to generate a surgical model.

[0126] The present invention obtains the surgical image data of the patient in real time to ensure that the model design can be carried out based on the actual surgical situation of the current patient. The feature node data of the historical segmented DICOM image data is used to perform feature node matching processing on the real-time surgical image data, identify key feature nodes in the real-time surgical image, such as the edge of the organ or other important anatomical structures, and match them with the historical data to establish a corresponding relationship between the real-time image and the historical segmented data, and generate feature node matching data according to the feature node matching processing. These data record the corresponding positions of the key feature nodes in the real-time surgical image in the historical segmented data, and provide key positioning information for subsequent model design. Based on the feature node matching data, the real-time surgical image data is mapped to the virtual anatomical structure model, and the spatial correspondence between the real-time image and the virtual anatomical structure model is achieved through the matching feature node information. The surgical model is designed on the basis of the virtual anatomical structure model, and the real-time surgical image data is combined with the virtual anatomical structure model to generate a surgical model that matches the patient's current anatomical structure, so as to better understand the patient's actual anatomical situation and improve the individualization and accuracy of surgical planning.

[0127] In an embodiment of the present invention, the image data of the patient is obtained in real time during the surgical operation, such as the image data of the patient's body surface irradiated by the monitoring equipment and the medical staff performing the surgical operation, so as to reflect the current surgical status. The feature node data of the historical segmented DICOM image data are used, and these feature nodes are the node information designed in the above steps, and the feature node matching processing is performed on the real-time surgical image data. For example, if specific vascular structure nodes are marked in the historical segmented DICOM image data, the skin surface nodes corresponding to the vascular structure nodes in the real-time image are matched by matching these nodes. Based on the feature node matching processing, matching data corresponding to the feature nodes of the historical segmented DICOM image data in the real-time surgical image data is generated, including coordinates, angles, sizes and other information in the real-time image. Using the generated feature node matching data, the real-time surgical image data is mapped to the virtual anatomical structure model, including matching a specific area in the real-time data with a corresponding area in the virtual model, so as to locate the position of the real-time surgical image in the virtual model. Based on the mapped real-time surgical image data and the virtual anatomical structure model, the surgical model is designed, including depicting the organ position, vascular structure, etc. in the real-time surgery in the virtual model, so as to form a virtual surgical model matching the actual surgery.

[0128] Preferably, the surgical assisted guidance path planning module includes the following functions:

[0129] Based on the preset U-Net model, the abnormal area of ​​the surgical model is analyzed and processed to generate the abnormal area data of the model;

[0130] The surgical assisted guidance path is designed based on the abnormal area data of the model to generate surgical assisted guidance path data.

[0131] The present invention uses a preset U-Net model to perform abnormal area analysis and processing on a surgical model. The U-Net model is a deep learning architecture, which is particularly suitable for image segmentation tasks. The pre-designed U-Net model is used to identify abnormal areas in a surgical model, such as lesions or risk areas, to generate model abnormal area data. These data represent the location and range of abnormal or critical areas identified in the surgical model, providing key information for subsequent guidance path design. According to the model abnormal area data, a surgical auxiliary guidance path design is performed, and an optimized guidance path is provided, such as a guidance path for the skin surface and internal anatomical structures, taking into account the location and characteristics of the abnormal area, to ensure that the operation can be more accurate and safe when performed in the abnormal area. Based on the surgical auxiliary guidance path design, the final surgical auxiliary guidance path data is generated. These data include geometric information, direction and corresponding guidance instructions of the guidance path, providing real-time and personalized surgical guidance for surgeons.

[0132] In an embodiment of the present invention, a preset U-Net model is used to analyze and process abnormal regions of a surgical model. The preset U-Net model is a model trained by a training set of abnormal regions of anatomical structures, and is used to segment abnormal regions of a surgical model, especially for detecting abnormal regions in medical images, such as tumors or other lesions. Based on the analysis results of the U-Net model, abnormal region data in the surgical model is generated, which is a binary image, in which the abnormal region is marked as a specific pixel value, and the normal region is another pixel value. Using the generated model abnormal region data, a surgical auxiliary guidance path is designed, and the design process includes: abnormal region positioning: determining which regions in the surgical model are considered abnormal regions, which can be obtained by performing region segmentation or contour extraction on the abnormal region data. Path design: based on the location of the abnormal region, one or more surgical auxiliary guidance paths are designed to guide the surgeon to better handle or avoid the abnormal region. The path can be a guidance path along a safe area, away from the abnormal region, or a specific incision, including the shortest guidance path information of the surface skin area, the guidance path information of the abnormal region inside the anatomical structure, etc. The designed surgical assisted guidance path is converted into data form, which may be a series of coordinate points, path curve equations, etc., so as to guide the actual operation.

[0133] Preferably, the real-time surgical auxiliary information analysis module includes the following functions:

[0134] Performing real-time surgical data acquisition based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information;

[0135] Mapping real-time surgical instrument depth information to a surgical model to perform a reference depth conversion of the surgical instrument, thereby generating reference depth data of the surgical instrument;

[0136] Perform effective area marking processing on the surgical model to generate effective area data of the model;

[0137] Perform abnormal depth analysis and processing of the surgical instrument based on the surgical instrument baseline depth data and the model effective area data to generate abnormal depth analysis data;

[0138] Mapping the real-time surgical execution path information to the surgical operation model to perform a reference distance conversion of the surgical execution path, and generating surgical execution path reference distance data;

[0139] The surgical auxiliary guidance path data is intelligently adjusted according to the surgical execution path benchmark distance data to generate intelligent surgical auxiliary guidance path data.

[0140] The present invention maps real-time surgical instrument depth information to a surgical model, performs a reference depth conversion of the surgical instrument, generates reference depth data of the surgical instrument, helps to align the depth information of the real-time surgical instrument with the model, and provides an accurate reference for subsequent abnormal depth analysis. The surgical model is marked with an effective area, and model effective area data is generated, which is used to mark the effective anatomical area in the model to ensure the accuracy of depth analysis and path adjustment. Based on the surgical instrument reference depth data and the model effective area data, the abnormal depth analysis of the surgical instrument is performed to generate abnormal depth analysis data, which helps to detect whether the surgical instrument deviates from the normal depth range and provide real-time abnormal depth information. The real-time surgical execution path information is mapped to the surgical model to perform a reference distance conversion of the surgical execution path, and the surgical execution path reference distance data is generated. By aligning the real-time execution path with the model, accurate distance conversion is achieved, and an accurate reference is provided for path intelligent adjustment. Based on the surgical execution path reference distance data, the surgical auxiliary guidance path data is intelligently adjusted, and intelligent surgical auxiliary guidance path data is generated, which helps to intelligently adjust the guidance path according to the actual surgical execution situation, and ensure that the guidance path is consistent with the actual surgery.

[0141] As an example of the present invention, refer to Figure 3 As shown, Figure 1The function flow diagram of the real-time surgical auxiliary information analysis module in this example includes:

[0142] S301: performing real-time surgical data acquisition according to real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information;

[0143] In an embodiment of the present invention, a monitoring device and a sensor device are used to perform depth perception or structural recognition on real-time surgical images to extract the depth information of real-time surgical instruments, including using stereo vision technology, depth sensors or other sensing devices to obtain the depth of surgical instruments relative to the surgical scene. By analyzing the real-time surgical image sequence and tracking the motion trajectory of the surgical instrument, real-time surgical execution path information is obtained. This can be achieved by calculating the position and direction changes of the surgical instrument in the image, or by tracking identifiable markers. The real-time surgical instrument depth information and the real-time surgical execution path information are integrated to form real-time surgical data, including recording the three-dimensional coordinates and depth of the instrument at each time point, as well as the motion trajectory of the surgical instrument on the execution path.

[0144] S302: Mapping the real-time surgical instrument depth information to the surgical operation model to perform reference depth conversion of the surgical instrument to generate reference depth data of the surgical instrument;

[0145] In an embodiment of the present invention, real-time surgical instrument depth information and a virtual anatomical structure model are used to map the depth information and convert the reference depth. The depth information of the real-time surgical instrument is mapped to the corresponding position of the virtual anatomical structure model, and the depth information is calibrated to obtain reference depth data. The mapped and converted depth information is integrated to generate reference depth data of the surgical instrument, which indicates the depth position of the surgical instrument relative to the virtual anatomical structure model, thereby providing basic data for subsequent surgical-assisted analysis.

[0146] S303: performing effective area marking processing on the surgical model to generate effective area data of the model;

[0147] In an embodiment of the present invention, a surgical model is used to determine the effective area during the operation according to the specific needs and type of the operation. The effective area may include the surgical target area, the drivable path area, important structures, etc. The marking process may be completed by a surgical professional or an algorithm preset by the system. The marking information of the effective area is embedded in the surgical model, and the marking process of the effective area is performed by adding marking tags or attributes to specific areas of the model, so as to accurately identify the effective area of ​​the model in subsequent analysis. After the marking process is completed, structured data containing the effective area data of the model is generated. These data may be the effectiveness mark of each point, each area or each structure in the surgical model.

[0148] S304: performing abnormal depth analysis processing of the surgical instrument based on the surgical instrument reference depth data and the model effective area data to generate abnormal depth analysis data;

[0149] In an embodiment of the present invention, the effective area data of the surgical model is used to ensure that the system understands which areas are considered to be effective guide paths that can be navigated, which areas need to be avoided, or which areas need special attention during the operation. By comparing the real-time surgical instrument reference depth and the model effective area data, the system can analyze whether there is an abnormal depth. For example, if the depth of the surgical instrument exceeds the preset safe depth range or enters an area that should be avoided, the system will determine this situation as an abnormal depth. In the case of detecting an abnormal depth, the system will generate abnormal depth analysis data, which includes specific information about the abnormal depth, such as the instrument position, depth offset, etc.

[0150] S305: Mapping the real-time surgical execution path information to the surgical operation model to perform a reference distance conversion of the surgical execution path, and generating surgical execution path reference distance data;

[0151] In the embodiment of the present invention, the real-time surgical execution path information is mapped to the surgical model, and the reference distance conversion of the surgical execution path is performed. This step ensures that the actual surgical execution path information can be accurately represented in the model and matched with the preset reference distance, and generates reference distance data containing the surgical execution path, including the relative position of the real-time surgical instrument in the surgical model, the length of the path and other information.

[0152] S306: Perform intelligent path adjustment processing on the surgical auxiliary guidance path data according to the surgical execution path benchmark distance data to generate intelligent surgical auxiliary guidance path data.

[0153] In an embodiment of the present invention, the surgical execution path reference distance data is used and applied to the surgical auxiliary guidance path data, which involves combining the surgical execution path reference distance data with the surgical auxiliary guidance path data. Based on the reference distance data of the surgical execution path, an intelligent adjustment algorithm is used to adjust the surgical auxiliary guidance path, and a path planning algorithm such as the Dijkstra algorithm is used to consider the reference distance of the surgical execution path to optimize the guidance path, ensuring that the path is more accurate and adaptable to the actual situation during the surgical execution process. Based on the adjusted path, the final intelligent surgical auxiliary guidance path data is generated, including updated path coordinates, direction and other information to ensure more accurate guidance during the operation.

[0154] Preferably, the real-time analysis and processing of abnormal depth of surgery based on surgical instrument reference depth data and model effective area data includes:

[0155] Perform real-time calculation and processing of the shortest edge distance of the effective area of ​​the surgical instrument benchmark depth data and the model effective area data to generate real-time shortest edge distance data;

[0156] When the real-time shortest edge distance data is less than a preset effective area edge threshold, the real-time surgical instrument depth information corresponding to the real-time shortest edge distance data is marked as abnormal depth data;

[0157] The abnormal depth data is analyzed and processed for abnormal depth of surgical instruments to generate abnormal depth analysis data.

[0158] The present invention performs calculation and processing of the shortest edge distance of the effective area in real-time operation for surgical instrument reference depth data and model effective area data, considers the position of the surgical instrument in the effective area, calculates the shortest distance between the instrument and the edge of the effective area, and the direction of the shortest distance, thereby generating real-time shortest edge distance data, which are used to record the distance of the surgical instrument relative to the edge of the effective area and provide real-time depth information. When the real-time shortest edge distance data is less than the preset effective area edge threshold, the corresponding real-time surgical instrument depth information is marked as abnormal depth data, and according to the pre-set threshold, it is determined whether the surgical instrument deviates from the expected effective area range, so as to achieve real-time marking of abnormal depth. The marked abnormal depth data is subjected to abnormal depth analysis processing of the surgical instrument, including further analysis of the specific situation of the abnormal depth, including specific information such as the degree and direction of the abnormal depth, so as to provide more detailed abnormal depth analysis data.

[0159] In an embodiment of the present invention, the real-time collected surgical instrument reference depth data and model effective area data are used to perform real-time calculation and processing of the shortest edge distance of the effective area, which involves the use of computational geometry methods, such as calculating the shortest distance between the surgical instrument contour and the edge of the model effective area. When the real-time shortest edge distance data is less than the preset effective area edge threshold, the real-time surgical instrument depth information corresponding to the real-time shortest edge distance data is marked as abnormal depth data to identify whether the surgical instrument has deviated from the expected effective area. The data marked as abnormal depth is subjected to abnormal depth analysis processing of the surgical instrument, including geometric analysis and morphological analysis of the position information analysis of the surgical instrument and the relative distance analysis from the dangerous area, so as to generate abnormal depth analysis data.

[0160] Preferably, the intelligent path adjustment processing of the surgical auxiliary guidance path data according to the surgical execution path reference distance data includes:

[0161] The surgical auxiliary guidance path data is updated according to the surgical execution path benchmark distance data, and the updated surgical auxiliary guidance path data is intelligently adjusted for the shortest path using the Dijkstra algorithm to generate intelligent surgical auxiliary guidance path data.

[0162] The present invention updates the surgical auxiliary guidance path data according to the surgical execution path benchmark distance data. The system considers the actual situation of the surgical execution path and dynamically adjusts the guidance path to adapt to the real-time surgical execution situation. The updated surgical auxiliary guidance path data is intelligently adjusted by the Dijkstra algorithm. The Dijkstra algorithm is a classic algorithm for calculating the shortest path between nodes in a graph. The guidance path is optimized by effectively selecting the path, making it more reasonable and efficient in actual surgical execution. Finally, intelligent surgical auxiliary guidance path data is generated, including the adjusted guidance path, to provide surgeons with more intelligent and real-time surgical guidance.

[0163] In an embodiment of the present invention, the surgical auxiliary guidance path data is updated based on the surgical execution path benchmark distance data, including path adjustment, node update or path segment recalculation to ensure that the path conforms to the actual surgical execution situation. The updated surgical auxiliary guidance path data is intelligently adjusted to the shortest path using the Dijkstra algorithm. The Dijkstra algorithm is a graph algorithm that can find the shortest path between multiple points. In this scenario, it is used to adjust the guidance path according to the actual surgical execution situation to make it as close to the shortest path as possible, and to generate intelligent surgical auxiliary guidance path data by combining the updated path and the adjustment result of the Dijkstra algorithm.

[0164] The beneficial effects of the present application are that the real-time image guidance system based on surgical operations of the present invention establishes a more detailed three-dimensional anatomical structure model by analyzing the multimodal images of DICOM, and can accurately analyze the patient's internal anatomical structure, thereby providing a sufficiently accurate anatomical guidance route, and can update the surgical guidance information in real time according to the executed anatomical path, as well as feedback abnormal information on the depth distance of the instrument during the surgical operation, so as to better assist medical personnel in performing subsequent surgical operations.

[0165] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0166] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A real-time image-guided system for surgical operations. It is characterized in that include: The historical image segmentation module is used to obtain the historical DICOM image data of patients in the medical database; perform image internal structure segmentation processing based on the historical DICOM image data to generate historical segmented DICOM image data; A virtual anatomical structure model building module is used to build a three-dimensional model of a virtual anatomical structure based on historical segmented DICOM image data to generate a virtual anatomical structure model; A surgical model design module is used to obtain real-time surgical imaging data of patients; a surgical model is designed based on the real-time surgical imaging data and a virtual anatomical structure model to obtain a surgical model; A surgical assisted guidance path planning module is used to perform surgical assisted guidance path planning based on a surgical operation model and generate surgical assisted guidance path data; A real-time surgical auxiliary information analysis module is used to collect real-time surgical data based on real-time surgical image data to generate real-time surgical data, wherein the real-time surgical data includes real-time surgical instrument depth information and real-time surgical execution path information; map the real-time surgical instrument depth information to the surgical model to perform a reference depth conversion of the surgical instrument to generate surgical instrument reference depth data; perform effective area marking processing on the surgical model to generate model effective area data; Perform abnormal depth analysis and processing of the surgical instrument based on the surgical instrument baseline depth data and the model effective area data to generate abnormal depth analysis data; Mapping the real-time surgical execution path information to the surgical operation model to perform a reference distance conversion of the surgical execution path, and generating surgical execution path reference distance data; Perform intelligent path adjustment processing on the surgical auxiliary guidance path data according to the surgical execution path benchmark distance data to generate intelligent surgical auxiliary guidance path data; The visual guidance feedback module is used to transmit abnormal depth analysis data and intelligent surgery assisted guidance path data to the terminal device to perform visual guidance feedback operations.

2. The real-time image guidance system for surgical operations according to claim 1, It is characterized in that The historical image segmentation module includes the following functions: Obtain historical DICOM image data of patients in the medical database; Perform noise reduction processing on historical DICOM image data to generate historical noise-reduced DICOM image data; perform image standardization processing on historical noise-reduced DICOM image data to generate historical standard DICOM image data; Mark the characteristic nodes of the historical standard DICOM image data and generate characteristic node data; Perform spatial calibration and matching processing of multi-modal images on historical standard DICOM image data according to feature node data to generate historical calibrated DICOM image data; The historical calibrated DICOM image data is segmented according to the internal structure of the image to generate historical segmented DICOM image data.

3. The real-time image guidance system for surgical operations according to claim 2, It is characterized in that The image internal structure segmentation processing of the historical calibration DICOM image data comprises: The edge detection algorithm is used to analyze the edge of the internal structure of the historical calibration DICOM image data to generate the internal structure edge data; Based on the threshold segmentation technology and the internal structure edge data, the historical calibrated DICOM image data is segmented to generate the historical segmented DICOM image data.

4. The real-time image-guided surgical operation system according to claim 1, It is characterized in that The virtual anatomy structure model building module includes the following functions: Perform slice data analysis and processing of DICOM images based on historical segmented DICOM image data to generate DICOM image slice data; According to the DICOM image slice data, the image space coordinate data is collected for the historical segmented DICOM image data to generate the image space coordinate data; A three-dimensional model of a virtual anatomical structure is established based on the historical segmentation DICOM image data and the corresponding image space coordinate data to generate a virtual anatomical structure model.

5. The real-time image guidance system for surgical operations according to claim 4, It is characterized in that The three-dimensional model establishment of the virtual anatomical structure according to the historical segmentation DICOM image data and the corresponding image space coordinate data includes: Establishing a three-dimensional model of the initial virtual anatomical structure according to the historical segmentation DICOM image data and the corresponding image space coordinate data, and generating an initial virtual anatomical structure model; The initial virtual anatomical structure model is filled and adjusted by using the interpolation method to generate an adjusted virtual anatomical structure model; The adjusted virtual anatomical structure model is subjected to model smoothing optimization processing to generate a virtual anatomical structure model.

6. The real-time image-guided surgical operation system according to claim 2, It is characterized in that The surgical model design module includes the following functions: Acquire real-time surgical imaging data of patients; Performing feature node matching processing on real-time surgical image data according to feature node data of historical segmented DICOM image data to generate feature node matching data; Based on the feature node matching data, real-time surgical image data is mapped to the virtual anatomical structure model for surgical model design to generate a surgical model.

7. The real-time image-guided surgical operation system according to claim 1, It is characterized in that The surgical assisted guidance path planning module includes the following functions: Based on the preset U-Net model, the abnormal area of ​​the surgical model is analyzed and processed to generate the abnormal area data of the model; The surgical assisted guidance path is designed based on the abnormal area data of the model to generate surgical assisted guidance path data.

8. The real-time image-guided surgical operation system according to claim 1, It is characterized in that The abnormal depth real-time analysis and processing of the surgery based on the surgical instrument reference depth data and the model effective area data includes: Perform real-time calculation and processing of the shortest edge distance of the effective area of ​​the surgical instrument benchmark depth data and the model effective area data to generate real-time shortest edge distance data; When the real-time shortest edge distance data is less than a preset effective area edge threshold, the real-time surgical instrument depth information corresponding to the real-time shortest edge distance data is marked as abnormal depth data; The abnormal depth data is analyzed and processed for abnormal depth of surgical instruments to generate abnormal depth analysis data.

9. The real-time image-guided surgical operation system according to claim 8, It is characterized in that The intelligent path adjustment processing of the surgical auxiliary guidance path data according to the surgical execution path reference distance data comprises: The surgical auxiliary guidance path data is updated according to the surgical execution path benchmark distance data, and the updated surgical auxiliary guidance path data is intelligently adjusted for the shortest path using the Dijkstra algorithm to generate intelligent surgical auxiliary guidance path data.

Citation Information

Cited By

  • Laser-assisted accurate angle measurement and display system in image interventional operation

    CN120859679A