Ai-assisted head and neck surgery robot system based on multi-depth learning algorithm

Abstract

The application discloses an AI-assisted head and neck surgery robot system based on a multi-depth learning algorithm, which is applied to head and neck surgery and comprises a data acquisition and preprocessing module, an anatomical structure recognition module and a real-time inference and feedback module; the anatomical structure recognition module is used for performing multi-model processing on an original image to generate a structure model, and labeling a target structure on the structure model; the data acquisition and preprocessing module is used for preprocessing data in each input stage and acquiring original data and real-time data of a user in a surgery process; and the real-time inference and feedback module is used for associating the anatomical structure recognition module and the data acquisition and preprocessing module to realize real-time feedback of information of the target structure to the user; through mutual cooperation of the above modules, informationization labeling and display of the target structure are realized, and efficient information feedback is generated in the surgery process to assist medical staff in judging the target structure.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and more particularly to an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms. Background Technology

[0002] Head and neck surgery is a specialized surgical procedure designed to treat diseases of the head and neck region, including tumors, trauma, congenital malformations, and infections. It involves complex anatomical areas such as the head, neck, face, mouth, pharynx, larynx, trachea, and esophagus. Compared to other types of surgery, the head and neck region is rich in blood vessels and nerves, including major blood vessels (common carotid artery, internal jugular vein) and important nerves (facial nerve, glossopharyngeal nerve, vagus nerve), and features complex vascular and nerve pathways and their adjacent relationships. Damage to blood vessels or nerves during surgery can lead to increased bleeding, paralysis of the affected nerve distribution area, and other adverse complications, increasing the risk of postoperative infection, delaying recovery, and significantly impacting the patient's postoperative quality of life. Therefore, head and neck surgery must be performed with high precision to minimize damage to these structures and avoid interfering with the patient's basic physiological functions. The unique characteristic of head and neck surgery compared to surgery in other areas lies in its profound impact on the patient's daily life and social interactions, particularly in basic functions such as speech, swallowing, and breathing. Damage to important anatomical structures during surgery can affect the patient's speech and respiratory functions, thereby impacting their social and psychological state.

[0003] Currently, medical staff rely on their own senses and experience to identify anatomical structures, increasing the risk and difficulty of surgery. Furthermore, head and neck surgeries are characterized by limited visibility and operating space, requiring medical staff to operate within confined areas, making it impossible to fully and clearly identify anatomical structures, further increasing surgical risks. Therefore, a more reasonable solution is urgently needed that can accurately identify anatomical structures while providing precise and efficient real-time feedback, thereby assisting medical staff in making appropriate judgments during head and neck surgeries. Summary of the Invention

[0004] To address the technical problem that existing head and neck surgeries lack suitable methods for identifying and labeling anatomical structures to assist medical staff in making reasonable judgments, and that existing technologies cannot provide accurate real-time feedback to medical staff during surgery to facilitate timely assessments, this invention provides a solution.

[0005] To achieve the above objectives, the present invention provides an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms, the system comprising:

[0006] The anatomical structure recognition module is used to perform multi-model processing on the original image to generate a structural model, and to annotate the target structure on the structural model.

[0007] It also includes a data acquisition and preprocessing module, which is used to preprocess the data at each input stage and to acquire the raw data and the user's real-time data during the surgery.

[0008] The real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to provide real-time feedback of the target structure information to the user.

[0009] Before the original image is input into the model for processing, it needs to undergo preprocessing. The preprocessing steps include:

[0010] Image standardization: Obtain the original image and perform normalization processing on the original image to obtain a first-level image;

[0011] Image enhancement: Performing data enhancement on a primary image to obtain a secondary image;

[0012] Image annotation: Labeling the secondary images and ensuring spatial consistency will yield training data.

[0013] The data augmentation methods include at least one of geometric transformation, color and brightness adjustment, noise addition, and local deformation.

[0014] The generation of the structural model includes the following steps:

[0015] The training data is input into the YOLO model for modeling processing to generate a structural model.

[0016] The YOLO model training process is as follows:

[0017] A1. Data preprocessing: Adjusting the training data to a fixed size to achieve standardized input;

[0018] A2. Object Detection: The YOLO model extracts image features through a convolutional neural network to generate bounding boxes of surgical targets and their corresponding class probabilities;

[0019] A3. Optimize training: Use cross-entropy loss function and IoU loss function to optimize the model and ensure that the convolutional network can produce localization and classification results during detection;

[0020] A4. Data Output: The output includes the location information of each target structure and the corresponding classification probability.

[0021] After obtaining the structural model, the process also includes the following steps:

[0022] The ResNet model is used to receive the output data of the YOLO model, and the surgical target location is classified and refined to output classification information.

[0023] The ResNet model training process is as follows:

[0024] B1. Feature Extraction and Classification: Input at least one location information, perform multi-layer convolution and pooling operations on the ResNet model to extract higher-level image features, and use fully connected layers to further classify anatomical structures so that each surgical target is assigned a probability value.

[0025] B2. Cross-entropy loss function: Optimize the ResNet model using the cross-entropy loss function;

[0026] B3. Training Optimization: The Adam optimizer is used for training, and the weight parameters in the model are adjusted to minimize the loss function;

[0027] B4. Data Output: The ResNet model outputs the category and confidence score of each target structure as classification information.

[0028] After completing the ResNet model processing, the process also includes:

[0029] Location and classification information are input into the Transformer model for processing. The Transformer model combines 3D image data to generate spatial relationship modeling.

[0030] The training process for the Transformer model is as follows:

[0031] C1. Encoder layer processing: The Transformer model uses an encoder to extract spatial features from the input image formed by the YOLO model, ResNet model, and 3D image data; the encoder models spatial characteristics through a multi-layer self-attention mechanism, generating spatial representation data for each target structure for output;

[0032] C2. Decoder layer processing: Based on the encoder output, the decoder further predicts the spatial relationships and relative positions between the various target structures;

[0033] C3. Loss Function Optimization: A preset loss function is used, combined with spatial relationships and the category of the target structure, to improve the accuracy of the model;

[0034] C4. Data Output: The output of the Transformer model includes the spatial coordinates of each anatomical target and their relative positional relationships.

[0035] It also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time reasoning and feedback module as sub-modules, and also includes an auxiliary sub-module, which is associated with the anatomical structure recognition module and the real-time reasoning and feedback module.

[0036] The beneficial effects of this invention are as follows: Compared with the prior art, this invention provides an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms, applied in head and neck surgery. The system includes a data acquisition and preprocessing module, an anatomical structure recognition module, and a real-time reasoning and feedback module. The anatomical structure recognition module performs multi-model processing on the original image to generate a structural model, and annotates the target structure on the structural model. The data acquisition and preprocessing module preprocesses the data at each input stage and acquires the original data and real-time data from the user during the surgery. The real-time reasoning and feedback module connects the anatomical structure recognition module and the data acquisition and preprocessing module to provide real-time feedback of the target structure information to the user. Through the cooperation of the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time reasoning and feedback module, the target structure is annotated and displayed with information, assisting medical personnel in making auxiliary judgments about the target structure. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the module connection of the present invention;

[0038] Figure 2 This is a schematic diagram of module connections according to another embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the anatomical structure recognition module of the present invention;

[0040] Figure 4 This is a schematic diagram illustrating the training steps of the YOLO model of the present invention;

[0041] Figure 5 This is a schematic diagram illustrating the training steps of the ResNet model of the present invention;

[0042] Figure 6 This is a schematic diagram illustrating the training steps of the Transformer model of the present invention;

[0043] Figure 7 This is a diagram illustrating the training data processing steps of the present invention;

[0044] Figure 8 This is a schematic diagram illustrating the optimized labeling method of the present invention;

[0045] Figure 9 This is a schematic diagram of the data augmentation process of the present invention. Detailed Implementation

[0046] To more clearly illustrate the present invention, the invention will be further described below with reference to the accompanying drawings.

[0047] In the following description, specific examples are given to provide a more in-depth understanding of the invention. It is obvious that the described embodiments are merely some, not all, of the embodiments of the invention. It should be understood that the specific embodiments described are for illustrative purposes only and are not intended to limit the scope of the invention.

[0048] It should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the said feature, integral, step, operation, element, or component, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, or combinations thereof.

[0049] To address the technical problem that current clinical anatomical procedures heavily rely on the experience and intuition of medical staff, increasing surgical risks and difficulties, and lacking a suitable solution to assist medical staff in making judgments and reducing surgical risks and difficulties, this application provides an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms. Please refer to the appendix. Figure 1 To be continued Figure 9 The system includes a data acquisition and preprocessing module, an anatomical structure recognition module, and a real-time reasoning and feedback module;

[0050] The anatomical structure recognition module is used to perform multi-model processing on the original image to generate a structural model, and to mark the target structure on the structural model; the data acquisition and preprocessing module is used to preprocess the data at each input stage, and to acquire the original data and the user's real-time data during the operation; the real-time reasoning and feedback module is used to connect the anatomical structure recognition module and the data acquisition and preprocessing module to realize the real-time feedback of the target structure information to the user.

[0051] By combining the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time reasoning and feedback module, the target structure is labeled and displayed in an informational way, which assists medical staff in judging the target structure and greatly reduces the reliance on the experience of medical staff.

[0052] To better understand this solution, the following section elaborates on specific implementation details regarding preliminary data collection and processing, model construction and optimization, and the collection and feedback of user data. In this application, "user" is understood to refer to the surgeon or medical personnel assisting in the surgery.

[0053] Regarding the initial data acquisition and processing, in the system of this application, the original images are medical imaging data, such as screenshots from various medical imaging data at a certain point in time, including CT and MRI scans, intraoperative real-time videos, and ultrasound. This data covers the head and neck anatomy of different patients, including not only normal anatomical structures but also variations or lesions that may be encountered during surgery. Due to the difficulty in acquiring the original images, sample size limitations, and anatomical differences among patients during surgery, primarily stemming from variations in machine models, firmware versions, and viewing angles during image acquisition, the original images can exhibit considerable complexity. Therefore, special preprocessing of the original images is performed. The preprocessing steps include:

[0054] Image standardization: Obtain the original image and normalize it to obtain a first-level image; normalize original images from different sources to ensure the consistency of input data;

[0055] Image augmentation: Data augmentation is performed on primary images to obtain secondary images; data augmentation techniques are used to process images to increase the diversity and robustness of the resulting training data, especially in the case of low-quality or missing data.

[0056] Image annotation: Labeling secondary images and ensuring spatial consistency yields training data; by having medical staff annotate secondary images, combined with automated tools, the accuracy of target structure labels is ensured, while the refinement and spatial consistency of labels are optimized, resulting in training data with strong representativeness and accuracy.

[0057] It is evident that data augmentation and labeling are crucial steps in ensuring the robustness and high accuracy of models in complex surgical scenarios. Data augmentation and labeling can improve the diversity and quality of training data, thereby enhancing the model's generalization ability and ensuring that the model can cope with various scenarios and uncertainties.

[0058] It is worth noting that data augmentation can significantly enhance the original image, making it compatible with a wider range of sources and quantities of original data, thus enhancing the reliability of the subsequently produced models. Therefore, in this embodiment, the specific image enhancement methods include:

[0059] Geometric transformation: Perform geometric transformations on the primary image, such as rotation, translation, scaling, and mirror flipping, to simulate different shooting angles and viewpoint changes, so that the subsequently generated structural model can adapt to different surgical viewpoints and postures;

[0060] Color and brightness adjustment: Taking into account the changes in lighting during surgery, methods such as brightness, contrast, and saturation adjustment are used to simulate image changes under different lighting conditions.

[0061] Noise addition: Simulate noise in the original image (such as Gaussian noise, salt and pepper noise, etc.) to enhance the robustness of the model, enabling it to accurately identify targets in unclear or low-quality images during surgery.

[0062] Local deformation: In order to simulate the deformation and local deformation of tissues during the operation, the image can be elastically deformed or affine transformed to improve the adaptability of the subsequent structural model to the target structure after morphological changes.

[0063] It is evident that data augmentation can significantly improve the diversity of original images, adapt to different surgical scenarios, and effectively optimize surgical scenarios. Specifically, regarding the diversity of data sources, by generating original images under different geometric states, data augmentation can expand the dataset and reduce the risk of overfitting. In terms of adapting to different surgical scenarios, conditions such as lighting, viewing angle, and image noise may change during surgery, and data augmentation can simulate these changes, improving the model's adaptability. More importantly, in terms of optimizing training effects, the augmented dataset enables the model to better learn the characteristics of target structures under different conditions, improving the ability to recognize smaller target structures and targets with blurred boundaries.

[0064] Regarding labeling, the target structures in the original images are highly specialized, and the accuracy of labeling is crucial for model training. Labeling helps ensure that each target structure is correctly and accurately labeled in the data. For example, medical personnel accurately label key anatomical structures of the head and neck (such as blood vessels, nerves, muscles, and bones) based on secondary images, ensuring that the bounding boxes and category labels of each structure are correct. A better approach is to perform multi-level labeling. Specifically, considering the details and layers of certain structures (e.g., nerve branches), multi-level labels can be used to label target structures, enabling subsequent models to identify different parts of the target structure (e.g., "superior branch of the facial nerve," "inferior branch of the facial nerve," etc.).

[0065] The specific optimization methods for labeling are as follows:

[0066] Label consistency check: Perform consistency checks on the annotations in the secondary images to ensure that the labels and bounding boxes of each target structure are accurate; optionally, reduce label bias through cross-validation and feedback from medical personnel.

[0067] Label expansion: Generate new labels for the same target structure (for example, target regions detected by the YOLO model can be further refined with labels), supplementing the deficiencies in the labels, especially in the annotation of subdivided anatomical structures (such as blood vessel branches, nerve branches, etc.).

[0068] Multi-label learning: In special cases, a target region may belong to multiple categories. For example, some target structures, due to their specific location, can be classified as either "glossopharyngeal nerve" or part of "facial nerve." Multi-label learning can optimize the model's performance in such multi-classification tasks.

[0069] Label calibration in relation to 3D space: By combining 3D annotation tools (such as 3D reconstruction technology), the spatial consistency of label annotation is optimized to ensure that the annotation of each target structure is accurate not only in the two-dimensional plane, but also in the three-dimensional space.

[0070] To achieve more representative and accurate training data, the following are also included:

[0071] a. Data segmentation: The dataset formed by the secondary images after data augmentation is divided into training set, validation set and test set to ensure the diversity and representativeness of each dataset and prevent the model from overfitting.

[0072] b. Batch Training and Validation: Perform batch training using the data-augmented training set, while simultaneously evaluating the model's performance on new data using the validation set. Adjust the data augmentation strategy based on the validation results to further improve the model's accuracy.

[0073] c. Label optimization and loss function: During training, in combination with label annotation and data augmentation, an appropriate loss function (such as Focal Loss) is used to improve the classification accuracy of difficult-to-identify samples (such as small target structures and blurred boundaries of target locations).

[0074] As a summary, and as is readily apparent, the data acquisition and preprocessing module of this application can effectively process the original images to generate highly representative and accurate training data as output. The diverse source data can simulate the varied conditions in various surgeries, resulting in a more adaptable and robust structural model. Furthermore, the module utilizes labeling and optimization methods to ensure accurate labeling of each anatomical structure, particularly enhancing the classification accuracy of subdivided anatomical structures. Data augmentation techniques further enhance the ability to identify smaller blood vessels and nerves. Label calibration based on associated 3D spatial construction improves navigation accuracy during subsequent surgeries, ultimately leading to more comprehensive and accurate data input into the structural model.

[0075] In the model construction and optimization section, this system will further process the training data obtained from the aforementioned processing using models, namely YOLO (You Only Look Once), ResNet (Residual Neural Network), and Transformer. First, the generation of the structural model includes the following steps:

[0076] Training data is input into the YOLO model for modeling processing to generate a structural model. This structural model serves as a crucial foundation for assisting surgery. The YOLO model first accepts training data from the data acquisition and preprocessing module, constructs a structural model, and the resulting structural model can intuitively and quickly represent the positional relationships of various anatomical structures within the head. These anatomical structures include key structures such as blood vessels, nerves, and bones. The YOLO model version is either YOLOv8 or YOLOv11, with YOLOv11 being preferred. Compared to YOLOv8, YOLOv11 employs a more efficient backbone network and a more refined decoding method, making it suitable for complex surgical scenarios and offering more comprehensive algorithmic accuracy.

[0077] Specifically, the YOLO model training process is as follows:

[0078] A1. Data preprocessing: Adjusting the training data to a fixed size to achieve standardized input;

[0079] A2. Object Detection: The YOLO model extracts image features through a convolutional neural network (CNN) to generate bounding boxes of surgical targets and the probabilities of their corresponding categories;

[0080] A3. Optimize training: Use the cross-entropy loss function and the IoU (Intersection over Union) loss function to optimize the model and ensure that the convolutional network can produce localization and classification results during detection;

[0081] A4. Data Output: The output includes the location information of each target structure and the corresponding classification probability.

[0082] As can be seen, the YOLO model extracts image features from input medical images through convolutional neural networks, thereby effectively detecting target structures. Its efficient feature extraction capability ensures real-time response during surgery, quickly providing the location and classification probability of important anatomical structures.

[0083] It's worth noting that the YOLO model doesn't just process training data. During surgery, the data acquisition and preprocessing module continuously transmits captured images or video streams to the YOLO model. The YOLO model then further detects target structures in each frame of the image and displays relevant classification information for each target structure. Medical staff can intuitively and quickly obtain relevant feedback from the structural model and classification information to assist in the surgery. The feedback can be provided through a surgical navigation subsystem, which displays information to medical staff via projectors, monitors, etc., allowing them to intuitively understand the spatial relationships between anatomical structures and target structures. Furthermore, during surgery, medical staff can make adjustments based on the output of the YOLO model. The system can also update anatomical structures and classification results in real time according to environmental changes, ensuring efficient and accurate completion of the surgery.

[0084] As can be seen, the aforementioned processing of training data and real-time acquired data using the YOLO model enables image analysis within milliseconds, ensuring the efficiency and safety of the surgery. It also performs well in small anatomical structures and overlapping target structures, making it more suitable for complex head and neck surgeries. On the other hand, the YOLO model is relatively lightweight, making it suitable for deployment on embedded hardware and applicable to the real-time computing needs of the surgical field.

[0085] In a further proposed solution, after obtaining the structural model, the following steps are also included:

[0086] The ResNet model receives the output data from the YOLO model, classifies and refines the surgical target locations to output classification information. After the YOLO model detects structural targets, the ResNet model is used to further classify and refine these target models. The residual connections of the ResNet model can solve the gradient vanishing problem in deep networks, improving the training efficiency and performance of structural models. The underlying principle is that the core of the ResNet model lies in its "residual block" design. This model structure can effectively solve the problem of information attenuation in deep networks through "skip connections," enabling the model network to maintain stable training performance while increasing its depth.

[0087] As can be seen, after the YOLO model is mainly responsible for detecting and locating anatomical and target structures, the ResNet model, based on this, utilizes its depth characteristics for image classification, refining the specific types of anatomical structures, such as distinguishing subtle differences like "common carotid artery" and "internal jugular vein." Through its deep network layers, the ResNet model can extract high-order features from complex images, providing more accurate recognition and classification when dealing with complex and intertwined anatomical regions of the head and neck. The residual connection design of the ResNet module effectively solves the gradient vanishing problem that may occur in deep neural networks, ensuring that the structural model can be trained with deeper network layers, thereby improving recognition accuracy. The ResNet model can improve the classification accuracy of the YOLO model output, especially when dealing with small, highly similar anatomical structures, providing more accurate classification.

[0088] In the specific solution, the ResNet model training process is as follows:

[0089] B0. Input and Preprocessing: After the YOLO model detects the location information of the target structure, it generates bounding boxes for each target structure. This location information is then passed as input to the ResNet model. In a preferred embodiment, before being passed to the ResNet model, the target region image is resized and standardized to meet the input requirements of the ResNet model. Standardization (such as subtracting the mean and dividing the standard deviation) ensures the consistency of the input data.

[0090] B1. Feature Extraction and Classification: Input the location information of at least one target structure, perform multi-layer convolution and pooling operations on the ResNet model to extract higher-level image features, and use fully connected layers to further classify anatomical structures so that each surgical target is assigned a probability value, thereby indicating what type of nerve, blood vessel or other unit the target structure is.

[0091] B2. Cross-entropy loss function: The cross-entropy loss function is used to optimize the ResNet model, ensuring that the ReNest model can make accurate classification predictions for each target structure.

[0092] B3. Training Optimization: The Adam optimizer is used for training, and the weight parameters in the model are adjusted to minimize the loss function;

[0093] B4. Data Output: The ResNet model outputs the category and confidence score of each target structure as classification information.

[0094] It is evident that the RSSNet and YOLO models work together to further enhance the system's analytical capabilities, enabling more detailed classification of surgical target structures. For instance, after the YOLO model identifies the facial nerve, the RSSNet model further distinguishes different branches of the facial nerve. Because the ResNet model can extract more detailed features from the data through its deep network structure, it is adapted to the identification tasks of more complex anatomical regions (such as vascular intersections and nerve entanglements). Furthermore, since many anatomical structures in head and neck surgery are small and have complex spatial relationships, the ResNet model, through residual connections, solves the gradient vanishing problem in deep neural network training, enhancing the identification ability of smaller target structures.

[0095] In conjunction with the surgical navigation subsystem, the YOLO and ResNet models work together during surgery. The YOLO model is responsible for quickly detecting targets and performing initial classification, while the ResNet model further refines the classification of anatomical structures based on the target regions output by YOLO, generating detailed surgical navigation information. Since the ResNet model outputs the category and confidence level of each target structure, it provides real-time navigation and feedback to the system, helping medical staff accurately locate target structures and avoid damage or misjudgment. Finally, a visual interface displays the classification results and location information of each anatomical structure and target structure during surgery, providing real-time alerts to medical staff on the risk level of the target area, ensuring efficient and safe surgical procedures.

[0096] A further step in the approach, after processing the ResNet model, also includes:

[0097] Positional and classification information are input into the Transformer model for processing. The Transformer model combines 3D image data to generate spatial relationship models. The Transformer model is mainly used to process the spatial relationships between anatomical structures and target structures. The Transformer model receives the outputs of the YOLO and ResNet models, and combines positional signals and classification information with 3D image data to model the spatial relationships between anatomical structures. It also focuses more on modeling the spatial relationships between multiple target structures within the visible bounding box. It is easy to understand that through the integration and optimization of global information, the Transformer model can provide more accurate spatial reasoning and interaction information, helping medical staff to better understand the relative positions of anatomical structures during surgery. Through its self-attention mechanism, the Transformer model helps to model the spatial relationships between target structures, especially in complex 3D image data, where it can accurately understand the relative position, direction, and distance of anatomical structures.

[0098] Specifically, the Transformer model training process is as follows:

[0099] C0. Input and Preprocessing: The Transformer model receives positional and classification information from the outputs of the YOLO and ResNet modules. In addition, 3D image data (such as CT scans and MRI images) is also an important input to the Transformer model. In a better approach, 3D image data is first sliced ​​and standardized to ensure the consistency of size among the data segments when input into the Transformer model. Furthermore, to enhance the model's understanding of complex spatial relationships, data augmentation (e.g., rotation, translation, flipping, etc.) can be applied to improve the data's suitability.

[0100] C1. Encoder layer processing: The Transformer model uses an encoder to extract spatial features from the input image formed by the YOLO model, ResNet model, and 3D image data; the encoder models spatial characteristics through a multi-layer self-attention mechanism to generate a spatial representation of each target structure for data output;

[0101] C2. Decoder layer processing: Based on the encoder's output, the decoder further predicts the spatial relationships and relative positions between various target structures; for example, the Transformer model can infer, record, and display the relative distance between the "facial nerve" and the "glossopharyngeal nerve" in the head and neck region.

[0102] C3. Loss Function Optimization: A pre-defined loss function is used, combined with spatial relationships and the category of the target structure, to improve the accuracy of the model; loss functions such as mean squared error loss and cross-entropy loss can effectively optimize the Transformer model;

[0103] C4. Data Output: The output of the Transformer model includes the spatial coordinates of each anatomical target and their relative positional relationships; it provides real-time visual interactive information about anatomical structures, such as displaying the risk level between the "common carotid artery" and the "glossopharyngeal nerve," which can assist medical staff in making judgments.

[0104] It is evident that the Transformer model can effectively model target structural relationships and provide global spatial awareness within the overall system design. In terms of target structural relationship modeling, the Transformer model can process the positional information output by YOLO, training, and ResNet models, and model the spatial relationships between various anatomical structures. For example, the Transformer model can analyze the distance relationship between the common carotid artery and the trachea, assisting medical personnel in predicting the risks of these target structures during surgery. Regarding global spatial awareness, Transformer modeling can not only understand the position of individual anatomical structures and target structures, but also analyze the spatial layout of the entire head and neck region, including the overlap and proximity relationships between anatomical structures and target structures, which is crucial for complex surgeries. Furthermore, the Transformer effectively processes 3D image data, generating more accurate spatial positioning information, which is essential for complex head and neck surgeries.

[0105] In conjunction with the surgical navigation subsystem, during surgery, the Transformer model receives real-time image data and, combined with the YOLO model, ResNet model, and 3D image data, analyzes the spatial relationships between anatomical structures in real time and updates the data in the surgical navigation subsystem. a. Navigation Feedback: Based on Transformer's spatial relationship modeling, the system can generate high-precision surgical navigation information, providing real-time feedback to surgical staff on the spatial distribution of head and neck anatomical structures. Staff can observe the relative positions of anatomical structures through a visual interface, enabling more accurate surgical decisions.

[0106] As an improvement of the present invention, it also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time inference and feedback module as sub-modules, and also includes an auxiliary sub-module, which is connected to the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time inference and feedback module. All modules, through efficient integration and collaboration, constitute a complete AI-assisted head and neck surgery robot system. The system integration module ensures smooth collaboration among the various subsystems and provides intelligent assistance functions through the auxiliary sub-modules, making the system operation more efficient, accurate, and sensitive.

[0107] For example, the auxiliary submodule includes a surgical navigation subsystem: this subsystem can comprehensively process the processing flow of the anatomical structure recognition module and the real-time acquired signal stream to generate accurate surgical navigation information for medical staff to use during the operation. The implemented visual interface facilitates the intuitive acquisition of information by medical staff. The specific details of the process have been mentioned in the aforementioned embodiments, so they will not be repeated here.

[0108] The auxiliary submodule also includes a surgical control feedback unit: this feedback unit adjusts the surgical robot's operating precision based on the operational feedback from medical staff to ensure the efficiency and accuracy of the surgical process. At the same time, the system can adjust the surgical path based on real-time images to optimize the surgical process.

[0109] The auxiliary submodule also includes a postoperative recovery tracking unit: after the surgery is completed, the postoperative recovery tracking unit continues to track the patient's recovery process, evaluates the surgical effect and provides recovery suggestions by comparing subsequent imaging data with intraoperative data.

[0110] This system achieves real-time identification of anatomical structures, spatial relationship modeling, and precise navigation during head and neck surgeries through the collaborative work of deep learning models and various AI algorithms. The system architecture includes multiple modules such as data acquisition and preprocessing, anatomical structure identification, data augmentation and label optimization, real-time inference and feedback, and system integration. These modules work together to ensure accurate identification and navigation of anatomical structures during surgery and can be dynamically adjusted according to the actual situation, minimizing surgical risks, reducing trauma, and improving the quality of postoperative recovery for patients.

[0111] The advantages of this invention are:

[0112] This system achieves real-time identification of anatomical structures, spatial relationship modeling, and precise navigation during head and neck surgery through the collaborative work of multiple deep learning models and various algorithms. The cooperation between the various modules of the system ensures accurate identification and navigation of anatomical structures during surgery and can be dynamically adjusted according to the actual situation to minimize surgical risks, reduce trauma, and improve the quality of postoperative recovery for patients.

[0113] The above-disclosed embodiments are merely a few specific examples of the present invention, but the present invention is not limited thereto. Any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. An AI-assisted head and neck surgery robot system based on multiple deep learning algorithms, characterized in that, The system includes: The anatomical structure recognition module is used to perform multi-model processing on the original image to generate a structural model, and to annotate the target structure on the structural model. It also includes a data acquisition and preprocessing module, which is used to preprocess the data at each input stage and to acquire the raw data and the user's real-time data during the surgery. The real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to enable the real-time feedback of the target structure information to the user. The generation of the structural model includes the following steps: The training data is input into the YOLO model for modeling processing to generate a structural model; After obtaining the structural model, the following steps are also included: The ResNet model is used to receive the output data of the YOLO model, and the surgical target location is classified and refined to output classification information; It also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module, and the real-time reasoning and feedback module as sub-modules, and also includes an auxiliary sub-module, which is associated with the anatomical structure recognition module and the real-time reasoning and feedback module. The auxiliary submodule includes the surgical navigation subsystem, which is used to comprehensively process the processing flow of the anatomical structure recognition module and the real-time acquired signal stream; The ResNet model training process is as follows: B1. Feature Extraction and Classification: Input at least one location information, perform multi-layer convolution and pooling operations on the ResNet model to extract higher-level image features, and use fully connected layers to further classify anatomical structures so that each surgical target is assigned a probability value. B2. Cross-entropy loss function: Optimize the ResNet model using the cross-entropy loss function; B3. Training Optimization: The Adam optimizer is used for training, and the weight parameters in the model are adjusted to minimize the loss function; B4. Data Output: The ResNet model outputs the category and confidence score of each target structure as classification information. After completing the ResNet model processing, the following is also included: Location and classification information are input into the Transformer model for processing. The Transformer model combines 3D image data to generate spatial relationship modeling.

2. The AI-assisted head and neck surgery robot system based on multiple deep learning algorithms according to claim 1, characterized in that, Before the original image is input into the model for processing, it needs to be preprocessed. The preprocessing steps include: Image standardization: Obtain the original image and perform normalization processing on the original image to obtain a first-level image; Image enhancement: Performing data enhancement on a primary image to obtain a secondary image; Image annotation: Labeling the secondary images and ensuring spatial consistency will yield training data.

3. The AI-assisted head and neck surgery robot system based on multiple deep learning algorithms according to claim 2, characterized in that, Data augmentation methods include at least one of geometric transformation, color and brightness adjustment, noise addition, and local deformation.

4. The AI-assisted head and neck surgery robot system based on multiple deep learning algorithms according to claim 1, characterized in that, The YOLO model training process is as follows: A1. Data preprocessing: Adjusting the training data to a fixed size to achieve standardized input; A2. Object Detection: The YOLO model extracts image features through a convolutional neural network to generate bounding boxes of surgical targets and their corresponding class probabilities; A3. Optimize training: Use cross-entropy loss function and IoU loss function to optimize the model and ensure that the convolutional network can produce localization and classification results during detection; A4. Data Output: The output includes the location information of each target structure and the corresponding classification probability.

5. The AI-assisted head and neck surgery robot system based on multiple deep learning algorithms according to claim 1, characterized in that, The training process for the Transformer model is as follows: C1. Encoder layer processing: The Transformer model uses an encoder to extract spatial features of the input image formed by the YOLO model, ResNet model, and 3D image data; The encoder models spatial characteristics through a multi-layer self-attention mechanism, generating spatial representation data for each target structure for output; C2. Decoder layer processing: Based on the encoder output, the decoder further predicts the spatial relationships and relative positions between the various target structures; C3. Loss Function Optimization: A preset loss function is used, combined with spatial relationships and the category of the target structure, to improve the accuracy of the model; C4. Data Output: The output of the Transformer model includes the spatial coordinates of each anatomical target and their relative positional relationships.

Images