AI-assisted head and neck surgery robot system based on multiple deep learning algorithms

Abstract

The invention discloses an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms, which is applied to head and neck surgery and comprises a data acquisition and preprocessing module, an anatomical structure recognition module and a real-time reasoning and feedback module, wherein the anatomical structure identification module is used for performing multi-model processing on an original image to generate a structure model, and marking a target structure on the structure model; the data acquisition and preprocessing module is used for preprocessing data of each input stage and acquiring original data and real-time data of a user in an operation process; the real-time reasoning and feedback module is used for associating the anatomical structure recognition module with the data acquisition and preprocessing module so as to feed back information of a target structure to a user in real time; through mutual cooperation of the modules, informatization labeling and showing of the target structure are achieved, and efficient information feedback is generated in the operation process so as to assist medical staff in judging the target structure.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms. Background Art

[0002] Head and neck surgery is a type of surgery that specifically treats head and neck diseases (including tumors, trauma, congenital malformations, infections, etc.). The anatomical areas involved include complex organs and tissues such as the head, neck, face, oral cavity, pharynx, larynx, trachea, and esophagus. Compared with other types of surgery, the head and neck region has many blood vessels and nerves, such as large blood vessels (common carotid artery, internal jugular vein), important nerves (facial nerve, glossopharyngeal nerve, vagus nerve), etc., and has the characteristics of complex blood vessels, nerve courses and adjacent relationships. If blood vessels or nerves are injured during surgery, it will cause adverse complications such as increased bleeding and paralysis of the relevant nerve innervation area, increase the risk of postoperative infection in patients, delay recovery time, and greatly affect the quality of life of patients after surgery. Therefore, head and neck surgery must be highly precise to minimize damage to these structures and avoid interference with the patient's basic physiological functions. The particularity of head and neck surgery compared to surgery in other parts of the body lies in its profound impact on patients' daily life and social interaction, especially in basic functions such as language, swallowing, and breathing. Damage to important anatomical structures during surgery can affect patients' speech and respiratory functions, and thus affect their social and psychological state.

[0003] At present, medical staff need to rely on their own senses and experience to identify anatomical structures, which increases the risk and difficulty of surgery. At the same time, head and neck surgery has the characteristics of limited vision and operating range. Medical staff need to operate in a limited space and cannot fully and clearly identify anatomical structures, which further increases the risk of surgery. Therefore, a more reasonable solution is urgently needed to accurately mark anatomical structures and provide accurate and efficient real-time feedback, so as to assist medical staff in making appropriate judgments in head and neck surgery. Summary of the invention

[0004] The present invention provides a solution to the technical problem that there is no suitable solution to identify and mark anatomical structures in existing head and neck surgeries to assist medical staff in making reasonable judgments; on the other hand, the existing technology cannot provide accurate feedback to medical staff in real time during the operation to facilitate medical staff to make timely judgments.

[0005] To achieve the above objectives, the present invention provides an AI-assisted head and neck surgery robot system based on a multi-depth learning algorithm, the system comprising: an anatomical structure recognition module, used for performing multi-model processing on the original image to generate a structure model, and marking the target structure on the structure model; It also includes a data acquisition and preprocessing module, which is used to preprocess the data at each input stage and to collect the original data and the real-time data of the user during the operation; The real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to realize real-time feedback of the target structure information to the user.

[0006] Among them, before the original image is input into the model for processing, it needs to be preprocessed. The preprocessing steps include: Image normalization: obtain the original image and perform normalization on the original image to obtain a primary image; Image enhancement: Perform data enhancement on the primary image to obtain the secondary image; Image annotation: Label the secondary images and ensure spatial consistency to obtain training data.

[0007] Among them, the data enhancement method includes at least one of geometric transformation, color and brightness adjustment, noise addition and local deformation.

[0008] The generation of the structural model includes the following steps: The training data is input into the YOLO model for modeling processing to generate a structural model.

[0009] Among them, the YOLO model training process is: A1. Data preprocessing: adjust the training data to a fixed size to achieve standardized input; A2. Target detection: The YOLO model extracts image features through a convolutional neural network to generate the bounding box of the surgical target and the probability of its corresponding category; A3. Optimize training: Use the cross entropy loss function and IoU loss function to optimize the model to ensure that the convolutional network can produce positioning and classification results during detection; A4. Data output: The output includes the location information of each target structure and the corresponding classification probability.

[0010] After the structural model is obtained, the following steps are also included: The ResNet model is used to receive the output data of the YOLO model, classify and refine the surgical target location to output classification information.

[0011] Among them, the ResNet model training process is: B1. Feature extraction and classification: Input at least one location information, perform multi-layer convolution and pooling layer operations on the ResNet model to extract higher-level image features, and use a fully connected layer to further classify the anatomical structure so that each surgical target will be assigned a probability value; B2. Cross entropy loss function: Use the cross entropy loss function to optimize the ResNet model; B3. Training optimization: Use the Adam optimizer for training and adjust the weight parameters in the model to minimize the loss function; B4. Data output: The ResNet model outputs the category of each target structure and the confidence of the category as classification information.

[0012] After completing the ResNet model processing, it also includes: The location information and classification information are input into the Transformer model for processing, and the Transformer model is combined with 3D image data to generate spatial relationship modeling.

[0013] Among them, the Transformer model training process is: C1. Encoder layer processing: The Transformer model uses the encoder to extract the spatial features of the input image formed by the YOLO model, ResNet model, and 3D impact data; the encoder models the spatial characteristics through a multi-layer self-attention mechanism and generates spatial representation data for each target structure for output; C2, decoder layer processing: The decoder further predicts the spatial relationship and relative position between each target structure based on the output of the encoder; C3, loss function optimization: using the preset loss function, combining the spatial relationship and the category of the target structure to improve the accuracy of the model; C4. Data output: The output of the Transformer model includes the spatial coordinates of each anatomical target and their relative positional relationships.

[0014] Among them, it also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module as sub-modules, and also includes an auxiliary sub-module, which is associated with the anatomical structure recognition module and the real-time reasoning and feedback module.

[0015] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention provides an AI-assisted head and neck surgical robot system based on a multi-deep learning algorithm, which is applied to head and neck surgery. The system includes a data acquisition and preprocessing module, an anatomical structure recognition module and a real-time reasoning and feedback module; wherein the anatomical structure recognition module is used to perform multi-model processing on the original image to generate a structural model, and mark the target structure on the structural model; the data acquisition and preprocessing module is used to preprocess the data at each input stage, and to collect the original data and the real-time data of the user during the operation; the real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to realize real-time feedback of the target structure information to the user; the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module cooperate with each other to realize the information-based labeling and display of the target structure, so as to assist medical staff in making auxiliary judgments on the target structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of module connection of the present invention; Figure 2 A schematic diagram of module connection according to another embodiment of the present invention; Figure 3 is a schematic diagram of an anatomical structure recognition module of the present invention; Figure 4 Schematic diagram of the training steps of the YOLO model of the present invention; Figure 5 Schematic diagram of the training steps of the ResNet model of the present invention; Figure 6 Schematic diagram of the training steps of the Transformer model of the present invention; Figure 7 A diagram showing the steps of processing training data of the present invention; Figure 8 This is a schematic diagram of the optimization means for labeling of the present invention; Fig. 9 Schematic diagram of data enhancement processing of the present invention. DETAILED DESCRIPTION

[0018] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with the accompanying drawings.

[0019] In the following description, the example details are given to provide a deeper understanding of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. It should be understood that the specific embodiments are only used to explain the present invention and are not used to limit the present invention.

[0020] It should be understood that when the terms "include" and / or "comprises" are used in this specification, they indicate the existence of the stated features, integers, steps, operations, elements or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components or combinations thereof.

[0021] In view of the technical problem that the existing clinical medical anatomy is extremely dependent on the experience and feeling of medical staff, which increases the risk and difficulty of surgery, and there is no suitable solution to assist medical staff in making judgments to reduce the risk and difficulty of surgery, this application provides an AI-assisted head and neck surgery robot system based on multiple deep learning algorithms. Please refer to the attached Figure 1 To Attachment Fig. 9 ,The system includes a data acquisition and preprocessing module, an anatomical structure recognition module, and a real-time reasoning and feedback module; Among them, the anatomical structure recognition module is used to perform multi-model processing on the original image to generate a structural model, and mark the target structure on the structural model; the data acquisition and preprocessing module is used to preprocess the data at each input stage, and to collect the original data and the real-time data of the user during the operation; the real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to realize real-time feedback of the target structure information to the user; Through the cooperation of the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module, the target structure can be annotated and displayed in an informationized manner, assisting medical staff in making auxiliary judgments on the target structure, greatly reducing the strictness of the medical staff's experience.

[0022] In order to better understand this solution, the following is combined with specific implementation details to further elaborate on the early data collection and processing, model construction and optimization, and implementation collection and feedback of user data; the user in this application is understood to be the surgeon or medical staff assisting in the operation.

[0023] In terms of early data collection and processing, in the system of the present application, the original image is the medical image data, such as screenshots of a certain time point from CT, MRI scan images, real-time intraoperative video, ultrasound and other medical image data. The above data covers the anatomical structure of the head and neck of different patients. The data content includes not only normal anatomical structures, but also variations or lesions that may be encountered during surgery. Due to the difficulty in obtaining the original image, the sample size limitation and the anatomical differences between different patients during the operation, mainly due to the differences in different models, firmware versions, different use angles, etc. when the machine collects the original image, the original image will appear more complicated. Therefore, the original image is preprocessed in particular, and the preprocessing steps include: Image standardization: obtain the original image and normalize it to obtain a first-level image; normalize the original images from different sources to ensure the consistency of the input data; Image augmentation: Perform data augmentation on primary images to obtain secondary images; use data augmentation techniques to process images to increase the diversity and robustness of the resulting training data, especially in the case of low-quality or missing data; Image annotation: Label the secondary images and ensure spatial consistency to obtain training data. Medical staff annotate the secondary images and use automated tools to assist in ensuring the accuracy of the labels of the target structures. At the same time, the refinement and spatial consistency of the labels are optimized, so that the training data is highly representative and accurate. It can be seen that data augmentation and label annotation are important steps to ensure that the model is robust and accurate in complex surgical scenarios. Data augmentation and label annotation can improve the diversity and quality of training data, thereby enhancing the generalization ability of the model and ensuring that the model can cope with various scenarios and uncertainties.

[0024] It is worth noting that enhancing the original image through data enhancement can significantly be compatible with more sources and quantities of original data, making the subsequently produced model more referenceable; therefore, in this embodiment, the specific means of image enhancement include: Geometric transformation: Perform geometric transformations on the primary image, such as rotation, translation, scaling, and mirror flipping, to simulate different shooting angles and perspective changes, so that the subsequently generated structural model can adapt to different surgical perspectives and postures; Color and brightness adjustment: Taking into account the changes in lighting during the operation, use brightness, contrast, saturation adjustment and other methods to simulate image changes under different lighting conditions.

[0025] Noise addition: Simulate the noise in the original image (such as Gaussian noise, salt and pepper noise, etc.) to enhance the robustness of the model so that it can still accurately identify targets in unclear or low-quality images during surgery.

[0026] Local deformation: In order to simulate the deformation during surgery and the local deformation of tissues, the image can be elastically deformed or affine transformed to improve the adaptability of the subsequent structural model to the target structure after the morphological change.

[0027] It can be seen that data enhancement can significantly improve the diversity of original images, adapt to different surgical scenarios and effectively optimize surgical scenarios; specifically, in terms of the diversity of data sources, by generating original images in different geometric states, data enhancement can expand the data set and reduce the risk of overfitting; in terms of adapting to different surgical scenarios, during surgery, conditions such as lighting, viewing angle, and image noise may change, and data enhancement can simulate these changes to improve the adaptability of the model; with emphasis on optimizing training effects, the enhanced data set enables the model to better learn the characteristics of target structures in different situations, and improve the recognition ability of smaller target structures and targets with blurred boundaries; Regarding label annotation processing, the target structure in the original image is highly professional, and the accuracy of label annotation is crucial to the training of the model; label annotation helps ensure that each target structure is correctly and accurately labeled in the data; for example, medical personnel accurately annotate the key anatomical structures of the head and neck (such as blood vessels, nerves, muscles, bones, etc.) based on the secondary images to ensure that the bounding box and category label of each structure are correct; in a better solution, the label annotation can be processed in a multi-level manner. Specifically, considering the details and levels of certain structures (for example, the branches of nerves), multi-level labels are used to annotate the target structure, so that the subsequent model can identify different parts of the target structure (such as "superior branch of facial nerve", "inferior branch of facial nerve", etc.); The specific optimization methods for label annotation are: Label consistency check: Perform consistency check on the annotations in the secondary images to ensure that the labels and bounding boxes of each target structure are accurate; optionally, reduce label bias through cross-validation and feedback from medical staff; Label expansion: Generate new labels for the same target structure (for example, the target areas detected by the YOLO model can further generate refined labels for these areas) to supplement the deficiencies in the labels, especially in the annotation of subdivided anatomical structures (such as vascular branches, nerve branches, etc.).

[0028] Multi-label learning: In special cases, a target region may belong to multiple categories. For example, some target structures can be classified as "glossopharyngeal nerve" or as part of "facial nerve" due to their location. Multi-label learning can optimize the performance of the model in such multiple classification tasks.

[0029] Associate three-dimensional space for label calibration: By combining 3D annotation tools (such as 3D reconstruction technology), the spatial consistency of label annotation is optimized to ensure that the annotation of each target structure is accurate not only in the two-dimensional plane but also in the three-dimensional space.

[0030] In order to achieve more representative and accurate training data, it also includes: a. Data segmentation: The data set formed by the secondary images after data augmentation processing is divided into training set, validation set and test set to ensure the diversity and representativeness of each data set and prevent overfitting of the model.

[0031] b. Batch training and validation: Use the data-enhanced training set for batch training, and use the validation set to evaluate the model's performance on new data. Adjust the data enhancement strategy based on the validation results to further improve the accuracy of the model.

[0032] c. Label optimization and loss function: During the training process, label annotation and data enhancement are combined, and appropriate loss functions (such as Focal Loss) are used to improve the classification accuracy of difficult-to-identify samples (such as small target structures and fuzzy boundaries of target locations).

[0033] As a partial summary, it is not difficult to know that the data acquisition and preprocessing module of the present application can effectively process the original image to generate training data with strong representativeness and accuracy as output; it is derived from a variety of original data as a data source, and can simulate the changing conditions in various types of operations, so that the subsequent structural model has strong adaptability and robustness; and it can ensure the accuracy of the label of each anatomical structure through label annotation and its optimization method, especially in the identification of subdivided anatomical structures, to enhance the classification accuracy of the model; and through data enhancement technology, it also further enhances the recognition ability of smaller blood vessels, nerves, etc.; associate three-dimensional space construction for label calibration, improve navigation accuracy in subsequent surgical processes, and ultimately achieve more complete and accurate data input into the structural model.

[0034] In the model construction and optimization part, this system will further process the training data obtained by the above processing through the model. The processed models are YOLO (You Only Look Once) model, ResNet (Residual Neural Network) model and Transformer model. First, the generation of the structural model includes the following steps: The training data is input into the YOLO model for modeling and processing to generate a structural model; by constructing the structural model as an important basis for auxiliary surgery, the YOLO model can first accept the training data input by the data acquisition and preprocessing module to construct the structural model. The obtained structural model can intuitively and quickly preliminarily represent the positional relationship of various anatomical structures in the head. The aforementioned anatomical structures are key structures such as blood vessels, nerves, and bones; the version of the YOLO model is YOLOv8 or YOLOv11, and YOLOv11 is preferably used. Compared with YOLOv8, YOLOv11 uses a more efficient backbone network (Backbone) and a more sophisticated decoding method (Decoding), which is suitable for complex surgical scenarios and has more complete algorithm accuracy.

[0035] Specifically, the YOLO model training process is: A1. Data preprocessing: adjust the training data to a fixed size to achieve standardized input; A2. Target detection: The YOLO model extracts image features through a convolutional neural network (CNN) to generate the bounding box of the surgical target and the probability of its corresponding category; A3. Optimize training: Use the cross entropy loss function and IoU (Intersection over Union) loss function to optimize the model to ensure that the convolutional network can produce positioning and classification results during detection; A4. Data output: The output includes the location information of each target structure and the corresponding classification probability.

[0036] It can be seen that the YOLO model extracts image features from the input medical images through convolutional neural networks, and can then effectively detect the target structure. Its efficient feature extraction capability ensures real-time response during surgery and quickly provides the location and classification probability of important anatomical structures.

[0037] It is worth noting that the YOLO model does not just process the training data. During the operation, the data acquisition and preprocessing module will also transmit the images or video streams collected at all times to the YOLO model. The YOLO model continues to detect the target structure in each frame of the image and displays relevant classification information for the target structure. Medical staff can intuitively and quickly obtain relevant information feedback from the structural model and classification information to assist in the operation. The feedback can be in the form of a corresponding surgical navigation subsystem, which is displayed to medical staff through projection, display, etc., so that they can intuitively understand the spatial relationship between the anatomical structure and the target structure. During the operation, medical staff can make operational adjustments based on the output of the YOLO model, and the system can also update the anatomical structure and classification results in real time according to environmental changes to ensure efficient and accurate completion of the operation.

[0038] It can be seen that the training data and real-time collected data processed by the YOLO model as described above can perform image analysis within milliseconds, ensuring the efficiency and safety of the operation, and also performs better in small anatomical structures and overlapping target structures, which is more suitable for complex head and neck surgeries. On the other hand, the YOLO model is relatively lightweight, suitable for deployment on embedded hardware, and is suitable for real-time computing needs at the surgical site.

[0039] In a further solution, after obtaining the structural model, the following steps are also included: The ResNet model is used to receive the output data of the YOLO model, classify and refine the surgical target positions to output classification information; after the YOLO model detects the structural target, the ResNet model is used to further classify and refine these target models. The residual connection of the ResNet model can solve the gradient vanishing problem in the deep network and improve the training efficiency and performance of the structural model; the basic principle is that the core of the ResNet model lies in its "residual block" design. This model structure can effectively solve the problem of information attenuation in the deep network through "jump connection", so that the model network can maintain stable training performance while increasing the depth; It can be seen that after the YOLO model is mainly responsible for detecting and locating anatomical structures and target structures, the ResNet model uses its deep characteristics to classify images and refine the specific types of anatomical structures, such as distinguishing the subtle differences between "common carotid artery" and "internal jugular vein"; through the deep network layer, the ResNet model can extract high-order features from complex images, and can provide more accurate recognition and classification when dealing with complex and intertwined anatomical regions of the head and neck. The residual connection design of the ResNet module effectively solves the gradient vanishing problem that may occur in deep neural networks, ensuring that the structural model can train deeper network layers, thereby improving the accuracy of recognition; the ResNet model can improve the classification accuracy of the YOLO model output, especially when dealing with anatomical structures with smaller structures and high similarity, and can provide more accurate classification.

[0040] In the specific solution, the ResNet model training process is: B0. Input and preprocessing: After the YOLO model detects the location information of the target structure, it generates a bounding box for each target structure. This location information will be passed as input to the ResNet model; in the preferred solution, before passing it to the ResNet model, the target area image is resized and standardized to meet the input requirements of the ResNet model, and the consistency of the input data is ensured by standardization (such as mean subtraction and standard deviation division); B1. Feature extraction and classification: Input the location information of at least one target structure, perform multi-layer convolution and pooling operations on the ResNet model to extract higher-level image features, and use a fully connected layer to further classify the anatomical structure so that each surgical target will be assigned a probability value, indicating what type of nerve, blood vessel or other unit the target structure is.

[0041] B2. Cross-entropy loss function: Use the cross-entropy loss function to optimize the ResNet model to ensure that the ReNest model can make accurate classification predictions for each target structure.

[0042] B3. Training optimization: Use the Adam optimizer for training and adjust the weight parameters in the model to minimize the loss function; B4. Data output: The ResNet model outputs the category of each target structure and the confidence of the category as classification information.

[0043] It can be seen that the RssNet model and the YOLO model work together to further improve the system's analytical capabilities, so as to achieve further detailed classification of surgical target structures. For example, after the YOLO model identifies the facial nerve, the RssNet model further distinguishes the different branches of the facial nerve; because the ResNet model can extract more detailed features from the data through its deep network structure, it can adapt to the recognition tasks of more complex anatomical areas (such as vascular interlacing, nerve entanglement, etc.); and because many anatomical structures in head and neck surgery are small and have complex spatial relationships, the ResNet model uses residual connections to solve the gradient vanishing problem in deep neural network training and enhance the recognition ability of smaller target structures.

[0044] In terms of cooperating with the surgical navigation subsystem, during the operation, the YOLO model works together with the ResNet model. The YOLO model is responsible for quickly detecting the target and performing preliminary classification, while the ResNet model further classifies the anatomical structure more finely according to the target area output by YOLO, and generates detailed surgical navigation information. Since the ResNet model outputs the category and confidence of each target structure, the system can perform real-time navigation and feedback to help medical staff accurately locate the target structure to avoid damage and misjudgment. Finally, the classification results and positioning information of each anatomical structure and target structure during the operation are displayed through a visual interface, and medical staff are prompted in real time to the risk level of the target area to ensure efficient and safe surgical operations.

[0045] In a further solution, after completing the ResNet model processing, it also includes: The position information and classification information are input into the Transformer model for processing. The Transformer model combines 3D image data to generate spatial relationship modeling. The Transformer model is mainly used to process the spatial relationship between anatomical structures and target structures. The Transformer model receives the output of the YOLO model and the ResNet model, and combines the position signal and classification information with the 3D image data to model the spatial relationship between anatomical structures, and also focuses on modeling the spatial relationship between multiple target structures within the visible border. It is not difficult to understand that through the integration and optimization of global information, the Transformer model can provide more accurate spatial reasoning and interaction information, helping medical staff to better understand the relative position of anatomical structures during surgery. The Transformer model helps model the spatial relationship between target structures through its self-attention mechanism, especially in complex 3D image data, and can accurately understand the relative position, direction and distance of anatomical structures. Specifically, the Transformer model training process is: C0. Input and preprocessing: The Transformer model receives position information and classification information from the output of the YOLO module and the ResNet module. In addition, 3D imaging data (such as CT scans and MRI images) are also important inputs to the Transformer model. In a better solution, the 3D imaging data is first sliced ​​and standardized to ensure the size consistency of each data fragment when it is input to the Transformer model. In addition, in order to enhance the model's understanding of complex spatial relationships, the data can also be enhanced through data augmentation processing (for example, rotation, translation, flipping, etc.) to enhance the suitability of the data. C1. Encoder layer processing: The Transformer model uses the encoder to extract the spatial features of the input image formed by the YOLO model, ResNet model, and 3D image data; the encoder models the spatial characteristics through a multi-layer self-attention mechanism and generates a spatial representation of each target structure for data output; C2, decoder layer processing: Based on the output of the encoder, the decoder further predicts the spatial relationship and relative position between the target structures; for example, the Transformer model can infer the relative distance between the facial nerve and the glossopharyngeal nerve in the head and neck area and record and display it; C3, Loss function optimization: Use the preset loss function, combined with the spatial relationship and the category of the target structure to improve the accuracy of the model; loss functions such as mean square error loss and cross entropy loss can effectively optimize the Transformer model; C4. Data output: The output of the Transformer model includes the spatial coordinates of each anatomical target and the relative positional relationship between them; it provides real-time visual interactive information about the anatomical structure, such as displaying the risk level between the "common carotid artery" and the "glossopharyngeal nerve", which can assist medical staff in making judgments; It can be seen that the Transformer model can have the effects of target structure relationship modeling and global spatial perception in the overall system solution. In terms of target structure relationship modeling, the Transformer model can process the position information output by the YOLO, training and ResNet models, and model the spatial relationship between various anatomical structures. For example, the Transformer model plays a role in analyzing the distance relationship between the "common carotid artery" and the "trachea", assisting medical staff in predicting the risks of these target structures during surgery; in terms of global spatial perception, Transformer modeling can not only understand the position of a single anatomical structure and target structure, but also analyze the spatial layout of the entire head and neck area, including the overlapping relationship and adjacent relationship between anatomical structures and target structures, which is crucial for complex surgeries; further, Transformer effectively processes 3D image data to generate more accurate spatial positioning information, which is crucial for complex head and neck surgeries.

[0046] In cooperation with the surgical navigation subsystem, during the operation, the Transformer model receives real-time image data, combines the YOLO model, ResNet model and 3D image data, analyzes the spatial relationship between anatomical structures in real time, and updates the data in the surgical navigation subsystem; a. Navigation feedback: Based on the spatial relationship modeling of Transformer, the system can generate high-precision surgical navigation information and provide real-time feedback to surgical medical staff on the spatial distribution of the anatomical structure of the head and neck. Medical staff can observe the relative positions of anatomical structures through a visual interface and make more accurate surgical decisions.

[0047] As an improved solution of the present invention, it also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module as submodules, and also includes an auxiliary submodule, which is commonly connected to the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module; all modules form a complete AI-assisted head and neck surgical robot system through efficient integration and collaboration, and the system integration module ensures the smooth collaboration of each subsystem, and provides intelligent auxiliary functions through the auxiliary submodule, so that the operation of the system is more efficient, accurate and sensitive; For example, the auxiliary submodule includes a surgical navigation subsystem: this subsystem can comprehensively process the processing flow of the anatomical structure recognition module and the real-time collected signal flow, and generate accurate surgical navigation information for medical staff to use during the operation. The realized visual interface facilitates medical staff to obtain information intuitively. The specific detailed processes are mentioned in the aforementioned embodiments, so they will not be elaborated on.

[0048] The auxiliary submodule also includes a surgical control feedback unit: this feedback unit adjusts the operating accuracy of the surgical robot according to the operational feedback from medical staff to ensure the efficiency and accuracy of the surgical process. At the same time, the system can adjust the surgical path according to real-time images to optimize the surgical process.

[0049] The auxiliary submodule also includes a postoperative recovery tracking unit: After the operation is completed, the postoperative recovery tracking unit continues to track the patient's recovery progress, compares subsequent imaging data with intraoperative data, evaluates the surgical effect and provides recovery suggestions.

[0050] This system realizes real-time recognition of anatomical structures, spatial relationship modeling and precise navigation during head and neck surgery through the collaboration of deep learning models and multiple AI algorithms. The system architecture includes multiple modules such as data acquisition and preprocessing, anatomical structure recognition, data enhancement and label optimization, real-time reasoning and feedback, and system integration. The modules work together to ensure accurate recognition and navigation of anatomical structures during surgery, and can be dynamically adjusted according to actual conditions to minimize surgical risks, reduce trauma, and improve the quality of postoperative recovery of patients.

[0051] The advantages of the present invention are: This system achieves real-time recognition of anatomical structures, spatial relationship modeling and precise navigation during head and neck surgery through the collaborative work of multiple deep learning models and multiple algorithms. The various modules of the system work together to ensure the accurate recognition and navigation of anatomical structures during surgery, and can be dynamically adjusted according to actual conditions to minimize surgical risks, reduce trauma, and improve the quality of patient recovery after surgery.

[0052] The above disclosures are only several specific embodiments of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. An AI-assisted head and neck surgical robot system based on multiple deep learning algorithms, characterized in that: The system includes: an anatomical structure recognition module, used for performing multi-model processing on the original image to generate a structure model, and marking the target structure on the structure model; It also includes a data acquisition and preprocessing module, which is used to preprocess the data at each input stage and to collect the original data and the real-time data of the user during the operation; The real-time reasoning and feedback module is used to associate the anatomical structure recognition module and the data acquisition and preprocessing module to realize real-time feedback of the target structure information to the user.

2. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 1 is characterized in that: Before the original image is input into the model for processing, it needs to be preprocessed. The preprocessing steps include: Image normalization: obtain the original image and perform normalization on the original image to obtain a primary image; Image enhancement: Perform data enhancement on the primary image to obtain the secondary image; Image annotation: Label the secondary images and ensure spatial consistency to obtain training data.

3. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 2 is characterized in that: The data enhancement method includes at least one of geometric transformation, color and brightness adjustment, noise addition and local deformation.

4. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 2 is characterized in that: The generation of the structural model includes the following steps: The training data is input into the YOLO model for modeling processing to generate a structural model.

5. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 4 is characterized in that: The YOLO model training process is: A1. Data preprocessing: adjust the training data to a fixed size to achieve standardized input; A2. Target detection: The YOLO model extracts image features through a convolutional neural network to generate the bounding box of the surgical target and the probability of its corresponding category; A3. Optimize training: Use the cross entropy loss function and IoU loss function to optimize the model to ensure that the convolutional network can produce positioning and classification results during detection; A4. Data output: The output includes the location information of each target structure and the corresponding classification probability.

6. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 5 is characterized in that: After obtaining the structural model, the following steps are also included: The ResNet model is used to receive the output data of the YOLO model, classify and refine the surgical target location to output classification information.

7. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 6 is characterized in that: The ResNet model training process is: B1. Feature extraction and classification: Input at least one location information, perform multi-layer convolution and pooling layer operations on the ResNet model to extract higher-level image features, and use a fully connected layer to further classify the anatomical structure so that each surgical target will be assigned a probability value; B2. Cross entropy loss function: Use the cross entropy loss function to optimize the ResNet model; B3. Training optimization: Use the Adam optimizer for training and adjust the weight parameters in the model to minimize the loss function; B4. Data output: The ResNet model outputs the category of each target structure and the confidence of the category as classification information.

8. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 7 is characterized in that: After completing the ResNet model processing, it also includes: The location information and classification information are input into the Transformer model for processing, and the Transformer model is combined with 3D image data to generate spatial relationship modeling.

9. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to claim 7 is characterized in that: The Transformer model training process is: C1. Encoder layer processing: The Transformer model uses the encoder to extract the spatial features of the input image formed by the YOLO model, ResNet model, and 3D impact data; The encoder models the spatial characteristics through a multi-layer self-attention mechanism and generates spatial representation data of each target structure for output; C2, decoder layer processing: The decoder further predicts the spatial relationship and relative position between each target structure based on the output of the encoder; C3, loss function optimization: using the preset loss function, combining the spatial relationship and the category of the target structure to improve the accuracy of the model; C4. Data output: The output of the Transformer model includes the spatial coordinates of each anatomical target and their relative positional relationships.

10. The AI-assisted head and neck surgical robot system based on multiple deep learning algorithms according to any one of claims 1 to 9, characterized in that: It also includes an integration module, which integrates the data acquisition and preprocessing module, the anatomical structure recognition module and the real-time reasoning and feedback module as sub-modules, and also includes an auxiliary sub-module, which is associated with the anatomical structure recognition module and the real-time reasoning and feedback module.

Images