Double-cavity tracheal catheter selection method and system based on multi-modal data fusion

Abstract

The invention provides a dual-cavity tracheal catheter selection method and system based on multi-modal data fusion. The method comprises the steps that basic information, neck ultrasonic images and chest CT data of audiences are collected; performing data fusion and preprocessing on the basic information, the neck ultrasonic image and the chest CT data to obtain multi-modal fusion data; a double-cavity tracheal catheter analysis result is obtained through deep learning; integrating the multi-modal fusion data and the double-lumen tracheal catheter analysis result to obtain a double-lumen tracheal catheter recommendation result; new tracheal intubation data are collected, and the new tracheal intubation data are used for continuously correcting and optimizing the double-cavity tracheal catheter recommendation result. According to the method, the double-cavity tracheal catheter is selected by adopting artificial intelligence and multi-modal fusion data, so that the purpose of correctly selecting a proper double-cavity tracheal catheter is achieved, and the selection accuracy of the double-cavity tracheal catheter can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence medical technology, and in particular to a double-lumen endotracheal tube selection method and system based on multimodal data fusion. Background Art

[0002] Double-lumen endotracheal tubes are used for endotracheal intubation. They are designed to isolate the lungs anatomically and physiologically and to provide selective independent ventilation for each lung. In thoracic surgery, double-lumen endotracheal tubes can collapse the lung on the surgical side, thereby providing a visual field for surgery, while ensuring mechanical ventilation of the non-surgical lung to ensure the patient's life safety. Correct placement of double-lumen endotracheal tubes can ensure the collapse of the lung on the surgical side and normal mechanical ventilation on the non-surgical side. Incorrect double-lumen endotracheal tubes can cause tracheal mucosal damage, tracheal lacerations, tension pneumothorax, mediastinal emphysema, inability to pass bronchoscopes, decreased blood oxygen saturation or carbon dioxide retention in patients, which may not only lead to surgical failure, but even endanger the patient's life in severe cases. Therefore, choosing the right model is very important for achieving effective lung isolation.

[0003] However, the morphology and size of the tracheobronchial tree vary significantly between individuals, making the selection of a double-lumen endotracheal tube complicated. Traditional double-lumen endotracheal tube selection methods usually rely on instrument assistance or experienced physicians to perform the operation, which is prone to deviations that can lead to serious consequences and medical disputes.

[0004] Therefore, there is a need to improve the current method of selecting double-lumen endotracheal tubes with the help of emerging technologies. Summary of the invention

[0005] The present invention provides a double-lumen endotracheal tube selection method and system based on multimodal data fusion, which are used to solve the defect of low precision in the selection operation of the double-lumen endotracheal tube in the prior art.

[0006] In a first aspect, the present invention provides a double-lumen endotracheal tube selection method based on multimodal data fusion, comprising: Collect the basic information of the audience, neck ultrasound images and chest CT data; Performing data fusion and preprocessing on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; Using deep learning to obtain double-lumen endotracheal tube analysis results; Combining the multimodal fusion data and the double-lumen endotracheal tube analysis results to obtain a double-lumen endotracheal tube recommendation result; New endotracheal intubation data is collected, and the new endotracheal intubation data is used to continuously revise and optimize the double-lumen endotracheal tube recommendation result.

[0007] According to a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention, data fusion and preprocessing are performed on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data, including: Convert data in different formats into a unified format, and perform image segmentation and labeling on the neck ultrasound image and the chest CT data; Among them, the labels include the glottis, endotracheal tube outline, carina, left and right bronchial outlines, and right superior bronchial outline.

[0008] According to a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention, image segmentation is performed on the neck ultrasound image and the chest CT data, including: Determine an AutoFocusFormer model as an image segmentation model, wherein the AutoFocusFormer model includes an encoder and a decoder; The encoder includes a multi-layer self-attention mechanism to extract multi-scale features of the input image through an adaptive focus attention module; The decoder includes gradually upsampling the multi-scale features extracted by the encoder, reshaping them to a size matching the original image, and fusing the low-level features in the encoder with the high-level features using skip connections; The contrast loss function is used to select the outputs of different enhanced images as positive and negative examples for contrast learning, and the final segmentation results are output.

[0009] According to a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention, a double-lumen endotracheal tube analysis result is obtained by deep learning, including: The U-Net++ model is used to determine an initial contour close to the target anatomical structure, and the initial contour is used as a starting point for model segmentation; A feature map is generated through convolutional layers and nested skip connections, and a predicted contour is output. The evolution of the predicted contour is driven by a preset energy function, and the tracheal diameter and length, the minimum diameter of the left and right bronchi, the angle between the left and right bronchi and the main trachea, and the distance between the right upper bronchus and the carina are calculated.

[0010] According to a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention, the multimodal fusion data and the double-lumen endotracheal tube analysis result are integrated to obtain a double-lumen endotracheal tube recommendation result, including: By learning the diameters and angles of different types of endotracheal tubes, specific information of different double-lumen endotracheal tubes is established; A database containing double-lumen endotracheal tubes of different models is constructed, and the first feature extracted from the image and the second feature of any tube model in the database are mapped to a new feature space through a Siamese network to generate a first embedding vector and a second embedding vector respectively; The distance between the first embedding vector and the second embedding vector is calculated using Euclidean distance, the catheter model with the smallest distance is selected as the most matching model, and a double-lumen endotracheal catheter model selection suggestion is output.

[0011] According to a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention, new endotracheal tube data is collected, and the double-lumen endotracheal tube recommendation result is continuously corrected and optimized using the new endotracheal tube data, including: Retraining the model using the new endotracheal intubation data to obtain model misjudgment data; The variational autoencoder algorithm is used to add label data to the model's misjudged data, perform correction and relearning, update the model weights and parameters, and obtain a corrected optimization result.

[0012] A double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention also includes: The double-lumen endotracheal tube recommendation result is presented through an interactive interface, and the user selects and adjusts parameters on the interactive interface to obtain a final decision result.

[0013] In a second aspect, the present invention further provides a double-lumen endotracheal tube selection system based on multimodal data fusion, comprising: The collection module is used to collect the basic information of the audience, neck ultrasound images and chest CT data; A preprocessing fusion module, used for performing data fusion and preprocessing on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; An analysis module, used to obtain double-lumen endotracheal tube analysis results using deep learning; A selection module, used for synthesizing the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain a double-lumen endotracheal tube recommendation result; The optimization module is used to collect new endotracheal intubation data and use the new endotracheal intubation data to continuously correct and optimize the double-lumen endotracheal tube recommendation result.

[0014] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for selecting a double-lumen endotracheal tube based on multimodal data fusion as described in any one of the above is implemented.

[0015] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a double-lumen endotracheal tube selection method based on multimodal data fusion as described in any one of the above.

[0016] The double-lumen endotracheal tube selection method and system based on multimodal data fusion provided by the present invention achieve the purpose of correctly selecting a suitable double-lumen endotracheal tube by adopting artificial intelligence and multimodal fusion data to select a double-lumen endotracheal tube, and can effectively improve the accuracy of double-lumen endotracheal tube selection. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a double-lumen endotracheal tube selection method based on multimodal data fusion provided by the present invention; Figure 2 is a model structure diagram of the AutoFocusFormer model provided by the present invention; Figure 3 It is a model structure diagram of the U-Net++ model provided by the present invention; Figure 4 It is a structural diagram of the Siamese model provided by the present invention; Figure 5 It is a user interface display diagram provided by the present invention; Figure 6 It is a structural schematic diagram of a double-lumen endotracheal tube selection system based on multimodal data fusion provided by the present invention; Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In view of the shortcomings of the prior art, the present invention proposes a double-lumen endotracheal tube selection method based on artificial intelligence and multimodal data fusion. Figure 1is a flow chart of a double-lumen endotracheal tube selection method based on multimodal data fusion provided by an embodiment of the present invention, such as Figure 1 As shown, including: Step 100: Collect basic information of the audience, neck ultrasound images and chest CT data; Step 200: fusing and preprocessing the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; Step 300: Obtaining double-lumen endotracheal tube analysis results using deep learning; Step 400: Combining the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain a double-lumen endotracheal tube recommendation result; Step 500: Collect new endotracheal intubation data, and use the new endotracheal intubation data to continuously revise and optimize the double-lumen endotracheal tube recommendation result.

[0020] Specifically, the double-lumen endotracheal tube selection method based on multimodal data fusion proposed in an embodiment of the present invention mainly includes a multimodal data input unit, a data fusion and preprocessing unit, an artificial intelligence analysis unit, a double-lumen endotracheal tube recommendation unit and a user interaction interface.

[0021] The multimodal data input unit includes the patient's basic information (height, weight, age, gender, previous airway surgery history, planned surgery), airway ultrasound images and chest CT images; the data fusion and preprocessing unit includes converting data in different formats into a unified format, segmenting and labeling chest CT images; the artificial intelligence analysis unit includes using deep learning to identify the trachea, calculate the tracheal diameter, length, the minimum diameter of the left and right bronchi, the angle between the left and right bronchi and the main trachea, and the distance between the right upper bronchus and the carina. At the same time, the patient's basic information is fused and different types of endotracheal tubes are input. The diameter and angle of the catheter are determined to establish different double-lumen endotracheal tube information; the double-lumen endotracheal tube recommendation unit includes providing doctors with double-lumen endotracheal tube selection models and insertion depths based on the results of artificial intelligence analysis, and evaluating the model prediction effect based on the double-lumen endotracheal intubation results. For the false detection part, label data is added again, and training is repeatedly strengthened; the present invention accurately segments the image, builds the model and realizes continuous optimization and upgrading of the model; the user interaction interface is used to allow doctors to view the analysis results and suggestions, make necessary adjustments, and record the final decision, and the doctor evaluates the accuracy of the double-lumen endotracheal tube selection.

[0022] Based on the above embodiment, data fusion and preprocessing are performed on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data, including: Convert data in different formats into a unified format, and perform image segmentation and labeling on the neck ultrasound image and the chest CT data; Among them, the labels include the glottis, endotracheal tube outline, carina, left and right bronchial outlines, and right superior bronchial outline.

[0023] The step of performing image segmentation on the neck ultrasound image and the chest CT data includes: Determine an AutoFocusFormer model as an image segmentation model, wherein the AutoFocusFormer model includes an encoder and a decoder; The encoder includes a multi-layer self-attention mechanism to extract multi-scale features of the input image through an adaptive focus attention module; The decoder includes gradually upsampling the multi-scale features extracted by the encoder, reshaping them to a size matching the original image, and fusing the low-level features in the encoder with the high-level features using skip connections; The contrast loss function is used to select the outputs of different enhanced images as positive and negative examples for contrast learning, and the final segmentation results are output.

[0024] Specifically, after collecting the patient's basic information, neck ultrasound images, and chest CT data, the chest CT data and neck ultrasound images were segmented and labeled, and the multi-feature fusion module in the convolutional neural network extracted features and obtained data. Then, the AutoFocusFormer model learning framework performed image segmentation processing to obtain image segmentation results. For the segmented image data, labels such as glottis, trachea, left bronchus, right bronchus, and carina were added, and the anatomy on the image was calibrated using annotation tool software.

[0025] In particular, if Figure 2As shown in the figure, the AutoFocusFormer model is designed to extract features from continuous image data with labels. The AutoFocusFormer model uses its adaptive focus attention mechanism and transformer architecture to perform efficient multi-scale feature extraction. The encoder part uses a multi-layer self-attention module to replace the traditional convolution layer, and generates rich multi-scale feature representations by capturing global context information and local details. In addition, the encoder's adaptive focus mechanism can dynamically adjust the focus area to more accurately extract key features. The decoder part is responsible for gradually upsampling the multi-scale features extracted by the encoder, reshaping them to a size that matches the original image, and generating the final segmentation result. During the decoding process, AutoFocusFormer uses jump connections to combine low-level features in the encoder with high-level features to restore details in the image and improve segmentation accuracy. The model can effectively process complex image content by performing continuous feature extraction and fusion between multiple frames of image data, and finally obtain high-quality feature image data. AutoFocusFormer uses a contrast loss function to select the outputs of different enhanced images as positive and negative examples for contrast learning:

[0026] in, is the distance between the feature outputs of the encoder on different augmented images, is the label (positive example is 1, negative example is 0), is the boundary threshold in the contrast loss. By optimizing this loss function, the model can more effectively distinguish the target area in the image, thereby achieving high-precision image segmentation. Based on the above embodiment, the double-lumen endotracheal tube analysis results are obtained by using deep learning, including: The U-Net++ model is used to determine an initial contour close to the target anatomical structure, and the initial contour is used as a starting point for model segmentation; A feature map is generated through convolutional layers and nested skip connections, and a predicted contour is output. The evolution of the predicted contour is driven by a preset energy function, and the tracheal diameter and length, the minimum diameter of the left and right bronchi, the angle between the left and right bronchi and the main trachea, and the distance between the right upper bronchus and the carina are calculated.

[0027] Specifically, the embodiment of the present invention adopts Figure 3 The U-Net++ model shown in the figure selects and predicts the initial contour by learning the labeled data of medical images. The model uses its convolutional layer And nested skip connections generate feature maps , and finally output the predicted contour , calculate the tracheal diameter, length, the minimum diameter of the left and right bronchi, the angle between the left and right bronchi and the main trachea, and the distance between the right upper bronchus and the carina.

[0028] After the feature extraction is processed by the graph convolution network module, the graph feature data of the continuous multi-frame multi-scale extraction under the graph convolution network module is obtained, and the feature image data of the continuous multi-frame multi-scale extraction under the neural convolution network module is obtained. The U-Net++ model realizes efficient feature extraction and segmentation by processing continuous image data with labels. The model uses a series of convolution layers and pooling layers in its encoder part to extract deep features of the input image layer by layer and generate multi-scale feature maps. U-Net++ fuses the different levels of features in the encoder and decoder parts through its dense skip connection structure to form a nested U-shaped network. In the decoder, U-Net++ gradually restores the spatial resolution through deconvolution (upsampling), reshapes the multi-scale feature map generated by the encoder to the same size as the input image, and generates accurate segmentation results. During the decoding process, the model uses skip connections to fuse low-level detail features and high-level semantic features, thereby improving segmentation accuracy and restoring image details.

[0029] Here, the embodiment of the present invention uses the Active Contour Model (ACM) algorithm for calculation. The U-Net++ model uses its nested skip connection mechanism to densely connect multi-level features between the encoder and the decoder to improve segmentation accuracy. The encoder part extracts multi-scale features from the input CT image and generates a high-resolution feature map. The decoder part gradually restores these feature maps to segmentation results that match the original image. In this process, the Dice loss function is used to measure the difference between the model output and the true label:

[0030] Where N is the number of samples, is the true label of the i-th pixel, The predicted value of the i-th pixel.

[0031] On imaging images, the U-Net++ model can automatically or manually select an initial contour close to the target anatomical structure (such as trachea, bronchus, etc.) This initial outline serves as the starting point for further segmentation of the model. Subsequently, the U-Net++ model extracts multi-scale features of the CT image through the convolutional and pooling layers of the encoder. Feature map generated by the encoder . It is an image representation at different levels, covering various features from low-level details to high-level semantic information. In the segmentation process, U-Net++ drives the evolution of the contour through an energy function E(C) composed of internal energy, image energy and external energy. The energy function is expressed as: By minimizing this energy function, the model makes the segmentation contour It gradually shrinks or expands in frame-by-frame iterations until it accurately fits the anatomical structures such as the trachea and bronchi in the CT image for further calculation of key anatomical parameters.

[0032] By inputting the diameter and angle of different types of double-lumen endotracheal tubes, the data information of different types of double-lumen endotracheal tubes is established, and the tracheal information obtained by deep learning CT is matched with the double-lumen endotracheal tube information, and suggestions are given. Through the study of the anatomy of endotracheal tubes, carinas, left and right bronchi, etc., the minimum diameter of the trachea, the bronchial diameter, the angle between the bronchus and the main trachea, and the distance from the carina to the glottis are calculated by comparing the glottis, endotracheal tube contours, carinas, left and right bronchi contours, right upper bronchial contours, CT rulers, and the scanning thickness of each layer of the image on CT images and neck ultrasound images.

[0033] Based on the above embodiment, the multimodal fusion data and the double-lumen endotracheal tube analysis result are combined to obtain a double-lumen endotracheal tube recommendation result, including: By learning the diameters and angles of different types of endotracheal tubes, specific information of different double-lumen endotracheal tubes is established; A database containing double-lumen endotracheal tubes of different models is constructed, and the first feature extracted from the image and the second feature of any tube model in the database are mapped to a new feature space through a Siamese network to generate a first embedding vector and a second embedding vector respectively; The distance between the first embedding vector and the second embedding vector is calculated using Euclidean distance, the catheter model with the smallest distance is selected as the most matching model, and a double-lumen endotracheal catheter model selection suggestion is output.

[0034] Specifically, the embodiment of the present invention fuses the processed data and clinical data into a comprehensive feature vector for use in artificial intelligence analysis.

[0035] By learning the diameter and angle of different types of endotracheal tubes, we can establish the specific information of different double-lumen endotracheal tubes. We can build a database containing different types of double-lumen endotracheal tubes through the Siamese network. Figure 4 As shown in the figure, the CT image is extracted Characteristics and characteristics of a catheter model in the database Features and are mapped to a new feature space to generate an embedding vector f( ) and f( ). Use the Euclidean distance to calculate the similarity between two embedding vectors: By calculating the distance between the CT image features and the features of all double-lumen endotracheal tube models in the database, the catheter model with the smallest distance is selected as the most matching model, and suggestions for selecting the left or right double-lumen endotracheal tube model are given.

[0036] Based on the above embodiment, new endotracheal intubation data is collected, and the double-lumen endotracheal tube recommendation result is continuously corrected and optimized using the new endotracheal intubation data, including: Retraining the model using the new endotracheal intubation data to obtain model misjudgment data; The variational autoencoder algorithm is used to add label data to the model's misjudged data, perform correction and relearning, update the model weights and parameters, and obtain a corrected optimization result.

[0037] Specifically, the embodiment of the present invention evaluates the prediction effect of the model based on the clinical double-lumen endotracheal intubation results. For the misjudged parts, label data is added again, and training is repeatedly strengthened. It is characterized in that correction and re-learning are performed through the Variational Auto-Encoder (VAE) algorithm, and the VAE is used to encode and decode the latent space z of these images to generate a representative corrected image that retains key features. . Its goal is to minimize the combined loss function of reconstruction error and KL divergence:

[0038] in, is the original image, To reconstruct the image, is the approximate posterior distribution, is the prior distribution, is the total loss function of VAE. Based on the generated corrected images, refined label data is added again to construct a new training set Subsequently, the reinforcement learning (RL) method is used to repeatedly strengthen the model training, by maximizing the long-term cumulative reward To optimize model performance and improve robustness against misdiagnosis and missed diagnosis.

[0039] The prediction effect of the model is evaluated based on the results of double-lumen endotracheal intubation, and new data from the operation process is continuously collected and used for model retraining. By continuously updating the weights and parameters of the model, the prediction accuracy and robustness of the model are improved, so that it can adapt to a variety of surgical scenarios. At the same time, technologies such as transfer learning are used to draw experience from similar surgical data to speed up the adaptation of the model. The present invention accurately segments the image, constructs the model, and realizes continuous optimization and upgrading of the model.

[0040] Based on the above embodiment, it also includes: The double-lumen endotracheal tube recommendation result is presented through an interactive interface, and the user selects and adjusts parameters on the interactive interface to obtain a final decision result.

[0041] Specifically, the tracheal diameter, length, minimum diameter of the left and right bronchi, angle between the left and right bronchi and the main trachea, distance of the right upper bronchus from the carina, and other information calculated by deep learning are matched with the input double-lumen tracheal tubes of different models, and artificial intelligence makes a decision, and the predicted double-lumen tracheal tube model and insertion depth are displayed on the user interface. The user interface allows doctors to adjust parameters and select the model of their choice, and a feedback mechanism is set to allow doctors to train and optimize the system based on actual results, such as Figure 5 The user interaction interface shown in the figure presents a result that recommends selecting the right double cavity.

[0042] The double-lumen endotracheal tube selection system based on multimodal data fusion provided by the present invention is described below. The double-lumen endotracheal tube selection system based on multimodal data fusion described below and the double-lumen endotracheal tube selection method based on multimodal data fusion described above can correspond to each other.

[0043] Figure 6 is a structural schematic diagram of a double-lumen endotracheal tube selection system based on multimodal data fusion provided by an embodiment of the present invention, such as Figure 6 As shown, it includes a collection module 61, a preprocessing and fusion module 62, an analysis module 63, a selection module 64 and an optimization module 65, wherein: The acquisition module 61 is used to collect the basic information of the audience, the neck ultrasound image and the chest CT data; the preprocessing fusion module 62 is used to perform data fusion and preprocessing on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; the analysis module 63 is used to obtain the double-lumen endotracheal tube analysis result by deep learning; the selection module 64 is used to integrate the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain the double-lumen endotracheal tube recommendation result; the optimization module 65 is used to collect new endotracheal intubation data, and use the new endotracheal intubation data to continuously correct and optimize the double-lumen endotracheal tube recommendation result.

[0044] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a double-lumen endotracheal tube selection method based on multimodal data fusion, the method comprising: collecting basic information, neck ultrasound images and chest CT data of the audience; performing data fusion and preprocessing on the basic information, the neck ultrasound images and the chest CT data to obtain multimodal fusion data; obtaining a double-lumen endotracheal tube analysis result by deep learning; synthesizing the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain a double-lumen endotracheal tube recommendation result; collecting new endotracheal tube data, and continuously revising and optimizing the double-lumen endotracheal tube recommendation result by using the new endotracheal tube data.

[0045] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0046] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the double-lumen endotracheal tube selection method based on multimodal data fusion provided by the above-mentioned methods, the method comprising: collecting basic information, neck ultrasound images and chest CT data of the audience; performing data fusion and preprocessing on the basic information, the neck ultrasound images and the chest CT data to obtain multimodal fusion data; obtaining a double-lumen endotracheal tube analysis result by using deep learning; combining the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain a double-lumen endotracheal tube recommendation result; collecting new endotracheal intubation data, and using the new endotracheal intubation data to continuously correct and optimize the double-lumen endotracheal tube recommendation result.

[0047] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0048] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A double-lumen endotracheal tube selection method based on multimodal data fusion, characterized in that: include: Collect the basic information of the audience, neck ultrasound images and chest CT data; Performing data fusion and preprocessing on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; Using deep learning to obtain double-lumen endotracheal tube analysis results; Combining the multimodal fusion data and the double-lumen endotracheal tube analysis results to obtain a double-lumen endotracheal tube recommendation result; New endotracheal intubation data is collected, and the new endotracheal intubation data is used to continuously revise and optimize the double-lumen endotracheal tube recommendation result.

2. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 1, characterized in that: The basic information, the neck ultrasound image and the chest CT data are subjected to data fusion and preprocessing to obtain multimodal fusion data, including: Convert data in different formats into a unified format, and perform image segmentation and labeling on the neck ultrasound image and the chest CT data; Among them, the labels include the glottis, endotracheal tube outline, carina, left and right bronchial outlines, and right superior bronchial outline.

3. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 2, characterized in that: Performing image segmentation on the neck ultrasound image and the chest CT data includes: Determine an AutoFocusFormer model as an image segmentation model, wherein the AutoFocusFormer model includes an encoder and a decoder; The encoder includes a multi-layer self-attention mechanism to extract multi-scale features of the input image through an adaptive focus attention module; The decoder includes gradually upsampling the multi-scale features extracted by the encoder, reshaping them to a size matching the original image, and fusing the low-level features in the encoder with the high-level features using skip connections; The contrast loss function is used to select the outputs of different enhanced images as positive and negative examples for contrast learning, and the final segmentation results are output.

4. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 1, characterized in that: Double-lumen endotracheal tube analysis results obtained using deep learning include: The U-Net++ model is used to determine an initial contour close to the target anatomical structure, and the initial contour is used as a starting point for model segmentation; A feature map is generated through convolutional layers and nested skip connections, and a predicted contour is output. The evolution of the predicted contour is driven by a preset energy function, and the tracheal diameter and length, the minimum diameter of the left and right bronchi, the angle between the left and right bronchi and the main trachea, and the distance between the right upper bronchus and the carina are calculated.

5. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 1, characterized in that: The multimodal fusion data and the double-lumen endotracheal tube analysis results are combined to obtain a double-lumen endotracheal tube recommendation result, including: By learning the diameters and angles of different types of endotracheal tubes, specific information of different double-lumen endotracheal tubes is established; A database containing double-lumen endotracheal tubes of different models is constructed, and the first feature extracted from the image and the second feature of any tube model in the database are mapped to a new feature space through a Siamese network to generate a first embedding vector and a second embedding vector respectively; The distance between the first embedding vector and the second embedding vector is calculated using Euclidean distance, the catheter model with the smallest distance is selected as the most matching model, and a double-lumen endotracheal catheter model selection suggestion is output.

6. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 1, characterized in that: Collect new endotracheal intubation data, and use the new endotracheal intubation data to continuously revise and optimize the double-lumen endotracheal tube recommendation results, including: Retraining the model using the new endotracheal intubation data to obtain model misjudgment data; The variational autoencoder algorithm is used to add label data to the model's misjudged data, perform correction and relearning, update the model weights and parameters, and obtain a corrected optimization result.

7. The double-lumen endotracheal tube selection method based on multimodal data fusion according to claim 1, characterized in that: Also includes: The double-lumen endotracheal tube recommendation result is presented through an interactive interface, and the user selects and adjusts parameters on the interactive interface to obtain a final decision result.

8. A double-lumen endotracheal tube selection system based on multimodal data fusion, characterized in that: include: The collection module is used to collect the basic information of the audience, neck ultrasound images and chest CT data; A preprocessing fusion module, used for performing data fusion and preprocessing on the basic information, the neck ultrasound image and the chest CT data to obtain multimodal fusion data; An analysis module, used to obtain double-lumen endotracheal tube analysis results using deep learning; A selection module, used for synthesizing the multimodal fusion data and the double-lumen endotracheal tube analysis result to obtain a double-lumen endotracheal tube recommendation result; The optimization module is used to collect new endotracheal intubation data and use the new endotracheal intubation data to continuously correct and optimize the double-lumen endotracheal tube recommendation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the double-lumen endotracheal tube selection method based on multimodal data fusion as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for selecting a double-lumen endotracheal tube based on multimodal data fusion as claimed in any one of claims 1 to 7 is implemented.

Images