Airway obstruction injury condition prediction method and system based on multi-modal transformer and feature selection

Abstract

The present application belongs to the technical field of medical artificial intelligence, and particularly relates to an airway obstruction injury condition prediction method and system based on multi-modal Transformer and feature selection. A composite neural network model is constructed, and differentiated feature extraction approaches are designed for time series vital sign data, structured detection data and initial diagnosis text records respectively: a full convolutional network and a squeeze excitation module are used to capture local patterns and recalibrate channels for time series signals, a multi-layer perceptron is used to mine high-order interaction relationships between structured indicators, a self-attention mechanism of a Transformer encoder is used to analyze text semantic information, and a staged cross-fusion strategy is used to form comprehensive representation and output hierarchical prediction results. The present application effectively breaks through the fusion bottleneck caused by the heterogeneity of multi-modal data, realizes adaptive enhancement of key features and suppression of non-key interference, and significantly improves the comprehensiveness and accuracy of airway obstruction severity prediction.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence technology, specifically relating to a method and system for predicting airway obstruction injuries based on multimodal Transformer and feature selection. Background Technology

[0002] Airway obstruction is a common critical condition in trauma emergency care and intensive care. Failure to promptly and accurately assess its severity will delay the implementation of airway management measures, significantly increasing the patient's risk of death and the incidence of complications. In current clinical practice, healthcare professionals typically rely on vital sign monitoring data such as trends in heart rate and respiratory rate, laboratory test indicators such as blood gas analysis results, and the patient's medical history at the initial consultation to comprehensively assess the injury. However, this information is scattered across different data sources, is vast in quantity, highly dynamic, and manual integration and analysis are inefficient and subject to significant subjective differences. With the development of medical informatization, using artificial intelligence technology to automate the analysis of multi-source heterogeneous data has become an important way to improve the accuracy of injury prediction. However, most existing prediction methods rely only on single-modality data, such as using time-series waveform data or static test results alone, failing to fully utilize the key information such as injury mechanisms and symptom descriptions contained in the initial consultation text records. This results in a limited perception range for the model, making it difficult to comprehensively capture the multidimensional pathological features related to airway obstruction.

[0003] In existing multimodal medical data processing solutions, recurrent neural networks or their variants are often used for modeling time-series vital sign data. However, these networks suffer from gradient decay when processing longer sequences and struggle to explicitly extract local patterns over time. For structured detection data, traditional methods typically employ shallow classifiers or linear regression, failing to fully uncover the higher-order nonlinear interactions between various detection indicators. For text processing, early technologies relied heavily on word frequency statistics or rule matching, lacking sufficient semantic expressive power and struggling to accurately extract subtle clues related to airway obstruction from natural language descriptions. More critically, existing systems often employ simple feature concatenation or weighted averaging when fusing multimodal features, lacking mechanisms for modeling deep intermodal relationships. Information from different modalities interferes with or becomes redundant during fusion, failing to achieve complementary enhancement.

[0004] Furthermore, most methods do not dynamically select and recalibrate based on the importance of different modal features, making the models susceptible to interference from irrelevant or noisy features, resulting in unstable generalization performance and difficulty in meeting the requirements of prediction accuracy and robustness in clinical applications. Summary of the Invention

[0005] To address the aforementioned shortcomings, this invention provides a method and system for predicting airway obstruction injuries based on multimodal Transformer and feature selection. By constructing a composite neural network model FSMT, differentiated feature extraction pathways are designed for temporal vital sign data, structured detection data, and initial diagnosis text records. Specifically, the collaborative design of a fully convolutional network and a squeeze excitation module enables local pattern capture and dynamic feature recalibration at the channel level for temporal signals, enhancing the response to key physiological trends. A multilayer perceptron module extracts complex dependencies between structured indicators through hierarchical nonlinear mapping. The Transformer encoder utilizes a self-attention mechanism to deeply mine semantic associations in the text, compensating for the shortcomings of unstructured information processing. At the feature fusion level, this invention adopts a staged cross-fusion strategy, first interactively fusing temporal and structured features, then integrating them with text features. This allows cross-modal information to gradually align and aggregate at different levels of abstraction, ultimately forming a comprehensive representation for severity grading prediction. This method fully considers the heterogeneity and complementarity of various data types, achieving a comprehensive and accurate assessment of airway obstruction injuries without relying on manual feature engineering, providing reliable technical support for emergency decision-making.

[0006] According to a first aspect of the present invention, the present invention claims protection for a method for predicting airway obstruction injury based on multimodal Transformer and feature selection, comprising: S1, acquire multi-dimensional medical data of the target object, including time-series vital sign data, structured test data, and initial diagnosis text records; S2, the temporal vital signs data is input into the fully convolutional network-squeeze excitation module, the local temporal patterns in the temporal vital signs data are extracted through one-dimensional convolution operation, and the convolution output is recalibrated at the channel level by the squeeze excitation mechanism to obtain the temporal feature vector; S3, the structured detection data is input into the multilayer perceptron module, and the detection indicators in the structured detection data are nonlinearly transformed through the multilayer fully connected network to extract the correlation features between the indicators and obtain the structured feature vector; S4, the initial diagnosis text record is input into the encoder part of the Transformer module, and the semantic dependency relationship between words at different positions in the initial diagnosis text record is captured through the self-attention mechanism, and the text semantic features are extracted to obtain the text feature vector; S5, cross-fuse the temporal feature vector with the structured feature vector to obtain the first fused feature, and then concatenate and fuse the first fused feature with the text feature vector to obtain the comprehensive representation vector; S6. Input the comprehensive representation vector into the output layer to obtain the predicted result of the airway obstruction severity of the target object.

[0007] Furthermore, in step S5, the temporal feature vector and the structured feature vector are cross-fused to obtain a first fused feature, including: The temporal feature vector and the structured feature vector are concatenated to form a concatenated vector; The spliced ​​vector is input into the first fully connected layer, and the spliced ​​vector is subjected to linear transformation and nonlinear activation through the first fully connected layer to obtain the first fused feature.

[0008] Furthermore, in step S5, the first fusion feature is concatenated and fused with the text feature vector to obtain a comprehensive representation vector, including: The first fused feature is concatenated with the text feature vector to form a comprehensive concatenated vector; The synthesized splicing vector is input into the second fully connected layer, and the synthesized splicing vector is subjected to linear transformation and nonlinear activation through the second fully connected layer to obtain the synthesized representation vector.

[0009] Furthermore, the fully convolutional network-squeeze excitation module includes multiple convolutional sub-modules connected in sequence, each of the convolutional sub-modules including a one-dimensional convolutional layer and a squeeze excitation block; The one-dimensional convolutional layer takes the temporal vital signs data or the output of the previous convolutional submodule as input, and performs convolution operations by sliding the convolutional kernel along the time dimension to generate the corresponding convolutional feature map. The squeeze excitation block performs a squeeze operation on the convolutional feature map and generates a global description vector in the channel dimension by performing global feature aggregation on the convolutional feature map in the time dimension. The squeeze excitation block performs an excitation operation on the global description vector, generates recalibration weights for each channel through the description values ​​of each channel in the global description vector, and applies each recalibration weight to the features of the corresponding channel in the convolutional feature map to obtain the recalibrated channel features. The recalibrated channel features output by each of the convolutional sub-modules are processed sequentially by subsequent convolutional sub-modules, and the temporal feature vector is output by the last convolutional sub-module.

[0010] Furthermore, the multilayer perceptron module includes a first fully connected layer, a second fully connected layer, and a third fully connected layer connected in sequence, wherein the number of neurons in the first fully connected layer is greater than the number of neurons in the second fully connected layer, and the number of neurons in the second fully connected layer is greater than the number of neurons in the third fully connected layer; The structured detection data is sequentially transformed nonlinearly through the first fully connected layer, the second fully connected layer, and the third fully connected layer. Each fully connected layer performs a linear transformation on the input and outputs it to the next layer after passing through an activation function. The third fully connected layer outputs the structured feature vector.

[0011] Furthermore, the encoder portion of the Transformer module includes multiple self-attention layers connected in sequence; Each self-attention layer performs multi-head self-attention operation on the input sequence data. Multiple attention heads calculate the attention weights between different positions in the input sequence in parallel. Each attention head performs weighted aggregation on the input sequence based on its own calculated attention weights to obtain the weighted output of each attention head. The weighted outputs of all attention heads are concatenated and linearly transformed to obtain the output of the self-attention layer, which is then passed to the next self-attention layer. The output of the last self-attention layer is aggregated along the position dimension to obtain the text feature vector.

[0012] Furthermore, the time-series vital sign data is time-series data composed of vital sign measurements at multiple time points, and the vital sign measurements at each time point include heart rate measurements and respiratory rate measurements; The one-dimensional convolution operation uses a convolution kernel with a set stride to perform sliding convolution along the time dimension of the time series data. Each time the kernel slides, it covers the vital sign measurement values ​​within a local time window in the time series data, performs weighted summation, and processes them through an activation function to generate the convolution result corresponding to that local time window. The convolution results of all local time windows constitute the convolution feature map. The squeezing excitation module performs global feature aggregation on the convolutional feature map in the time dimension, and performs aggregation operation on the feature values ​​of each channel in the convolutional feature map at all time steps to obtain the global description value of each channel. The global description values ​​of all channels constitute the global description vector. The squeezing excitation module generates the recalibration weights for each channel by nonlinearly transforming the global description vector, and multiplies each recalibration weight with the feature value of the corresponding channel at each time step in the convolutional feature map to obtain the recalibrated channel features.

[0013] Furthermore, the concatenation of the temporal feature vector and the structured feature vector includes sequentially combining the elements of each dimension of the temporal feature vector with the elements of each dimension of the structured feature vector to form a concatenated vector whose dimension number is the sum of the dimension number of the temporal feature vector and the dimension number of the structured feature vector. When the first fully connected layer performs a linear transformation on the concatenated vector, it takes the elements of each dimension of the concatenated vector as input, maps the concatenated vector to the target dimension space through a weight matrix, and then processes it through an activation function to obtain the first fused feature.

[0014] Furthermore, the step of concatenating the first fusion feature with the text feature vector includes sequentially combining the elements of each dimension of the first fusion feature with the elements of each dimension of the text feature vector to form a comprehensive concatenated vector whose number of dimensions is the sum of the number of dimensions of the first fusion feature and the number of dimensions of the text feature vector. When the second fully connected layer performs a linear transformation on the composite splicing vector, it takes the elements of each dimension of the composite splicing vector as input, maps the composite splicing vector to the target dimension space through the weight matrix, and then processes it through the activation function to obtain the composite representation vector. The output layer is a fully connected classification layer. The fully connected classification layer takes the comprehensive representation vector as input and maps the comprehensive representation vector to an output dimension equal to the number of airway obstruction severity levels through a weight matrix. The value of each output dimension corresponds to the prediction confidence of the target object belonging to each airway obstruction severity level. The prediction result of the airway obstruction severity of the target object is determined based on each prediction confidence.

[0015] According to a second aspect of the present invention, the present invention claims protection for an airway obstruction injury prediction system based on multimodal Transformer and feature selection, comprising: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the airway obstruction injury prediction method based on multimodal Transformer and feature selection.

[0016] This invention belongs to the field of medical artificial intelligence technology, specifically relating to a method and system for predicting airway obstruction severity based on multimodal Transformer and feature selection. A composite neural network model is constructed, and differentiated feature extraction approaches are designed for time-series vital sign data, structured detection data, and initial diagnosis text records: a fully convolutional network and a squeezing excitation module are used to capture local patterns and recalibrate channels in the time-series signals; a multilayer perceptron is used to mine high-order interaction relationships between structured indicators; the self-attention mechanism of the Transformer encoder is used to parse textual semantic information; and a staged cross-fusion strategy is employed to form a comprehensive representation and output hierarchical prediction results. This invention effectively overcomes the fusion bottleneck caused by the heterogeneity of multimodal data, achieving adaptive enhancement of key features and suppression of non-key interference, significantly improving the comprehensiveness and accuracy of airway obstruction severity prediction. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for predicting airway obstruction injury based on multimodal Transformer and feature selection, as claimed in the embodiments of the present invention. Figure 2 The second workflow diagram of a method for predicting airway obstruction injury based on multimodal Transformer and feature selection, as claimed in an embodiment of the present invention; Figure 3 The third flowchart of a method for predicting airway obstruction injury based on multimodal Transformer and feature selection, as claimed in an embodiment of the present invention, is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0019] The technical solutions disclosed in the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] According to the first embodiment of the present invention, referring to Figure 1 This invention claims protection for a method for predicting airway obstruction injury based on multimodal Transformer and feature selection, comprising: S1, acquire multi-dimensional medical data of the target object, including time-series vital sign data, structured test data, and initial diagnosis text records; S2, the temporal vital signs data is input into the fully convolutional network-squeeze excitation module, the local temporal patterns in the temporal vital signs data are extracted through one-dimensional convolution operation, and the convolution output is recalibrated at the channel level by the squeeze excitation mechanism to obtain the temporal feature vector; S3, the structured detection data is input into the multilayer perceptron module, and the detection indicators in the structured detection data are nonlinearly transformed through the multilayer fully connected network to extract the correlation features between the indicators and obtain the structured feature vector; S4, the initial diagnosis text record is input into the encoder part of the Transformer module, and the semantic dependency relationship between words at different positions in the initial diagnosis text record is captured through the self-attention mechanism, and the text semantic features are extracted to obtain the text feature vector; S5, cross-fuse the temporal feature vector with the structured feature vector to obtain the first fused feature, and then concatenate and fuse the first fused feature with the text feature vector to obtain the comprehensive representation vector; S6. Input the comprehensive representation vector into the output layer to obtain the predicted result of the airway obstruction severity of the target object.

[0021] In this embodiment, multivariate medical data of the target subject is acquired through a medical information interface or data entry terminal. This multivariate medical data includes at least three types: time-series vital sign data, structured detection data, and initial diagnosis text records. Time-series vital sign data refers to a sequence of vital sign measurements collected from the target subject at multiple consecutive time points. The sampling interval can be preset according to clinical monitoring needs, such as recording heart rate and respiratory rate values ​​at intervals of one second or one minute, forming a numerical sequence arranged in chronological order. This invention does not limit the sampling interval to one minute; the specific sampling interval can be selected based on factors such as the target subject's disease risk and clinical monitoring conditions. Structured test data refers to the laboratory test results of the target object, which includes the names of multiple test indicators and their corresponding values, such as the partial pressure of oxygen, partial pressure of carbon dioxide, and pH value in blood gas analysis, as well as blood lactate and blood glucose value in biochemical tests. These indicators are organized in the form of key-value pairs, forming a set of structured numerical data. Initial diagnosis text records refer to the natural language text recorded by medical staff when making a preliminary diagnosis of the target object, which includes unstructured text information such as description of the cause of injury, chief complaint of symptoms, and brief description of past medical history.

[0022] After data acquisition, the time-series vital signs data are input into a pre-constructed fully convolutional network-squeeze activation module, namely the FCN-SE module. This module contains multiple sequentially stacked one-dimensional convolutional layers and squeeze activation blocks. Specifically, the time-series vital signs data are input as a two-dimensional tensor, where the first dimension is the time step and the second dimension is the number of feature channels, such as heart rate and respiratory rate channels. The one-dimensional convolutional layers slide their kernels along the time dimension, with each convolution operation covering data from several consecutive time points within the current time window. The data from each channel within the window are weighted and summed, and a convolutional response value is output via an activation function. As the convolutional kernels gradually slide along the time axis, a convolutional feature map corresponding to the entire time series is generated. This feature map retains local pattern information in the time dimension, and the number of channels is determined by the number of convolutional kernels. Subsequently, the squeeze excitation block performs channel recalibration on the convolutional feature map: first, it globally aggregates the response values ​​of all time steps for each channel in the time dimension to obtain the global description value of each channel; then, it generates the multiplicative weights of each channel based on these global description values ​​through a gating mechanism; finally, it multiplies the multiplicative weights of each channel with the original feature map of that channel element by element to obtain the recalibrated features; after being processed step by step by multiple convolutional sub-modules, the last sub-module outputs a fixed-dimensional temporal feature vector, which gathers the key pattern information of the temporal vital signs data after channel recalibration.

[0023] Simultaneously, the structured detection data is input into the Multilayer Perceptron (MLP) module. This module consists of multiple fully connected layers connected sequentially, each containing several neurons. The input structured detection data is first expanded into a one-dimensional vector, with each detection index value serving as an element of the vector's dimension. The first fully connected layer performs a linear transformation on this vector, mapping the input vector to a higher-dimensional latent space through a weight matrix and introducing nonlinearity through an activation function, outputting an intermediate activation vector. This intermediate activation vector is then input into the second fully connected layer, where it undergoes another linear mapping and nonlinear transformation, progressively reducing or adjusting the vector dimension while extracting the complex interactions between the detection indices. After processing by the final fully connected layer, a fixed-dimensional structured feature vector is output, representing the correlation patterns between indices contained in the structured detection data.

[0024] Meanwhile, the initial diagnostic text record is input into the encoder part of the Transformer module. This encoder part contains multiple sequentially connected self-attention layers. Before input, the initial diagnostic text record is first segmented into a word sequence, and each word is mapped to a word embedding vector. At the same time, positional encoding is added to the position of each word to preserve the sequence order information. The word embedding sequence passes through the first self-attention layer in sequence. In this layer, multiple attention heads are computed in parallel: each attention head calculates the attention weight of each word with all other words in the sequence, and then performs a weighted sum of the embeddings of all words according to these weights to obtain a new representation of each word. The outputs of all attention heads are concatenated and linearly transformed to form the output sequence of this layer. This output sequence continues to be passed to the next self-attention layer, and the above multi-head self-attention calculation is repeated. After the step-by-step processing of all self-attention layers, the deep semantic representation sequence of each word is finally obtained. This sequence is then aggregated in the sequence dimension, for example, by taking the average or maximum value of all word representations, to obtain a fixed-dimensional text feature vector, which concentrates the key semantic information in the initial diagnostic text record.

[0025] After obtaining the temporal feature vector, structured feature vector, and text feature vector, the feature fusion stage begins. First, a cross-fusion process is performed: the temporal feature vector and the structured feature vector are concatenated end-to-end along their vector dimensions to form a concatenated vector with a dimension equal to the sum of the dimensions of the two vectors. This concatenated vector is then input into a fully connected layer, which performs a linear mapping to compress the dimension to a preset fusion dimension and outputs the first fused feature via a non-linear activation function. This cross-fusion operation allows the temporal and structured features to interact within the same feature space, forming a joint representation.

[0026] Then, the second stage of splicing and fusion is performed: the first fusion feature obtained in the first stage is spliced ​​together with the text feature vector output by the Transformer module to form a comprehensive splicing vector; this comprehensive splicing vector is input into another fully connected layer, which linearly maps it to the final prediction feature space and outputs a comprehensive representation vector; this comprehensive representation vector contains the fusion information of three types of data: temporal vital signs, structured detection, and initial diagnosis text.

[0027] Finally, the comprehensive representation vector is input into the output layer; the output layer adopts a fully connected classification structure, and the number of its output nodes is the same as the number of preset airway obstruction severity levels; this fully connected layer maps the comprehensive representation vector to the raw score corresponding to each level, and these raw scores are converted into a probability distribution by an activation function such as the Softmax function. The probability value of each category represents the confidence level of the target object belonging to that severity level; based on the level corresponding to the maximum probability, the predicted result of the airway obstruction severity of the target object is determined.

[0028] Furthermore, in step S5, the temporal feature vector and the structured feature vector are cross-fused to obtain a first fused feature, including: The temporal feature vector and the structured feature vector are concatenated to form a concatenated vector; The spliced ​​vector is input into the first fully connected layer, and the spliced ​​vector is subjected to linear transformation and nonlinear activation through the first fully connected layer to obtain the first fused feature.

[0029] In this embodiment, the temporal feature vector and the structured feature vector are first obtained, both of which are fixed-length one-dimensional tensors. The temporal feature vector is output by the FCN-SE module, and its dimension is denoted as A. The structured feature vector is output by the MLP module, and its dimension is denoted as B. When the system performs the concatenation operation, the first element to the Ath element of the temporal feature vector are arranged sequentially, followed by the first element to the Bth element of the structured feature vector, thereby forming a one-dimensional concatenated vector with dimension A+B. This concatenation operation only involves rearranging the elements and does not change the value of each element itself.

[0030] Subsequently, the system inputs the concatenated vector into the first fully connected layer; this first fully connected layer has a preset input dimension of A+B and an output dimension denoted as C, i.e., the dimension of the first fused feature; inside this fully connected layer, the system maintains a weight matrix with rows equal to the output dimension C and columns equal to the input dimension A+B, and also maintains a bias vector of length C; the system calculates the matrix-vector multiplication of the concatenated vector and the weight matrix, that is, performing a dot product between each row of the weight matrix and each element of the concatenated vector, and adding the corresponding bias value from the bias vector, to obtain a linear transformation result vector of length C; this structure Each element of the result vector represents a weighted combination of the dimensions of the concatenated vector. Subsequently, the system applies a nonlinear activation function, such as the ReLU function, to each element of the linear transformation result vector, setting all negative values ​​to zero and keeping positive values ​​unchanged, thereby obtaining a nonlinear first fusion feature. This activation operation enhances the expressive power of the feature, enabling subsequent fusion to capture more complex patterns. Finally, the first fully connected layer outputs a first fusion feature vector with dimension C, which integrates feature information from both temporal vital signs and structured detection data, providing an intermediate representation for subsequent fusion with text features.

[0031] Furthermore, in step S5, the first fusion feature is concatenated and fused with the text feature vector to obtain a comprehensive representation vector, including: The first fused feature is concatenated with the text feature vector to form a comprehensive concatenated vector; The synthesized splicing vector is input into the second fully connected layer, and the synthesized splicing vector is subjected to linear transformation and nonlinear activation through the second fully connected layer to obtain the synthesized representation vector.

[0032] In this embodiment, the first stage of fusion has been completed, resulting in a first fused feature vector with dimension C. Simultaneously, a text feature vector with dimension D is obtained from the Transformer module. The system then performs a second-stage concatenation operation: arranging all the dimensional elements of the first fused feature vector in their original order first, followed by arranging all the dimensional elements of the text feature vector in their original order last, forming a combined concatenated vector with dimensions C+D. This concatenation operation does not involve any numerical calculations; it only involves the sequential combination of the elements of the two vectors.

[0033] Next, the system inputs the synthesized vector into the second fully connected layer. This second fully connected layer has a preset input dimension of C+D and an output dimension of E, which is the dimension of the synthesized representation vector. Internally, this fully connected layer contains a weight matrix with E rows and C+D columns, and a bias vector of length E. The system performs a linear transformation: summing the dot products of each element of the synthesized vector with the corresponding rows of the weight matrix, and adding the bias vector element by element to obtain a linear output vector of length E. Each element of this linear output vector is a weighted linear combination of all dimensions of the synthesized vector. Subsequently, the system applies a non-linear activation function, such as ReLU or Tanh, to this linear output vector to introduce non-linearity. The activated vector is the synthesized representation vector, with a dimension of E. This synthesized representation vector uses a phased fusion strategy, first fusing temporal features with structured features, and then fusing the fusion result with textual features. This allows for the gradual integration of data information from three different modalities at the feature level, ultimately forming a synthesized representation vector that covers all the key information of the input data, providing input for subsequent severity prediction.

[0034] Furthermore, referring to Figure 2 The fully convolutional network-squeeze excitation module includes multiple convolutional sub-modules connected in sequence, and each convolutional sub-module includes a one-dimensional convolutional layer and a squeeze excitation block. One-dimensional convolutional layers take temporal vital sign data or the output of the previous convolutional submodule as input, and perform convolution operations by sliding the convolutional kernel along the time dimension to generate corresponding convolutional feature maps. The squeeze excitation block performs a squeeze operation on the convolutional feature map, and generates a global description vector in the channel dimension by performing global feature aggregation on the convolutional feature map in the time dimension. The squeezed excitation block performs an excitation operation on the global description vector, generates recalibration weights for each channel through the description values ​​of each channel in the global description vector, and applies each recalibration weight to the features of the corresponding channel in the convolutional feature map to obtain the recalibrated channel features. The recalibrated channel features output by each convolutional submodule are processed sequentially by subsequent convolutional submodules, and the temporal feature vector is output by the last convolutional submodule.

[0035] In this embodiment, the fully convolutional network-squeeze excitation module is composed of multiple convolutional sub-modules stacked sequentially. Each convolutional sub-module contains a one-dimensional convolutional layer and a squeeze excitation block. The input of the first convolutional sub-module receives the original temporal vital signs data, and the input of each subsequent convolutional sub-module is connected to the output of the previous convolutional sub-module.

[0036] For any one-dimensional convolutional layer in a convolutional submodule, it contains multiple convolutional kernels, each of which is a fixed-length one-dimensional filter. The number of filters determines the number of output channels. This convolutional layer receives input data, which is a two-dimensional tensor. The first dimension is the time step, i.e., the length of the time series, and the second dimension is the number of channels, i.e., the number of vital sign types, such as heart rate and respiratory rate. The convolutional layer slides each convolutional kernel along the time dimension with a fixed stride. At each sliding position, the convolutional kernel covers all channel data within the current time window. The weights of each channel data and the corresponding position of the convolutional kernel are multiplied element-wise and summed. After adding a bias term, a response value at that position is obtained. After each convolutional kernel has slid across the entire time axis, a one-dimensional response sequence corresponding to that convolutional kernel is generated. The response sequences generated by all convolutional kernels are stacked along the channel dimension to form a convolutional feature map. The time dimension length of this convolutional feature map is determined by the input sequence length, the convolutional kernel size, and the stride. The number of channels is equal to the number of convolutional kernels.

[0037] The convolutional feature map is then fed into the squeeze excitation block of the current submodule; the squeeze excitation block first performs a squeeze operation: for each channel of the convolutional feature map, a global aggregation is performed in the time dimension; specifically, for the c-th channel, the squeeze excitation block extracts the response value of the channel at all time steps, calculates the arithmetic mean of these response values, and obtains a scalar value that represents the global response level of the channel; the same operation is performed on all channels to generate a global description vector of length equal to the number of channels, where each element of the vector corresponds to the global response level of a channel.

[0038] Subsequently, the squeezed excitation block performs an excitation operation: the global description vector is input into a small network containing two fully connected layers; the first fully connected layer maps the global description vector to a lower dimension, i.e., performs dimensionality reduction, and outputs an intermediate vector after dimensionality reduction; the second fully connected layer maps this intermediate vector back to the original channel number dimension, and outputs a weight vector with the same length as the global description vector; each element of this weight vector is a value between 0 and 1 constrained by the Sigmoid function, representing the importance weight of the corresponding channel. The system performs a channel-by-channel multiplication operation on this weight vector with the original convolutional feature map: for each channel, the weight value of the channel is multiplied by the response value of the channel at all time steps to obtain the recalibrated channel features; this recalibration operation enhances the responses of important channels and suppresses the responses of unimportant channels.

[0039] After processing by the current convolutional submodule, the output recalibrated channel features are still two-dimensional tensors, although the time dimension may change. The number of channels remains unchanged and serves as the input for the next convolutional submodule. After all convolutional submodules have been processed sequentially, the last convolutional submodule outputs the final temporal feature vector. This temporal feature vector is obtained by globally aggregating the final output two-dimensional tensor in the time dimension, such as by averaging or max pooling, compressing the time dimension, and retaining only the channel dimension to obtain a one-dimensional vector as the temporal feature vector.

[0040] Furthermore, the multilayer perceptron module includes a first fully connected layer, a second fully connected layer, and a third fully connected layer connected in sequence, wherein the number of neurons in the first fully connected layer is greater than the number of neurons in the second fully connected layer, and the number of neurons in the second fully connected layer is greater than the number of neurons in the third fully connected layer; The structured detection data is sequentially transformed nonlinearly through the first fully connected layer, the second fully connected layer, and the third fully connected layer. Each fully connected layer performs a linear transformation on the input and outputs it to the next layer after passing through an activation function. The third fully connected layer outputs the structured feature vector.

[0041] In this embodiment, the multilayer perceptron module includes three fully connected layers connected in sequence: a first fully connected layer, a second fully connected layer, and a third fully connected layer. The number of neurons in these three fully connected layers decreases sequentially, i.e., the number of neurons in the first fully connected layer is greater than the number of neurons in the second fully connected layer, and the number of neurons in the second fully connected layer is greater than the number of neurons in the third fully connected layer.

[0042] Structured detection data is input in key-value pair format. The system first extracts the values ​​of each detection index according to a preset index order to form a one-dimensional input vector. The dimension of this vector is equal to the total number of detection indexes. This input vector is fed into the first fully connected layer. The first fully connected layer has the same number of input nodes as the input vector dimension, and a large number of output nodes, i.e., neurons. Inside this layer, the system multiplies each element of the input vector with all weights in the corresponding column of the weight matrix. Each output node receives the weighted sum of all input elements and adds its bias term. Subsequently, the weighted sum of each output node is processed by the ReLU activation function, which truncates negative values ​​to zero and keeps positive values ​​unchanged, forming the first hidden layer vector.

[0043] The first hidden layer vector is then input to the second fully connected layer. The number of input nodes in the second fully connected layer is equal to the number of output nodes in the first fully connected layer, and its number of output nodes is less than that of the first fully connected layer but greater than that of the third fully connected layer. The second fully connected layer performs the same linear weighted summation operation as the first layer, that is, it calculates the product of the input vector and the weight matrix and adds a bias, and then applies the ReLU activation function to obtain the second hidden layer vector. This process further reduces the feature dimension and extracts higher-level abstract features.

[0044] The second hidden layer vector is ultimately input to the third fully connected layer. The number of input nodes in the third fully connected layer is equal to the number of output nodes in the second fully connected layer, and its output node count is minimized, which is the preset dimension of the structured feature vector. The third fully connected layer also performs linear weighted summation on the input vector and adds a bias, but the activation function can be ReLU or other functions. The third fully connected layer outputs the structured feature vector. In the prediction phase after training, this three-layer network performs forward computation with fixed weights, reducing the data dimension layer by layer, and extracting the complex interaction relationships between various detection indicators through non-linear activation functions, finally outputting a condensed structured feature representation.

[0045] Furthermore, the encoder portion of the Transformer module includes multiple self-attention layers connected in sequence; Each self-attention layer performs multi-head self-attention operation on the input sequence data. Multiple attention heads calculate the attention weights between different positions in the input sequence in parallel. Each attention head performs weighted aggregation on the input sequence based on its own calculated attention weights to obtain the weighted output of each attention head. The weighted outputs of all attention heads are concatenated and linearly transformed to obtain the output of the self-attention layer, which is then passed to the next self-attention layer. The output of the last self-attention layer is aggregated along the position dimension to obtain the text feature vector.

[0046] In this embodiment, the encoder portion consists of multiple self-attention layers stacked sequentially. Each self-attention layer includes a multi-head self-attention sub-layer and a feedforward network sub-layer, and is equipped with residual connections and layer normalization after each sub-layer.

[0047] After the initial diagnostic text record is input, the system first performs text preprocessing: the text is segmented into tokens by spaces or punctuation marks, and each token is mapped to a fixed-dimensional word embedding vector through a query embedding matrix; at the same time, the system generates a position embedding for each token, uses sine and cosine functions to generate a vector with the same dimension as the word embedding, and adds it element by element to the word embedding vector to obtain an input embedding sequence with position information.

[0048] The embedding sequence serves as the input to the first self-attention layer. Within this layer, a multi-head self-attention sublayer is configured with multiple attention heads. Each attention head independently performs the following operations: for each word in the sequence, it generates a query vector, key vector, and value vector based on its embedding vector, using different linear projection matrices. Then, the attention head calculates the dot product between the query vector of each word and the key vectors of all words to obtain an attention score matrix, which is then scaled and normalized using Softmax to obtain an attention weight matrix. Finally, the attention head performs a weighted summation of the value vectors of all words based on the attention weight matrix to obtain a weighted output for each word. The outputs of all attention heads are concatenated along the feature dimension and then transformed by a linear projection layer to obtain the output of the self-attention sublayer. This output is then added to the input embedding sequence via residual connections and normalized by layers before being fed into a feedforward sublayer. The feedforward sublayer contains two linear transformations with non-linear activation in between, and also performs residual connections and layer normalization to complete all computations of the self-attention layer.

[0049] The output sequence of the self-attention layer serves as the input to the next self-attention layer, and the above calculation process is repeated for each subsequent self-attention layer. Each self-attention layer independently learns the semantic dependencies between words in the sequence. As the number of layers increases, the extracted semantic information becomes more abstract and global. After processing by all self-attention layers, the deep semantic representation sequence of each word is finally output. In order to obtain a fixed-length text feature vector, the system performs aggregation operations on the sequence dimension, such as calculating the average value of all word representations or taking the maximum value for pooling, to obtain a one-dimensional vector, i.e., the text feature vector.

[0050] Furthermore, referring to Figure 3 The time-series vital sign data is time-series data composed of vital sign measurements at multiple time points, and the vital sign measurements at each time point include heart rate and respiratory rate measurements. One-dimensional convolution operation uses a convolution kernel with a set stride to slide convolution along the time dimension of the time series data. Each time the kernel slides, it covers the vital sign measurement values ​​within a local time window in the time series data. The values ​​are weighted, summed, and processed by an activation function to generate the convolution result corresponding to that local time window. The convolution results of all local time windows constitute the convolution feature map. The squeezing excitation module performs global feature aggregation on the convolutional feature map in the time dimension, and performs aggregation operation on the feature values ​​of each channel in the convolutional feature map at all time steps to obtain the global description value of each channel. The global description values ​​of all channels constitute the global description vector. The squeezing excitation module generates recalibration weights for each channel by nonlinearly transforming the global description vector. Each recalibration weight is then multiplied by the feature values ​​of the corresponding channel at each time step in the convolutional feature map to obtain the recalibrated channel features.

[0051] In this embodiment, the time-series vital signs data is a time series composed of heart rate and respiratory rate measurements collected at fixed sampling intervals, such as every second or every minute; each sampling moment corresponds to a two-dimensional data point, containing heart rate and respiratory rate values; all sampling moments are arranged in chronological order to form two parallel sequences: heart rate time series and respiratory rate time series; the system organizes these two sequences into a two-dimensional matrix, with the number of rows being the length of the time series and the number of columns being 2, corresponding to heart rate and respiratory rate respectively.

[0052] One-dimensional convolution is performed on this two-dimensional matrix; the convolution kernel is a one-dimensional filter of length L, each kernel has 2 channels, consistent with the number of columns in the input data; the convolution kernel slides along the time axis with a stride S; at each sliding position, the convolution kernel covers the time window from time t to time t+L-1; each channel of the convolution kernel is element-wise multiplied with the value of the corresponding channel of the input data within the window, the product of the two channels is added together, and a bias is added to obtain a convolution response value at that window position; as the convolution kernel slides from the beginning to the end of the sequence, a response sequence with a length of sequence length - L+1 / S is generated. If the convolutional layer is configured with K convolution kernels, K response sequences are generated, stacked column-wise to form a convolutional feature map with K channels and a time dimension length equal to the length of the aforementioned response sequence.

[0053] The convolutional feature map is input to the squeeze excitation block. The squeeze excitation block first performs global feature aggregation in the time dimension, i.e., a squeeze operation; for each channel in the convolutional feature map (a total of K channels), the system extracts the response value of that channel at all time steps; the system calculates the arithmetic mean of these response values ​​as the global description value of that channel; this operation is repeated for all K channels to obtain a global description vector of length K; the k-th element of this vector corresponds to the average global response level of the k-th channel.

[0054] Subsequently, the activation operation begins. The system takes the global description vector as input and passes it through a gated network consisting of two fully connected layers. The first fully connected layer maps the global description vector from dimension K to a smaller dimension, such as K / r, where r is the reduction ratio, outputting a reduced vector and applying the ReLU activation function. The second fully connected layer remaps the reduced vector back to dimension K, outputting an inactive weight vector, which is then compressed to between 0 and 1 using the Sigmoid function to form a recalibrated weight vector. The k-th element of this weight vector is the multiplicative weight of the k-th channel. The system multiplies the multiplicative weight of each channel element-wise with the original convolutional feature map of that channel: for the k-th channel, the response value of that channel at all time steps is multiplied by the k-th value in the weight vector to obtain the recalibrated channel feature. This operation scales the response value of each channel according to its global importance, strengthening the signal of key channels and weakening the signal of non-key channels. Finally, the recalibrated channel feature retains the original time dimension and number of channels as the output of this convolutional submodule and is passed to the next layer.

[0055] Furthermore, the concatenation of the temporal feature vector and the structured feature vector includes sequentially combining the elements of each dimension of the temporal feature vector with the elements of each dimension of the structured feature vector to form a concatenated vector whose dimension number is the sum of the dimension number of the temporal feature vector and the dimension number of the structured feature vector. When the first fully connected layer performs a linear transformation on the concatenated vector, it takes the elements of each dimension of the concatenated vector as input, maps the concatenated vector to the target dimension space through a weight matrix, and then processes it through an activation function to obtain the first fused feature.

[0056] In this embodiment, the system obtains a temporal feature vector, assuming its dimension is A, containing A elements, arranged from index 1 to A. Simultaneously, it obtains a structured feature vector, with dimension B, containing B elements, arranged from index 1 to B. The concatenation operation follows a fixed order: temporal first, then structured. The system creates a new empty vector with dimension A+B; it places the first element of the temporal feature vector in the first position of the new vector, the second element in the second position, and so on, until the Ath element is placed in the Ath position; subsequently, it places the first element of the structured feature vector in the (A+1)th position of the new vector, the second element in the (A+2)th position, and so on, until the Bth element is placed in the (A+B)th position. After these operations, the first A elements of the new vector come from the temporal feature vector, and the last B elements come from the structured feature vector; the original order of both vectors is preserved.

[0057] The concatenated vector is input to the first fully connected layer. This first fully connected layer maintains a weight matrix W1 with rows equal to the preset output dimension C and columns equal to the input dimension A+B. It also maintains a bias vector b1 of length C. When calculating the linear transformation, the system treats the concatenated vector as a column vector with dimension A+B×1, multiplying it by the weight matrix W1 to obtain an intermediate result of dimension C×1. Specifically, for the i-th node of the output dimension, the system multiplies each element of the concatenated vector in pairs, using the weight values ​​(A+B) in the i-th row of W1. The values ​​are accumulated and then added to the i-th bias value of b1 to obtain the linear output value of the node. This process is repeated for i from 1 to C to obtain a linear output vector of length C. Subsequently, the system applies the ReLU activation function to each element of the linear output vector: if the element value is greater than 0, it remains unchanged; if the element value is less than or equal to 0, it is set to 0. The activated vector is the first fused feature. This feature retains the original information of the temporal and structural features and achieves the interactive combination of the two in the feature space through the learnable weights of the fully connected layer.

[0058] Furthermore, the step of concatenating the first fusion feature with the text feature vector includes sequentially combining the elements of each dimension of the first fusion feature with the elements of each dimension of the text feature vector to form a comprehensive concatenated vector whose number of dimensions is the sum of the number of dimensions of the first fusion feature and the number of dimensions of the text feature vector. When the second fully connected layer performs a linear transformation on the composite splicing vector, it takes the elements of each dimension of the composite splicing vector as input, maps the composite splicing vector to the target dimension space through the weight matrix, and then processes it through the activation function to obtain the composite representation vector. The output layer is a fully connected classification layer. The fully connected classification layer takes the comprehensive representation vector as input and maps the comprehensive representation vector to an output dimension equal to the number of airway obstruction severity levels through a weight matrix. The value of each output dimension corresponds to the prediction confidence of the target object belonging to each airway obstruction severity level. The prediction result of the airway obstruction severity of the target object is determined based on each prediction confidence.

[0059] In this embodiment, the system obtains a first fused feature vector with dimension C and elements numbered sequentially from the 1st to the Cth; simultaneously, it obtains a text feature vector with dimension D and elements numbered sequentially from the 1st to the Dth. During the concatenation operation, the system creates a new comprehensive concatenation vector with dimension C+D; the system copies all elements of the first fused feature vector to the starting position of the comprehensive concatenation vector in their original order, i.e., positions 1 to C are filled with the elements of the first fused feature vector sequentially; then, the system copies all elements of the text feature vector to subsequent positions in their original order, i.e., positions C+1 to C+D are filled with the elements of the text feature vector sequentially; after concatenation, the first half of the comprehensive concatenation vector represents the temporal-structured fused features, and the second half represents the text features.

[0060] The synthesized vector is input to the second fully connected layer. This second fully connected layer has a weight matrix W2 with rows equal to the preset output dimension E and columns equal to the input dimension C+D, and a bias vector b2 with a length of E. The system performs a linear transformation: calculating the product of W2 and the synthesized vector, adding b2, to obtain a linear output vector of length E. Specifically, for the j-th node in the output dimension, the system performs a dot product of all elements in the j-th row of W2 containing C+D weights and the synthesized vector, and adds the j-th component of b2 to obtain the value of that node. After completing the calculation for all E nodes, a linear output vector is obtained. Subsequently, the system applies an activation function, such as the Tanh function, with an output range of -1 to 1 to this linear output vector, calculating element by element to obtain a synthesized representation vector. This synthesized representation vector has a dimension of E, and its value contains fusion information from the three types of modal data. Through the second-stage fully connected layer, the first fusion feature, which already contains temporal and structured information, interacts and combines with textual features at a higher level to form the final comprehensive feature representation used for prediction.

[0061] The output layer is a fully connected classification layer, and its input is a comprehensive representation vector with dimension E. The fully connected classification layer maintains a weight matrix W_out, with the number of rows equal to the number of preset airway obstruction severity levels, denoted as N, and the number of columns equal to the input dimension E. It also maintains a bias vector b_out with a length of N.

[0062] The system inputs the comprehensive representation vector into the fully connected classification layer and first performs a linear transformation: calculating the product of W_out and the comprehensive representation vector to obtain an original score vector of length N, where each score corresponds to a severity level; specifically, for the r-th level, the system performs a dot product of the r-th row of W_out (containing E weights) with all elements of the comprehensive representation vector, and adds the r-th component of b_out to obtain the original score for that level; after completing the calculation for all N levels, the original score vector is obtained, where the value of each element can be positive or negative, representing the unnormalized intensity of the comprehensive representation vector mapped to each level space.

[0063] The system then inputs the original score vector into the Softmax activation function. The Softmax function converts each original score into a probability value between 0 and 1, and the sum of the probabilities of all levels is 1. The conversion process is as follows: for the r-th level, calculate the exponent of its original score, divide it by the sum of the exponents of the original scores of all levels, and obtain the probability of that level. This probability represents the prediction confidence that the severity of the airway obstruction of the target object belongs to that level.

[0064] The system determines the final prediction result based on the probability values ​​of all N levels: it identifies the level index with the highest probability value and uses that level as the predicted severity level of airway obstruction; if multiple levels have the same maximum probability of occurring very rarely, they can be selected according to a preset level priority, such as from severe to mild or randomly; finally, the system outputs the level identifier, such as the level name or code, as the predicted severity of airway obstruction for the target object; this result can be directly used for clinical decision support, providing medical staff with an objective reference for severity assessment.

[0065] According to a second embodiment of the present invention, the present invention claims protection for an airway obstruction injury prediction system based on multimodal Transformer and feature selection, comprising: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the airway obstruction injury prediction method based on multimodal Transformer and feature selection.

[0066] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for predicting airway obstruction injury based on multimodal Transformer and feature selection, characterized in that, include: S1, acquire multi-dimensional medical data of the target object, including time-series vital sign data, structured test data, and initial diagnosis text records; S2, the temporal vital signs data is input into the fully convolutional network-squeeze excitation module, the local temporal patterns in the temporal vital signs data are extracted through one-dimensional convolution operation, and the convolution output is recalibrated at the channel level by the squeeze excitation mechanism to obtain the temporal feature vector; S3, the structured detection data is input into the multilayer perceptron module, and the detection indicators in the structured detection data are nonlinearly transformed through the multilayer fully connected network to extract the correlation features between the indicators and obtain the structured feature vector; S4, the initial diagnosis text record is input into the encoder part of the Transformer module, and the semantic dependency relationship between words at different positions in the initial diagnosis text record is captured through the self-attention mechanism, and the text semantic features are extracted to obtain the text feature vector; S5, cross-fuse the temporal feature vector with the structured feature vector to obtain the first fused feature, and then concatenate and fuse the first fused feature with the text feature vector to obtain the comprehensive representation vector; S6. Input the comprehensive representation vector into the output layer to obtain the predicted result of the airway obstruction severity of the target object.

2. The method according to claim 1, characterized in that, In step S5, the temporal feature vector and the structured feature vector are cross-fused to obtain a first fused feature, including: The temporal feature vector and the structured feature vector are concatenated to form a concatenated vector; The spliced ​​vector is input into the first fully connected layer, and the spliced ​​vector is subjected to linear transformation and nonlinear activation through the first fully connected layer to obtain the first fused feature.

3. The method according to claim 2, characterized in that, In step S5, the first fusion feature is then concatenated and fused with the text feature vector to obtain a comprehensive representation vector, including: The first fused feature is concatenated with the text feature vector to form a comprehensive concatenated vector; The synthesized splicing vector is input into the second fully connected layer, and the synthesized splicing vector is subjected to linear transformation and nonlinear activation through the second fully connected layer to obtain the synthesized representation vector.

4. The method according to claim 1, characterized in that, The fully convolutional network-squeeze excitation module includes multiple convolutional sub-modules connected in sequence, and each convolutional sub-module includes a one-dimensional convolutional layer and a squeeze excitation block. The one-dimensional convolutional layer takes the temporal vital signs data or the output of the previous convolutional submodule as input, and performs convolution operations by sliding the convolutional kernel along the time dimension to generate the corresponding convolutional feature map. The squeeze excitation block performs a squeeze operation on the convolutional feature map and generates a global description vector in the channel dimension by performing global feature aggregation on the convolutional feature map in the time dimension. The squeeze excitation block performs an excitation operation on the global description vector, generates recalibration weights for each channel through the description values ​​of each channel in the global description vector, and applies each recalibration weight to the features of the corresponding channel in the convolutional feature map to obtain the recalibrated channel features. The recalibrated channel features output by each of the convolutional sub-modules are processed sequentially by subsequent convolutional sub-modules, and the temporal feature vector is output by the last convolutional sub-module.

5. The method according to claim 1, characterized in that, The multilayer perceptron module includes a first fully connected layer, a second fully connected layer, and a third fully connected layer connected in sequence. The number of neurons in the first fully connected layer is greater than the number of neurons in the second fully connected layer, and the number of neurons in the second fully connected layer is greater than the number of neurons in the third fully connected layer. The structured detection data is sequentially transformed nonlinearly through the first fully connected layer, the second fully connected layer, and the third fully connected layer. Each fully connected layer performs a linear transformation on the input and outputs it to the next layer after passing through an activation function. The third fully connected layer outputs the structured feature vector.

6. The method according to claim 1, characterized in that, The encoder portion of the Transformer module includes multiple self-attention layers connected in sequence; Each self-attention layer performs multi-head self-attention operation on the input sequence data. Multiple attention heads calculate the attention weights between different positions in the input sequence in parallel. Each attention head performs weighted aggregation on the input sequence based on its own calculated attention weights to obtain the weighted output of each attention head. The weighted outputs of all attention heads are concatenated and linearly transformed to obtain the output of the self-attention layer, which is then passed to the next self-attention layer. The output of the last self-attention layer is aggregated along the position dimension to obtain the text feature vector.

7. The method according to claim 4, characterized in that, The time-series vital sign data is time-series data composed of vital sign measurements at multiple time points, and the vital sign measurements at each time point include heart rate and respiratory rate measurements. The one-dimensional convolution operation uses a convolution kernel with a set stride to perform sliding convolution along the time dimension of the time series data. Each time the kernel slides, it covers the vital sign measurement values ​​within a local time window in the time series data, performs weighted summation, and processes them through an activation function to generate the convolution result corresponding to that local time window. The convolution results of all local time windows constitute the convolution feature map. The squeezing excitation module performs global feature aggregation on the convolutional feature map in the time dimension, and performs aggregation operation on the feature values ​​of each channel in the convolutional feature map at all time steps to obtain the global description value of each channel. The global description values ​​of all channels constitute the global description vector. The squeezing excitation module generates the recalibration weights for each channel by nonlinearly transforming the global description vector, and multiplies each recalibration weight with the feature value of the corresponding channel at each time step in the convolutional feature map to obtain the recalibrated channel features.

8. The method according to claim 2, characterized in that, The step of concatenating the temporal feature vector with the structured feature vector includes sequentially combining the elements of each dimension of the temporal feature vector with the elements of each dimension of the structured feature vector to form a concatenated vector whose dimension number is the sum of the dimension number of the temporal feature vector and the dimension number of the structured feature vector. When the first fully connected layer performs a linear transformation on the concatenated vector, it takes the elements of each dimension of the concatenated vector as input, maps the concatenated vector to the target dimension space through a weight matrix, and then processes it through an activation function to obtain the first fused feature.

9. The method according to claim 3, characterized in that, The step of concatenating the first fusion feature with the text feature vector includes sequentially combining the elements of each dimension of the first fusion feature with the elements of each dimension of the text feature vector to form a comprehensive concatenated vector whose number of dimensions is the sum of the number of dimensions of the first fusion feature and the number of dimensions of the text feature vector. When the second fully connected layer performs a linear transformation on the composite splicing vector, it takes the elements of each dimension of the composite splicing vector as input, maps the composite splicing vector to the target dimension space through the weight matrix, and then processes it through the activation function to obtain the composite representation vector. The output layer is a fully connected classification layer. The fully connected classification layer takes the comprehensive representation vector as input and maps the comprehensive representation vector to an output dimension equal to the number of airway obstruction severity levels through a weight matrix. The value of each output dimension corresponds to the prediction confidence of the target object belonging to each airway obstruction severity level. The prediction result of the airway obstruction severity of the target object is determined based on each prediction confidence.

10. An airway obstruction injury prediction system based on multimodal Transformer and feature selection, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the airway obstruction injury prediction method based on multimodal Transformer and feature selection according to any one of claims 1 to 9.

Images