Intelligent decision support system for emergency based on multi-modal medical health data

Abstract

The application discloses a sudden event intelligent decision support system based on multi-modal medical and health data and belongs to the technical field of medical information processing. The system comprises the following modules: a data fusion module, which performs heterogeneous adaptive inference processing on electronic medical record text data, vital sign time series numerical data and medical image data, and generates a unified structured data tensor; a decoupling analysis module, which performs entropy weight method-based time series and spatial domain coupling feature decoupling analysis on the unified structured data tensor, and generates independent feature components and cross feature components after decoupling; an event graph module, which maps the independent feature components and the cross feature components to a preset clinical event ontology network, and generates a dynamically weighted event correlation representation graph; and a decision deduction module, which performs a hybrid deduction operation based on Monte Carlo tree search and deep Q network on the event correlation representation graph, and generates an optimal emergency decision sequence and a resource scheduling list.

Description

Technical Field

[0001] This invention relates to the field of medical information processing technology, specifically to an intelligent decision support system for emergencies based on multimodal medical and health data. Background Technology

[0002] In public health emergencies or hospital emergency scenarios, decision-making requires the integration of multiple modalities of data, including electronic medical record text, time-series vital signs data, and medical images. Existing emergency decision support systems typically construct decision models using single-modal data analysis or simple multi-source data stitching. For multi-scale information such as unstructured text descriptions in electronic medical records, continuous time-series monitored vital signs waveforms, and tomographic images, the mainstream approach is to extract features separately and directly input them into classifiers or rule engines, lacking mechanisms for fine-grained alignment and joint representation of cross-modal spatiotemporal relationships.

[0003] Existing solutions of this kind have significant shortcomings when dealing with highly dynamic and strongly coupled medical data. First, vital signs and imaging features exhibit complex evolutionary patterns along the time dimension, while the descriptions of illnesses in textual records are intertwined with these temporal and spatial features. Existing methods fail to effectively decouple the coupled features under the constraints of time series dependence and spatial structure, resulting in state representations used for decision-making containing redundant and interfering components, failing to accurately reflect the dominant factors and inter-factor relationships in the evolution of injuries or illnesses. Second, the decision generation process often relies on predefined emergency procedure libraries or static reinforcement learning strategies. When the actual execution results deviate from the predictions, the system lacks the ability to adjust the strength of inter-event relationships online based on feedback signals, and cannot re-optimize decision paths in dynamically changing environments, causing emergency solutions to gradually become ineffective in the continuously evolving situation. Summary of the Invention

[0004] The technical problems to be solved by this invention include: the high coupling of temporal and spatial domain features in multimodal medical and health data makes it difficult for existing fusion methods to separate the independent features of the dominant symptom evolution and the cross-interaction features between modalities, resulting in the distortion of event state expression; the decision graph of sudden events relies on fixed weights and cannot be corrected online and restarted according to the real-time feedback of decision execution, making it unsuitable for dynamically changing emergency environments.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides an intelligent decision support system for emergencies based on multimodal medical and health data. This system includes a data fusion module, a decoupling analysis module, an event graph module, a decision inference module, and a self-feedback iteration module. The data fusion module receives heterogeneous data sources from medical institutions, including electronic medical record text data, vital sign time-series numerical data, and medical image data. It performs heterogeneous adaptive inference processing on the heterogeneous data sources to generate a unified structured data tensor, thereby eliminating semantic gaps and scale differences between different modalities and providing a consistent data foundation for subsequent analysis.

[0006] Preferably, the data fusion module extracts semantic feature vectors from electronic medical record text data, waveform feature vectors from vital sign time-series numerical data, and texture and morphological feature vectors from medical image data; performs spatiotemporal alignment processing on the above feature vectors, mapping them to a unified time stamp dimension; constructs a learnable heterogeneous attention fusion network, inputs the aligned feature vectors into the network, calculates the cross-attention weights between each modality of data through a multi-head attention mechanism; and performs a weighted summation of the feature vectors of each modality of data according to the cross-attention weights to generate the unified structured data tensor, so that the fused tensor can retain the unique information of each modality and fully express the complementary and correlated relationships between modalities.

[0007] The decoupling analysis module performs temporal and spatial domain coupling feature decoupling analysis on the unified structured data tensor based on the entropy weight method, generating decoupled independent feature components and cross feature components, thereby separating the complex coupled spatiotemporal pattern into core features that can be interpreted independently and cross features that reflect the interaction, improving the accuracy of subsequent event correlation analysis.

[0008] As a technical solution of the present invention, the decoupling analysis module performs a sliding window slicing operation on the unified structured data tensor along the time axis to obtain multiple time slice sub-tensors; calculates the local entropy value in the spatial domain for each time slice sub-tensor to generate an entropy distribution matrix for each time slice sub-tensor; based on the entropy distribution matrix, extracts the principal component with the largest information content in each time slice sub-tensor as the independent feature component using the principal component analysis algorithm; calculates the correlation matrix for the high-order residual terms remaining after deducting the independent feature components in each time slice sub-tensor, and identifies the combination of residual terms with a correlation exceeding a preset threshold as the cross feature component. This achieves robust decoupling guided by information content measurement, avoiding deviations caused by manually set parameters.

[0009] The event graph module maps the independent and cross-feature components to a preset clinical event ontology network, generating a dynamically weighted event association representation graph. This transforms real-time multimodal data into an event network with clear clinical semantics, allowing the evolution of sudden events to be presented in the form of a structured graph.

[0010] Preferably, the event graph module pre-stores a clinical event ontology network containing nodes and edges, where nodes represent clinical event types and edges represent the relationships between events; it calculates the semantic matching degree between independent feature components and cross feature components and the event type defined for each node; it uses the component with the highest semantic matching degree as the event state input for the corresponding node and updates the state attribute value of that node; based on the relationship strength value in the cross feature components, it updates the weight attribute value of the edges connecting different nodes, generating the dynamically weighted event association representation graph, thereby ensuring that the graph can dynamically reflect the current patient state and adaptively reflect changes in the intensity of influence between events.

[0011] The decision extrapolation module performs a hybrid extrapolation operation based on Monte Carlo tree search and deep Q-network on the event association representation map, generating the optimal emergency decision sequence and the corresponding resource scheduling list. This hybrid extrapolation operation deeply integrates the long-term planning capability of Monte Carlo tree search with the evaluation capability of deep Q-network, enabling the rapid identification of high-value decision paths on large-scale, high-uncertainty clinical event maps.

[0012] Furthermore, the decision-making module uses the current dynamically weighted event association representation graph as the state node for Monte Carlo tree search. During each Monte Carlo tree search simulation phase, a pre-trained deep Q-network is invoked. This deep Q-network takes the current state node and a set of candidate actions as input, outputs the state-action value of each candidate action, and selects an action on the simulation path based on this state-action value. In the expansion phase, child nodes are added to the event association representation graph based on the actions and state transition results on the simulation path, and the number of visits and cumulative reward values ​​of the child nodes are recorded. In the backpropagation phase, the reward value at the end of the simulation is backpropagated along the search path, updating the number of visits and cumulative reward values ​​of all nodes on the path. When the search reaches the preset computational budget, the optimal action is selected based on the number of visits and cumulative reward values ​​of the root node's child nodes, and the optimal emergency decision sequence and resource scheduling list are generated based on the state change corresponding to the optimal action. This achieves online real-time simulation and dynamic decision optimization, significantly improving the timeliness and rationality of emergency response plans.

[0013] The self-feedback iteration module is used to receive feedback time-series data after executing the optimal emergency decision sequence, update the weights of the dynamically weighted event association representation graph online based on the feedback time-series data, and trigger the decision inference module to re-execute the inference, so that the system forms a closed-loop self-learning mechanism and continuously adapts to changes in the situation and execution deviations during the emergency response process.

[0014] In its implementation, the self-feedback iteration module monitors and collects the medical staff response records, equipment usage records, and bed occupancy records generated by the medical system after executing the resource scheduling list, using these as the feedback time-series data. This feedback time-series data is then input into a time-series residual analysis model, which calculates the residual vector between the feedback time-series data and the ideal state sequence predicted by the decision-making inference module during the inference process. A weight adjustment gradient is generated based on the residual vector to indicate which edges in the event association representation graph require positive or negative weight adjustments. The time-series residual analysis model preferably employs a long short-term memory network structure and decomposes the residual vector into system deviation components and random noise components. The weight adjustment gradient is generated only based on the system deviation components, thereby avoiding random disturbances from misleading the graph update and improving the stability of the feedback control.

[0015] Preferably, the self-feedback iteration module calculates the weight update amount for each edge in the event association representation graph based on the weight adjustment gradient; it uses an online stochastic gradient descent optimizer to superimpose the weight update amount onto the current weight value of the corresponding edge to generate an updated weight value; it performs a clamping operation between the updated weight value and preset upper and lower weight limits to ensure that the weight value does not exceed the limits; and it replaces the original weight value with the clamped weight value to complete the online update. After completing the online update of the graph, the self-feedback iteration module generates a deduction restart signal and sends the deduction restart signal and the updated graph to the decision deduction module; upon receiving the deduction restart signal, it clears the current search tree of the Monte Carlo tree search and uses the updated graph as the new initial state node to restart the hybrid deduction operation, thereby realizing a continuous cycle of "perception-decision-feedback-adjustment" to ensure that the system maintains its decision advantage in a dynamically changing environment of sudden events.

[0016] In a preferred embodiment of the present invention, the system further includes a decision result visualization and interaction module, used to receive the optimal emergency decision sequence and resource scheduling list; to perform structured rendering of the optimal emergency decision sequence according to the timeline, generating a Gantt chart containing a timeline, decision nodes, and execution branches; to overlay and display the current status and scheduling path identifier of each resource in the resource scheduling list on the Gantt chart; and to generate a modified decision version in response to the user's drag-and-drop or modification operation on any decision node in the Gantt chart, and to resubmit the modified version as input to the decision deduction module for deduction and verification, thereby organically integrating human domain experience with the system's intelligent decision-making, significantly enhancing the interpretability and flexible adjustment capability of the solution.

[0017] The technical effects and advantages provided by the present invention in the above technical solution are as follows: To address the issue of highly coupled temporal and spatial domain features in multimodal healthcare data, a decoupling analysis module is used to perform entropy-weighted decoupling analysis on the temporal and spatial domain coupling features of the unified structured data tensor. Local entropy values ​​in the spatial domain are calculated for sub-tensors sliced ​​along the time axis, and principal component analysis is used to extract the principal components with the highest information content as independent feature components. Simultaneously, correlation matrices are calculated for higher-order residuals after deducting independent features, and combinations of residuals with correlations exceeding a preset threshold are identified as cross-feature components. This processing method decouples the mixed state obtained from electronic medical records, vital sign waveforms, and images into independent features reflecting the dominant trend of a single modality and cross-features characterizing the nonlinear correlation between modalities. This avoids the dominant signal being submerged by coupling noise, enabling the state components in subsequent input event maps to correspond to the evolution trajectory of major lesions and complication or multi-organ correlation patterns, thereby obtaining a clearer and more attributable situational awareness in the early stages of an emergency.

[0018] To address the issue of the inability to dynamically adjust the emergency decision-making graph based on execution feedback, a self-feedback iteration module collects medical staff response records, equipment usage records, and bed occupancy records generated after executing the optimal emergency decision sequence as feedback time-series data. A time-series residual analysis model is then used to calculate the residual vector between the feedback time-series data and the ideal state sequence predicted by the decision-making simulation module. Based on the system bias component in the residual, a weight adjustment gradient is generated, and the weights of each edge in the dynamically weighted event association representation graph are updated online using stochastic gradient descent. After the weight update, the Monte Carlo tree search tree is cleared, and the updated graph is used as the new initial state to restart the hybrid simulation operation. This closed-loop mechanism allows the correlation strength between different clinical events to be continuously adjusted based on the actual execution effect, avoiding the problem of initial weights becoming invalid in a dynamic environment. This enables subsequent simulations to inherit experience adjustments from the actual treatment process, continuously optimizing the timeliness and relevance of emergency decision sequences and resource scheduling schemes without interrupting system operation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0020] Figure 1 This is a schematic diagram of the structure of an intelligent decision support system for emergencies based on multimodal medical and health data; Figure 2 This is a flowchart of the decoupling analysis of temporal and spatial domain coupling characteristics based on the entropy weight method; Figure 3 This is a flowchart of the online update and inference interaction of event graph weights based on self-feedback iterative temporal residual analysis; Figure 4 It is the curve showing the change of eigenvalues ​​of multimodal eigenvectors and unified structured data tensors over time. Figure 5 This is a graph showing the entropy values ​​and correlation analysis of the independent and cross-feature components of the time slice tensor; Figure 6 This is a graph showing the change in semantic matching degree of clinical event types over time slices. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See Figure 1This invention provides an intelligent decision support system for emergencies based on multimodal medical and health data, comprising a data fusion module, a decoupling analysis module, an event graph module, a decision inference module, and a self-feedback iteration module. The data fusion module receives three heterogeneous data sources: electronic medical record text data, vital sign time-series numerical data, and medical image data from medical institutions. It performs heterogeneous adaptive inference processing on these heterogeneous data sources to generate a unified structured data tensor. The decoupling analysis module performs entropy-weighted decoupling analysis on the unified structured data tensor to decouple temporal and spatial domain coupling features, generating decoupled independent and cross-feature components. The event graph module maps the independent and cross-feature components to a preset clinical event ontology network, generating a dynamically weighted event association representation graph. The decision inference module performs a hybrid inference operation based on Monte Carlo tree search and deep Q-networks on the event association representation graph to generate an optimal emergency decision sequence and a corresponding resource scheduling list. The self-feedback iteration module is used to receive feedback time-series data after executing the optimal emergency decision sequence, update the weights of the dynamically weighted event association representation graph online based on the feedback time-series data, and trigger the decision inference module to re-execute the inference.

[0023] Example 1:

[0024] In specific implementation, the data fusion module performs heterogeneous adaptive inference processing on multi-source data fusion of electronic medical record text data, vital sign time-series numerical data, and medical image data to generate a unified structured data tensor. The process of extracting semantic feature vectors from electronic medical record text data is as follows: A pre-trained clinical language model is used. The pre-trained clinical language model is a bidirectional encoder representation model based on the Transformer architecture. The bidirectional encoder representation model based on the Transformer architecture contains 12 Transformer encoding layers, each with 12 heads of the multi-head self-attention mechanism and a hidden layer dimension of 768. The bidirectional encoder representation model based on the Transformer architecture is pre-trained on a large-scale clinical text corpus through masked language model tasks and next sentence prediction tasks. When extracting semantic feature vectors from electronic medical record text data, the electronic medical record text data is first segmented into words. The segmented electronic medical record text data is converted into a word sequence. A classification marker word is added at the beginning of the word sequence, and a separator marker word is added at the end of the word sequence to form the input word sequence. The input word sequence is input into the pre-trained clinical language model, and the hidden state vector corresponding to the classification tag word output by the last Transformer encoding layer of the pre-trained clinical language model is taken as the semantic feature vector of the electronic medical record text data. The semantic feature vector has a dimension of 768.

[0025] In specific implementation, the process of extracting waveform feature vectors from vital sign time-series numerical data is as follows: A waveform feature extraction network is constructed, consisting of a one-dimensional convolutional neural network and a long short-term memory network connected in series. The first part of the waveform feature extraction network consists of three one-dimensional convolutional layers. The first one-dimensional convolutional layer has a kernel size of 7, 64 output channels, and a stride of 1; the second one-dimensional convolutional layer has a kernel size of 5, 128 output channels, and a stride of 1; the third one-dimensional convolutional layer has a kernel size of 3, 128 output channels, and a stride of 1. Each one-dimensional convolutional layer is followed by a batch normalization layer and a ReLU activation function layer. The second part of the waveform feature extraction network consists of two stacked long short-term memory networks. Each long short-term memory network has 128 hidden units, and a random deactivation operation is used between the two long short-term memory networks, with a random deactivation probability set to 0.2. The time-series numerical data of vital signs are arranged in chronological order to form a one-dimensional input sequence. The one-dimensional input sequence is then fed into a waveform feature extraction network. After three layers of one-dimensional convolutional layers, local waveform patterns are extracted to obtain a convolutional feature sequence. The convolutional feature sequence is then fed into a two-layer stacked long short-term memory network. The hidden state vector of the last time step of the last layer of the long short-term memory network is taken as the waveform feature vector of the time-series numerical data of vital signs. The waveform feature vector has a dimension of 128.

[0026] In specific implementation, the process of extracting texture and morphological feature vectors from medical image data is as follows. A medical image encoder based on a residual network is adopted. The medical image encoder based on a residual network is a ResNet-50 structure. The ResNet-50 structure includes an initial convolutional layer and four residual stages. The four residual stages contain 3, 4, 6, and 3 residual blocks, respectively. Each residual block is composed of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer stacked sequentially. An identity skip connection is established between the input and output of the residual block. The medical image data is adjusted to a uniform size of 224×224 pixels, and the pixel values ​​of the medical image data are normalized to the [0,1] interval. The adjusted and normalized medical image data is input into the medical image encoder based on the residual network. After passing through four residual stages of the ResNet-50 structure, a feature map is obtained. Global average pooling is performed on the feature map to obtain a 2048-dimensional vector. Then, a fully connected layer is used to map the 2048-dimensional vector to a 256-dimensional vector. The 256-dimensional vector is used as the texture and morphological feature vector of the medical image data.

[0027] In specific implementation, the semantic feature vector, waveform feature vector, and texture and morphology feature vector undergo spatiotemporal alignment processing. The process of mapping these three types of feature vectors to a unified time stamp dimension is as follows: Using the time sampling points of vital sign time-series numerical data as the alignment reference time series, the time sampling frequency of the vital sign time-series numerical data is once per minute, and each time sampling point corresponds to a unified timestamp. For the semantic feature vector of electronic medical record text data, the record timestamp corresponding to each record in the electronic medical record text data is extracted. The record timestamp corresponding to each record is matched with the unified timestamp of the vital sign time-series numerical data. If the absolute time difference between the record timestamp and a certain unified timestamp is less than 30 seconds, the corresponding semantic feature vector is directly assigned to the unified timestamp. If the record timestamp is located between two adjacent unified timestamps and the absolute time difference with both adjacent unified timestamps is greater than or equal to 30 seconds, a forward padding method is used to assign the corresponding semantic feature vector to the earlier unified timestamp. For texture and morphological feature vectors of medical image data, the acquisition timestamp of the medical image data is extracted. This acquisition timestamp is then matched to the nearest unified timestamp. Using the nearest neighbor allocation principle, the texture and morphological feature vectors are assigned to the unified timestamp with the smallest absolute time difference from the acquisition timestamp. If two unified timestamps have the same absolute time difference, the texture and morphological feature vector is assigned to the earlier unified timestamp. After alignment, each unified timestamp has one semantic feature vector, one waveform feature vector, and one texture and morphological feature vector. For unified timestamps without a corresponding original data, zero vectors are used for padding.

[0028] In specific implementation, the process of constructing a learnable heterogeneous attention fusion network is as follows. The learnable heterogeneous attention fusion network includes three modality-specific linear projection layers: a semantic modality linear projection layer, a waveform modality linear projection layer, and an image modality linear projection layer. These three layers map the semantic feature vector, waveform feature vector, and texture and morphology feature vector to a 128-dimensional unified embedding space, respectively. The network also includes a multi-head cross-attention module with four attention heads. After inputting the aligned semantic feature vector, waveform feature vector, and texture and morphology feature vector into the heterogeneous attention fusion network, the process of calculating the cross-attention weights between the modal data using the multi-head attention mechanism is as follows: The aligned semantic feature vector, waveform feature vector, and texture and morphology feature vector are passed through their respective modality-specific linear projection layers to obtain the semantic modality embedding vector, waveform modality embedding vector, and image modality embedding vector. The embedding vectors of the three modalities are paired in pairs to form three modal pairings: semantic modality and waveform modality, semantic modality and image modality, and waveform modality and image modality. For each modal pairing, the embedding vector of one modality is used as the query input, and the embedding vector of the other modality is used as the key and value input, respectively, and input to the multi-head cross-attention module. In each attention head of the multi-head cross-attention module, a linear transformation is performed on the query input to obtain the query matrix. Perform a linear transformation on the key input to obtain the key matrix. Perform a linear transformation on the input values ​​to obtain the value matrix. The linear transformations are all implemented using learnable weight matrices, with each attention head having its own independent learnable weight matrix. For attention head index, The values ​​are integers from 1 to 4. A total of 12 independent attention head calculations are performed for the three modality pairings. Within each attention head, the cross-attention weights are calculated using a scaled dot product attention mechanism.

[0029] in, Indicates the first The query matrix is ​​in each attention head, with dimension [missing information]. , The length of the query sequence, For each attention head, the feature dimensions, The value is 32; Indicates the first Each attention head has a key matrix with dimension [ ]. , The length of the key sequence; Indicates the first The median matrix of each attention head has dimensions of . ; Key matrix The transpose of the matrix; This represents the scaling factor, with a value of [value missing]. ; This represents the softmax normalization function applied row by row; Indicates the first The cross-attention weighted value output by each attention head, with dimension [missing information]. The outputs of all attention heads are concatenated along the feature dimension to obtain a concatenated cross-attention representation. This concatenated cross-attention representation is then passed through a learnable output projection matrix. Perform a linear transformation to output the cross-attention weights, and the output projection matrix. The dimension is ,in The value is 128. For the three modality pairings, the above process is followed to calculate three sets of cross-attention weights.

[0030] In specific implementation, the process of generating a unified structured data tensor by weighting and summing the feature vectors of each modality based on the cross-attention weights is as follows: For each unified timestamp, the semantic modality embedding vector, waveform modality embedding vector, and image modality embedding vector, as well as three sets of cross-attention weights, are obtained. The semantic modality embedding vector is weighted according to the cross-attention weights obtained by pairing semantic modality with waveform modality and cross-attention weights obtained by pairing semantic modality with image modality to obtain the enhanced semantic feature representation; the waveform modality embedding vector is weighted according to the cross-attention weights obtained by pairing semantic modality with waveform modality and cross-attention weights obtained by pairing waveform modality with image modality to obtain the enhanced waveform feature representation; the image modality embedding vector is weighted according to the cross-attention weights obtained by pairing semantic modality with image modality and cross-attention weights obtained by pairing waveform modality with image modality to obtain the enhanced image feature representation. The enhanced semantic feature representation, enhanced waveform feature representation, and enhanced image feature representation are added element-wise to obtain a fused feature vector. All fused feature vectors with the same timestamp are then stacked along the time axis to form a unified structured data tensor. The dimension of the unified structured data tensor is [missing information]. ,in This indicates the total number of unified timestamps.

[0031] In specific implementation, the learnable heterogeneous attention fusion network needs to be pre-trained. The pre-training process is as follows: A trimodal alignment training dataset is constructed, containing electronic medical record text data, vital sign time-series numerical data, and medical image data. Each training sample in the trimodal alignment training dataset contains an electronic medical record text fragment, vital sign time series, and medical image of the same patient within the same time period, with alignment annotations in the time dimension. Pre-training adopts a reconstruction self-supervised task. The trimodal alignment training dataset is input into the learnable heterogeneous attention fusion network to generate a unified structured data tensor. Then, three modality-specific decoders are added to the unified structured data tensor: a text decoder, a time-series decoder, and an image decoder. The text decoder is used to reconstruct the semantic feature vector of the original electronic medical record text from the unified structured data tensor. The time-series decoder is used to reconstruct the waveform feature vector of the original vital sign time-series numerical data from the unified structured data tensor. The image decoder is used to reconstruct the texture and morphological feature vector of the original medical image data from the unified structured data tensor. The pre-training loss function is a weighted sum of the reconstruction errors of the three modalities. Mean squared error (MSE) is used for text reconstruction errors, time-series reconstruction errors, and image reconstruction errors, with each modality's weight set to 1.0. The Adam optimizer is used for parameter optimization, with a learning rate of 0.0001 and a batch size of 32. Pre-training is performed for 100 epochs until the loss function converges. After pre-training, the three modality-specific decoders are removed, while the parameters of the learnable heterogeneous attention fusion network are retained for subsequent inference.

[0032] See Figure 4 In the graph, the horizontal axis represents the unified timestamp sequence number, ranging from 0 to 600, and the vertical axis represents the eigenvalues ​​of each feature vector, ranging from approximately -0.2 to 0.8. The legend indicates the eigenvalue trends of the four curves corresponding to the semantic feature vector (10th dimension), waveform feature vector (50th dimension), texture and morphology feature vector (200th dimension), and unified structured data tensor (60th dimension), respectively. Each curve represents a sampled value at discrete time points and is distinguished by different line types and colors.

[0033] Specifically, the blue solid line represents the 10th dimension of the semantic feature vector. Its feature value exhibits significant fluctuations over time, with strong periodicity and alternating peaks and troughs. The maximum value is close to 0.6, and the minimum is close to -0.2, indicating that the semantic features of the electronic medical record text data have relatively obvious dynamic changes over time. The orange dashed line represents the 50th dimension of the waveform feature vector. The feature value varies considerably, reaching a maximum of approximately 0.75 and a minimum close to -0.1. Moreover, the fluctuation frequency is slightly different from that of the semantic features, showing that the waveform features of the vital signs time-series numerical data have a unique temporal pattern.

[0034] The green dotted line represents the 200th dimension of the texture and morphology feature vector. The curve shows equally dramatic changes, with clear peak-to-valley transitions. The maximum eigenvalue is approximately 0.75, and the minimum is close to zero, reflecting the non-stationary and dynamic changes in the texture and morphology features of medical image data over time. The red dotted line represents the 60th dimension of the unified structured data tensor. As the output of multimodal fusion, its eigenvalue variation trend lies between the three types of features mentioned above, with a maximum value of approximately 0.5 and a minimum value of approximately 0. The curve is relatively smooth, indicating that the fused feature tensor achieves effective fusion and balance of various modal features, mitigating the dramatic fluctuations of individual modal features.

[0035] Example 2:

[0036] In specific implementation, please refer to Figure 2 The decoupling analysis module performs temporal and spatial domain coupling feature decoupling analysis on the unified structured data tensor based on the entropy weight method, generating decoupled independent feature components and cross-feature components. The process of obtaining multiple time-slice sub-tensors by performing a sliding window slicing operation along the time axis on the unified structured data tensor is as follows. The dimension of the unified structured data tensor is... ,in This represents the total number of unified timestamps. This represents the dimension of the fused feature vector at each timestamp. It also sets the window width of the sliding window. For a fixed value, The value is 12, which sets the sliding step size of the sliding window. For a fixed value, The value is set to 4. The sliding window is moved from the beginning of the time axis of the unified structured data tensor. For the _th ... A sliding window is used to extract data from a unified structured data tensor, starting from the time index. Time Index All rows form a time slice subtensor, the first... The dimension of the tensor of each time slice is ,in The value is a positive integer. If the remaining time step is insufficient during the sliding window capture process... The remaining portion is then discarded. Through the sliding window slicing operation, a total of [number] portions were obtained. Tensor of a time slice The calculation method is as follows ,in This indicates the floor function. This represents the total number of time slice tensors obtained in the final result.

[0037] In practice, the process of calculating the local entropy value in the spatial domain for each time slice sub-tensor and generating the entropy distribution matrix for each time slice sub-tensor is as follows. For the th The tensor of the nth time slice, the nth The dimension of the tensor of each time slice is , will the The time slice tensor can be viewed as a two-dimensional matrix, where the row index corresponds to the first time slice within the time window. Each time step The value range is 1 to The column index of the two-dimensional matrix corresponds to the first eigenvector of the fused feature vector. Each feature dimension The value range is 1 to For the first The position of each element in the time slice tensor Defined by element position The local neighborhood centered on the center is a rectangular region, and the row index range of the rectangular region is from... arrive The column index range of the rectangular area is from arrive ,in The local neighborhood radius, The value of is 2, for values ​​exceeding the first . The boundaries of the tensors in each time slice are processed using mirror filling. The values ​​of all elements within the local neighborhood are extracted to form a neighborhood value set, and the range of element values ​​in the neighborhood value set is evenly divided into... Each interval The value of is 10. The number of elements falling into each interval in the neighborhood value set is counted to obtain a frequency distribution histogram. The th element in the frequency distribution histogram... The frequency of each interval is denoted as , The value range is 1 to Calculating element positions based on frequency distribution histograms Local entropy at Local entropy The calculation formula is:

[0038] in, Indicates the first The time slice tensor is located at the time step index. Feature Dimension Index The local entropy value at that point, The index of the time slice subtensor, with values ​​ranging from 1 to... Integers; This is the time step index, with values ​​ranging from 1 to... Integers; This is the feature dimension index, with values ​​ranging from 1 to... Integers; This represents the index of the interval in the frequency distribution histogram, with values ​​ranging from 1 to... Integers; This indicates that the value of an element in the local neighborhood falls within the first... The frequency of each interval The value of is a real number between 0 and 1. The sum is 1; To prevent the logarithm from being negative infinity, Values ; This represents a logarithmic operation with base 2. For the... Positions of all elements in the tensor of each time slice The local entropy value was calculated in the manner described above, and all the calculated local entropy values ​​were then processed. According to element position Arrange to form the first The entropy distribution matrix of the tensor of the nth time slice, the th The dimension of the entropy distribution matrix of the time slice sub-tensors is For all Each time slice sub-tensor undergoes the above local entropy calculation operation to generate... A matrix of entropy distribution values.

[0039] In practical implementation, based on the entropy distribution matrix, the process of extracting the principal component with the largest information content from the tensor of each time slice as an independent feature component using the principal component analysis algorithm is as follows. For the th The entropy distribution matrix of the time slice sub-tensor is flattened into a row vector. row vector The dimension is row vector Each element corresponds to a local entropy value at a specific location in the entropy distribution matrix. All The row vectors corresponding to the time slice tensors Arranged in rows, forming a data matrix. Data matrix The dimension is Data matrix Each row corresponds to the entropy distribution information of a time slice sub-tensor. For the data matrix... For each column, the mean is removed by calculating the mean of all rows in each column and subtracting the mean of the corresponding column from each element, resulting in a mean-removed data matrix. Calculate the mean-removed data matrix. covariance matrix covariance matrix The dimension is covariance matrix The Middle Line number The elements of the column are the mean-reduced data matrix. The Middle Column and number The covariance of the column, and The value range is 1 to For the covariance matrix Perform eigenvalue decomposition to obtain Each eigenvalue and its corresponding Given eigenvectors, arrange the eigenvalues ​​in descending order, and select the eigenvector corresponding to the largest eigenvalue as the first principal component direction. The first principal component direction has one dimension. The vector. The first The row vectors corresponding to the entropy distribution matrix of the tensor of each time slice Projecting onto the direction of the first principal component yields the projected value. Projection value It is a scalar value. The projected value... Multiplying by the direction of the first principal component yields a dimension of The vector, rearrange this vector into a The matrix is ​​obtained to get the first... The principal component matrix of the tensor of the nth time slice. The nth time slice tensor... Each row of the principal component matrix of the tensor of the nth time slice is used as an independent feature representation of a time step, and the nth time slice is used as the principal component matrix of the tensor of the nth time slice. The principal component matrix of the tensor of the nth time slice is merged along the time dimension to form the nth time slice tensor. Each time slice tensor has independent feature components, and the dimension of each independent feature component is... ,in The feature dimension representing the independent feature components is numerically equal to For all Each time slice tensor is processed using the above process to extract independent feature components, resulting in... Each independent feature component.

[0040] In practical implementation, the process of calculating the correlation matrix for the high-order residuals remaining after deducting independent feature components from the tensor of each time slice is as follows. For the th The time slice tensor will be the first time slice tensor. The original matrix of the tensor of the nth time slice minus the nth time slice tensor The principal component matrix of the tensor of the nth time slice is obtained. The residual matrix of the tensor of each time slice residual matrix The dimension is residual matrix It contains higher-order residual terms that the independent eigencomponents could not explain. The residual matrix... Each column is considered as a residual sequence, and the residual matrix is... Total of There are n residual sequences, each with a length of n. For the residual matrix Any two rearranged indexes of different dimensions and , and The value range is 1 to and Extract the first The residual sequence and the first Calculate the residual sequence. The residual sequence and the first Pearson correlation coefficient among residual sequences The Pearson correlation coefficient is calculated by dividing the covariance of the two residual sequences by the product of their standard deviations. For the residual matrix... Pair all residual sequences together and calculate the Pearson correlation coefficient to obtain the first... The correlation matrix of the tensors of each time slice Correlation matrix The dimension is Correlation matrix The Middle Line number The elements of the column are Correlation matrix It is a symmetric matrix with all diagonal elements being 1. For all The residual matrix of each time slice sub-tensor is subjected to the above correlation matrix calculation operation to obtain A correlation matrix.

[0041] In practical implementation, the process of identifying combinations of residual terms with correlation exceeding a preset threshold as cross-feature components is as follows: Preset threshold The value is 0.7. The threshold of 0.7 is set based on the statistical criteria for strong correlation: when the absolute value of the Pearson correlation coefficient is greater than or equal to 0.7, it indicates a strong correlation between the two variables. For the first... Correlation matrices Traverse the correlation matrix For all elements except the diagonal, for the first... Line number Column element values ,like If the absolute value is greater than or equal to a preset threshold of 0.7, then the residual matrix will be... The Middle The residual sequence and the first Each residual sequence is identified as a set of residual terms that satisfy the correlation requirement. All residual term combinations that satisfy the correlation requirement are aggregated. The aggregation method is as follows: for each residual term combination that satisfies the correlation requirement, the residual matrix is... The Middle The residual sequence and the first Multiplying each residual sequence element-wise yields a sequence of length ... If the same residual sequence appears in multiple combinations of residual terms that satisfy the correlation requirement, then the generated aggregate sequences are summed element-wise. The aggregated sequences are then arranged according to the feature dimension to obtain a sequence with dimension . The matrix, as the first The cross-feature components of the time slice sub-tensor, where The feature dimension representing the cross feature components. The value of is equal to the number of different residual sequences involved in the combination of residual terms that satisfies the correlation requirement. Less than or equal to For all Each time slice sub-tensor is identified and cross-feature components are generated according to the above process, obtaining... Each cross-feature component.

[0042] See Figure 5 The horizontal axis in the figure represents the index of the time slice tensor. The range is from 0 to 500, reflecting the sequence position of multiple time segments obtained by applying a sliding window slicing operation along the time axis to the unified structured data tensor in Example 2. The left ordinate corresponds to the mean local entropy value and the projection value of independent feature components, in the form of dimensionless feature values. The right ordinate corresponds to the number of combinations of cross feature components, representing the number of combinations of residual terms in the higher-order residual terms that satisfy the correlation threshold condition.

[0043] The blue solid line curve represents the mean local entropy value, reflecting the average level of entropy values ​​within the local neighborhood of the tensor in each time slice. The curve shows a gradual upward trend, indicating that as the time series progresses, the overall entropy level increases, and the feature complexity or information content gradually strengthens. Several peaks and troughs exist on the curve, reflecting the dynamic changes in local entropy values, suggesting that the feature performance after multimodal data fusion exhibits periodic or phased changes across different time periods.

[0044] The orange dashed curve represents the projection values ​​of independent feature components, i.e., the projection intensity of the principal information components in the tensors of each time slice extracted based on principal component analysis. The curve fluctuates little between the numbers 0 and 100, then shows a significant peak in the range of approximately 100 to 300, after which it gradually declines and tends to stabilize. This indicates that the information content of the independent feature components reaches its highest level in this range, reflecting that the main independent feature dimensions of the multimodal fusion data are quite significant during this time period, and there is a clear process of information aggregation and release over time.

[0045] The green dotted-line curve represents the number of combinations of cross-feature components, i.e., satisfying the Pearson correlation coefficient threshold condition. The curve shows a steady upward trend, increasing from approximately 10 to approximately 35, indicating that the number of strongly correlated feature combinations in the residual terms increases significantly over time. This reflects the gradual strengthening of complex relationships after multimodal data fusion, and the increasingly rich and complex structure of cross-feature components.

[0046] Example 3:

[0047] In practice, the event graph module pre-stores a clinical event ontology network containing nodes and edges. The clinical event ontology network is defined as a directed graph. ,in Represents a set of nodes. Represents the set of edges. Set of nodes. Each node in This indicates a predefined clinical event type. For node indexing, The value range is 1 to , This represents the total number of nodes in the clinical event ontology network. Clinical event types include disease diagnosis events, symptom presentation events, laboratory test events, treatment plan events, complication events, outcome events, and resource scheduling events. Each clinical event type corresponds to a unique event type identifier and a semantic description text. Edge set. Each edge in Indicates from node Pointing to node The relationship between them To and Different node indices represent different relationship types, including causal relationships, temporal relationships, concurrent relationships, and conditional triggering relationships. Each edge... Store an initial weight value Initial weight values The value of is a real number between [0,1], and the initial weight value is... The settings are based on the event co-occurrence frequency and causal relationship strength statistically analyzed in the medical knowledge base. For node pairs that do not have a direct relationship, the corresponding edge either does not exist or the edge weight value is set to 0.

[0048] In practice, the process of calculating the semantic matching degree between independent feature components and cross feature components and the event type defined by each node in the clinical event ontology network is as follows: Extracting the semantic matching degree between each node in the clinical event ontology network... The event type semantic description text is input into a pre-trained text encoder. The pre-trained text encoder uses the same Transformer-based bidirectional encoder representation model as the semantic feature vectors extracted from the electronic medical record text data in the data fusion module. This Transformer-based bidirectional encoder representation model contains 12 Transformer encoding layers with a hidden layer dimension of 768. This Transformer-based bidirectional encoder representation model is pre-trained on a large-scale clinical text corpus through masked language modeling and next-sentence prediction tasks. For the event type semantic description text, word segmentation is performed, and classification and delimiter tokens are added. The processed text is then input into the pre-trained text encoder. The hidden state vector corresponding to the classification tokens output from the last Transformer encoding layer is taken as the node semantic vector, which has a dimension of 768. The node semantic vector is then dimensionality-reduced using a linear projection layer, mapping the 768-dimensional node semantic vector to a 128-dimensional node semantic representation vector. The weight matrix of the linear projection layer has a dimension of The bias vector dimension is 128.

[0049] For the output of the decoupling analysis module The independent and cross feature components corresponding to the time slice sub-tensor are combined into a unified feature representation vector. Specifically, for the ... Each time slice tensor has an independent feature component dimension of . The dimension of the cross feature components is The independent feature components and the cross feature components are concatenated along the feature dimension to obtain a dimension of The concatenated feature matrix is ​​obtained. The average of each row of the concatenated feature matrix along the time dimension is taken to obtain a matrix with dimension . The average feature vector is then mapped to a 128-dimensional first-order feature vector through a fully connected layer. A unified event representation vector of time slice tensors The weight matrix of the fully connected layer has a dimension of . For all Perform the above operation on each time slice tensor to obtain A unified event representation vector.

[0050] For each unified event representation vector and the semantic representation vector of each node Calculate the semantic matching degree between the two. semantic matching degree Cosine similarity is used for measurement:

[0051] in, Indicates the first The unified event representation vector of the time slice tensor and the first time slice tensor The semantic matching degree between the node semantic representation vectors of each node. The index of the time slice subtensor. The value range is 1 to , This serves as an index for nodes in the clinical event ontology network. The value range is 1 to ; Indicates the first The unified event representation vector of each time slice tensor has a dimension of 128. Indicates the first The node semantic representation vector of each node has 128 dimensions; This represents the vector dot product operation; Represents a unified event representation vector The L2 norm; Represents the semantic representation vector of a node. The L2 norm. For each uniform event representation vector. ,calculate With all The semantic matching degree of each node's semantic representation vector is used to obtain a matching degree vector. .

[0052] In practice, the component with the highest semantic matching degree among the independent feature components and cross feature components is used as the event state input of the corresponding node, and the process of updating the node state attribute value is as follows. For each node... In all The tensor corresponding to each time slice Find the node in the unified event representation vector. The tensor of the time slice with the highest semantic matching degree. Specifically, for the node... Determine the index ,Right now In order to make To obtain the maximum value Value. The first A unified event representation vector of time slice tensors As a node The event status input. Node The state attribute values ​​are taken from the event state vector. This represents the event state vector. The dimension is 128. The method for updating node state attribute values ​​is to update the event state vector. Replace directly with ,Right now For all Each node performs the above-mentioned state attribute value update operation.

[0053] In practical implementation, the process of updating the weight attribute values ​​of edges connecting different nodes in the clinical event ontology network based on the correlation strength values ​​in the cross-feature components, and generating a dynamically weighted event association representation graph, is as follows. For the output of the decoupling analysis module... The cross-feature components of the time slice tensors, with dimensions of , are . The cross-feature components are averaged along the time dimension to obtain a dimension of The averaged cross-feature vector is then input into an association strength extraction network. This network is a fully connected network containing two fully connected layers. The first fully connected layer has 64 neurons and uses the ReLU activation function, while the second fully connected layer has… n neurons and use the Sigmoid activation function, where This represents the total number of edges in the clinical event ontology network. The association strength extraction network outputs an association strength vector. Association strength vector The dimension is Association strength vector Each element takes the value of a real number between (0,1), and each element corresponds to the association strength value of an edge in the clinical event ontology network.

[0054] For all Each time slice sub-tensor is processed through an association strength extraction network to obtain the corresponding association strength vector. The average of each element in the association strength vector is taken to obtain an average association strength vector. Average association strength vector The dimension is For each edge in the clinical event ontology network In the average correlation strength vector Find the edge Corresponding association strength value , will the side The original weight values Update to new weight values The update method is ,in For smoothing coefficients, The value of is 0.3. The smoothing coefficient is set to 0.3 based on a grid search on the validation set. Candidate values ​​between 0.1 and 0.9 with a step size of 0.1 are scanned on the validation set, and a value that balances the stability of the graph structure and the response sensitivity is selected. Here, 0.3 is the optimal value in the grid search. After performing the above weight update operation on all edges, the clinical event ontology network is transformed into a dynamically weighted event association representation graph. The dynamically weighted event association representation graph is defined by the updated node state attribute values ​​and the updated edge weight values.

[0055] In practical implementation, the association strength extraction network needs to be pre-trained. The pre-training process is as follows: A training dataset is constructed containing independent feature components, cross-feature components, and corresponding event association annotation information. The event association annotation information consists of three levels of annotations by medical experts regarding the existence and strength of an association between two clinical events: strong association, weak association, and no association, with corresponding numerical labels of 1.0, 0.5, and 0.0, respectively. After generating a unified event representation vector from the independent and cross-feature components in the training dataset as described above, this vector is input into the association strength extraction network in units of time slice sub-tensors. The association strength extraction network outputs an association strength vector, and the mean squared error loss between the association strength vector and the numerical label is calculated. The Adam optimizer is used for parameter optimization, with a learning rate of 0.001 and a batch size of 16. Training is performed for 50 epochs until the loss function converges. After pre-training, the parameters of the association strength extraction network are retained for subsequent online inference.

[0056] See Figure 6 The horizontal axis in the figure represents the index of the time slice tensor. The range is from 0 to 500, and the vertical axis represents the semantic matching degree, ranging from 0.80 to 0.86. There are six curves in the figure, corresponding to the semantic matching degree of five preset clinical event types—disease diagnosis event, symptom manifestation event, laboratory test event, treatment plan event, and complication event—changing over time.

[0057] Within the tensor sequence number range of time slices from 0 to 100, the semantic matching degree between examination and testing events (green dashed lines) and treatment plan events (red dashed lines) shows a slow upward trend, reaching a maximum of nearly 0.84, reflecting a high degree of semantic consistency between the characteristics of examination and testing events and the corresponding nodes in the clinical event ontology network during this stage. The semantic matching degree of symptom manifestation events (orange dashed lines) and complication events (purple dashed lines) fluctuates less, remaining between 0.82 and 0.835, indicating that the semantic matching of related events is relatively stable during this stage.

[0058] Within the tensor sequence number range of time slices from 100 to 300, the semantic matching degree of disease diagnosis events (blue solid line) rises rapidly, reaching a peak of approximately 0.845, indicating that the semantic features of diagnosis events in this time period match the corresponding nodes more strongly. The semantic matching degrees of symptom manifestation events and laboratory test events also improve significantly, reaching highs of approximately 0.844 and 0.853 respectively, reflecting the dynamic changes in clinical event characteristics, with laboratory test and symptom events being particularly prominent.

[0059] Within the tensor sequence number range of time slices 300 to 450, the semantic matching degree of complication events significantly improved, peaking at approximately 0.846, indicating that the semantic features of complication-related clinical events in this stage were highly consistent with the ontology nodes. The semantic matching degree of treatment plan events, however, showed a downward trend, decreasing to below 0.81 at sequence number 450, reflecting a weakening of the semantic representation of these events in the event graph. The matching degree of laboratory test events also decreased, but remained around 0.825.

[0060] Within the tensor sequence number range of time slices 450 to 500, the semantic matching degree for symptom-related events rebounded, remaining around 0.842, indicating a renewed strengthening of the dynamic characteristics of this event type during this phase. The semantic matching degree for disease diagnosis and complication events decreased, but overall remained within the range of 0.82 to 0.83. The semantic matching degree for treatment plan events continued to decline, reaching a low of nearly 0.80, indicating relatively weak semantic performance.

[0061] Example 4:

[0062] In practical implementation, the decision-making inference module performs a hybrid inference operation based on Monte Carlo tree search and deep Q-network on a dynamically weighted event association representation graph, generating the optimal emergency decision sequence and the corresponding resource scheduling list. The dynamically weighted event association representation graph is generated by the event graph module and includes... There are 10 nodes, each with an event state vector. Event state vector The event association representation graph has 128 dimensions and is dynamically weighted. It also includes a set of edges that define the association relationships, with each edge having an updated weight value. The entire dynamically weighted event association representation graph is represented as a global state vector. Global state vector The construction method is to put all Event state vector of each node Concatenate the nodes in node index order and assign weights to all edges. Concatenate the edges according to their index order, then concatenate the concatenated node state vector and edge weight vector to obtain a fixed-dimensional global state vector. The global state vector As the root node state of the Monte Carlo tree search, the search tree node corresponding to the root node state is denoted as... ,node The attributes include nodes Number of visits and nodes Cumulative reward value During node initialization Number of visits Set to 0, node Cumulative reward value Set to 0.

[0063] In practice, the process of calling a pre-trained deep Q-network during each Monte Carlo tree search simulation phase is as follows. The pre-trained deep Q-network consists of an input layer, three fully connected hidden layers, and an output layer. The number of neurons in the input layer is... The dimension of the deep Q-network is the sum of the dimensions of the candidate action encoding vectors. The number of neurons in the three fully connected hidden layers are 256, 128, and 64, respectively. Each fully connected hidden layer is followed by a ReLU activation function layer and a batch normalization layer. The output layer has only one neuron and does not use an activation function. The input of the pre-trained deep Q-network contains the global state vector corresponding to the current simulation state node and a set of candidate action encoding vectors. The set of candidate actions is defined as... ,in The total number of candidate actions, which include adding a new clinical event node to the event association graph, modifying the state attribute values ​​of existing nodes in the event association graph, adding or deleting edges between nodes in the event association graph, and triggering a resource scheduling operation. Each candidate action... Represented as a candidate action encoding vector Candidate action encoding vector The dimension is 64. For the index of candidate actions, The value range is 1 to The deep Q-network uses the global state vector corresponding to the current state node. and candidate action encoding vector The concatenated vector is used as input, and the state-action value of each candidate action is output. ,in This represents all trainable parameters of the deep Q-network. In each simulation phase, for each node in the current Monte Carlo tree search tree, the upper confidence bound for each candidate action at that node is calculated based on the statistical information stored in the node. The upper confidence bound is calculated as follows:

[0064] in, Indicates candidate actions The upper confidence threshold; Indicates the execution of candidate actions Later arrived child node The cumulative reward value; Indicates the execution of candidate actions Later arrived child node Number of visits; To prevent small constants with a denominator of zero, the value is taken as... ; To explore the coefficient, The value of 1.5 is based on the empirical setting of the exploration coefficient of 1.5 in several classic Monte Carlo tree search applications to balance exploration and exploitation. This empirical setting is adopted directly here. Candidate actions for output by a deep Q-network In the global state vector The value of actions in the following state; This represents the total number of visits to all child nodes under the current node. During the simulation phase of Monte Carlo tree search, starting from the root node, a candidate action with the largest upper confidence bound is selected at each search tree node until a node that has not yet been expanded is reached or a preset simulation termination condition is met.

[0065] In its implementation, the pre-training process of the Deep Q-Network is as follows: A historical emergency decision-making dataset is constructed, containing snapshots of event association representation maps collected during historical emergency response processes, actual actions performed, and corresponding decision effectiveness scores. These scores are determined by medical experts based on the timeliness of the event response, the rationality of resource utilization, and the patient's prognosis, with scores ranging from 1 to 10. For each record in the historical emergency decision-making dataset, the event association representation map snapshot is converted into a global state vector, actions are encoded into candidate action encoding vectors, and the decision effectiveness score is normalized to the [0,1] interval and used as the reward value. The Deep Q-Network is trained offline in batches, using the mean squared error loss function, the Adam optimizer, a learning rate of 0.0005, a batch size of 64, and a training epoch of 100. During training, the target network parameters are updated every 1000 training steps. The target network has the same structure as the Deep Q-Network, but its parameters are maintained independently.

[0066] In practical implementation, during the expansion phase of the Monte Carlo tree search, the process of adding child nodes to the event association representation graph based on the actions and state transitions along the simulation path is as follows: When the simulation phase reaches a node that has not yet been expanded, the expansion operation is performed at that node. For the global state vector corresponding to that node... Enumerate all There are 1 candidate action, and for each candidate action... Execute candidate actions The corresponding state transition operation includes: if the candidate action To add a new clinical event node, a new node is created in the event association representation graph. The initial event state vector of the new node is obtained by aggregating the event state vectors of related nodes in the current graph through attention. If the candidate action... To modify the state attribute value of an existing node, update the event state vector of the corresponding node; if the candidate action... To add or delete an edge, modify the weight value of the corresponding edge or add a new edge; if the candidate action... To trigger resource scheduling operations, the state attribute values ​​of relevant nodes are updated to reflect changes in resource allocation. After performing a state transition operation, an updated event association representation graph is obtained, which is then transformed into a new global state vector. ,by As a newly added child node, create a new child node in the search tree. Adding child nodes After creation, record the newly added child nodes. Number of visits Initialize to 0, and record newly added child nodes. Cumulative reward value Initialize to 0.

[0067] In practical implementation, during the backpropagation phase of the Monte Carlo tree search, the reward value at the end of the simulation is backpropagated along the search path, and the process of updating the visit count and cumulative reward value of all nodes on the path is as follows: When the simulation terminates along the selected action path during the simulation phase, a reward value at the end of the simulation is obtained. Reward value The calculation method involves evaluating the clinical situation reflected in the current event association representation map at the simulation termination time. The evaluation uses a pre-trained situation assessment network, which takes the global state vector at the termination time as input and outputs a scalar value between [0,1] as the reward value. The closer the reward value is to 1, the better the clinical condition. The simulation terminates when the simulation depth reaches the preset maximum simulation depth. Alternatively, the event correlation graph may trigger a termination node indicating successful resolution of a sudden event or the triggering of an irreversible deterioration, with the maximum simulation depth. The value is 20. When the simulation ends, starting from the termination node, the simulation propagates backward along the simulation path to the root node, layer by layer. For each node traversed on the backward propagation path... ,node Number of visits Updated to ,node Cumulative reward value Updated to The backpropagation operation continues until the root node, updating the statistical information of all nodes along the simulation path.

[0068] In practical implementation, once the search reaches the preset computational budget, the optimal action is selected based on the number of visits to the root node's child nodes and the cumulative reward value. The process of generating the optimal emergency decision sequence and resource scheduling list based on the state changes corresponding to the optimal action is as follows. The preset computational budget is set to the total number of simulations performed in the Monte Carlo tree search. Total number of simulations The value is 5000. The search process terminates when the cumulative number of Monte Carlo tree search simulations reaches 5000. After the search terminates, the root node is obtained. All direct child nodes of the root node Each direct child node Get direct child nodes Number of visits The direct child node with the highest number of visits is selected as the optimal child node, and the candidate action to be performed when reaching the optimal child node is selected as the optimal action. Execute the optimal action. The corresponding state transition operation updates the event association representation graph from the root node state to the optimal child node state, and uses the optimal child node as the new root node. This process is repeated, each time starting from the current new root node, performing a new round of Monte Carlo tree search within the remaining computational budget, and recording the optimal actions selected each time in sequence to form an optimal action sequence. The iterative process continues until the preset maximum length of the emergency decision sequence is reached or a termination node indicating successful resolution of the emergency event is triggered. The maximum length of the emergency decision sequence is set to 20. The recorded optimal action sequence is converted into an optimal emergency decision sequence. Each decision step in the optimal emergency decision sequence includes a specific clinical operation description, the target node, and operation parameters. Simultaneously, for decision steps in the optimal action sequence that trigger resource scheduling operations, the corresponding resource scheduling information is extracted. This information includes the number and departments of medical personnel to be called, the type and quantity of equipment to be allocated, the type and quantity of beds to be allocated, and the scheduling sequence. All resource scheduling information is then summarized to generate a resource scheduling list.

[0069] Example 5:

[0070] In specific implementation, please refer to Figure 3 The self-feedback iteration module receives feedback time-series data after executing the optimal emergency decision sequence, updates the weights of the dynamically weighted event correlation representation graph online based on the feedback time-series data, and triggers the decision inference module to re-execute the inference. The decision result visualization and interaction module receives the optimal emergency decision sequence and resource scheduling list, and provides interactive modification and verification functions.

[0071] In practical implementation, the self-feedback iteration module monitors and collects the medical system's response records, equipment usage records, and bed occupancy records generated after the execution of the resource scheduling list, using these as feedback time-series data. The self-feedback iteration module connects to the medical staff scheduling and attendance database, the medical equipment IoT management platform, and the bed management system database through the hospital information system's standard data interface. The medical staff response records include the time each medical staff member receives the scheduling instruction, the time they confirm the response, the time they arrive at the designated location, and the start and end times of the operation. All times are recorded in Unix timestamp format with a precision of seconds. The equipment usage records include the time each scheduled device switches from standby to running state, the time-series data of the device's actual operating parameters, and the time the device completes use and is released. The sampling frequency for the time-series data of the device's actual operating parameters is once per minute. The bed occupancy records include the allocation time of each scheduled bed, the patient's admission time, the patient's transfer or discharge time, and the time the bed cleaning and disinfection is completed. The self-feedback iterative module performs time alignment processing on the medical staff response records, equipment usage records, and bed occupancy records on the same time axis. The time alignment processing uses a uniform time interval of one minute. Within each uniform time interval, the occurrence status of various events in the medical staff response records is encoded into a binary state vector. The dimension of the medical staff state vector for a uniform time interval is... , The total number of medical staff participating in the response; if the medical staff are on the [number]th [day / month]... If a device is in a response state within a uniform time interval, the corresponding element has a value of 1; otherwise, it has a value of 0. The device usage record for each device in the [number]th [time interval] is [set to 1]. Whether a bed is in an operational state within a unified time interval is encoded as a binary status vector; the status of each bed in the bed occupancy record is then recorded in the first... Whether a time interval is occupied is encoded as a binary state vector. The three state vectors are concatenated in a fixed order to form the [missing information]. Feedback state vectors with uniform time intervals, with dimension . ,in This represents the total number of devices being scheduled. Let be the total number of beds to be scheduled. Stack all feedback state vectors with uniform time intervals in chronological order to form a feedback time series data matrix. The time length of the feedback time series data matrix is... A unified time interval, with dimension [missing information]. .

[0072] In its implementation, the self-feedback iteration module inputs the feedback time-series data into a time-series residual analysis model. The time-series residual analysis model employs a long short-term memory (LSTM) network structure, which comprises two stacked LSM unit layers. The first LSM unit layer has 256 hidden units, and the second LSM unit layer has 128 hidden units. A random deactivation layer with a probability of 0.2 is set between the first and second LSM unit layers. The input to the time-series residual analysis model consists of two parts: the first part is the feedback time-series data matrix, which the model sequentially inputs into the first LSM unit layer; the second part is the ideal state sequence predicted by the decision-making inference module during the inference process. The ideal state sequence is generated as follows: During the hybrid inference operation of Monte Carlo tree search and deep Q-network in the decision inference module, the module records the state of the event association representation graph after each candidate action is executed. It extracts the ideal state vector for each time step within the same time length as the feedback time series data. The ideal state vector has a dimension of 128. It is obtained by concatenating the event state vectors of all nodes in the dynamically weighted event association representation graph at the corresponding time step and then reducing the dimension through a fully connected layer. The weight matrix of the fully connected layer has a dimension of [missing information]. The ideal state sequence is mapped to a 128-dimensional vector through a linear projection layer. This mapped ideal state sequence is then concatenated with the hidden state sequence output from the first long short-term memory (LSM) layer along the feature dimension. The resulting vector has the following dimensions: The concatenated vector is input into the second long short-term memory (LSM) layer. The second LSM layer outputs a hidden state vector at each time step. The hidden state vector at the last time step is used as the fused representation vector, which has a dimension of 128.

[0073] In practical implementation, the process by which the temporal residual analysis model calculates the residual vector between the feedback time-series data and the ideal state sequence predicted by the decision-making inference module during the inference process is as follows: The fused representation vector is input into a fully connected layer for residual computation. This fully connected layer contains 128 neurons and does not use an activation function. The output of the fully connected layer for residual computation is the residual vector. residual vector The dimension is 128. Within the time-series residual analysis model, the residual vector is further... Decomposed into systematic deviation components and random noise components The decomposition process is implemented by two parallel fully connected transformation layers: a system bias transformation layer and a random noise transformation layer. The input to the system bias transformation layer is the residual vector. The output is the system deviation component. The system bias transformation layer has 128 neurons and uses a linear activation function. The trainable parameters of the system bias transformation layer include a 128×128 weight matrix. And a 128-dimensional bias vector The input to the random noise transformation layer is also a residual vector. The output is a random noise component. The random noise transformation layer has 128 neurons and uses a linear activation function. The trainable parameters of the random noise transformation layer include a 128×128 weight matrix. And a 128-dimensional bias vector The decomposition process satisfies the residual vector. The time-series residual analysis model promotes the separation of the two components by constraining the predictive ability of the system bias component on the effect of subsequent decisions during training.

[0074] The time-series residual analysis model is based only on the system bias component. Generate weighted adjustment gradients. Incorporate system bias components. The input is fed into a gradient generation network, which contains two fully connected layers. The first fully connected layer has 64 neurons and uses the ReLU activation function. The second fully connected layer has [number of neurons missing]. And the Tanh activation function is used. This represents the total number of edges in a dynamically weighted event association graph. The output of the gradient generation network is the weight-adjusted gradient vector. Weight adjustment gradient vector The dimension is Weight adjustment gradient vector Each element The value ranges from -1 to 1. The first event in the event association representation graph The direction and magnitude of the weight adjustment for the strip edges. A positive value indicates that the weight of the corresponding edge needs to be adjusted positively. A negative value indicates that the weight of the corresponding edge needs to be adjusted negatively.

[0075] In practical implementation, the time-series residual analysis model needs to be pre-trained. The training dataset is constructed as follows: multiple complete event closed-loop data are extracted from historical emergency response records. Each event closed-loop data includes feedback time-series data after the execution of the emergency decision sequence, the ideal state sequence predicted by the decision inference module, and manually evaluated graph edge weight correction reference values. The manually evaluated graph edge weight correction reference values ​​are marked by medical experts according to the event response effect, indicating the magnitude by which the weight of each edge in the event association representation graph should be increased or decreased. The marked magnitudes are mapped to the interval [-1,1]. The training dataset is divided into a training set and a validation set according to time order, with a ratio of 8:2. The loss function of the time-series residual analysis model consists of a weighted sum of three parts: the first part is the gradient prediction loss, which calculates the mean square error between the weight adjustment gradient vector and the manually labeled edge weight correction reference values; the second part is the system bias prediction loss, which calculates the system bias component... The system uses an auxiliary prediction head to predict the medium- to long-term trend of subsequent feedback time-series data, and calculates the mean square error between the predicted trend and the actual trend; the third part is a noise regularization term, which reduces the random noise components. The L2 norm is minimized to compress the energy of random noise components. The weights of the three parts are set to 1.0, 0.5, and 0.1, respectively. Training uses the Adam optimizer with a learning rate of 0.0002, a batch size of 8, and 200 training epochs. Training is terminated early when the total loss on the validation set does not decrease for 10 consecutive epochs. After training, all parameters of the temporal residual analysis model are retained.

[0076] In practical implementation, based on the weight adjustment gradient, the weight update amount of each edge in the event association representation graph is calculated, and the online update process is performed using an online stochastic gradient descent optimizer, as follows. For each edge in the dynamically weighted event association representation graph... ,side The current weight value is denoted as Current weight value To dynamically weight the edges in the event association representation graph before executing the current round of emergency decision sequence and receiving feedback time series data from the self-feedback iterative module. Stored weight values. Adjust the gradient vector from the weights. Extract the corresponding edges Weight adjustment gradient The learning rate of the online stochastic gradient descent optimizer The learning rate was set to 0.05. This setting was based on a balance between convergence stability and response speed determined through simulation experiments in online time series model updates. Four candidate learning rates (0.01, 0.05, 0.1, and 0.2) were tested in an offline simulation environment. A learning rate of 0.05 showed the highest average cumulative return over five iterations after weight updates without any weight oscillations. The weight update amount was calculated as follows: An online stochastic gradient descent optimizer is used to add the updated weights to the current weights of the corresponding edges in the dynamically weighted event association graph, generating updated weights. The updated weights are then compared with a preset upper limit. and weight lower limit Perform a clamping operation, with a maximum weight. Set to 1.0, lower limit of weight Set to 0.0. The final weight value after the clamping operation. The calculation method is as follows:

[0077] in, This indicates that the update was performed online and the following was restricted. The final weight value; Representing an edge Current weight value before update; This represents the learning rate of the online stochastic gradient descent optimizer, with a value of 0.05. Representing an edge The corresponding weight adjustment gradient takes values ​​in the range of [-1, 1] (real numbers). This represents the upper limit of the weight, with a value of 1.0. This represents the lower limit of the weight, with a value of 0.0. This indicates the operation of finding the maximum value. This indicates the minimum value operation. The final weight value after the clamping operation is used. Alternative dynamically weighted event association representation graph corresponding edges The original weight values ​​are used to complete the online update operation for all edges.

[0078] In practical implementation, after completing the online update of the dynamically weighted event association representation graph, the self-feedback iteration module generates a deduction restart signal and simultaneously sends the restart signal and the updated dynamically weighted event association representation graph to the decision deduction module. Upon receiving the restart signal, the decision deduction module performs a search tree clearing operation, which includes releasing the memory space occupied by all nodes in the current Monte Carlo tree search tree and resetting the root node pointer to null. After the clearing operation is completed, the decision deduction module uses the received updated dynamically weighted event association representation graph as the new initial state node and converts the updated dynamically weighted event association representation graph into a new global state vector. A new root node for the Monte Carlo tree search is constructed, with the number of visits and cumulative reward value of the root node initialized to 0. The decision inference module restarts the hybrid inference operation based on Monte Carlo tree search and deep Q-network on the new root node, generating a new round of optimal emergency decision sequence and corresponding resource scheduling list.

[0079] In practical implementation, the decision result visualization and interaction module receives the optimal emergency decision sequence and resource scheduling list generated by the decision deduction module. This module includes a front-end rendering engine and a user interaction event processing unit. The front-end rendering engine renders the optimal emergency decision sequence in a structured timeline, generating a Gantt chart. The horizontal axis of the Gantt chart represents time, with a granularity of minutes. The start time of the horizontal axis is the execution time of the first decision node in the optimal emergency decision sequence, and the end time is the estimated completion time of the last decision node. The vertical axis of the Gantt chart lists each decision node in the optimal emergency decision sequence. Each decision node is represented by a bar in the Gantt chart. The starting position of the bar corresponds to the planned start time of the decision node, the length of the bar corresponds to the planned duration of the decision node, and the color of the bar is distinguished according to the clinical operation type corresponding to the decision node. Clinical operation types include diagnostic operations, treatment operations, resource scheduling operations, and transfer operations, and each clinical operation type is mapped to a preset color value. On the Gantt chart, if there are execution dependencies between the bars of each decision node, they are connected by arrowed lines pointing to subsequent dependent nodes. The front-end rendering engine overlays the current status and scheduling path identifier of each resource in the resource scheduling list onto the Gantt chart. For each resource in the resource scheduling list, a resource icon and resource scheduling path identifier are displayed above the corresponding time point on the Gantt chart's timeline. The resource scheduling path identifier represents the path and estimated time for the resource to move from its current location to its target location in a line graph format. The current status of the resource includes available status, occupied status, and maintenance status, distinguished by different fill colors. The user interaction event handling unit listens for user drag-and-drop or modification operations on any decision node in the Gantt chart. Drag operations are captured via mouse press, move, and release events. When a user drags a decision node's bar, the front-end rendering engine updates the bar's position on the timeline in real time and simultaneously updates the schedules of subsequent decision nodes that depend on it. Modification operations are achieved by double-clicking a decision node's bar to bring up a decision parameter editing panel. This panel allows users to modify the decision node's planned start time, planned duration, responsible person, and required resources. In response to user drag-and-drop or modification operations on any decision node in the Gantt chart, the decision result visualization and interaction module generates a modified decision version. The modified version maintains the structure of the original optimal emergency decision sequence, but the time and resource parameters of the modified decision node and its subsequent nodes within its influence range have been updated.The decision result visualization and interaction module resubmits the modified decision version as input to the decision inference module. The decision inference module uses the currently updated dynamically weighted event association representation map as the initial state to perform inference verification on the modified decision version. The verification process is the same as the hybrid inference operation. The module outputs the verified optimal emergency decision sequence and resource scheduling list and sends them back to the decision result visualization and interaction module for display.

[0080] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An intelligent decision support system for emergencies based on multimodal medical and health data, characterized in that: include: The data fusion module is used to receive heterogeneous data sources from medical institutions, including electronic medical record text data, vital sign time-series numerical data, and medical image data, and to perform heterogeneous adaptive inference processing on the heterogeneous data sources to generate a unified structured data tensor. The decoupling analysis module is used to perform temporal and spatial domain coupling feature decoupling analysis on the unified structured data tensor based on the entropy weight method, and generate decoupled independent feature components and cross feature components. The event graph module is used to map the independent feature components and cross feature components to a preset clinical event ontology network to generate a dynamically weighted event association representation graph. The decision inference module is used to perform a hybrid inference operation based on Monte Carlo tree search and deep Q network on the event association representation map to generate the optimal emergency decision sequence and the corresponding resource scheduling list; The self-feedback iteration module is used to receive feedback time-series data after executing the optimal emergency decision sequence, update the weights of the dynamically weighted event association representation graph online based on the feedback time-series data, and trigger the decision inference module to re-execute the inference.

2. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 1, characterized in that, The data fusion module performs heterogeneous adaptive inference processing on the heterogeneous data sources, specifically including: Semantic feature vectors of the electronic medical record text data, waveform feature vectors of the vital signs time-series numerical data, and texture and morphological feature vectors of the medical image data are extracted respectively. The semantic feature vector, waveform feature vector, and texture and morphology feature vector are spatiotemporally aligned to map them to a unified temporal stamp dimension. A learnable heterogeneous attention fusion network is constructed. The aligned semantic feature vector, waveform feature vector, and texture and morphological feature vector are input into the heterogeneous attention fusion network. The cross-attention weights between the data of each modality are calculated through a multi-head attention mechanism. The feature vectors of each modality are weighted and summed according to the cross-attention weights to generate the unified structured data tensor.

3. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 2, characterized in that, The decoupling analysis module performs temporal and spatial domain coupling feature decoupling analysis on the unified structured data tensor based on the entropy weight method, specifically including: The unified structured data tensor is sliced ​​along the time axis using a sliding window to obtain multiple time slice sub-tensors; For each time slice sub-tensor, the local entropy value in the spatial domain is calculated to generate the entropy distribution matrix of each time slice sub-tensor; Based on the entropy distribution matrix, the principal component with the largest information content in the tensor of each time slice is extracted as the independent feature component by the principal component analysis algorithm. The high-order residuals remaining after deducting the independent feature components from the subtensor of each time slice are used to calculate the correlation matrix, and the combination of residuals with correlation exceeding a preset threshold is identified as the cross feature components.

4. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 3, characterized in that, The event graph module maps the independent and cross-feature components to a preset clinical event ontology network, specifically including: A clinical event ontology network containing nodes and edges is pre-stored, where nodes represent clinical event types and edges represent the relationships between events; Calculate the semantic matching degree between the independent feature components and cross feature components and the event type defined by each node in the clinical event ontology network; The component with the highest semantic matching degree among the independent feature components and cross feature components is used as the event state input of the corresponding node, and the state attribute value of the node is updated. Based on the correlation strength values ​​in the cross-feature components, the weight attribute values ​​of the edges connecting different nodes in the clinical event ontology network are updated to generate the dynamically weighted event association representation graph.

5. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 4, characterized in that, The decision inference module performs a hybrid inference operation based on Monte Carlo tree search and deep Q-network on the event association representation map, specifically including: Use the current dynamically weighted event association representation graph as the state node for Monte Carlo tree search; In each simulation phase of the Monte Carlo tree search, a pre-trained deep Q-network is invoked. The deep Q-network takes the current state node and a set of candidate actions as input, outputs the state-action value of each candidate action, and selects the action on the simulation path based on the state-action value. In the extended phase of Monte Carlo tree search, based on the action and state transition results on the simulated path, a new child node is added to the event association representation graph, and the number of visits and cumulative reward value of the child node are recorded. In the backpropagation phase of the Monte Carlo tree search, the reward value at the end of the simulation is backpropagated along the search path to update the number of visits and the cumulative reward value of all nodes on the path. Once the search reaches the preset computational budget, the optimal action is selected based on the number of visits to the child nodes of the root node and the cumulative reward value. Based on the state change corresponding to the optimal action, the optimal emergency decision sequence and resource scheduling list are generated.

6. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 5, characterized in that, The self-feedback iteration module receives feedback time-series data after executing the optimal emergency decision sequence, specifically including: The medical system monitors and collects medical staff response records, equipment usage records, and bed occupancy records generated after the execution of the resource scheduling list, and uses these as the feedback time-series data. The feedback time series data is input into a time series residual analysis model, and the time series residual analysis model calculates the residual vector between the feedback time series data and the ideal state sequence predicted by the decision inference module during the inference process; Based on the residual vector, a weight adjustment gradient is generated, which is used to indicate which edges in the event association representation graph need to have their weights adjusted positively or negatively.

7. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 6, characterized in that, The time-series residual analysis model adopts a long short-term memory network structure and decomposes the residual vector into a system bias component and a random noise component, and generates the weight adjustment gradient based only on the system bias component.

8. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 6, characterized in that, The self-feedback iteration module updates the weights of the dynamically weighted event association representation graph online based on the feedback time series data, specifically including: Based on the weight adjustment gradient, calculate the weight update amount for each edge in the event association representation graph; An online stochastic gradient descent optimizer is used to superimpose the weight update amount onto the current weight value of the corresponding edge in the dynamically weighted event association representation graph to generate the updated weight value; The updated weight value is clamped to the preset upper and lower weight limits to ensure that the updated weight value does not exceed the upper and lower weight limits. The original weight values ​​of the corresponding edges in the dynamically weighted event association representation graph are replaced with the weight values ​​after the clamping operation to complete the online update.

9. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 8, characterized in that, The self-feedback iteration module triggers the decision deduction module to re-execute the deduction, specifically including: After completing the online update of the dynamically weighted event association representation map, a deduction restart signal is generated; The simulation restart signal and the updated dynamically weighted event association representation map are simultaneously sent to the decision simulation module; Upon receiving the inference restart signal, the current search tree of the Monte Carlo tree search is cleared, and the updated dynamically weighted event association representation graph is used as the new initial state node to restart the hybrid inference operation based on Monte Carlo tree search and deep Q network.

10. The intelligent decision support system for emergencies based on multimodal medical and health data according to claim 1, characterized in that, The system also includes a decision result visualization and interaction module, used for: Receive the optimal emergency decision sequence and resource scheduling list generated by the decision deduction module; The optimal emergency decision sequence is rendered in a structured manner according to the timeline to generate a Gantt chart containing the timeline, decision nodes, and execution branches; The current status and scheduling path identifier of each resource in the resource scheduling list are overlaid on the Gantt chart; In response to the user's drag-and-drop or modification operation on any decision node on the Gantt chart, a modified decision version is generated, and the modified decision version is resubmitted to the decision deduction module as input for deduction and verification.

Images