Cerebral stroke patient recurrence risk early prediction method and related equipment

Through the multi-scale data fusion model, the problem of failure to fully integrate imaging and clinical data in the existing technology is solved, the accuracy of prediction of stroke recurrence risk is improved, and a comprehensive analysis of complex clinical scenarios is achieved .

CN118969292AInactive Publication Date: 2024-11-15NEW SMART MEDICAL TECH (NINGBO) CO LTD

Patent Information

Application Number
CN202411452075.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-17
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing stroke recurrence risk prediction model relies on traditional risk factors and fails to fully integrate imaging and clinical data, resulting in low prediction accuracy. Most deep learning models focus on single modal data, making it difficult to achieve comprehensive comprehensiveness of complex clinical scenarios. analyze.

Method used

A multi-scale data fusion model is proposed, and the multi-modal examination data (clinical indicators, electrocardiograms, imaging reports) is obtained for pre-processing, and the data fusion is fusion using multi-head attention module and multi-layer perceptron to build a target prediction model to realize Effective integration of imaging and clinical data and accurate prediction of recurrence risk.

Benefits of technology

It improves the prediction accuracy of stroke recurrence risk, realizes effective integration of multimodal data and comprehensive analysis of complex clinical scenarios, and can more accurately identify the risk factors for stroke recurrence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969292A_ABST
    Figure CN118969292A_ABST
Patent Text Reader

Abstract

The invention discloses a cerebral apoplexy patient recurrence risk early prediction method and related equipment, and relates to the field of cerebral apoplexy patient recurrence risk early prediction, and the method comprises the steps: obtaining multi-modal examination data of a patient, and carrying out the preprocessing of the examination data to obtain to-be-predicted data; inputting to-be-predicted data into the target prediction model, and linearly transforming the to-be-predicted data through a multi-head attention module in the target prediction model to obtain a parameter vector group corresponding to each modal type; executing attention calculation on the corresponding parameter vector group through attention heads, splicing calculation results of the attention heads to form an attention fusion vector, and inputting the attention fusion vector into a multi-layer sensor through a normalization module; predicting a probability value corresponding to each recurrence risk level through a multi-layer sensor; according to the method, the parameter vector group corresponding to each modal type is acquired, and the corresponding parameter vector group is independently calculated through the attention head, so that accurate probability prediction of different recurrence risk levels of the stroke patient is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of early prediction of recurrence risk in stroke patients, and in particular to an early prediction method and related equipment for recurrence risk in stroke patients. Background Art

[0002] Stroke recurrence is an important factor affecting the prognosis of patients, especially LAA ischemic stroke. Existing risk prediction models mainly rely on traditional risk factors, such as blood pressure, blood sugar level, blood lipid status, etc. These models often do not fully consider the textual diagnostic information of imaging (such as MRI, CT, etc.) and other complex clinical data (such as electrocardiogram, laboratory test results, etc.). Therefore, these models have certain limitations in prediction accuracy.

[0003] With the advancement of artificial intelligence technology, especially the development of deep learning, multimodal data fusion has become possible. Multimodal data fusion refers to the combination of data from different sources (such as imaging, clinical indicators, biomarkers, etc.) for disease diagnosis or prognosis assessment. This method can more comprehensively reflect the patient's health status, thereby improving the accuracy of prediction. However, most of the current deep learning models are still mainly focused on the application of single-modal data. Although the application of single-modal data can achieve good results in specific tasks, it is still insufficient for the comprehensive analysis of complex clinical scenarios. For example, it is difficult to fully capture the multifaceted characteristics of a disease by relying solely on imaging data or clinical indicator data.

[0004] In summary, existing risk prediction models are mostly based on traditional risk factors and lack effective integration of imaging and clinical data, resulting in low prediction accuracy. With the development of artificial intelligence technology, multimodal data fusion has broad application prospects in medical image analysis. However, most current deep learning models still focus on single-modality data, making it difficult to achieve comprehensive analysis of complex clinical scenarios. Summary of the invention

[0005] In order to achieve effective integration of imaging and clinical data and improve model prediction accuracy, the present invention proposes an early prediction method for the recurrence risk of stroke patients, comprising: Acquire multimodal examination data of the patient and preprocess it to obtain data to be predicted; the modal types of the examination data include: clinical indicator data, electrocardiogram signals in electrocardiograms, and text diagnosis data in imaging reports; the clinical indicator data are high-dimensional feature data; the data to be predicted include: clinical indicator feature data, electrocardiogram features in electrocardiograms, and hidden layer vectors corresponding to each word in text diagnosis data; A multi-scale data fusion model is constructed based on an input module, a multi-head attention module including multiple attention heads, a normalization module and a multi-layer perceptron, and a target prediction model is obtained by training the multi-scale data fusion model; The data to be predicted is input into the target prediction model, and the data to be predicted is linearly transformed through the multi-head attention module in the target prediction model to obtain the parameter vector group corresponding to each modal type; the parameter vector group corresponds to the attention head one by one; the attention head performs attention calculation on the corresponding parameter vector group, splices the calculation results of each attention head to form an attention fusion vector, and inputs it into the multilayer perceptron through the normalization module; the probability value corresponding to each recurrence risk level is predicted through the multilayer perceptron, and output through the output node of the multilayer perceptron; the recurrence risk level corresponds to the output node one by one; any of the parameter vector groups includes: Query vector, Key vector and Value vector.

[0006] Furthermore, the method of acquiring multimodal examination data of the patient and preprocessing it to obtain the data to be predicted specifically includes: processing the text diagnosis data through the BERT model to obtain the hidden layer vector corresponding to each word in the text diagnosis data; extracting the ECG features in the ECG signal through the wavelet transform method; mapping the high-dimensional clinical indicator data to the low-dimensional space through the t-SNE method to obtain the clinical indicator feature data represented in the low-dimensional space.

[0007] Furthermore, the ECG features in the ECG signal are extracted by wavelet transform method, specifically: Decomposing the ECG signal by wavelet transform, obtaining multiple sub-signals and wavelet coefficient sets corresponding to each sub-signal; setting the wavelet coefficients in each wavelet coefficient set that are smaller than a preset threshold to zero to achieve denoising; Perform inverse wavelet transform on the denoised sub-signal to obtain the reconstructed ECG signal; The electrocardiogram characteristics are obtained through the reconstructed electrocardiogram signal; the electrocardiogram characteristics include: heart rate, heart rhythm and heart conduction value.

[0008] Furthermore, the clinical index data includes: basic information of the patient, clinical information, specific indexes in blood tests and scores of the nervous system; The clinical indicator feature data include: low-dimensional representation features of the patient's basic information, low-dimensional representation features of the patient's clinical information, low-dimensional representation features of specific indicators in blood tests, and low-dimensional representation features of nervous system scores.

[0009] Further, the normalization module includes: a first normalization unit and a second normalization unit; In the multi-scale data fusion model: the input module, the first normalization unit, the multi-head attention module, the second normalization unit and the multi-layer perceptron are connected in sequence; wherein the output end of the input module is also residually connected to the input end of the second normalization unit, and the output end of the multi-head attention module is residually connected to the input end of the multi-layer perceptron; The input module is used to input the data to be predicted into the first normalization unit and the second normalization unit; The first normalization unit is used to normalize the data to be predicted so that the feature distribution of the data to be predicted in each data dimension is consistent, and input the normalized data to be predicted into the multi-head attention module; The multi-head attention module is used to linearly transform the normalized data to be predicted, obtain the parameter vector group corresponding to each modal type, perform attention calculation on the corresponding parameter vector group through the attention head, splice the calculation results of each attention head, form an attention fusion vector, and input it into the second normalization unit and the multi-layer perceptron; The second normalization unit is used to normalize the attention fusion vector and the data to be predicted input by the input module to obtain normalized data, and input the normalized data into the multilayer perceptron; The multilayer perceptron is used to predict the probability value corresponding to each recurrence risk level according to the normalized data and the attention fusion vector, and output it through the output node of the multilayer perceptron.

[0010] Furthermore, the multi-head attention module includes: An attention head corresponding to the parameter vector group one by one; any of the attention heads comprises: a first linear transformation unit, a second linear transformation unit, a third linear transformation unit and a scaled dot product attention module; Splicing modules; Linear transformation module; where: The first linear transformation unit is used to transform the Query vector into a first vector conforming to a preset format; the second linear transformation unit is used to transform the Key vector into a second vector conforming to a preset format; the third linear transformation unit is used to transform the Value vector into a third vector conforming to a preset format; the preset format is a form conforming to attention calculation; The scaled dot product attention module is used to perform attention calculation on the first vector, the second vector and the third vector, and input the calculation results into the splicing module; the splicing module is used to splice the calculation results input by the scaled dot product attention module in each attention head to obtain a splicing vector, and input the splicing vector into the linear transformation module; The linear transformation module is used to map the splicing vector to a predefined final output space through a preset weight matrix, thereby forming an attention fusion vector.

[0011] Furthermore, the target prediction model is obtained by training the multi-scale data fusion model, specifically including: Collect multiple initial data samples and preprocess them to obtain target data samples corresponding to the initial data samples one by one; the target data samples correspond to the patients one by one; the target data samples include: clinical indicator feature data, ECG features in ECG signals, and hidden layer vectors corresponding to each word in text diagnosis data; Label each target data sample with a risk label; the risk label includes: a probability value corresponding to each recurrence risk level; A training set is formed by annotated target data samples, and a multi-scale data fusion model is trained by the training set and the cross entropy loss function to obtain a target prediction model.

[0012] Furthermore, the scaled dot product attention module is specifically used to calculate the attention weight between the first vector and the second vector, and perform weighted sum calculation on the third vector according to the attention weight, and input the calculation result into the splicing module.

[0013] The present invention further proposes an electronic device, comprising: a processor, and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the method described above.

[0014] The present invention also proposes a non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described above.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention obtains multimodal examination data of patients and preprocesses the data to be predicted; constructs a multiscale data fusion model and obtains a target prediction model by training the multiscale data fusion model; inputs the data to be predicted into the target prediction model, linearly transforms the data to be predicted through the multi-head attention module in the target prediction model, and obtains a parameter vector group corresponding to each modality type; performs attention calculation on the corresponding parameter vector group through the attention head, splices the calculation results of each attention head to form an attention fusion vector, and inputs it into a multilayer perceptron through a normalization module; predicts the probability value corresponding to each recurrence risk level through the multilayer perceptron, and outputs it through the output node of the multilayer perceptron; that is, the present invention obtains the parameter vector group corresponding to each modality type, and separately calculates the corresponding parameter vector group through the attention head, thereby realizing the effective integration of multimodal examination data, and on this basis, realizes the accurate probability prediction of different recurrence risk levels of stroke patients through the multilayer perceptron; (2) The present invention improves the prediction accuracy of the target prediction model for the risk of stroke recurrence by making full use of information from multiple data sources such as clinical indicator data, electrocardiogram and imaging reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for early prediction of recurrence risk in stroke patients; Figure 2 It is a structural diagram of the multi-scale data fusion model; Figure 3 This is a schematic diagram of the structure of the multi-head attention module; Figure 4 This is a schematic diagram of the structure of the feature extraction module in the BERT model; Figure 5 FIG. 4 is a schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0017] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.

[0018] Embodiment 1

[0019] Existing risk prediction models are mostly based on traditional risk factors and lack effective integration of imaging and clinical data, resulting in low prediction accuracy. In addition, most current deep learning models still focus on single-modality data, making it difficult to achieve comprehensive analysis of complex clinical scenarios. Multimodal data fusion can overcome the limitations of single-modality data and greatly improve the generalization ability and prediction accuracy of the model by integrating different types of data. For example, combining imaging features with clinical indicators can more accurately identify risk factors for stroke recurrence. Based on this, Figure 1 As shown, the present invention proposes a method for early prediction of recurrence risk in stroke patients, comprising: Acquire multimodal examination data of the patient and preprocess it to obtain data to be predicted; the modal types of the examination data include: clinical indicator data, electrocardiogram signals in electrocardiograms, and text diagnosis data in imaging reports; the clinical indicator data are high-dimensional feature data; the data to be predicted include: clinical indicator feature data, electrocardiogram features in electrocardiograms, and hidden layer vectors corresponding to each word in text diagnosis data; The clinical indicator data include: basic information of the patient, clinical information, specific indicators in blood tests and nervous system scores (specifically including NIHSS scores and mRS scores); In this embodiment, the patient's clinical information includes: basic indicator information such as age, height, weight and blood pressure; specific indicators in blood tests include: myoglobin, total cholesterol (TCHO), triglycerides (TG), blood sugar, etc. in blood tests.

[0020] The method of obtaining multimodal examination data of patients and preprocessing the data to be predicted specifically includes: processing text diagnosis data through a BERT model to obtain a hidden layer vector corresponding to each word in the text diagnosis data; extracting ECG features in ECG signals through a wavelet transform method; and mapping high-dimensional clinical indicator data to a low-dimensional space through a t-SNE method to obtain clinical indicator feature data represented in the low-dimensional space.

[0021] It should be explained that t-SNE is a nonlinear dimensionality reduction technique. Its core idea is to construct a probability distribution between data points in a high-dimensional space, and then try to find the corresponding probability distribution in a low-dimensional space so that the two are as similar as possible. Specifically, the similarity between data points in the high-dimensional space is modeled by a Gaussian distribution, while in the low-dimensional space it is modeled by a probability distribution that follows a t-distribution. Dimensionality reduction is achieved by optimizing the positions of data points in the low-dimensional space so that the distribution of similarity between them matches the distribution in the high-dimensional space as much as possible.

[0022] The low-dimensional representation features obtained by the t-SNE method in this embodiment can better understand the relationship between data and facilitate visualization. These low-dimensional representation features retain important feature information in the original data, but eliminate unnecessary noise and redundant information.

[0023] The clinical indicator feature data include: low-dimensional representation features of the patient's basic information, low-dimensional representation features of the patient's clinical information, low-dimensional representation features of specific indicators in blood tests, and low-dimensional representation features of nervous system scores.

[0024] This embodiment preprocesses multi-modal inspection data so that the inspection data of different modalities conform to the input format of the target prediction model, thereby achieving differentiated interpretation of data of different modalities.

[0025] It should be noted that the BERT model is a natural language processing pre-training technology based on Transformer. The input part of the BERT model designs three types of embeddings: Token Embedding: refers to the process of converting text into digital vectors, that is, after the text is tokenized, each word (or symbol) corresponds to a vector representation. During processing, a special token CLS is inserted at the beginning of the sequence for subsequent classification tasks, and a SEP token is inserted at the end of the sequence for sentence segmentation.

[0026] Segment Embedding: When dealing with tasks such as a pair of sentences, this type of embedding is used to distinguish the order of the two sentences. In the BERT model, this information is used in the Next SentencePrediction (NSP) training task.

[0027] Position Embedding: Since the encoder layer in the Transformer architecture cannot automatically capture the position information in the sequence like a recurrent neural network, position encoding needs to be added so that the model can understand the position of each word in the input sequence.

[0028] In this embodiment, the BERT model also includes a feature extraction module, such as Figure 4 As shown, the feature extraction module includes: A first Transformer layer and a second Transformer layer including a plurality of bidirectional encoders, wherein the bidirectional encoders in the first Transformer layer correspond one-to-one to the bidirectional encoders in the second Transformer layer; A hidden layer, which includes multiple hidden units; the hidden units correspond one-to-one to the bidirectional encoders in the second Transformer layer.

[0029] In this embodiment, the specific steps of processing text diagnosis data through the BERT model include: 1. Input preparation: Text diagnostic data needs to be preprocessed before entering the BERT model, including tokenization, which is to divide the text into a series of tokens (words or symbols).

[0030] A special tag CLS is inserted at the beginning of the sequence after word segmentation, which is usually used for classification tasks, such as determining whether a piece of text belongs to a certain category.

[0031] Another special marker SEP is inserted at the end of the sequence to mark the end of the sentence. This marker is particularly important when dealing with the relationship between two sentences.

[0032] 2. Processing of the Embedding layer (the final output of the Embedding layer is the vector corresponding to each token in the text diagnosis data): Token Embedding: Map each word or symbol after word segmentation into a vector space of fixed dimension.

[0033] Segment Embedding: If the input text consists of two segments (e.g., two sentences used for training in BERT), each word is given a tag indicating whether it belongs to the first or second segment.

[0034] Position Embedding: Add a fixed vector to the word at each position to preserve the relative or absolute position information of the word in the text.

[0035] 3. Processing of feature extraction module, such as Figure 4 As shown (in the figure: E1, E2, ...E N Represents the vector corresponding to each token, N represents the number of vectors, Trm represents the bidirectional encoder, represents the i-th hidden unit), which includes: (1) Processing of the first Transformer layer: After being processed by the Embedding layer, the vector corresponding to each token is input into each bidirectional encoder in the first Transformer layer. The bidirectional encoder contains a self-attention mechanism to capture long-term dependencies and contextual information in the input sequence.

[0036] (2) Processing of the second Transformer layer: The output of each bidirectional encoder in the first Transformer layer is input into each bidirectional encoder in the second Transformer layer.

[0037] (3) Processing of hidden layer: The output of the bidirectional encoder in the corresponding second Transformer layer is processed by the hidden unit, and the hidden layer vector is output, so as to obtain the hidden layer vector corresponding to each word.

[0038] The ECG features in the ECG signal are extracted by wavelet transform method, specifically: Decomposing the ECG signal by wavelet transform, obtaining multiple sub-signals and wavelet coefficient sets corresponding to each sub-signal; setting the wavelet coefficients in each wavelet coefficient set that are smaller than a preset threshold to zero to achieve denoising; Perform inverse wavelet transform on the denoised sub-signal to obtain the reconstructed ECG signal; The electrocardiogram characteristics are obtained through the reconstructed electrocardiogram signal; the electrocardiogram characteristics include: heart rate, heart rhythm and heart conduction value.

[0039] Based on the input module, the multi-head attention module containing multiple attention heads, the normalization module and the multi-layer perceptron, Figure 2 The multi-scale data fusion model shown in the figure is used to obtain the target prediction model by training the multi-scale data fusion model; The normalization module includes: a first normalization unit and a second normalization unit; In the multi-scale data fusion model: the input module, the first normalization unit, the multi-head attention module, the second normalization unit and the multi-layer perceptron are connected in sequence; wherein the output end of the input module is also residually connected to the input end of the second normalization unit, and the output end of the multi-head attention module is residually connected to the input end of the multi-layer perceptron; It should be explained that the output end of the input module is residually connected to the input end of the second normalization unit, and the output end of the multi-head attention module is residually connected to the input end of the multi-layer perceptron, which helps to transfer information and alleviate the gradient disappearance problem in deep networks.

[0040] The input module is used to input the data to be predicted into the first normalization unit and the second normalization unit; The first normalization unit is used to normalize the data to be predicted so that the feature distribution of the data to be predicted in each data dimension is consistent, and input the normalized data to be predicted into the multi-head attention module; It should be noted that when processing multimodal data, different modal types of data or feature types can be regarded as different data dimensions (or channels). For example: text data, electrocardiogram data, image data, etc. can be regarded as different dimensions (channels). In this embodiment, the first normalization unit ensures that the feature distribution in each data dimension is consistent (i.e., effective alignment of different modal embeddings is achieved) by normalizing the predicted data, which helps to eliminate the differences between different data dimensions and makes the feature distribution of different modal embeddings in each data dimension tend to be consistent, so as to facilitate the subsequent deep fusion of feature information.

[0041] The multi-head attention module is used to linearly transform the normalized data to be predicted, obtain the parameter vector group corresponding to each modal type, perform attention calculation on the corresponding parameter vector group through the attention head, splice the calculation results of each attention head, form an attention fusion vector, and input it into the second normalization unit and the multi-layer perceptron; like Figure 3 As shown, the multi-head attention module includes: An attention head corresponding to the parameter vector group one by one; any of the attention heads comprises: a first linear transformation unit, a second linear transformation unit, a third linear transformation unit and a scaled dot product attention module; Splicing modules; Linear transformation module; where: The first linear transformation unit is used to transform the Query vector into a first vector conforming to a preset format; the second linear transformation unit is used to transform the Key vector into a second vector conforming to a preset format; the third linear transformation unit is used to transform the Value vector into a third vector conforming to a preset format; the preset format is a form conforming to attention calculation; The scaled dot product attention module is used to perform attention calculation on the first vector, the second vector and the third vector, and input the calculation results into the splicing module; the splicing module is used to splice the calculation results input by the scaled dot product attention module in each attention head to obtain a splicing vector, and input the splicing vector into the linear transformation module; The scaled dot product attention module is specifically used to calculate the attention weight between the first vector and the second vector, and perform weighted sum calculation on the third vector according to the attention weight, and input the calculation result into the splicing module.

[0042] The linear transformation module is used to map the splicing vector to a predefined final output space through a preset weight matrix, thereby forming an attention fusion vector.

[0043] This embodiment adds a second normalization unit after the multi-head attention module to ensure that the features output by the multi-head attention module have been normalized before entering the next processing stage (multi-layer perceptron), thereby ensuring the consistency and stability of the features.

[0044] The second normalization unit is used to normalize the attention fusion vector and the data to be predicted input by the input module to obtain normalized data, and input the normalized data into the multilayer perceptron; The multilayer perceptron is used to predict the probability value corresponding to each recurrence risk level according to the normalized data and the attention fusion vector, and output it through the output node of the multilayer perceptron.

[0045] The target prediction model is obtained by training the multi-scale data fusion model, specifically including: Collect multiple initial data samples and preprocess them to obtain target data samples corresponding to the initial data samples one by one; the target data samples correspond to the patients one by one; the target data samples include: clinical indicator feature data, ECG features in ECG signals, and hidden layer vectors corresponding to each word in text diagnosis data; It should be noted that the initial data sample includes multimodal examination data of patients.

[0046] Label each target data sample with a risk label; the risk label includes: a probability value corresponding to each recurrence risk level; A training set is formed by annotated target data samples, and a multi-scale data fusion model is trained by the training set and the cross entropy loss function to obtain a target prediction model.

[0047] It should be noted that when training a multi-scale data fusion model, loss calculation is a key step to measure the gap between the model's prediction results and the actual risk labels. Specifically: The multiscale data fusion model generates prediction results based on the input data. In this embodiment, the output of the model is the predicted probability corresponding to different recurrence risk levels of stroke patients. The risk label is the actual probability value corresponding to each recurrence risk level, which is usually determined by experts or clinical data. The loss function is used to quantify the difference between the model prediction results and the risk label. The loss function used in this embodiment is the cross-entropy loss, which is used to measure the gap between the probability distribution predicted by the model and the probability distribution of the risk label; the formula expression of the cross-entropy loss function is: ; In the formula, The number represents the level of risk of recurrence; Represents the one-hot encoding of the risk label, that is, for the recurrence risk level , , other recurrence risk levels ; Represents the probability of the i-th recurrence risk level in the probability distribution predicted by the multi-scale data fusion model.

[0048] In addition, this embodiment calculates the gradient of the cross entropy loss function with respect to the model parameters through back propagation, and uses an optimization algorithm (such as stochastic gradient descent SGD, Adam, etc.) to update the model parameters to minimize the loss function.

[0049] In this embodiment, the training of the multi-scale data fusion model specifically includes: 1. Input the target data sample into the multi-scale data fusion model to obtain the prediction result output by the model, that is, the probability value corresponding to each recurrence risk level; 2. Calculate the value of the cross entropy loss function based on the prediction results output by the model and the risk label corresponding to the current target data sample; 3. Calculate the gradient of the cross entropy loss function with respect to the model parameters through the chain rule; 4. Use an optimization algorithm (such as stochastic gradient descent (SGD), Adam, etc.) to update the model parameters using the calculated gradients to minimize the value of the cross entropy loss function.

[0050] The data to be predicted is input into the target prediction model, and the data to be predicted is linearly transformed through the multi-head attention module in the target prediction model to obtain the parameter vector group corresponding to each modal type; the parameter vector group corresponds to the attention head one by one; the attention head performs attention calculation on the corresponding parameter vector group, splices the calculation results of each attention head to form an attention fusion vector, and inputs it into the multilayer perceptron through the normalization module; the probability value corresponding to each recurrence risk level is predicted through the multilayer perceptron, and output through the output node of the multilayer perceptron; the recurrence risk level corresponds to the output node one by one; any of the parameter vector groups includes: Query vector, Key vector and Value vector.

[0051] The present invention obtains multimodal examination data of patients and preprocesses the data to be predicted; constructs a multiscale data fusion model, and obtains a target prediction model by training the multiscale data fusion model; inputs the data to be predicted into the target prediction model, linearly transforms the data to be predicted through a multi-head attention module in the target prediction model, and obtains a parameter vector group corresponding to each modality type; performs attention calculation on the corresponding parameter vector group through the attention head, splices the calculation results of each attention head to form an attention fusion vector, and inputs it into a multilayer perceptron through a normalization module; predicts the probability value corresponding to each recurrence risk level through the multilayer perceptron, and outputs it through the output node of the multilayer perceptron; that is, the present invention obtains the parameter vector group corresponding to each modality type, and separately calculates the corresponding parameter vector group through the attention head, thereby realizing the effective integration of multimodal examination data, and on this basis realizes the accurate probability prediction of different recurrence risk levels of stroke patients through the multilayer perceptron.

[0052] Embodiment 2 An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0053] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

[0054] An embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method of the embodiment of the present invention.

[0055] The relevant devices in the present invention are the electronic device, non-transitory machine-readable medium and computer program product in this embodiment.

[0056] refer to Figure 5 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0057] like Figure 5 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 to a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of the electronic device can also be stored. The computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0058] Multiple components in the electronic device are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device, and the input unit 406 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a disk, an optical disk. The communication unit 409 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0059] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0060] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0061] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0062] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0063] In addition, in the present invention, descriptions such as "first", "second", "one", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0064] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A method for early prediction of recurrence risk in stroke patients, characterized in that: include: Acquire multimodal examination data of patients and preprocess them to obtain data to be predicted; The modal types of the examination data include: clinical indicator data, electrocardiogram signals in electrocardiograms, and text diagnosis data in imaging reports; the clinical indicator data are high-dimensional feature data; the data to be predicted include: clinical indicator feature data, electrocardiogram features in electrocardiogram signals, and hidden layer vectors corresponding to each word in text diagnosis data; A multi-scale data fusion model is constructed based on an input module, a multi-head attention module including multiple attention heads, a normalization module and a multi-layer perceptron, and a target prediction model is obtained by training the multi-scale data fusion model; The data to be predicted is input into the target prediction model, and the data to be predicted is linearly transformed through the multi-head attention module in the target prediction model to obtain the parameter vector group corresponding to each modal type; the parameter vector group corresponds to the attention head one by one; the attention head performs attention calculation on the corresponding parameter vector group, splices the calculation results of each attention head to form an attention fusion vector, and inputs it into the multilayer perceptron through the normalization module; the probability value corresponding to each recurrence risk level is predicted through the multilayer perceptron, and output through the output node of the multilayer perceptron; the recurrence risk level corresponds to the output node one by one; any of the parameter vector groups includes: Query vector, Key vector and Value vector.

2. The method for early prediction of recurrence risk in stroke patients according to claim 1, characterized in that: The method of obtaining multimodal examination data of patients and preprocessing the data to be predicted specifically includes: processing text diagnosis data through a BERT model to obtain a hidden layer vector corresponding to each word in the text diagnosis data; extracting ECG features in ECG signals through a wavelet transform method; and mapping high-dimensional clinical indicator data to a low-dimensional space through a t-SNE method to obtain clinical indicator feature data represented in the low-dimensional space.

3. The method for early prediction of recurrence risk in stroke patients according to claim 2, characterized in that: The ECG features in the ECG signal are extracted by wavelet transform method, specifically: Decomposing the ECG signal by wavelet transform, obtaining multiple sub-signals and wavelet coefficient sets corresponding to each sub-signal; setting the wavelet coefficients in each wavelet coefficient set that are smaller than a preset threshold to zero to achieve denoising; Perform inverse wavelet transform on the denoised sub-signal to obtain the reconstructed ECG signal; Obtaining ECG features through reconstructed ECG signals; The electrocardiogram characteristics include: heart rate, heart rhythm and heart conduction value.

4. The method for early prediction of recurrence risk in stroke patients according to claim 3, characterized in that: The clinical index data include: basic information of the patient, clinical information, specific indexes in blood tests and scores of the nervous system; The clinical indicator feature data include: low-dimensional representation features of the patient's basic information, low-dimensional representation features of the patient's clinical information, low-dimensional representation features of specific indicators in blood tests, and low-dimensional representation features of nervous system scores.

5. The method for early prediction of recurrence risk in stroke patients according to claim 1, characterized in that: The normalization module includes: a first normalization unit and a second normalization unit; In the multi-scale data fusion model: the input module, the first normalization unit, the multi-head attention module, the second normalization unit and the multi-layer perceptron are connected in sequence; wherein the output end of the input module is also residually connected to the input end of the second normalization unit, and the output end of the multi-head attention module is residually connected to the input end of the multi-layer perceptron; The input module is used to input the data to be predicted into the first normalization unit and the second normalization unit; The first normalization unit is used to normalize the data to be predicted so that the feature distribution of the data to be predicted in each data dimension is consistent, and input the normalized data to be predicted into the multi-head attention module; The multi-head attention module is used to linearly transform the normalized data to be predicted, obtain the parameter vector group corresponding to each modal type, perform attention calculation on the corresponding parameter vector group through the attention head, splice the calculation results of each attention head, form an attention fusion vector, and input it into the second normalization unit and the multi-layer perceptron; The second normalization unit is used to normalize the attention fusion vector and the data to be predicted input by the input module to obtain normalized data, and input the normalized data into the multilayer perceptron; The multilayer perceptron is used to predict the probability value corresponding to each recurrence risk level according to the normalized data and the attention fusion vector, and output it through the output node of the multilayer perceptron.

6. The method for early prediction of recurrence risk in stroke patients according to claim 5, characterized in that: The multi-head attention module includes: An attention head corresponding to the parameter vector group one by one; any of the attention heads comprises: a first linear transformation unit, a second linear transformation unit, a third linear transformation unit and a scaled dot product attention module; Splicing modules; Linear transformation module; where: The first linear transformation unit is used to transform the Query vector into a first vector conforming to a preset format; the second linear transformation unit is used to transform the Key vector into a second vector conforming to a preset format; the third linear transformation unit is used to transform the Value vector into a third vector conforming to a preset format; the preset format is a form conforming to attention calculation; The scaled dot product attention module is used to perform attention calculation on the first vector, the second vector and the third vector, and input the calculation results into the splicing module; the splicing module is used to splice the calculation results input by the scaled dot product attention module in each attention head to obtain a splicing vector, and input the splicing vector into the linear transformation module; The linear transformation module is used to map the splicing vector to a predefined final output space through a preset weight matrix, thereby forming an attention fusion vector.

7. The method for early prediction of recurrence risk in stroke patients according to claim 6, characterized in that: The target prediction model is obtained by training the multi-scale data fusion model, specifically including: Collect multiple initial data samples and preprocess them to obtain target data samples corresponding to the initial data samples one by one; the target data samples correspond to the patients one by one; the target data samples include: clinical indicator feature data, ECG features in ECG signals, and hidden layer vectors corresponding to each word in text diagnosis data; Label each target data sample with a risk label; the risk label includes: a probability value corresponding to each recurrence risk level; A training set is formed by annotated target data samples, and a multi-scale data fusion model is trained by the training set and the cross entropy loss function to obtain a target prediction model.

8. The method for early prediction of recurrence risk in stroke patients according to claim 7, characterized in that: The scaled dot product attention module is specifically used to calculate the attention weight between the first vector and the second vector, and perform weighted sum calculation on the third vector according to the attention weight, and input the calculation result into the splicing module.

9. An electronic device, comprising: A processor and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

10. A non-transitory machine-readable medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Transform algorithm-based single-mode label generation and multi-mode emotion discrimination method

    CN114386515A

  • Cerebral stroke early-stage auxiliary diagnosis method, system and device and storage medium

    CN116612886A

  • Upper limb function reconstruction system for stroke patient

    CN116983185A

  • Multi-modal false news detection method based on adaptive fusion

    CN117251795A

  • Method and system for predicting recurrence of cerebral apoplexy

    CN118430819A

Cited By

  • Method for training stroke recurrence risk prediction model and related product

    CN120413003A

  • Medical data processing method and system based on large model and related equipment

    CN120727302A