Rare disease incidence probability prediction method, device and electrocardiogram analysis system

Abstract

The application provides a rare disease occurrence probability prediction method and device and an electrocardiogram analysis system. The method comprises the following steps: acquiring a plurality of to-be-evaluated electrocardiogram data and corresponding to-be-evaluated medical record text data; inputting the to-be-evaluated electrocardiogram data into an electrocardiogram pre-training model to obtain a target electrocardiogram feature vector; inputting the medical record text data corresponding to the to-be-evaluated electrocardiogram data into a medical text pre-training model to obtain a target text feature vector; inputting the to-be-evaluated electrocardiogram data into a preset risk detection model to obtain an electrocardiogram rare disease occurrence probability coefficient; presetting a rare disease feature dictionary; determining the occurrence probability of each rare disease according to the similarity between the target electrocardiogram feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the electrocardiogram rare disease occurrence probability coefficient. The occurrence probability of the rare disease can be predicted without training a large number of electrocardiogram samples of the rare disease.

Description

Technical Field

[0001] This application relates to the field of electrocardiogram signal recognition technology, and more specifically, to a method, device, and electrocardiogram analysis system for predicting the probability of rare diseases. Background Technology

[0002] Electrocardiography (ECG) is an ideal tool for examining cardiac function and assessing cardiovascular disease risk due to its numerous advantages, including being non-invasive, simple, rapid, cost-effective, and widely available. ECG-based cardiovascular disease identification and risk assessment are of great significance for promoting early screening and intervention of cardiovascular diseases, improving patient prognosis, and alleviating the pressure on chronic disease management in my country. Currently, with the development of artificial intelligence technology, ECG-based intelligent report interpretation and disease risk identification methods have made significant progress and are playing an important role in improving the quality of disease diagnosis and treatment and the health of the people. However, the identification of rare diseases still faces challenges. Limited by the availability of sample data, how to more effectively utilize large amounts of unlabeled data to learn the association patterns with diseases, and thus achieve early risk identification of rare diseases, has become an urgent problem to be solved.

[0003] Current rare disease identification technologies primarily rely on supervised training using limited labeled data. Due to the small data volume, models often learn insufficiently, resulting in low recognition accuracy. Several studies have demonstrated that self-supervised pre-training and supervised fine-tuning can alleviate reliance on labeled data and improve the utilization of unlabeled data. Specifically, a large amount of unlabeled electrocardiogram (ECG) data is used to train the model, allowing it to fully learn, understand, and mine ECG features; then, supervised fine-tuning is applied to transfer the model's capabilities to specific disease identification tasks, improving performance. However, practice shows that even during supervised fine-tuning, a certain number of labeled samples are still required. For many rare disease scenarios, the number of labeled samples is extremely limited, often insufficient for effective model training. Furthermore, the ECG manifestations of many important rare diseases are often non-specific, making it difficult to uncover disease associations in complex populations and scenarios using only ECG input. How to extract disease knowledge from a large amount of historical medical records and, with a very small sample size, learn to associate it with electrocardiogram (ECG) manifestations to further improve the performance of rare disease identification based on ECG remains to be explored. Summary of the Invention

[0004] The main objective of this application is to provide a method, device, and electrocardiogram (ECG) analysis system for predicting the probability of rare disease occurrence, so as to at least solve the problem that the training of ECG for rare disease identification requires a large number of ECG samples of rare diseases.

[0005] To achieve the above objectives, according to one aspect of this application, a method for predicting the probability of rare disease occurrence is provided, comprising: acquiring multiple electrocardiogram (ECG) data to be evaluated and corresponding medical record text data to be evaluated, wherein the medical record text data to be evaluated includes at least patient information and ECG report conclusions, and the ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data; inputting the ECG data to be evaluated into an ECG pre-training model to obtain a target ECG feature vector, wherein the ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data; and inputting the medical record text data corresponding to the ECG data to be evaluated into a medical document... This pre-trained model obtains a target text feature vector; the ECG data to be evaluated is input into a preset risk detection model to obtain the ECG rare disease occurrence probability coefficient; a preset rare disease feature dictionary is prepared, which includes text feature vectors corresponding to each rare disease; based on the similarity between the target ECG feature vector and the reference text feature vectors of each rare disease, the similarity between the target text feature vector and the reference text feature vectors of each rare disease, and the ECG rare disease occurrence probability coefficient, the occurrence probability of each rare disease is determined, where the reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0006] Optionally, the medical text pre-training model is trained based on multiple historical medical record text data, specifically including: acquiring multiple historical medical record text data, wherein the historical medical record text data includes at least patient information, electrocardiogram report conclusions, and disease tags; performing masking processing on partial words of the historical medical record text data to obtain a first masked text, using the first masked text to perform masked pre-training on the text pre-training model to obtain an initial medical text pre-training model; performing masking processing on partial modules of the historical medical record text data to obtain a second masked text, using the second masked text to perform masked pre-training on the initial medical text pre-training model to obtain the medical text pre-training model, wherein the patient information includes gender, age, chief complaint, and present medical history, and the module is one of the chief complaint, the present medical history, the electrocardiogram report conclusions, and the disease tags.

[0007] Optionally, a partial word masking process is performed on the historical medical record text data to obtain a first masked text. The first masked text is then used to pre-train the text pre-training model to obtain an initial medical text pre-training model. This includes: performing text segmentation and serialization on the historical medical record text data to obtain a target vocabulary and multiple first formatted sequences; performing dynamic masking on partial words of the multiple first formatted sequences to obtain multiple second formatted sequences, using the second formatted sequences as the first masked text; obtaining the first label vector corresponding to the dynamically masked words through the target vocabulary; and converting the multiple second formatted sequences into multiple first text vectors, each first text vector consisting of a first word embedding vector, a first position embedding vector, and a first segment embedding vector, whereby the first word embedding vector represents the second formatting. The semantic information of words in the sequence is obtained. The first position embedding vector is used to represent the position of the word in the second formatted sequence in the sentence, and the first segment embedding vector is used to represent the position of the word in the second formatted sequence in the sentence. Multiple first text vectors are input into the encoder of the text pre-training model to obtain multiple second text vectors. The multiple second text vectors are input into the prediction head of the text pre-training model to obtain multiple first prediction vectors. Based on the first prediction vectors, the first prediction vector corresponding to the dynamically masked word is obtained. The sum of the losses of the multiple first prediction vectors corresponding to the dynamically masked words and the corresponding first label vectors is calculated to obtain the first loss function value. The text pre-training model is iteratively trained to minimize the first loss function value to obtain the initial medical text pre-training model.

[0008] Optionally, a portion of the historical medical record text data is masked to obtain a second masked text. This second masked text is then used to pre-train the initial medical text pre-training model, resulting in the medical text pre-training model. This process includes: performing text segmentation and serialization on the historical medical record text data to obtain a target vocabulary and multiple first formatted sequences; performing dynamic masking on the words of any module in the multiple first formatted sequences to obtain multiple third formatted sequences, using these third formatted sequences as the second masked text; obtaining the second label vector corresponding to the masked module through the target vocabulary; and converting the multiple third formatted sequences into multiple third text vectors, each composed of a second word embedding vector, a second position embedding vector, and a second segment embedding vector. The second word embedding vector is used to characterize the third formatted sequence. The semantic information of the words in the column, the second position embedding vector is used to represent the position of the words in the third formatted sequence in the sentence, the second segment embedding vector is used to represent the position of the module in which the words in the third formatted sequence are located, the multiple third text vectors are input into the encoder of the initial medical text pre-training model to obtain multiple fourth text vectors; the multiple fourth text vectors are input into the prediction head of the text pre-training model to obtain multiple second prediction vectors; based on the second prediction vectors, the second prediction vector corresponding to the module that is dynamically masked is obtained; the sum of the losses of the multiple second prediction vectors corresponding to the modules that are dynamically masked and the corresponding second label vectors is calculated to obtain the second loss function value; the initial medical text pre-training model is iteratively trained to minimize the second loss function value to obtain the medical text pre-training model.

[0009] Optionally, the training process of the ECG pre-training model includes: performing masked pre-training on the ECG self-supervised pre-training model using multiple non-rare disease ECG data and multiple rare disease ECG data to obtain a first ECG pre-training model, and determining the encoder of the first ECG pre-training model as the initial ECG pre-training model; performing comparative training on the initial ECG pre-training model using the historical medical record text data, the corresponding historical ECG data, and the medical text pre-training model to obtain the ECG pre-training model, wherein the loss function of the ECG pre-training model is at least used to calculate the loss of the text feature vector of the historical medical record text data and the ECG feature vector of the corresponding historical ECG data, and the loss includes single-modal contrast loss and cross-modal contrast loss.

[0010] Optionally, the preset risk detection model is trained using non-rare disease ECG data, including: performing masked pre-training on a self-supervised ECG pre-training model using multiple non-rare disease ECG data to obtain a second ECG pre-training model; constructing the preset risk detection model based on the second ECG pre-training model and its loss function, wherein the preset risk detection model is a model that calculates the probability coefficient of rare ECG occurrence using the loss function value of the second ECG pre-training model.

[0011] Optionally, a second ECG pre-training model is obtained by masking a self-supervised ECG pre-training model using multiple non-rare disease ECG data sets. This includes: segmenting the non-rare disease ECG data into multiple segments along the time dimension; masking some segments to obtain non-rare disease masked ECG data; inputting the non-rare disease masked ECG data into the self-supervised ECG pre-training model to obtain reconstructed ECG data; calculating the loss between the reconstructed ECG data and the non-rare disease ECG data to obtain a third loss function value; and iteratively training the self-supervised ECG pre-training model to minimize the third loss function value to obtain the second ECG pre-training model.

[0012] Optionally, the probability of occurrence of each rare disease is determined based on the similarity between the target ECG feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the ECG rare disease occurrence probability coefficient. This includes: a first calculation step, calculating the maximum similarity between the target ECG feature vector and each text feature vector of the target rare disease in the rare disease feature dictionary to obtain a first similarity, and calculating the maximum similarity between the target text feature vector and each text feature vector of the target rare disease in the rare disease feature dictionary to obtain a second similarity, wherein the target rare disease is any one of the rare diseases; a second calculation step, aggregating the first similarity and the second similarity to obtain the probability of occurrence of the target rare disease; repeating the first calculation step and the second calculation step at least once in sequence until the probability of occurrence of all rare diseases is obtained.

[0013] Optionally, aggregating the first similarity and the second similarity to obtain the probability of occurrence of the target rare disease includes: inputting the electrocardiogram data to be evaluated into the preset risk detection model to obtain the probability coefficient of occurrence of the rare electrocardiogram disease; and aggregating the first similarity and the second similarity using the probability coefficient of occurrence of the rare electrocardiogram disease to obtain the probability of occurrence of the target rare disease.

[0014] Optionally, the initial ECG pre-training model is trained by comparing the historical medical record text data, the corresponding historical ECG data, and the medical text pre-training model to obtain the ECG pre-training model. This includes: inputting the historical medical record text data into the medical text pre-training model to obtain training text feature vectors; inputting the ECG data corresponding to the historical medical record text data into the initial ECG pre-training model to obtain training ECG feature vectors; calculating the loss between the training text feature vectors and the corresponding training ECG feature vectors to obtain a cross-modal contrast loss; processing the training ECG feature vectors using two data augmentation methods to obtain a first ECG feature vector and a second ECG feature vector; calculating the loss between the first ECG feature vector and the second ECG feature vector to obtain a single-modal contrast loss; performing a weighted summation of the cross-modal contrast loss and the single-modal contrast loss to obtain a joint contrast loss; and iteratively training the initial ECG pre-training model to minimize the joint contrast loss to obtain the ECG pre-training model.

[0015] Optionally, the rare disease feature dictionary is constructed based on historical medical record text data, including: for each rare disease, inputting the historical medical record text data of the corresponding patient into the medical text pre-training model to obtain the text feature vector corresponding to the rare disease; multiple text feature vectors corresponding to the rare diseases constitute the rare disease feature dictionary.

[0016] According to another aspect of this application, an apparatus for intelligent identification of rare disease risk using electrocardiograms is provided, comprising: a first acquisition unit, configured to acquire multiple electrocardiogram data to be evaluated and corresponding medical record text data to be evaluated, wherein the medical record text data to be evaluated includes at least patient information and electrocardiogram report conclusions, and the electrocardiogram data to be evaluated includes non-rare disease electrocardiogram data and rare disease electrocardiogram data; a first input unit, configured to input the electrocardiogram data to be evaluated into an electrocardiogram pre-training model to obtain a target electrocardiogram feature vector, wherein the electrocardiogram pre-training model is trained based on multiple historical medical record text data and corresponding historical electrocardiogram data; and a second input unit, configured to input the medical record text data to be evaluated corresponding to the electrocardiogram data to be evaluated into a medical text pre-training model. The model is trained to obtain the target text feature vector; the third input unit is used to input the ECG data to be evaluated into a preset risk detection model to obtain the ECG rare disease occurrence probability coefficient; the fourth input unit is used to call a preset rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; the determination unit is used to determine the occurrence probability of each rare disease based on the similarity between the target ECG feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the ECG rare disease occurrence probability coefficient, wherein the reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0017] According to another aspect of this application, an electrocardiogram (ECG) analysis system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of the methods described.

[0018] By applying the technical solution of this application, the above-mentioned method for intelligent identification of rare disease risks using electrocardiograms employs a collaborative architecture of three-level pre-training models: an electrocardiogram pre-training model, a medical text pre-training model, and a risk detection model. This model is jointly trained across two modalities: text and electrocardiogram signals. Disease knowledge is associated with electrocardiogram signal information, and combined with an electrocardiogram rare disease risk score, thereby enhancing the rare disease identification performance centered on electrocardiograms. A qualified model can be trained using relatively few rare disease patient sample data to determine the probability of rare disease occurrence, significantly reducing the need for rare disease patient sample data. It eliminates the need for a large number of rare disease electrocardiogram samples for training, solving the problem in existing electrocardiogram rare disease identification training that requires a large number of rare disease electrocardiogram samples. Furthermore, cross-modal training helps to fully utilize a large amount of clinical data to deeply explore potential disease patterns hidden in electrocardiogram signals, improving rare disease identification performance. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 A hardware structure block diagram of a mobile terminal for performing a rare disease occurrence probability prediction method is shown in an embodiment of this application; Figure 2 A flowchart illustrating a method for predicting the probability of occurrence of a rare disease according to an embodiment of this application is shown. Figure 3 A flowchart illustrating a method for predicting the probability of occurrence of a rare disease according to an embodiment of this application is shown. Figure 4 A structural block diagram of a rare disease occurrence probability prediction device provided according to an embodiment of this application is shown.

[0020] The above figures include the following reference numerals: 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] As described in the background section, the training of rare disease identification using existing electrocardiograms requires a large number of rare disease electrocardiogram samples. To solve this technical problem, embodiments of this application provide a method, apparatus, and electrocardiogram analysis system for predicting the probability of rare disease occurrence.

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0026] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a rare disease occurrence probability prediction method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the rare disease occurrence probability prediction method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one instance, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] This embodiment provides a method for predicting the probability of rare disease occurrence that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 2 This is a flowchart of a rare disease occurrence probability prediction method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps: Step S201: Obtain multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The aforementioned medical record text data to be evaluated includes at least patient information and ECG report conclusions. The aforementioned ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Specifically, to obtain the medical record text data to be evaluated, at least two parts of the target subject's medical record text data need to be collected and concatenated: patient information and electrocardiogram (ECG) report conclusion. Patient information includes, but is not limited to, gender, age, chief complaint, and present medical history. Some missing information is allowed. The text should be organized according to a fixed format. The template for the medical record text data to be evaluated is as follows: "Gender is {A1}, Age is {A2}. Chief complaint is {C}. Present medical history is {P}. ECG report conclusion is {R}". Here, A1, A2, C, P, and R represent different modules. An example text is as follows: "Gender is {Male}, Age is {56 years old}. Chief complaint is {Recently frequent chest tightness and shortness of breath, with a significant decrease in exercise tolerance in the past month}. Present medical history is {Hypertension for 10 years}. ECG report conclusion is {Sinus rhythm, mild ST segment depression}." In addition, 12-lead 10s 500Hz ECG signal data should be collected from the target subject to obtain a 12×5000 digital signal matrix, which will then yield the corresponding ECG data to be evaluated.

[0030] Step S202: Input the above-mentioned ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The above-mentioned ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Specifically, the historical medical record text data and historical electrocardiogram (ECG) data are anonymized versions of the medical record text data and ECG data from the database. The historical medical record text data includes disease labels, which are presented as a separate module to describe the disease diagnosis. Example text is as follows: {D0, D1, D2}, where D0 represents a rare disease, and D1 and D2 represent common diseases. The historical medical record text data and historical ECG data are also organized according to the template format of the ECG data and medical record text data to be evaluated. The ECG pre-trained model trained using the historical medical record text data and the corresponding historical ECG data can transform the ECG data to be evaluated into a target ECG feature vector to extract features related to rare diseases.

[0031] Step S203: Input the above-mentioned medical record text data corresponding to the above-mentioned ECG data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Specifically, the aforementioned medical text pre-training model is trained based on multiple historical medical record text data. The medical text pre-training model is used to transform the aforementioned medical record text data to be evaluated into target text feature vectors. Through mask training, the ability to capture language structure is enhanced. Even if some information of the aforementioned medical record text data to be evaluated is missing, a target text feature vector that is basically the same as the text feature vector corresponding to the medical record text data to be evaluated with complete information can be obtained.

[0032] Step S204: Input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Specifically, the ECG data to be evaluated is input into a preset risk detection model to obtain the probability coefficient of rare ECG diseases, so as to assess the risk of developing rare diseases.

[0033] Step S205: Preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; Specifically, a rare disease feature dictionary is pre-defined, which includes text feature vectors corresponding to each rare disease. The method for constructing the rare disease feature dictionary will be explained in detail later.

[0034] Step S206: Based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease, determine the probability of occurrence of each of the rare diseases. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0035] Specifically, the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases is the cross-modal similarity, and the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases is the unimodal similarity. The probability of occurrence of each rare disease is determined by comprehensively considering the cross-modal similarity, unimodal similarity, and ECG rare disease occurrence probability coefficient. Compared with the discrimination model based on unimodal feature similarity, the discrimination performance is enhanced by dynamically weighting the cross-modal feature similarity and ECG rare disease occurrence probability coefficient.

[0036] In this embodiment, the above-mentioned method for intelligent identification of rare disease risks using electrocardiograms employs a collaborative architecture of three-level pre-training models: an electrocardiogram pre-training model, a medical text pre-training model, and a risk detection model. This model is jointly trained across two modalities: text and electrocardiogram signals. Disease knowledge is associated with electrocardiogram signal information, and combined with an electrocardiogram rare disease risk score, enhancing the rare disease identification performance centered on electrocardiograms. A qualified model can be trained using relatively few rare disease patient samples to determine the probability of rare disease occurrence, significantly reducing the need for large amounts of rare disease patient sample data. This solves the problem of existing electrocardiogram rare disease identification training requiring a large number of rare disease electrocardiogram samples. Furthermore, cross-modal training helps to fully utilize a large amount of clinical data to deeply explore potential disease patterns hidden in electrocardiogram signals, improving rare disease identification performance.

[0037] To train a pre-trained medical text model, one optional implementation involves training the model based on multiple historical medical record text data, specifically including: Step S301: Obtain multiple historical medical record text data, which include at least patient information, electrocardiogram report conclusions, and disease tags; Step S302: Use a portion of the words from the aforementioned historical medical record text data for masking to obtain the first masked text. Use the first masked text to perform masked pre-training on the text pre-training model to obtain the initial medical text pre-training model. Step S303: Masking is performed on a portion of the aforementioned historical medical record text data to obtain a second masked text. The second masked text is then used to perform masked pre-training on the aforementioned initial medical text pre-training model to obtain the aforementioned medical text pre-training model. The aforementioned patient information includes gender, age, chief complaint, and present medical history. The aforementioned module is one of the aforementioned chief complaint, the aforementioned present medical history, the aforementioned electrocardiogram report conclusion, and the aforementioned disease label.

[0038] In the above implementation, compared to the medical record text data to be evaluated, the historical medical record text data also includes disease labels. That is, the template of the historical medical record text data also includes module D. Other modules are the same, but the content is different. Module D includes multiple disease labels, for example, {D0, D1, D2}, where D0 represents a rare disease, and D1 and D2 represent common diseases. For example, the disease labels are {coronary atherosclerotic heart disease, unstable angina}. The text pre-training model is used to convert the medical record text data into text feature vectors. A partial word masking process is performed on the aforementioned historical medical record text data to obtain the first masked text. The text pre-training model is then pre-trained using the first masked text. Some input words are randomly masked, and the text pre-training model needs to predict the masked content based on the context. This method forces the text pre-training model to deeply understand bidirectional semantic relationships, enhances its ability to capture language structure, and thus improves its performance in text classification. This allows the trained initial medical text pre-training model to convert medical record text data even when some words are missing. To obtain the text feature vector corresponding to medical record text data without missing words, a portion of the aforementioned historical medical record text data is masked to obtain a second masked text. This second masked text is then used to pre-train the initial medical text pre-training model. Some modules are randomly masked, and the text pre-training model needs to predict the masked content based on the relationships between modules. This approach forces the text pre-training model to deeply understand the relationships between modules, enabling the trained medical text pre-training model to transform medical record text data into text feature vectors corresponding to medical record text data without missing modules, even when some modules are missing. Thus, even when there is information missing in the medical record text data to be evaluated, it is still possible to predict the probability of rare diseases. The word and module-level text model training method effectively integrates the relationships between diseases and the relationship between diseases and ECG report conclusions. Moreover, the diseases are not limited to rare diseases, which helps to fully utilize clinically known prior knowledge of diseases and enhance the model's intrinsic understanding and feature representation of diseases.

[0039] To achieve word-level mask pre-training, in one optional implementation, step S302 above includes: Step S3021: Perform text segmentation and serialization on the above-mentioned historical medical record text data to obtain a target vocabulary and multiple first-formatted sequences; In the above implementation, the dataset composed of multiple historical medical record text data is denoted as . First, text segmentation is performed to obtain the target vocabulary and the initial word sequence for each text. Specifically, for the dataset... Each sample text in the code is segmented (e.g., using the jieba library) to generate an initial word sequence. Obtain the preset vocabulary of the text pre-trained model LM0. The sample texts refer to individual historical medical record text data. Each sample text corresponds to an initial word sequence. Based on the initial word sequences after word segmentation of all sample texts, a summarized vocabulary is obtained. Based on vocabulary Expanded vocabulary The glossary Add to vocabulary In this process, the final target vocabulary is obtained. For words In the middle, if If, then it is called the original word, if Then it is called a new word.

[0040] Secondly, the initial word sequence of each sample text is formatted. This includes inserting category and delimiter tags at the beginning and end to generate a formatted sequence. If the sample text has multiple sentences, a separator [SEP] is added to the end of each sentence. If the maximum sequence length supported by the pre-trained text model LM0 is L, if the sequence is too long or too short, it is truncated or padded to a preset length L, with the padding character being [PAD]. [CLS], [SEP], and [PAD] are all special markers, which can be used to obtain multiple first-formatted sequences.

[0041] Step S3022: Perform dynamic masking on some words of the multiple first formatted sequences to obtain multiple second formatted sequences, and use the second formatted sequences as the first masked text; For the first formatted sequence A certain percentage (e.g., 15%) of non-special marker words are randomly selected as the masked words, forming a masked word set. For each selected word Based on probability, the following processing is performed: 80% probability of being replaced with [MASK], 10% probability of being replaced with other words. 10% probability of retaining the original word Unchanged. Dynamic masking can enhance the generalization ability of the model. The formatted sequence after dynamic masking is denoted as... This is the second formatted sequence, with a length of L. The second formatted sequence is used as the first mask text.

[0042] Step S3023: Obtain the first label vector corresponding to the masked word through the target vocabulary. .

[0043] Specifically, a mapping table from words to indices is established based on the target vocabulary. For each word being masked, its corresponding index in the mapping table is obtained, and the corresponding first label vector is retrieved based on the index. The label vector is a one-hot vector, meaning the dimension of the vector is equal to the size of the vocabulary, with the index position set to 1 and the remaining positions set to 0.

[0044] Step S3024: Convert the multiple second formatted sequences into multiple first text vectors. The first text vectors are composed of a first word embedding vector, a first position embedding vector, and a first segment embedding vector. The first word embedding vector is used to represent the semantic information of the words in the second formatted sequence. The first position embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. The first segment embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. Input the multiple first text vectors into the encoder of the text pre-training model to obtain multiple second text vectors. Based on multiple The sequence (second formatted sequence) is used as the first mask text and converted into multiple first text vectors, with one second formatted sequence corresponding to one first text vector. Specifically, it constructs... The sequence corresponds to the first text vector , where L is Sequence length, which is the number of words contained in the second-formatted sequence; The input vector corresponding to each word is the first text sub-vector, which includes the first word embedding vector. First position embedding vector and the first segment embedding vector And the first text subvector First word embedding vector To help the model understand the semantic and contextual information of each word, the original words are input into the text, and the pre-trained model LM0 obtains the word embedding vectors of the original words. The word embedding vector of a new word can be initialized in several ways, such as by initializing it to the average value of existing word embedding vectors; the first position embedding vector... This is used to represent the positional information of each word in the input text, enabling the model to capture the relative positional relationships of words in a sentence, thereby better understanding the semantics of the sentence. The positional embedding is a learnable parameter, and its initial value can be set through random initialization; the first segment embedding vector To distinguish different sentences in the input text, the position of each word in the sentence can be marked. For example, all words in the first sentence would be marked as 0, all words in the second sentence would be marked as 1, and so on, for each of the above second formatted sequences. Convert to multiple first text vectors Multiple first text vectors are input into the encoder of the pre-trained text model to obtain multiple second text vectors. The second text vector has the same dimension as the first text vector. Step S3025: Input the plurality of second text vectors into the prediction head of the text pre-training model to obtain a plurality of first prediction vectors; During training, the second text vector is input batch by batch into the prediction head of the pre-trained text model LM0 to obtain the first prediction vector P. The dimension of the first prediction vector P is d. V It is the same size as the target vocabulary.

[0045] Step S3026: Obtain the first prediction vector corresponding to the word being dynamically masked based on the first prediction vector; The first prediction vector includes the first prediction vector corresponding to the unmasked words and the first prediction vector corresponding to the words that are dynamically masked. During the previous masking, it was recorded which positions were masked. The first prediction vector corresponding to the words that are dynamically masked can be filtered out by directly obtaining the corresponding prediction vector based on the mask position (or index).

[0046] Step S3027: Calculate the sum of the losses of the first prediction vectors and the corresponding first label vectors of the multiple words that have been dynamically masked, and obtain the first loss function value; Step S3028: Iteratively train the above-mentioned text pre-trained model to minimize the value of the first loss function and obtain the above-mentioned initial medical text pre-trained model.

[0047] The loss function for model training is the average loss of the masked words. For each masked word, the first label vector corresponding to the masked word is used as the input vector, and the first prediction vector corresponding to the masked word is used as the output vector. The cross-entropy loss of the input and output vectors is calculated separately. Based on the cross-entropy loss, the first loss function value is calculated, i.e. , in, The set of words that are masked. Representative set The number of mask words in the text Represents the cross-entropy loss function. Indicator The label vector, Indicator The predicted vector. The above text pre-trained model is iteratively trained to minimize the value of the first loss function, thus obtaining the initial medical text pre-trained model.

[0048] To achieve module-level mask pre-training, in one optional implementation, step S303 above includes: Step S3031: Perform text segmentation and serialization on the above-mentioned historical medical record text data to obtain a target vocabulary and multiple first-formatted sequences; Step S3032: Dynamically mask the words in any module of the multiple first formatted sequences to obtain multiple third formatted sequences, and use the third formatted sequences as the second masked text; Step S3033: Obtain the second tag vector corresponding to the masked module through the target vocabulary; Step S3034: Convert the multiple third formatted sequences into multiple third text vectors. The third text vectors are composed of a second word embedding vector, a second position embedding vector, and a second segment embedding vector. The second word embedding vector is used to represent the semantic information of the words in the third formatted sequence. The second position embedding vector is used to represent the position of the words in the third formatted sequence in the sentence. The second segment embedding vector is used to represent the position of the module in which the words in the third formatted sequence are located. Input the multiple third text vectors into the encoder of the initial medical text pre-training model to obtain multiple fourth text vectors. Step S3035: Input the above-mentioned multiple fourth text vectors into the prediction head of the above-mentioned text pre-training model to obtain multiple second prediction vectors; Step S3036: Obtain the second prediction vector corresponding to the module being dynamically masked based on the second prediction vector described above; Step S3037: Calculate the sum of the losses of the second prediction vectors and the corresponding second label vectors of the multiple modules that have undergone dynamic masking to obtain the second loss function value; Step S3038: Iteratively train the initial medical text pre-training model to minimize the second loss function value to obtain the medical text pre-training model.

[0049] In the above embodiments, random module masking is performed on multiple first formatted sequences. For the four modules of chief complaint C, present medical history P, ECG report conclusion R, and disease label D in the multiple first formatted sequences, any one module of each first formatted sequence is randomly masked to obtain multiple third formatted sequences. All words in this module are masked, and the word masking method is also random masking. There is an 80% probability that the word will be replaced with [MASK], a 10% probability that it will be replaced with other words, and a 10% probability that it will remain unchanged. Similar to word masking pre-training, the obtained third text vector is obtained by summing three sets of vectors: the second word embedding vector, the second segment embedding vector, and the second position embedding vector. In particular, the second segment embedding vector is used to identify the module to which the word belongs. For example, it can be labeled according to the module order. The same training method is used to iteratively train the above-mentioned initial medical text pre-training model to obtain the above-mentioned medical text pre-training model.

[0050] To enhance discrimination performance, in one optional implementation, the training process of the above-mentioned ECG pre-training model includes: Step S401: Use multiple non-rare disease ECG data and multiple rare disease ECG data to perform mask pre-training on the ECG self-supervised pre-training model to obtain the first ECG pre-training model, and determine the encoder of the first ECG pre-training model as the initial ECG pre-training model. Step S402: The initial ECG pre-training model is compared and trained using the aforementioned historical medical record text data, the corresponding historical ECG data, and the aforementioned medical text pre-training model to obtain the aforementioned ECG pre-training model. The loss function of the aforementioned ECG pre-training model is used to calculate at least the loss of the text feature vector of the aforementioned historical medical record text data and the ECG feature vector of the corresponding historical ECG data. The aforementioned loss includes single-modal comparison loss and cross-modal comparison loss.

[0051] In the above implementation, multiple non-rare disease ECG data and multiple rare disease ECG data are used to perform masked pre-training on the ECG self-supervised pre-training model to obtain the first ECG pre-training model. Some input ECG data are randomly masked, and the model needs to predict the masked content based on the ECG data before and after the time series. The mean square error between the actual values ​​and predicted values ​​of the non-rare disease ECG data and the multiple rare disease ECG data is calculated as the loss function. When the function is lower than a preset threshold, the masked pre-training of the ECG self-supervised pre-training model is completed. This method can fully explore the inherent correlation and local feature dependence of time series waveforms, effectively increasing... The strong model's ability to learn representations of abnormal ECG waveforms and hidden pathological patterns was demonstrated by comparing the initial ECG pre-training model with the aforementioned historical medical record text data, corresponding historical ECG data, and the aforementioned medical text pre-training model. The resulting ECG pre-training model's loss function includes not only single-modal contrast loss but also cross-modal contrast loss. Compared to the single-modal feature similarity discrimination model, dynamic weighting of cross-modal feature similarity helps to fully utilize a large amount of clinical data to deeply explore potential disease patterns hidden in ECG signals, thereby enhancing discrimination performance.

[0052] To construct a pre-defined risk detection model, in one optional implementation, the pre-defined risk detection model is trained using non-rare disease electrocardiogram data, including: Step S501: Use multiple non-rare disease ECG data to perform masked pre-training on the ECG self-supervised pre-training model to obtain the second ECG pre-training model. Step S502: Construct the preset risk detection model based on the second ECG pre-training model and the loss function of the second ECG pre-training model. The preset risk detection model is a model that calculates the probability coefficient of rare ECG diseases using the loss function value of the second ECG pre-training model.

[0053] In the above implementation, firstly, a dataset containing only non-rare disease electrocardiogram signals is collected and formed. For example, 12-lead, 10-second, 500Hz electrocardiogram (ECG) signal data are collected, resulting in a 12×5000-dimensional digital signal matrix for each sample, which serves as the input to the ECG self-supervised pre-training model. Secondly, multiple sets of the aforementioned non-rare disease ECG data are used for masked pre-training of the ECG self-supervised pre-training model, resulting in a second ECG pre-training model. The output loss function value is denoted as... The probability coefficient r of rare electrocardiogram diseases is calculated based on the loss function value, and the formula is as follows: αr lossHere, α is a hyperparameter used to constrain the range of values ​​for the probability coefficient of rare ECG diseases. The specific value is usually obtained by optimizing the rare disease identification performance on the validation set, which contains ECG signal data of rare diseases. Therefore, the aforementioned pre-set risk detection model for calculating the probability coefficient of rare ECG diseases can be constructed based on the second ECG pre-trained model and its loss function. Since the training samples for the ECG rare disease risk detection model are non-rare ECG signal datasets... Therefore, the rare disease risk score generated based on the ECG rare disease risk detection model requires less rare disease sample data.

[0054] To achieve masked pre-training of the ECG self-supervised pre-training model, in one optional implementation, step S501 above includes: Step S5011: Divide the above non-rare disease ECG data into multiple segments along the time dimension; Step S5012: Masking is performed on a portion of the data to obtain masked ECG data for non-rare diseases; Step S5013: Input the above non-rare disease masked ECG data into the above ECG self-supervised pre-training model to obtain reconstructed ECG data; Step S5014: Calculate the loss between the reconstructed ECG data and the non-rare disease ECG data to obtain the third loss function value; Step S5015: Iteratively train the above ECG self-supervised pre-trained model to minimize the value of the third loss function and obtain the above second ECG pre-trained model.

[0055] In the above implementation, the ECG self-supervised pre-training model is self-supervised to obtain the second ECG pre-training model ER. The model includes an encoder and a decoder. When processing non-rare disease ECG data, such as 12×5000 ECG signals, the data is divided into several non-overlapping segments (e.g., 200 segments) along the time dimension, and randomly divided into masked segments and unmasked segments proportionally (e.g., 25% of the segments are masked). The masking operation is performed simultaneously on all 12 leads. The non-rare disease masked ECG data is input into the ECG self-supervised pre-training model. The goal of the pre-training is to reconstruct the segmentally normalized masked segments using the ECG signals from the unmasked segments. The third loss function is the mean squared error, which is the mean squared error between the true value and the predicted value of the ECG signal from the unmasked segment. The above ECG self-supervised pre-training model is iteratively trained to minimize the value of the third loss function, resulting in the second ECG pre-training model.

[0056] In order to calculate the probability of occurrence of rare diseases, in one optional implementation, step S206 above includes: Step S2061: Perform the first calculation step, calculate the maximum similarity between the target ECG feature vector and each of the text feature vectors of the target rare disease in the rare disease feature dictionary, and obtain the first similarity; calculate the maximum similarity between the target text feature vector and each of the text feature vectors of the target rare disease in the rare disease feature dictionary, and obtain the second similarity; the target rare disease is any of the rare diseases mentioned above. Step S2062: Perform the second calculation step to aggregate the first similarity and the second similarity to obtain the probability of occurrence of the target rare disease. Step S2063: Repeat the first calculation step and the second calculation step above at least once, until the occurrence probability of all the above rare diseases is obtained.

[0057] In the above embodiments, the rare disease feature dictionary is: Where J is the total number of rare diseases, d j For the jth rare disease, Let be the text feature vector corresponding to the j-th rare disease; for example, the text feature vector of the i-th rare disease is . , Let e ​​represent the number of samples for the i-th rare disease, and calculate the target electrocardiogram feature vector e and the text feature vector for the i-th rare disease. The cosine similarity is used to select the maximum value to obtain the first similarity score. Calculate the feature vector t of the target text and the feature vector of the i-th rare disease text. The cosine similarity is used to select the maximum value to obtain the second similarity. Calculate the similarity after aggregation , which represents the probability of the occurrence of the i-th rare disease. By assessing the probability, doctors can be assisted in diagnosing rare diseases and improve their diagnostic accuracy.

[0058] To improve the accuracy of rare disease incidence probability, in one optional implementation, step S2062 includes: Step S20621: Input the ECG data to be evaluated into the above-mentioned preset risk detection model to obtain the probability coefficient of rare ECG diseases. Step S20622: The first similarity and the second similarity are aggregated using the above-mentioned electrocardiogram rare disease occurrence probability coefficient to obtain the occurrence probability of the target rare disease.

[0059] In the above implementation, the similarity after risk-enhanced aggregation is calculated. ,in, The probability of occurrence of the i-th rare disease is obtained, which is the probability of occurrence of the target rare disease. The correlation between electrocardiogram signals and rare diseases is explored, and a risk enhancement algorithm is used to achieve high-precision prediction of rare disease risk, thereby promoting early identification and intervention management of diseases.

[0060] In an optional implementation of training the ECG pre-training model, step S402 above includes: Step S4021: Input the above-mentioned historical medical record text data into the above-mentioned medical text pre-training model to obtain the training text feature vector; Step S4022: Input the ECG data corresponding to the historical medical record text data into the initial ECG pre-training model to obtain the trained ECG feature vector. Step S4023: Calculate the loss of the above-mentioned training text feature vector and the corresponding above-mentioned training ECG feature vector to obtain the cross-modal contrast loss; Step S4024: Two data augmentation methods are used to process the above-mentioned trained ECG feature vectors to obtain the first ECG feature vector and the second ECG feature vector. Step S4025: Calculate the loss of the first ECG feature vector and the second ECG feature vector to obtain the single-modal contrast loss; Step S4026: The cross-modal contrast loss and the single-modal contrast loss are weighted and summed to obtain the joint contrast loss; Step S4027: Iteratively train the above initial ECG pre-trained model to minimize the above joint contrast loss, and obtain the above ECG pre-trained model.

[0061] In the above embodiments, the training process of the above-mentioned ECG pre-training model is as follows: Figure 3 As shown, historical medical record text data is input into a medical text pre-training model to obtain the target text feature vector. Where N is the number of samples, historical ECG data is input into the ECG pre-training model to obtain the target ECG feature vector. Data augmentation methods 1 and 2 are used to process the target ECG feature vector to obtain the first ECG feature vector. Second ECG feature vector The loss of the first ECG feature vector and the second ECG feature vector is calculated to obtain the single-modal contrast loss. ,Right now ,in, , The cross-modal contrast loss is obtained by calculating the loss between the training text feature vector and the corresponding training ECG feature vector. ,Right now ,in, , The weighted sum of the cross-modal contrast loss and the single-modal contrast loss yields the joint contrast loss L, which is the loss function value of the ECG pre-training model. ,in, and For hyperparameters, Here, is the cosine similarity calculation function, N is the number of samples, and i and k are both positive integers between 1 and N. Let be the first ECG feature vector of the k-th sample. Let's consider the second ECG feature vector of the k-th sample. The training objective includes two aspects: firstly, to enhance the ECG feature vector of the same sample. and The first principle is to make the ECG feature vectors as similar as possible to those of other enhanced samples, while also ensuring they differ as much as possible from the enhanced ECG feature vectors of other samples. Text feature vectors paired with it The model should be as similar as possible to other samples, with as many differences as possible between the ECG feature vectors and text feature vectors. Therefore, by minimizing the joint contrast loss, the initial ECG pre-trained model can be iteratively trained to obtain the ECG pre-trained model.

[0062] Specifically, the process of obtaining training samples for the aforementioned ECG pre-training model involves: multiple historical medical record text data. Composed of multiple historical electrocardiogram data We collected paired electrocardiogram signals and medical record text data to form an initial signal-text paired dataset. The text is also organized according to the following template: "Gender is {A1}, Age is {A2}, Chief Complaint is {C}, Present Illness is {P}, ECG Report Conclusion is {R}, Disease Label is {D}". Optionally, to enhance the learning and understanding of knowledge related to rare diseases, the dataset... For samples related to rare disease diagnosis, resampling is performed using the same sampling strategy as above, where the sampling ratio can be set based on the diagnostic distribution in the actual scenario data. Optionally, the sampling strategy can be set as follows: For new samples generated through resampling, other module data remain unchanged, but the disease labels are randomized. Specifically, taking disease labels {D0, D1, D2} as an example, where D0 represents a rare disease and D1 and D2 represent common diseases, the disease labels of the new samples can be randomized to any one of these four types: {D0}, {D0, D1}, {D0, D2}, {D0, D1, D2}, to enhance the richness of the samples and strengthen the model's semantic information understanding. The final paired dataset is denoted as... .

[0063] To construct a rare disease feature dictionary, in one optional implementation, the rare disease feature dictionary is constructed based on historical medical record text data, including: Step S601: For each rare disease, input the above-mentioned historical medical record text data of the corresponding patient into the above-mentioned medical text pre-training model to obtain the text feature vector corresponding to the above-mentioned rare disease. Step S602: The text feature vectors of multiple rare diseases constitute the above-mentioned rare disease feature dictionary.

[0064] In the above embodiments, the medical record text dataset is based on multiple historical medical record text data. For each rare disease of interest, the historical medical records of the corresponding patient population are input into the medical text pre-training model LM to obtain text feature vectors, forming a text feature vector library for that rare disease. Multiple text feature vector libraries for that rare disease constitute a rare disease feature dictionary. }, where J represents the total number of rare diseases. This represents the number of samples for the j-th rare disease. m for the jth rare disease j A set of text feature vectors. Dict={d1,…d J}, where J represents the total number of rare diseases, and for a specific rare disease, such as the i-th rare disease, the corresponding text feature vector is} ,in This represents the number of samples corresponding to rare disease i.

[0065] This application also provides a rare disease occurrence probability prediction device. It should be noted that the rare disease occurrence probability prediction device of this application embodiment can be used to execute the rare disease occurrence probability prediction method provided in this application embodiment. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0066] The following describes the rare disease occurrence probability prediction device provided in the embodiments of this application.

[0067] Figure 4 This is a schematic diagram of a rare disease occurrence probability prediction device according to an embodiment of this application. Figure 4 As shown, the device includes: The first acquisition unit 10 is used to acquire multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The medical record text data to be evaluated includes at least patient information and ECG report conclusions. The ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Specifically, to obtain the medical record text data to be evaluated, at least two parts of medical record text data need to be collected and spliced ​​together: patient information and electrocardiogram (ECG) report conclusion. Patient information includes, but is not limited to, gender, age, chief complaint, and present medical history. Some missing information is allowed. The text should be organized according to a fixed format. The template for the medical record text data to be evaluated is as follows: "Gender is {A1}, Age is {A2}. Chief complaint is {C}. Present medical history is {P}. ECG report conclusion is {R}". Here, A1, A2, C, P, and R represent different modules. An example text is as follows: "Gender is {Male}, Age is {56 years old}. Chief complaint is {Recently frequent chest tightness and shortness of breath, with a significant decrease in exercise tolerance in the past month}. Present medical history is {Hypertension for 10 years}. ECG report conclusion is {Sinus rhythm, mild ST segment depression}." In addition, 12-lead 10s 500Hz ECG signal data is collected from the target subject to obtain a 12×5000 digital signal matrix, which yields the ECG data to be evaluated.

[0068] The first input unit 20 is used to input the above-mentioned ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Specifically, the historical medical record text data and historical electrocardiogram (ECG) data are anonymized versions of the medical record text data and ECG data from the database. The historical medical record text data includes disease tags for rare diseases. These disease tags are presented as a separate module to describe the disease diagnosis, with an example text as follows: {D0, D1, D2}, where D0 represents a rare disease, and D1 and D2 represent common diseases. The historical medical record text data and historical ECG data are also organized according to the template format of the ECG data and medical record text data to be evaluated. The ECG pre-trained model trained using the historical medical record text data and the corresponding historical ECG data can transform the ECG data to be evaluated into a target ECG feature vector to extract features related to rare diseases.

[0069] The second input unit 30 is used to input the above-mentioned medical record text data corresponding to the above-mentioned electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Specifically, the aforementioned medical text pre-training model is trained based on multiple historical medical record text data. The medical text pre-training model is used to transform the aforementioned medical record text data to be evaluated into target text feature vectors. Through mask training, the ability to capture language structure is enhanced. Even if some information of the aforementioned medical record text data to be evaluated is missing, a target text feature vector that is basically the same as the text feature vector corresponding to the medical record text data to be evaluated with complete information can be obtained.

[0070] The third input unit 40 is used to input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Specifically, the ECG data to be evaluated is input into a preset risk detection model to obtain the probability coefficient of rare ECG diseases, so as to assess the risk of developing rare diseases.

[0071] The fourth input unit 50 is used to preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease. Specifically, a rare disease feature dictionary is pre-defined, which includes text feature vectors corresponding to each rare disease. The method for constructing the rare disease feature dictionary will be explained in detail later.

[0072] The determining unit 60 is used to determine the probability of occurrence of each of the rare diseases based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0073] Specifically, the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases is the cross-modal similarity, and the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases is the unimodal similarity. The probability of occurrence of each rare disease is determined by comprehensively considering the cross-modal similarity, unimodal similarity, and ECG rare disease occurrence probability coefficient. Compared with the discrimination model based on unimodal feature similarity, the discrimination performance is enhanced by dynamically weighting the cross-modal feature similarity and ECG rare disease occurrence probability coefficient.

[0074] In this embodiment, the aforementioned device for intelligent identification of rare disease risks using electrocardiograms employs a collaborative architecture of three-level pre-training models: an electrocardiogram pre-training model, a medical text pre-training model, and a risk detection model. This architecture enables joint training across both text and electrocardiogram signal modalities, linking disease knowledge with electrocardiogram signal information. Combined with an electrocardiogram rare disease risk score, this enhances the rare disease identification performance centered on electrocardiograms. A qualified model can be trained using relatively few rare disease patient sample data to determine the probability of rare disease occurrence, significantly reducing the need for rare disease patient sample data. It eliminates the need for a large number of rare disease electrocardiogram samples for training, solving the problem of existing electrocardiogram rare disease identification training requiring a large number of rare disease electrocardiogram samples. Furthermore, cross-modal training helps to fully utilize a large amount of clinical data to deeply explore potential disease patterns hidden in electrocardiogram signals, thereby improving rare disease identification performance.

[0075] In an optional implementation of the medical text pre-training model, the medical text pre-training model is trained based on multiple historical medical record text data. The apparatus further includes: The second acquisition unit is used to acquire multiple historical medical record text data, which include at least patient information, electrocardiogram report conclusions and disease labels. The first masking unit is used to mask a portion of the historical medical record text data to obtain the first mask text. The first mask text is then used to perform mask pre-training on the text pre-training model to obtain the initial medical text pre-training model. The second masking unit is used to perform masking processing on a portion of the aforementioned historical medical record text data to obtain a second mask text. The second mask text is then used to perform mask pre-training on the aforementioned initial medical text pre-training model to obtain the aforementioned medical text pre-training model. The aforementioned patient information includes gender, age, chief complaint, and present medical history. The aforementioned module is one of the aforementioned chief complaint, the aforementioned present medical history, the aforementioned electrocardiogram report conclusion, and the aforementioned disease label.

[0076] In the above implementation, compared to the medical record text data to be evaluated, the historical medical record text data also includes disease labels. That is, the template of the historical medical record text data also includes module D. Other modules are the same, but the content is different. Module D includes multiple disease labels, for example, {D0, D1, D2}, where D0 represents a rare disease, and D1 and D2 represent common diseases. For example, the disease labels are {coronary atherosclerotic heart disease, unstable angina}. The text pre-training model is used to convert the medical record text data into text feature vectors. A partial word masking process is performed on the aforementioned historical medical record text data to obtain the first masked text. The text pre-training model is then pre-trained using the first masked text. Some input words are randomly masked, and the text pre-training model needs to predict the masked content based on the context. This method forces the text pre-training model to deeply understand bidirectional semantic relationships, enhances its ability to capture language structure, and thus improves its performance in text classification. This allows the trained initial medical text pre-training model to convert medical record text data even when some words are missing. To obtain the text feature vector corresponding to medical record text data without missing words, a portion of the aforementioned historical medical record text data is masked to obtain a second masked text. This second masked text is then used to pre-train the initial medical text pre-training model. Some modules are randomly masked, and the text pre-training model needs to predict the masked content based on the relationships between modules. This approach forces the text pre-training model to deeply understand the relationships between modules, enabling the trained medical text pre-training model to transform medical record text data into text feature vectors corresponding to medical record text data without missing modules, even when some modules are missing. Thus, even when there is information missing in the medical record text data to be evaluated, it is still possible to predict the probability of rare diseases. The word and module-level text model training method effectively integrates the relationships between diseases and the relationship between diseases and ECG report conclusions. Moreover, the diseases are not limited to rare diseases, which helps to fully utilize clinically known prior knowledge of diseases and enhance the model's intrinsic understanding and feature representation of diseases.

[0077] To achieve word-level mask pre-training, in one optional implementation, the first mask unit includes: The first splitting module is used to perform text segmentation and serialization on the above-mentioned historical medical record text data to obtain a target vocabulary and multiple first-formatted sequences. The first masking module is used to dynamically mask part of the words in multiple first formatted sequences to obtain multiple second formatted sequences, and use the second formatted sequences as the first masked text. The first tag vector acquisition module obtains the first tag vector corresponding to the words that are dynamically masked through the target vocabulary; The first conversion module is used to convert multiple second formatted sequences into multiple first text vectors. The first text vectors are composed of a first word embedding vector, a first position embedding vector, and a first segment embedding vector. The first word embedding vector is used to represent the semantic information of the words in the second formatted sequence. The first position embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. The first segment embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. The multiple first text vectors are input into the encoder of the pre-trained text model to obtain multiple second text vectors. The first prediction module is used to input the above-mentioned multiple second text vectors into the prediction head of the above-mentioned text pre-training model to obtain multiple first prediction vectors; The first filtering module is used to obtain the first prediction vector corresponding to the word that is dynamically masked based on the first prediction vector mentioned above. The first calculation module is used to calculate the sum of the losses of the first prediction vectors and the corresponding first label vectors of the multiple words that have been dynamically masked, and to obtain the first loss function value. The first training module is used to iteratively train the above-mentioned text pre-trained model to minimize the value of the first loss function and obtain the above-mentioned initial medical text pre-trained model.

[0078] In the above implementation, the dataset composed of multiple historical medical record text data is denoted as . First, text segmentation is performed to obtain a vocabulary and an initial word sequence for each text. Specifically, for the dataset... Each sample text in the dataset is segmented (e.g., using the jieba library) to generate an initial word sequence Seq; the vocabulary of the pre-trained text model LM0 is then obtained. The summarized vocabulary is obtained based on the word sequences after word segmentation of all sample texts. Based on vocabulary Expanded vocabulary This yields the final vocabulary list. For words In the middle, if If, then it is called the original word, if These are then called new words. Next, the initial word sequence of each text is formatted. This includes inserting category and delimiter tags at the beginning and end to generate a formatted sequence. If the sample text has multiple sentences, add a separator [SEP] to the end of each sentence. If the maximum sequence length supported by the pre-trained text model LM0 is L, if the sequence is too long or too short, truncate or pad it to the preset length L, using [PAD] as the padding character. [CLS], [SEP], and [PAD] are all special markers, thus obtaining multiple first-formatted sequences. For the first-formatted sequences... A certain percentage (e.g., 15%) of non-special marker words are randomly selected as the masked words, forming a masked word set. For each selected word Based on probability, the following processing is performed: 80% probability of being replaced with [MASK], 10% probability of being replaced with other words. 10% probability of retaining the original word Unchanged. Dynamic masking can enhance the generalization ability of the model. The formatted sequence after dynamic masking is denoted as... , that is, the second formatted sequence, which has a length of L.

[0079] based on The sequence is used as the first mask text, which is then converted into multiple first text vectors. Specifically, this constructs the input for each sample. , where L is Sequence length, which is the number of words contained in the second-formatted sequence; The input vector corresponding to each word, i.e., the first text sub-vector, includes the first word embedding vector. First position embedding vector and the first segment embedding vector And the first text subvector First word embedding vector To help the model understand the semantic and contextual information of each word, the original words are input into the text, and the pre-trained model LM0 obtains the word embedding vectors of the original words. The word embedding vector of a new word can be initialized in several ways, such as by initializing it to the average value of existing word vectors; the first position embedding vector This is used to represent the positional information of each word in the input text, enabling the model to capture the relative positional relationships of words in a sentence, thereby better understanding the semantics of the sentence. The positional embedding is a learnable parameter, and its initial value can be set through random initialization; the first segment embedding vector To distinguish different sentences in the input text, the position of each word in the sentence can be marked. For example, all words in the first sentence would be marked as 0, all words in the second sentence would be marked as 1, and so on, for multiple of the above first formatted sequences. Convert to multiple first text vectors During training, the first text vector of each sample is... In the encoder of the batch-trained input text model LM0, the second text vector obtained is denoted as... The output vector has the same dimension as the input vector. During training, the second text vector is input batch by batch into the prediction head of the pre-trained model LM0 to obtain the first prediction vector P. The dimension of the first prediction vector P is d. V The size is the same as the target vocabulary. The loss function for model training is the average loss of the masked words. For each masked word, the first label vector corresponding to the masked word is used as the input vector, and the first prediction vector corresponding to the masked word is used as the output vector. The cross-entropy loss of the input vector and the output vector are calculated respectively. The first loss function value is obtained based on the cross-entropy loss, i.e. ,in, The set of words that are masked. Representative set The number of mask words in the text Represents the cross-entropy loss function. Indicator The label vector, Indicator The predicted vector. The above text pre-trained model is iteratively trained to minimize the value of the first loss function, thus obtaining the initial medical text pre-trained model.

[0080] To achieve module-level mask pre-training, in one optional implementation, the second mask unit includes: The second splitting module is used to perform text segmentation and serialization on the above-mentioned historical medical record text data to obtain a target vocabulary and multiple first-formatted sequences. The second masking module is used to dynamically mask the words in any of the modules of the multiple first formatting sequences to obtain multiple third formatting sequences, and the third formatting sequences are used as the second masking text. The second tag vector acquisition module is used to obtain the second tag vector corresponding to the masked module through the target vocabulary; The second conversion module converts the multiple third formatted sequences into multiple third text vectors. Each third text vector consists of a second word embedding vector, a second position embedding vector, and a second segment embedding vector. The second word embedding vector is used to represent the semantic information of the words in the third formatted sequence. The second position embedding vector is used to represent the position of the words in the third formatted sequence within the sentence. The second segment embedding vector is used to represent the position of the module in which the words in the third formatted sequence are located. The multiple third text vectors are then input into the encoder of the initial medical text pre-training model to obtain multiple fourth text vectors. The second prediction module is used to input the above-mentioned multiple fourth text vectors into the prediction head of the above-mentioned text pre-training model to obtain multiple second prediction vectors; The second filtering module obtains the second prediction vector corresponding to the module that is dynamically masked based on the aforementioned second prediction vector. The second calculation module calculates the sum of the losses of the second prediction vectors and the corresponding second label vectors of the multiple modules that have undergone dynamic masking to obtain the second loss function value. The second training module is used to iteratively train the initial medical text pre-trained model to minimize the value of the second loss function and obtain the medical text pre-trained model.

[0081] In the above embodiments, random module masking is performed on multiple first formatted sequences. For the four modules of chief complaint C, present medical history P, ECG report conclusion R, and disease label D in the multiple first formatted sequences, any one module of each first formatted sequence is randomly masked to obtain multiple third formatted sequences. All words in this module are masked, and the word masking method is also random masking. There is an 80% probability that the word will be replaced with [MASK], a 10% probability that it will be replaced with other words, and a 10% probability that it will remain unchanged. Similar to word masking pre-training, the obtained third text vector is obtained by summing three sets of vectors: the second word embedding vector, the second segment embedding vector, and the second position embedding vector. In particular, the second segment embedding vector is used to identify the module to which the word belongs. For example, it can be labeled according to the module order. The same training method is used to iteratively train the above-mentioned initial medical text pre-training model to obtain the above-mentioned medical text pre-training model.

[0082] To enhance the discrimination performance, in an optional embodiment, the above-mentioned device further includes: The first training unit is used to perform mask pre-training on the ECG self-supervised pre-training model using multiple non-rare disease ECG data and multiple rare disease ECG data to obtain the first ECG pre-training model, and the encoder of the first ECG pre-training model is determined as the initial ECG pre-training model. The second training unit is used to compare and train the initial ECG pre-training model with the aforementioned historical medical record text data, the corresponding historical ECG data, and the aforementioned medical text pre-training model to obtain the aforementioned ECG pre-training model. The loss function of the aforementioned ECG pre-training model is used to calculate at least the loss of the text feature vector of the aforementioned historical medical record text data and the ECG feature vector of the corresponding historical ECG data. The aforementioned loss includes single-modal comparison loss and cross-modal comparison loss.

[0083] In the above embodiments, multiple non-rare disease ECG data and multiple rare disease ECG data are used to perform masked pre-training on the ECG self-supervised pre-training model to obtain the first ECG pre-training model. Some input ECG data is randomly masked, and the model needs to predict the masked content based on ECG data before and after the time series. This approach forces the model to deeply understand the characteristics of rare disease ECG data and non-rare disease ECG data, enhancing its ability to capture rare disease features. This ensures that even if the ECG data to be evaluated has some missing parts, it will not affect the identification of rare diseases. The above-mentioned historical medical record text data, the corresponding historical ECG data, and the above-mentioned medical text pre-training model are used to compare and train the above-mentioned initial ECG pre-training model to obtain the above-mentioned ECG pre-training model. The loss function of the ECG pre-training model includes not only single-modal contrast loss but also cross-modal contrast loss. Compared with the discrimination model based on single-modal feature similarity, the dynamic weighting of cross-modal feature similarity helps to fully utilize a large amount of clinical data to deeply explore the potential disease patterns hidden in the ECG signal, thereby enhancing the discrimination performance.

[0084] To construct a preset risk detection model, in one optional implementation, the preset risk detection model is trained using non-rare disease electrocardiogram data, and the device further includes: The third training unit is used to perform masked pre-training of the ECG self-supervised pre-training model using multiple ECG data of the above-mentioned non-rare diseases to obtain the second ECG pre-training model. The construction unit is used to construct the preset risk detection model based on the second ECG pre-training model and the loss function of the second ECG pre-training model. The preset risk detection model is a model that calculates the probability coefficient of rare ECG diseases using the loss function value of the second ECG pre-training model.

[0085] In the above implementation, firstly, a dataset containing only non-rare disease electrocardiogram signals is collected and formed. For example, 12-lead, 10-second, 500Hz electrocardiogram (ECG) signal data are collected, resulting in a 12×5000-dimensional digital signal matrix for each sample, which serves as the input to the ECG self-supervised pre-training model. Secondly, multiple sets of the aforementioned non-rare disease ECG data are used for masked pre-training of the ECG self-supervised pre-training model, resulting in a second ECG pre-training model. The output loss function value is denoted as... The probability coefficient r of rare electrocardiogram diseases is calculated based on the loss function value, as shown in the following formula: Here, α is a hyperparameter used to constrain the range of values ​​for the probability coefficient of rare ECG diseases. The specific value is usually obtained by optimizing the rare disease identification performance on the validation set. That is, the aforementioned pre-set risk detection model for calculating the probability coefficient of rare ECG diseases is constructed based on the aforementioned second ECG pre-trained model and its loss function. Since the training samples for the ECG rare disease risk detection model are non-rare disease ECG signal datasets... Therefore, the rare disease risk score generated based on the ECG rare disease risk detection model requires less rare disease sample data.

[0086] To achieve masked pre-training of the ECG self-supervised pre-training model, in one optional implementation, the third training unit includes: The segmentation module is used to divide the above non-rare disease ECG data into multiple segments along the time dimension; The third masking module is used to mask certain segments to obtain masked ECG data for non-rare diseases. The reconstruction module is used to input the above-mentioned non-rare disease masked ECG data into the above-mentioned ECG self-supervised pre-trained model to obtain reconstructed ECG data. The third calculation module is used to calculate the loss between the reconstructed ECG data and the non-rare disease ECG data to obtain the third loss function value. The third training module is used to iteratively train the above-mentioned ECG self-supervised pre-training model to minimize the value of the third loss function and obtain the above-mentioned second ECG pre-training model.

[0087] In the above implementation, the ECG self-supervised pre-training model is self-supervised to obtain the second ECG pre-training model ER. The model includes an encoder and a decoder. When processing non-rare disease ECG data, such as 12×5000 ECG signals, the data is divided into several non-overlapping segments (e.g., 200 segments) along the time dimension, and randomly divided into masked segments and unmasked segments proportionally (e.g., 25% of the segments are masked). The masking operation is performed simultaneously on all 12 leads. The non-rare disease masked ECG data is input into the ECG self-supervised pre-training model. The goal of the pre-training is to reconstruct the segmentally normalized masked segments using the ECG signals from the unmasked segments. The third loss function is the mean squared error, which is the mean squared error between the true value and the predicted value of the ECG signal from the unmasked segment. The above ECG self-supervised pre-training model is iteratively trained to minimize the value of the third loss function, resulting in the second ECG pre-training model.

[0088] In an optional implementation for calculating the probability of rare diseases, the determining unit includes: The fourth calculation module is used to perform the first calculation step, calculate the maximum similarity between the target ECG feature vector and each of the text feature vectors of the target rare disease in the rare disease feature dictionary to obtain the first similarity, and calculate the maximum similarity between the target text feature vector and each of the text feature vectors of the target rare disease in the rare disease feature dictionary to obtain the second similarity, wherein the target rare disease is any of the rare diseases mentioned above. The fifth calculation module is used to perform the second calculation step, which aggregates the first similarity and the second similarity to obtain the probability of occurrence of the target rare disease. The repeat module is used to repeat the first calculation step and the second calculation step at least once in sequence until the occurrence probability of all the above-mentioned rare diseases is obtained.

[0089] In the above implementation, based on the rare disease feature dictionary Where J is the total number of rare diseases, d j For the jth rare disease, Let be the text feature vector corresponding to the j-th rare disease; for example, the text feature vector of the i-th rare disease is . , Let e ​​represent the number of samples for the i-th rare disease, and calculate the target electrocardiogram feature vector e and the text feature vector for the i-th rare disease. The cosine similarity is used to select the maximum value to obtain the first similarity score. Calculate the feature vector t of the target text and the feature vector of the i-th rare disease text. The cosine similarity is used to select the maximum value to obtain the second similarity. Calculate the similarity after aggregation , which represents the probability of the occurrence of the i-th rare disease. By assessing the probability, doctors can be assisted in diagnosing rare diseases and improve their diagnostic accuracy.

[0090] To improve the accuracy of rare disease incidence probability calculations, in one optional implementation, the fifth calculation module includes: The input submodule is used to input the ECG data to be evaluated into the above-mentioned preset risk detection model to obtain the probability coefficient of rare ECG diseases. The aggregation submodule is used to aggregate the first similarity and the second similarity using the above-mentioned electrocardiogram rare disease occurrence probability coefficient to obtain the occurrence probability of the target rare disease.

[0091] In the above implementation, the similarity after risk-enhanced aggregation is calculated. ,in, The probability of occurrence of the i-th rare disease is obtained, which is the probability of occurrence of the target rare disease. The correlation between electrocardiogram signals and rare diseases is explored, and a risk enhancement algorithm is used to achieve high-precision prediction of rare disease risk, thereby promoting early identification and intervention management of diseases.

[0092] In one optional implementation for training the ECG pre-training model, the second training unit includes: The fourth training module is used to input the aforementioned historical medical record text data into the aforementioned medical text pre-training model to obtain the training text feature vector; The fifth training module is used to input the ECG data corresponding to the historical medical record text data into the initial ECG pre-training model to obtain the trained ECG feature vector. The calculation module is used to calculate the loss of the above-mentioned training text feature vector and the corresponding above-mentioned training ECG feature vector to obtain the cross-modal contrast loss; The enhancement processing module is used to process the above-mentioned training ECG feature vectors using two data augmentation processing methods to obtain the first ECG feature vector and the second ECG feature vector. The sixth calculation module is used to calculate the loss of the first ECG feature vector and the second ECG feature vector to obtain the single-modal contrast loss. The seventh calculation module is used to perform a weighted summation of the cross-modal contrast loss and the single-modal contrast loss to obtain the joint contrast loss; The fourth training module is used to iteratively train the above-mentioned initial ECG pre-trained model to minimize the above-mentioned joint contrast loss and obtain the above-mentioned ECG pre-trained model.

[0093] In the above embodiments, the training process of the above-mentioned ECG pre-training model is as follows: Figure 3 As shown, historical medical record text data is input into a medical text pre-training model to obtain the target text feature vector. Where N is the number of samples, historical ECG data is input into the ECG pre-training model to obtain the target ECG feature vector. Data augmentation methods 1 and 2 are used to process the target ECG feature vector to obtain the first ECG feature vector. Second ECG feature vector The loss of the first ECG feature vector and the second ECG feature vector is calculated to obtain the single-modal contrast loss. ,Right now ,in, , The cross-modal contrast loss is obtained by calculating the loss between the training text feature vector and the corresponding training ECG feature vector. ,Right now ,in, , The weighted sum of the cross-modal contrast loss and the single-modal contrast loss yields the joint contrast loss L, which is the loss function value of the ECG pre-training model. ,in, and For hyperparameters, Here, is the cosine similarity calculation function, N is the number of samples, and i and k are both positive integers between 1 and N. Let be the first ECG feature vector of the k-th sample. Let's consider the second ECG feature vector of the k-th sample. The training objective includes two aspects: firstly, to enhance the ECG feature vector of the same sample. and The first principle is to make the ECG feature vectors as similar as possible to those of other enhanced samples, while also ensuring they differ as much as possible from the enhanced ECG feature vectors of other samples. Text feature vectors paired with it The model should be as similar as possible to other samples, with as many differences as possible between the ECG feature vectors and text feature vectors. Therefore, by minimizing the joint contrast loss, the initial ECG pre-trained model can be iteratively trained to obtain the ECG pre-trained model.

[0094] Specifically, the process of obtaining training samples for the aforementioned ECG pre-training model involves: multiple historical medical record text data. Multiple historical electrocardiogram data constitute D ecg We collected paired electrocardiogram signals and medical record text data to form an initial signal-text paired dataset. The text is also organized according to the following template: "Gender is {A1}, Age is {A2}, Chief Complaint is {C}, Present Illness is {P}, ECG Report Conclusion is {R}, Disease Label is {D}". Optionally, to enhance the learning and understanding of knowledge related to rare diseases, the dataset... For samples related to rare disease diagnosis, resampling is performed using the same sampling strategy as above, where the sampling ratio can be set based on the diagnostic distribution in the actual scenario data. Optionally, the sampling strategy can be set as follows: For new samples generated through resampling, other module data remain unchanged, but the disease labels are randomized. Specifically, taking disease labels {D0, D1, D2} as an example, where D0 represents a rare disease and D1 and D2 represent common diseases, the disease labels of the new samples can be randomized to any one of these four types: {D0}, {D0, D1}, {D0, D2}, {D0, D1, D2}, to enhance the richness of the samples and strengthen the model's semantic information understanding. The final paired dataset is denoted as... .

[0095] To construct a rare disease feature dictionary, in one optional implementation, the rare disease feature dictionary is constructed based on historical medical record text data, and the apparatus further includes: The input unit is used to input the historical medical record text data of the corresponding patient into the medical text pre-training model for each rare disease to obtain the text feature vector corresponding to the rare disease. The construction unit is used to construct the rare disease feature dictionary from multiple text feature vectors corresponding to the above rare diseases.

[0096] In the above embodiments, the medical record text dataset is based on multiple historical medical record text data. For each rare disease of interest, the historical medical records of the corresponding patient population are input into the medical text pre-training model LM to obtain text feature vectors, forming a text feature vector library for that rare disease. Multiple text feature vector libraries for that rare disease constitute a rare disease feature dictionary. }, where J represents the total number of rare diseases. This represents the number of samples for the j-th rare disease. m for the jth rare disease j A set of text feature vectors. Dict={d1,…d J}, where J represents the total number of rare diseases, and for a specific rare disease, such as the i-th rare disease, the corresponding text feature vector is} .

[0097] The aforementioned rare disease occurrence probability prediction device includes a processor and a memory. The first acquisition unit, first input unit, second input unit, third input unit, fourth input unit, and determination unit are all stored as program units in the memory. The processor executes these program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0098] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of requiring a large number of rare disease ECG samples for training in existing ECG recognition techniques.

[0099] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0100] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the rare disease occurrence probability prediction method.

[0101] Specifically, methods for predicting the probability of rare disease occurrence include: Step S201: Obtain multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The aforementioned medical record text data to be evaluated includes at least patient information and ECG report conclusions. The aforementioned ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Step S202: Input the ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Step S203: Input the above-mentioned medical record text data corresponding to the above-mentioned electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Step S204: Input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Step S205: Preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; Step S206: Based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease, determine the probability of occurrence of each of the rare diseases. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0102] This invention provides a processor for running a program, wherein the program executes the above-mentioned rare disease occurrence probability prediction method.

[0103] Specifically, methods for predicting the probability of rare disease occurrence include: Step S201: Obtain multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The aforementioned medical record text data to be evaluated includes at least patient information and ECG report conclusions. The aforementioned ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Step S202: Input the ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Step S203: Input the above-mentioned medical record text data corresponding to the above-mentioned electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Step S204: Input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Step S205: Preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; Step S206: Based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease, determine the probability of occurrence of each of the rare diseases. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0104] This invention provides an electrocardiogram (ECG) analysis system, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps: Step S201: Obtain multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The aforementioned medical record text data to be evaluated includes at least patient information and ECG report conclusions. The aforementioned ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Step S202: Input the ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Step S203: Input the above-mentioned medical record text data corresponding to the above-mentioned electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Step S204: Input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Step S205: Preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; Step S206: Based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease, determine the probability of occurrence of each of the rare diseases. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0105] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps: Step S201: Obtain multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The aforementioned medical record text data to be evaluated includes at least patient information and ECG report conclusions. The aforementioned ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. Step S202: Input the ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Step S203: Input the above-mentioned medical record text data corresponding to the above-mentioned electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. Step S204: Input the above-mentioned ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. Step S205: Preset a rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; Step S206: Based on the similarity between the target ECG feature vector and the reference text feature vector of each of the rare diseases, the similarity between the target text feature vector and the reference text feature vector of each of the rare diseases, and the probability coefficient of occurrence of the rare ECG disease, determine the probability of occurrence of each of the rare diseases. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

[0106] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0107] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0113] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0116] As can be seen from the above description, the embodiments of this application achieve the following technical effects: 1) In the method for intelligent identification of rare disease risk using electrocardiogram (ECG) in this application, a three-level pre-training model collaborative architecture of ECG pre-training model, medical text pre-training model and risk detection model is used to jointly train across two modalities: text and ECG signal. Disease knowledge is associated with ECG signal information, and combined with ECG rare disease risk score, the rare disease identification performance with ECG as the core is enhanced. A qualified model can be trained with a small amount of rare disease patient sample data to determine the probability of rare disease occurrence, which greatly reduces the need for rare disease patient sample data. There is no need to use a large number of rare disease ECG samples for training, which solves the problem that the ECG rare disease identification training of existing technology requires a large number of rare disease ECG samples. In addition, cross-modal training helps to make full use of a large amount of clinical data to deeply explore the potential disease patterns hidden in ECG signals and improve the rare disease identification performance.

[0117] 2) In the ECG-based intelligent rare disease risk identification device of this application, a three-level pre-training model collaborative architecture of ECG pre-training model, medical text pre-training model and risk detection model is used to jointly train across two modalities: text and ECG signal. This associates disease knowledge with ECG signal information and combines it with ECG rare disease risk scores to enhance the rare disease identification performance centered on ECG. A qualified model can be trained with a small amount of rare disease patient sample data to determine the probability of rare disease occurrence, which greatly reduces the need for rare disease patient sample data. It eliminates the need for a large number of rare disease ECG samples for training, solving the problem that the ECG rare disease identification training in the prior art requires a large number of rare disease ECG samples. In addition, cross-modal training helps to make full use of a large amount of clinical data to deeply explore the potential disease patterns hidden in ECG signals and improve the rare disease identification performance.

[0118] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for predicting the probability of occurrence of rare diseases, characterized in that, include: Acquire multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The medical record text data to be evaluated includes at least patient information and ECG report conclusions. The ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. The ECG data to be evaluated is input into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. Input the medical record text data corresponding to the ECG data to be evaluated into the medical text pre-training model to obtain the target text feature vector; The ECG data to be evaluated is input into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. A pre-defined rare disease feature dictionary is provided, which includes text feature vectors corresponding to each rare disease. The occurrence probability of each rare disease is determined based on the similarity between the target ECG feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the ECG rare disease occurrence probability coefficient. The reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

2. The method according to claim 1, characterized in that, The medical text pre-training model was trained based on multiple historical medical record text data, specifically including: Acquire multiple historical medical record text data, wherein the historical medical record text data includes at least patient information, electrocardiogram report conclusions, and disease tags; A partial word of the historical medical record text data is masked to obtain a first masked text. The first masked text is then used to perform masked pre-training on the text pre-training model to obtain an initial medical text pre-training model. A portion of the historical medical record text data is masked to obtain a second masked text. The second masked text is then used to perform masked pre-training on the initial medical text pre-training model to obtain the medical text pre-training model. The patient information includes gender, age, chief complaint, and present medical history. The module is one of the chief complaint, the present medical history, the electrocardiogram report conclusion, and the disease label.

3. The method according to claim 2, characterized in that, A partial word masking process is performed on the historical medical record text data to obtain a first masked text. The first masked text is then used to perform masked pre-training on the text pre-training model to obtain an initial medical text pre-training model, including: The historical medical record text data is segmented and serialized to obtain a target vocabulary and multiple first-formatted sequences; Dynamically mask a portion of the words in multiple first formatted sequences to obtain multiple second formatted sequences, and use the second formatted sequences as the first masked text; Obtain the first label vector corresponding to the words that are dynamically masked by using the target vocabulary; Multiple second formatted sequences are converted into multiple first text vectors. Each first text vector consists of a first word embedding vector, a first position embedding vector, and a first segment embedding vector. The first word embedding vector is used to represent the semantic information of the words in the second formatted sequence. The first position embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. The first segment embedding vector is used to represent the position of the words in the second formatted sequence in the sentence. Multiple first text vectors are input into the encoder of the text pre-training model to obtain multiple second text vectors. The plurality of second text vectors are input into the prediction head of the text pre-training model to obtain a plurality of first prediction vectors; Based on the first prediction vector, obtain the first prediction vector corresponding to the word that is dynamically masked; The sum of the losses of the first prediction vectors and the corresponding first label vectors of the multiple words that have been dynamically masked is calculated to obtain the first loss function value; The text pre-trained model is iteratively trained to minimize the value of the first loss function, thereby obtaining the initial medical text pre-trained model.

4. The method according to claim 3, characterized in that, A portion of the historical medical record text data is masked to obtain a second masked text. This second masked text is then used to pre-train the initial medical text pre-training model, resulting in the medical text pre-training model, which includes: The historical medical record text data is segmented and serialized to obtain a target vocabulary and multiple first-formatted sequences; Dynamic masking is performed on the words of any module in multiple first formatted sequences to obtain multiple third formatted sequences, and the third formatted sequences are used as the second masked text; Obtain the second tag vector corresponding to the masked module using the target vocabulary; Multiple third-formatted sequences are converted into multiple third-text vectors. Each third-text vector consists of a second word embedding vector, a second position embedding vector, and a second segment embedding vector. The second word embedding vector is used to represent the semantic information of the words in the third-formatted sequence. The second position embedding vector is used to represent the position of the words in the third-formatted sequence in the sentence. The second segment embedding vector is used to represent the position of the module in which the words in the third-formatted sequence are located. Multiple third-text vectors are input into the encoder of the initial medical text pre-training model to obtain multiple fourth-text vectors. The plurality of fourth text vectors are input into the prediction head of the text pre-training model to obtain a plurality of second prediction vectors; The second prediction vector is obtained based on the second prediction vector and the second prediction vector corresponding to the module that is dynamically masked; The sum of the losses of the second prediction vectors and the corresponding second label vectors of the multiple modules that are dynamically masked is calculated to obtain the second loss function value; The initial medical text pre-trained model is iteratively trained to minimize the value of the second loss function, thereby obtaining the medical text pre-trained model.

5. The method according to claim 1, characterized in that, The training process of the ECG pre-training model includes: The ECG self-supervised pre-training model is pre-trained with a mask using multiple non-rare disease ECG data and multiple rare disease ECG data to obtain a first ECG pre-training model. The encoder of the first ECG pre-training model is determined as the initial ECG pre-training model. The initial ECG pre-training model is trained by comparing the historical medical record text data, the corresponding historical ECG data, and the medical text pre-training model to obtain the ECG pre-training model. The loss function of the ECG pre-training model is used to calculate at least the loss of the text feature vector of the historical medical record text data and the ECG feature vector of the corresponding historical ECG data. The loss includes single-modal comparison loss and cross-modal comparison loss.

6. The method according to claim 1, characterized in that, The preset risk detection model is trained using non-rare disease electrocardiogram data and includes: The ECG self-supervised pre-training model was pre-trained with a mask using multiple non-rare disease ECG data to obtain a second ECG pre-training model. The preset risk detection model is constructed based on the second ECG pre-training model and the loss function of the second ECG pre-training model. The preset risk detection model is a model that uses the loss function value of the second ECG pre-training model to calculate the probability coefficient of rare ECG diseases.

7. The method according to claim 6, characterized in that, The ECG self-supervised pre-training model was pre-trained using multiple non-rare disease ECG data sets to obtain a second ECG pre-training model, including: The non-rare disease electrocardiogram data were segmented into multiple segments along the time dimension; Masking was performed on some segments to obtain masked ECG data for non-rare diseases; The non-rare disease masked ECG data is input into the ECG self-supervised pre-trained model to obtain reconstructed ECG data; The loss between the reconstructed ECG data and the non-rare disease ECG data is calculated to obtain the value of the third loss function. The ECG self-supervised pre-trained model is iteratively trained to minimize the value of the third loss function, thereby obtaining the second ECG pre-trained model.

8. The method according to claim 1, characterized in that, The probability of occurrence of each rare disease is determined based on the similarity between the target ECG feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the probability coefficient of occurrence of the rare ECG disease, including: The first calculation step involves calculating the maximum similarity between the target ECG feature vector and each text feature vector of the target rare disease in the rare disease feature dictionary to obtain a first similarity. Then, the maximum similarity between the target text feature vector and each text feature vector of the target rare disease in the rare disease feature dictionary is calculated to obtain a second similarity. The target rare disease can be any one of the rare diseases. The second calculation step involves aggregating the first similarity and the second similarity to obtain the probability of occurrence of the target rare disease. Repeat the first calculation step and the second calculation step at least once in sequence until the occurrence probability of all the rare diseases is obtained.

9. The method according to claim 8, characterized in that, Aggregating the first similarity and the second similarity yields the probability of occurrence of the target rare disease, including: Input the ECG data to be evaluated into the preset risk detection model to obtain the probability coefficient of rare ECG diseases. The probability of occurrence of the target rare disease is obtained by aggregating the first similarity and the second similarity using the rare disease occurrence probability coefficient of electrocardiogram.

10. The method according to claim 5, characterized in that, The initial ECG pre-training model is trained by comparing the historical medical record text data, the corresponding historical ECG data, and the medical text pre-training model to obtain the ECG pre-training model, including: The historical medical record text data is input into the medical text pre-training model to obtain the training text feature vector; Input the electrocardiogram (ECG) data corresponding to the historical medical record text data into the initial ECG pre-training model to obtain the trained ECG feature vector; Calculate the loss between the training text feature vector and the corresponding training ECG feature vector to obtain the cross-modal contrast loss; Two data augmentation methods are used to process the trained ECG feature vectors to obtain a first ECG feature vector and a second ECG feature vector. Calculate the loss of the first ECG feature vector and the second ECG feature vector to obtain the single-modal contrast loss; The joint contrast loss is obtained by weighted summing of the cross-modal contrast loss and the single-modal contrast loss. The initial ECG pre-trained model is iteratively trained to minimize the joint contrast loss, thereby obtaining the ECG pre-trained model.

11. The method according to claim 1, characterized in that, The rare disease feature dictionary is constructed based on historical medical record text data and includes: For each rare disease, the historical medical record text data of the corresponding patient is input into the medical text pre-training model to obtain the text feature vector corresponding to the rare disease; The text feature vectors corresponding to multiple rare diseases constitute the rare disease feature dictionary.

12. A device for intelligent identification of rare disease risk using electrocardiogram (ECG), characterized in that, include: The first acquisition unit is used to acquire multiple ECG data to be evaluated and corresponding medical record text data to be evaluated. The medical record text data to be evaluated includes at least patient information and ECG report conclusions. The ECG data to be evaluated includes non-rare disease ECG data and rare disease ECG data. The first input unit is used to input the ECG data to be evaluated into the ECG pre-training model to obtain the target ECG feature vector. The ECG pre-training model is trained based on multiple historical medical record text data and corresponding historical ECG data. The second input unit is used to input the medical record text data corresponding to the electrocardiogram data to be evaluated into the medical text pre-training model to obtain the target text feature vector. The third input unit is used to input the ECG data to be evaluated into a preset risk detection model to obtain the probability coefficient of rare ECG diseases. The fourth input unit is used to call a preset rare disease feature dictionary, which includes text feature vectors corresponding to each rare disease; The determining unit is configured to determine the probability of occurrence of each rare disease based on the similarity between the target ECG feature vector and the reference text feature vector of each rare disease, the similarity between the target text feature vector and the reference text feature vector of each rare disease, and the probability coefficient of occurrence of the rare ECG disease, wherein the reference text feature vector is the text feature vector corresponding to the rare disease in the rare disease feature dictionary.

13. An electrocardiogram (ECG) analysis system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 10.

Images