A chronic kidney disease risk early warning method and system

Abstract

The present invention provides a chronic kidney disease risk warning method and system, which obtains values ​​corresponding to keywords from electronic medical records to form a <keyword, value, importance> triple, splices the word embedding code of the value and the word embedding code of the importance to obtain the encoding result corresponding to each keyword; divides the sorted encoding results into three subsets according to the importance; calculates the attention of each subset using a multi-head attention mechanism to obtain the first attention, and calculates the attention of the encoding results corresponding to all keywords using a multi-head attention mechanism to obtain the second attention; obtains the input of the encoder according to the importance, the first attention, and the second attention, and uses the output of the encoder as the input of the MLP to obtain the risk probability of chronic kidney disease. The present invention calculates attention in a grouping manner for different factors, and inputs the first attention into each layer of the encoder, thereby improving the accuracy of prediction.

Description

Technical Field

[0001] The present invention relates to the medical field, and in particular to a chronic kidney disease risk early warning method and system. Background Art

[0002] Chronic kidney disease (CKD) is a progressive kidney disease that can be caused by a variety of reasons, such as high blood pressure, diabetes, kidney infection, or factors in the environment that damage kidney tissue. The main way to determine chronic kidney disease is through blood tests, which measure the level of serum creatinine in the blood. Chronic kidney disease is called chronic because the process can last for years or even decades. It is not easy to detect in the early stages and has no obvious symptoms. However, as the disease progresses, patients will experience fatigue and edema. If it develops to the stage of kidney failure, dialysis or even kidney transplantation will be required. Early diagnosis and early intervention are the key to managing chronic kidney disease, especially for some people with high blood pressure, diabetes, or other diseases that may cause kidney damage. How to early warn of the risk of chronic kidney disease is very important. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a chronic kidney disease risk early warning method, the method comprising the following steps:

[0004] Preset keywords related to chronic kidney disease and the importance of the keywords, obtain the values ​​corresponding to the keywords from the electronic medical records, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain the encoding result corresponding to each keyword;

[0005] The encoding results are sorted according to the importance, and the sorted encoding results are divided into three subsets according to the importance; for each subset, attention is calculated using a multi-head attention mechanism to obtain a first attention, and the encoding results corresponding to all keywords are calculated using a multi-head attention mechanism to obtain a second attention;

[0006] The input of an encoder is obtained according to the importance, the first attention, and the second attention, and the output of the encoder is used as the input of the MLP to obtain the risk probability of chronic kidney disease.

[0007] Preferably, obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically:

[0008] Get the subset where the keyword is located and get the average importance of the subset ;

[0009] Calculate the average importance of all keywords ;

[0010] According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder.

[0011] Preferably, the encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure;

[0012] Among them, the input of the first LN layer in the nth layer is ;

[0013] in Represents the input of the nth layer, SubLayber( ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n=1, 2, …, N.

[0014] Preferably, the multi-head attention mechanism is used to calculate the attention of each subset to obtain the first attention, specifically:

[0015] For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector ;in, represents the scaling factor;

[0016] The multiple feature vectors corresponding to each subset Concatenate to get the feature vector , then Concatenate to get the eigenvector Z, and combine the eigenvector Z and the weight matrix Multiply them together and you get the first attention;

[0017] Among them, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

[0018] Preferably, the keywords include at least: gender, weight, age, blood sugar, blood creatinine, cystatin, and genetic history.

[0019] Preferably, the importance of blood creatinine, cystatin, blood sugar, gender, age, weight, and genetic history decreases in sequence, and the importance is between (0, 1].

[0020] In addition, the present invention also provides a chronic kidney disease risk early warning system, the system comprising the following units:

[0021] An encoding unit is used to pre-set keywords related to chronic kidney disease and the importance of the keywords, obtain values ​​corresponding to the keywords from the electronic medical record, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain an encoding result corresponding to each keyword;

[0022] An attention calculation unit is used to sort the encoding results according to the importance, and divide the sorted encoding results into three subsets according to the importance; calculate the attention of each subset using a multi-head attention mechanism to obtain a first attention, and calculate the attention of the encoding results corresponding to all keywords using a multi-head attention mechanism to obtain a second attention;

[0023] A risk prediction unit is used to obtain the input of an encoder according to the importance, the first attention, and the second attention, and use the output of the encoder as the input of the MLP to obtain the risk probability of chronic kidney disease.

[0024] Preferably, obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically:

[0025] Get the subset where the keyword is located and get the average importance of the subset ;

[0026] Calculate the average importance of all keywords ;

[0027] According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder.

[0028] Preferably, the encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure;

[0029] Among them, the input of the first LN layer in the nth layer is ;

[0030] in Represents the input of the nth layer, SubLayber( ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n=1, 2, …, N.

[0031] Preferably, the multi-head attention mechanism is used to calculate the attention of each subset to obtain the first attention, specifically:

[0032] For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector ;in, represents the scaling factor;

[0033] The multiple feature vectors corresponding to each subset Concatenate to get the feature vector , then Concatenate to get the eigenvector Z, and combine the eigenvector Z and the weight matrix Multiply them together and you get the first attention;

[0034] Among them, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

[0035] Preferably, the keywords include at least: gender, weight, age, blood sugar, blood creatinine, cystatin, and genetic history.

[0036] Preferably, the importance of blood creatinine, cystatin, blood sugar, gender, age, weight, and genetic history decreases in sequence, and the importance is between (0, 1].

[0037] Finally, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0038] Early detection of chronic kidney disease and early intervention in stage 1 or 2 are important ways to prevent chronic kidney disease from developing into dialysis and kidney transplantation. The present invention divides the indicators into three categories according to the indicators with different impacts on chronic kidney disease in electronic medical records, assigns higher importance to important indicators, and then calculates the attention for each subset to obtain the first attention. In order to prevent the neural network from degenerating, the result of the first attention is input into the residual layer of each layer in the form of residual connection, which has high fitting. Combined with sample enhancement, the risk situation of chronic kidney disease can be well obtained by training the neural network proposed in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is a flow chart of Embodiment 1;

[0041] Figure 2 This is a schematic diagram of the fusion of the first attention and the second attention;

[0042] Figure 3 This is the encoder structure diagram;

[0043] Figure 4 Schematic diagram of the first attention calculation;

[0044] Figure 5 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0045] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The present invention provides a chronic kidney disease risk early warning method, such as Figure 1 As shown, the method comprises the following steps:

[0048] Preset keywords related to chronic kidney disease and the importance of the keywords, obtain the values ​​corresponding to the keywords from the electronic medical records, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain the encoding result corresponding to each keyword;

[0049] Chronic kidney disease is related to many factors. In the past, the judgment of chronic kidney disease was mostly based on serum creatinine calculation, but this judgment method is mostly made when serum creatinine is abnormal, which is already in stage 2 or later. The present invention obtains the value corresponding to the keyword from the electronic medical record, and sets the keyword and importance in advance. For example, a keyword is "serum creatinine", the value is "1.2", the importance is "1", the second keyword is "blood sugar", the value is "110", the importance is "0.8", etc.

[0050] This constitutes a <keyword, value, importance> triple. Taking the above data as an example, we get two triples <serum creatinine, 1.2, 1> and <blood sugar, 110, 0.8>. Then, embedding is performed on the value and importance respectively. The so-called word embedding coding refers to mapping words to vector space. Assuming that the word embedding dimension is 128, the word encoding result corresponding to 1.2 is obtained. Similarly, the word encoding result corresponding to 1 can be obtained. The word encoding result of the value and the word encoding result of the importance are concatenated, such as the concat operation in Python, and the encoding result corresponding to the keyword is . ()

[0051] The encoding results are sorted according to the importance, and the sorted encoding results are divided into three subsets according to the importance; for each subset, attention is calculated using a multi-head attention mechanism to obtain a first attention, and the encoding results corresponding to all keywords are calculated using a multi-head attention mechanism to obtain a second attention;

[0052] After obtaining the encoding results corresponding to each keyword, the encoding results are sorted according to the importance, and then the sorted encoding results are divided into three subsets according to the importance. In a specific embodiment, the importance ranges corresponding to the three subsets are [1, 0.6), [0.6, 0.3), and [0.3, 0). If the importance corresponding to a keyword is 0.7, it will be divided into the first subset. When the keywords and importance are determined, each subset is also determined.

[0053] Since the three subsets have different effects on the prediction of chronic kidney disease risk, for example, serum creatinine is an important indicator, so it will be classified into the first subset. Based on this, the present invention performs multi-head attention mechanism calculation on each subset to prevent other low-importance indicators from affecting high-importance indicators. After each subset obtains the first attention mechanism, it is spliced ​​row by row to obtain the first attention. In order to increase the fit, the present invention also performs multi-head attention mechanism calculation on all keywords to obtain the second attention.

[0054] The input of an encoder is obtained according to the importance, the first attention, and the second attention, and the output of the encoder is used as the input of the MLP to obtain the risk probability of chronic kidney disease.

[0055] After obtaining the first and second attentions, the input of the encoder is obtained according to the importance. The output of the encoder is connected to an MLP, and the probability value of the MLP output is used as the risk probability. During training, there are two labeling results for the sample, 1 for chronic kidney disease and 0 for no chronic kidney disease. The probability of the MLP output result being 1 is the risk probability.

[0056] Regarding the labeling of samples, in a specific embodiment, no manual labeling is required, and the samples can be obtained from the user's electronic medical records. For example, if the sample is finally determined to be chronic kidney disease, it will be displayed in the electronic medical record, so the labeling information can be obtained through the electronic medical record.

[0057] In a specific embodiment, obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically:

[0058] Get the subset where the keyword is located and get the average importance of the subset ;

[0059] Calculate the average importance of all keywords ;

[0060] According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder.

[0061] For example, the keyword "serum creatinine" is in the first subset, the average importance of the first subset is 0.8, and the average importance of all keywords is 0.6. Then, the product of 0.8 and the first attention and the product of 0.6 and the second attention are added bit by bit to obtain the final attention, which is the input of the encoder. After normalization, it is used as the input of the encoder. The process is as Figure 2 shown.

[0062] The encoder is used to encode information. In a specific embodiment, the encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure;

[0063] Among them, the input of the first LN layer in the nth layer is ;

[0064] in Represents the input of the nth layer, SubLayber( ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n=1, 2, …, N. When n=1, .

[0065] like Figure 3 As shown, by fusing the first attention layer by layer into the result obtained by the multi-head attention mechanism in a residual manner, the network is prevented from being too deep, resulting in the problem of gradient disappearance or gradient explosion. In addition, in the present invention, the (1-n / N) method is used to reduce the number of layers. Excessive interference to subsequent networks. It indicates the output of the previous layer of the first Add&Norm layer. Since the previous layer of Add&Norm is a multi-head attention mechanism layer in the present invention, it is the output of the multi-head attention mechanism in this layer. It should be noted that the numbering of the N layers of the same structure is from input to output, and the layer number increases successively. Figure 3 This is just an example, and other structures may also be included. For example, a transformation layer is included before the MLP layer to flatten the output of the encoder into a one-dimensional vector.

[0066] Each subset adopts a multi-head attention mechanism. In one embodiment, each subset is treated as an independent subset to complete all calculations of the multi-head attention mechanism, that is, the query weight matrix, key weight matrix, value weight matrix and weight matrix of each subset are different, that is, they are not shared.

[0067] In another embodiment, the multi-head attention mechanism is used to calculate the attention of each subset to obtain the first attention, specifically:

[0068] For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector ;in, represents a scaling factor, T represents a transpose, and in a more specific embodiment, Represents the dimension of the input features.

[0069] The multiple feature vectors corresponding to each subset Concatenate to get the feature vector , then Concatenate to get the eigenvector Z, and combine the eigenvector Z and the weight matrix Multiply them together and you get the first attention;

[0070] In a specific embodiment, multiple feature vectors Concatenate to get the feature vector It is concatenated using the np.concatenate((a,b),axis=1) method in Python; The concatenation of the eigenvector Z is performed by using np.concatenate((a,b),axis=0) in Python. The concatenation and weight matrix The multiplication process is as follows Figure 4 shown.

[0071] Among them, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

[0072] The multi-head attention mechanism includes multiple independent self-attention heads, each head corresponds to its own QKV matrix. If there are 10 heads, there will be 10 QKV matrices. After calculation by the above formula, 10 attention heads will be obtained, where the eigenvector Also called attention head .

[0073] To ensure successful splicing, each subset uses the same number of heads in the multi-head attention mechanism, for example, 5 heads are obtained in the end. The weight matrix is ​​reduced in the above way The number of parameters in , which provides training and prediction speed.

[0074] Chronic kidney disease is related to many factors. As much information as possible should be obtained from the electronic medical records so that accurate prediction can be made. In a specific embodiment, the keywords at least include: gender, weight, age, blood sugar, blood creatinine, cystatin, and genetic history. Of course, the keywords may also include other content, such as blood pressure, heart rate, blood type, etc.

[0075] In a specific embodiment, the importance of serum creatinine, cystatin, blood sugar, gender, age, weight, and genetic history decreases in sequence, and the importance is in the range of (0,1]. The more relevant the keyword is to chronic kidney disease, the greater the corresponding weight is. For example, serum creatinine is an important indicator, and its importance can be set to 1, while heart rate or blood type is not closely related to chronic kidney disease and can be set to 0.2 or 0.1.

[0076] In addition, the present invention also provides a chronic kidney disease risk early warning system, the system comprising the following units:

[0077] An encoding unit is used to pre-set keywords related to chronic kidney disease and the importance of the keywords, obtain values ​​corresponding to the keywords from the electronic medical record, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain an encoding result corresponding to each keyword;

[0078] An attention calculation unit is used to sort the encoding results according to the importance, and divide the sorted encoding results into three subsets according to the importance; calculate the attention of each subset using a multi-head attention mechanism to obtain a first attention, and calculate the attention of the encoding results corresponding to all keywords using a multi-head attention mechanism to obtain a second attention;

[0079] A risk prediction unit is used to obtain the input of an encoder according to the importance, the first attention, and the second attention, and use the output of the encoder as the input of the MLP to obtain the risk probability of chronic kidney disease.

[0080] Preferably, obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically:

[0081] Get the subset where the keyword is located and get the average importance of the subset ;

[0082] Calculate the average importance of all keywords ;

[0083] According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder.

[0084] Preferably, the encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure;

[0085] Among them, the input of the first LN layer in the nth layer is ;

[0086] in Represents the input of the nth layer, SubLayber( ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n=1, 2, …, N.

[0087] Preferably, the multi-head attention mechanism is used to calculate the attention of each subset to obtain the first attention, specifically:

[0088] For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector ;in, represents a scaling factor. In a more specific embodiment, Represents the dimension of the input features.

[0089] The multiple feature vectors corresponding to each subset Concatenate to get the feature vector , then Concatenate to get the eigenvector Z, and combine the eigenvector Z and the weight matrix Multiply them together and you get the first attention;

[0090] Among them, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

[0091] Preferably, the keywords include at least: gender, weight, age, blood sugar, blood creatinine, cystatin, and genetic history.

[0092] Preferably, the importance of blood creatinine, cystatin, blood sugar, gender, age, weight, and genetic history decreases in sequence, and the importance is between (0, 1].

[0093] Embodiment 3: The present invention further provides a Transformer-based encoding system, the system comprising the following units:

[0094] The encoding unit is used to pre-set keywords and the importance corresponding to the keywords to form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain the encoding result corresponding to each keyword; the value is the value corresponding to the keyword;

[0095] In a specific embodiment, the keywords are from sentences or texts;

[0096] An attention calculation unit is used to sort the encoding results according to the importance, and divide the sorted encoding results into three subsets according to the importance; calculate the attention of each subset using a multi-head attention mechanism to obtain a first attention, and calculate the attention of the encoding results corresponding to all keywords using a multi-head attention mechanism to obtain a second attention;

[0097] A result output unit is used to obtain the input of the encoder of the Transformer network model according to the importance, the first attention, and the second attention, use the output of the encoder as the input of the MLP, and use the output of the MLP as the final result.

[0098] In a specific embodiment, the final result is used to classify sentences or texts.

[0099] Preferably, obtaining the input of the encoder of the Transformer network model according to the importance, the first attention, and the second attention is specifically:

[0100] Get the subset where the keyword is located and get the average importance of the subset ;

[0101] Calculate the average importance of all keywords ;

[0102] According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder.

[0103] Preferably, the encoder comprises N layers of identical structure;

[0104] Among them, the input of the first LN layer in the nth layer is ;

[0105] in Represents the input of the nth layer, SubLayber( ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n=1, 2, …, N.

[0106] Preferably, the multi-head attention mechanism is used to calculate the attention of each subset to obtain the first attention, specifically:

[0107] For each subset, multiple <query weight matrix, key weight matrix, value weight matrix> triplets are set respectively. Each subset is passed through a <query weight matrix, key weight matrix, value weight matrix> according to the formula Get the feature vector ;

[0108] The multiple feature vectors corresponding to each subset Concatenate to get the feature vector , then Concatenate to get the eigenvector Z, and combine the eigenvector Z and the weight matrix Multiply them together and you get the first attention;

[0109] Among them, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

[0110] Embodiment 4, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the method described in Embodiment 1 is implemented.

[0111] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution can essentially or contribute to the prior art in the form of a computer product, and the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. In addition, if the samples and other data mentioned in the present invention involve patient or personal information, desensitization operations will be uniformly performed, or other methods that can remove personal information will be used; if there are situations that require the patient or user to know or agree, the consent of the patient or user will be obtained.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A chronic kidney disease risk early warning method, characterized in that: The method comprises the following steps: Preset keywords related to chronic kidney disease and the importance of the keywords, obtain the values ​​corresponding to the keywords from the electronic medical records, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain the encoding result corresponding to each keyword; The encoding results are sorted according to the importance, and the sorted encoding results are divided into three subsets according to the importance; for each subset, attention is calculated using a multi-head attention mechanism to obtain a first attention, and the encoding results corresponding to all keywords are calculated using a multi-head attention mechanism to obtain a second attention; Obtaining an input of an encoder according to the importance, the first attention, and the second attention, and using an output of the encoder as an input of the MLP to obtain a risk probability of chronic kidney disease; The step of obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically: Get the subset where the keyword is located and get the average value μ1 of the importance of the subset; Calculate the average importance of all keywords μ2; According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder; The encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure; Among them, the input of the first LN layer in the nth layer is Where X n Represents the input of the nth layer, SubLayber(X n ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n = 1, 2, …, N; The first attention is obtained by respectively calculating the attention of each subset using the multi-head attention mechanism, specifically: For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector in, represents the scaling factor; The multiple feature vectors corresponding to each subset Concatenate to get the feature vector Z j , then Z j Concatenate the feature vector Z and the weight matrix W 0 Multiply them together and you get the first attention; Wherein, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

2. The method according to claim 1, characterized in that The keywords at least include: gender, weight, age, blood sugar, blood creatinine, cystatin, and genetic history.

3. The method according to claim 2, characterized in that The importance of blood creatinine, cystatin, blood sugar, gender, age, weight, and genetic history decreases in turn, and the importance is between (0,1).

4. A chronic kidney disease risk early warning system, characterized in that: The system comprises the following units: An encoding unit is used to pre-set keywords related to chronic kidney disease and the importance of the keywords, obtain values ​​corresponding to the keywords from the electronic medical record, form a <keyword, value, importance> triple, and concatenate the word embedding code of the value and the word embedding code of the importance to obtain an encoding result corresponding to each keyword; An attention calculation unit is used to sort the encoding results according to the importance, and divide the sorted encoding results into three subsets according to the importance; calculate the attention of each subset using a multi-head attention mechanism to obtain a first attention, and calculate the attention of the encoding results corresponding to all keywords using a multi-head attention mechanism to obtain a second attention; a risk prediction unit, configured to obtain an input of an encoder according to the importance, the first attention, and the second attention, and use an output of the encoder as an input of the MLP to obtain a risk probability of chronic kidney disease; The step of obtaining the input of the encoder according to the importance, the first attention, and the second attention is specifically: Get the subset where the keyword is located and get the average value μ1 of the importance of the subset; Calculate the average importance of all keywords μ2; According to the formula The encoder input is calculated; where, Indicates the first attention, represents the second attention, and C represents the input of the encoder; The encoder is an encoder of a Transformer network model, and the encoder includes N layers of the same structure; Among them, the input of the first LN layer in the nth layer is Where X n Represents the input of the nth layer, SubLayber(X n ) represents the output of the multi-head attention mechanism in the nth layer, N is an integer between [6,12], n = 1, 2, …, N; The first attention is obtained by respectively calculating the attention of each subset using the multi-head attention mechanism, specifically: For each subset, multiple <query weight matrix Q, key weight matrix K, value weight matrix V> triplets are set respectively. Each subset is passed through a <query weight matrix Q, key weight matrix K, value weight matrix V> according to the formula Get the feature vector in, represents the scaling factor; The multiple feature vectors corresponding to each subset Concatenate to get the feature vector Z j , then Z j Concatenate the feature vector Z and the weight matrix W 0 Multiply them together and you get the first attention; Wherein, j represents the sequence number of the subset, j=1 or 2 or 3, k represents the sequence number of the triplet, and k is a positive integer.

Images