A method for processing a historical visit sequence and application

Abstract

The application relates to the technical field of kidney disease prediction, and discloses a history visit sequence processing method and application, which comprises the following steps: obtaining a history visit sequence of a subject; extracting history medical codes and history non-medical information from the history visit sequence, merging the history medical codes and the history non-medical information, and obtaining a visit vector sequence; calculating an embedding vector based on the visit vector sequence; inputting the history medical codes into a history encoder to output a hidden state sequence; obtaining a feature vector sequence according to the embedding vector and the hidden state sequence, and obtaining an attention weight sequence according to the feature vector sequence; inputting the embedding vector, the feature vector sequence and the attention weight sequence into a collaborative attention mechanism to obtain a context vector; and obtaining a processing result according to the context vector. The application can improve the accuracy of chronic kidney disease prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of kidney disease prediction, in particular to a processing method of historical visit sequences and application. BACKGROUND

[0002] Risk models are of great importance in disease prognosis, diagnosis, and intervention. Currently, most risk models rely on expert-selected predictors and are constructed using traditional statistical models. However, with the gradual acquisition of electronic health records, especially longitudinal electronic health records, new risk prediction models can be developed using patients' health trajectories. These new models can capture important predictors and their long-term interdependencies, resulting in more accurate risk predictions. By integrating more comprehensive and dynamic health information, these models have the potential to provide more personalized and precise healthcare services, helping doctors better assess patients' risk levels and prevent and intervene in potential health problems in advance, thereby improving patients' overall health and quality of life.

[0003] Electronic health records provide clinicians with up-to-date and comprehensive information about patients, enabling them to assess the entire patient trajectory and utilize actual available clinical practice content. Despite the complexity and heterogeneity of electronic health records, deep learning models have shown superior performance in various complex risk prediction tasks, especially when using large-scale electronic health records. However, current applications typically only utilize a small amount of information from the dataset, mainly focusing on diseases and medications in hospital records. This limits the number of patient records and sequence length, while actual needs require more comprehensive information. Therefore, it is necessary to establish risk prediction models to handle thousands of electronic health record sequences or even longer to avoid losing important historical information. SUMMARY

[0004] The primary object of the present application is to overcome the problems existing in the prior art and provide a processing method of historical visit sequences. The present application can improve the accuracy of chronic kidney disease prediction.

[0005] To achieve the above object, the present application provides a processing method of historical visit sequences, which comprises:

[0006] S1: obtaining a historical visit sequence of a subject;

[0007] S2: extracting historical medical codes and historical non-medical information from the historical visit sequence, and merging the historical medical codes and historical non-medical information to obtain a visit vector sequence;

[0008] S3: calculating an embedding vector based on the visit vector sequence, specifically comprising:

[0009] S3.1: The medical visit vector sequence is mapped to obtain a vector array, specifically as follows:

[0010]

[0011] in, For learnable embedding matrices, For time Medical visit vector For the total number of features, It is a vector array;

[0012] S3.2: Perform a linear transformation on the vector array to obtain the input feature representation, specifically:

[0013]

[0014] in, For input feature representation;

[0015] S3.3: Input the input feature representation into the Transformer to obtain the output feature representation, specifically as follows:

[0016]

[0017] in, This represents the output features.

[0018] S3.4: Aggregate the embeddings of all features occurring at the current time t, and average them to obtain the first embedding vector. Specifically:

[0019]

[0020] in, In time The number of characteristics that occur, For the first Embedding vectors of feature vectors;

[0021] S3.5: Sequence The input is fed into an encoding recurrent neural network layer, and the output is a patient embedding vector. Specifically:

[0022]

[0023] in, To convert the output of a recurrent neural network layer into an embedding vector, and For time The hidden states of a recurrent neural network layer are used to capture temporal information in a sequence. Determined by the following formula:

[0024]

[0025] wherein, is a forward propagation function of a recurrent neural network layer, is a time previous hidden state;

[0026] S4: inputting the historical medical code into a historical encoder to output a hidden state sequence;

[0027] S5: obtaining a feature vector sequence according to the embedding vector and the hidden state sequence, and obtaining an attention weight sequence according to the feature vector sequence;

[0028] S6: inputting the embedding vector, the feature vector sequence and the attention weight sequence into a collaborative attention mechanism to obtain a context vector;

[0029] S7: obtaining a processing result according to the context vector.

[0030] Further, the inputting the historical medical code into a historical encoder to output a hidden state sequence specifically comprises:

[0031] performing mapping processing on the historical medical code to obtain a mapped output vector, specifically:

[0032]

[0033] wherein, is a time medical code, is a weight matrix, is a time mapped output vector;

[0034] inputting the mapped output vector into a recurrent neural network layer to obtain a hidden state sequence, specifically:

[0035]

[0036] wherein, is an update function in the recurrent neural network, used for updating a hidden state according to an mth layer hidden state of a previous time and a mapped output vector of a current time , is an mth layer hidden state of the recurrent neural network layer at a time .

[0037] Further, a feature vector sequence is obtained based on the embedding vector and the hidden state sequence, and an attention weight sequence is obtained based on the feature vector sequence, specifically including:

[0038] The embedding vector and each hidden state in the hidden state sequence are combined to obtain a feature vector sequence, specifically:

[0039]

[0040] in, For embedding vectors, For time eigenvectors;

[0041] Calculate the score corresponding to each feature vector based on the feature vector sequence to obtain the score sequence;

[0042] The attention weight corresponding to each score is calculated based on the score sequence to obtain the attention weight sequence.

[0043] Furthermore, the step of calculating the score corresponding to each feature vector based on the feature vector sequence to obtain the score sequence is specifically calculated as follows:

[0044]

[0045] in, For use in transforming feature vectors The weight matrix; It is the bias vector; This is the hyperbolic tangent function, used to compress the input value to... Within the range, To calculate the weight vector for the score, For time The score.

[0046] Furthermore, the attention weight corresponding to each score is calculated based on the score sequence to obtain the attention weight sequence. The specific calculation method is as follows:

[0047]

[0048] in, The normalization factor is calculated by summing the scores over all time steps after exponential operation.

[0049] Further, the embedding vector, the feature vector sequence, and the attention weight sequence are...

[0050] The collaborative attention mechanism is used to obtain the context vector, and the specific calculation method is as follows:

[0051]

[0052] is a feature vector of all time steps is weighted and summed, is a context vector.

[0053] Further, the processing result is obtained according to the context vector, and the specific calculation manner is:

[0054]

[0055] wherein, is a weight vector, is a bias term, is a sigmoid function, is a prediction probability.

[0056] The application also provides an application of the processing method of the historical visit sequence, which is used for predicting chronic kidney disease, and the obtained processing result can predict chronic kidney disease.

[0057] Finally, the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the processing method of the historical visit sequence.

[0058] Compared with the prior art, the application has the beneficial effects that:

[0059] The application calculates an embedding vector through a visit vector sequence, introduces multi-level feature extraction and embedding based on a Transformer model, can comprehensively analyze the visit history of a patient, better solves the comprehensive analysis of the visit history of the patient, improves the algorithm interpretability, and thus accurately predicts whether the patient will be diagnosed as chronic kidney disease for the first time at the next visit, thereby providing strong support for early diagnosis and treatment of chronic kidney disease. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a flowchart of the processing method of the historical visit sequence according to an embodiment of the application;

[0061] Figure 2 is a representation of non-medical coding information and medical coding information;

[0062] Figure 3 is an architecture diagram of an online network and a target network;

[0063] Figure 4 is a process of an electronic medical record enhancement model and a segmented enhancement model;

[0064] Figure 5 is a process of a sliding window;

[0065] Figure 6 Figure 1 is a schematic diagram of a processing method of a historical visit sequence according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] The specific embodiments of the present application will be further described in conjunction with the drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0067] In the description of the present application, it should be noted that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0068] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0069] In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0070] Example 1

[0071] As shown in Figures 1-6 , the historical visit sequence processing method of the preferred embodiment of the present application comprises:

[0072] S1: obtaining a historical visit sequence of a subject;

[0073] In one embodiment, the medical history of a patient is represented by a time series of visit records, each record containing medical codes and clinical observations; medical codes are mainly represented in the form of multi-hot vectors, and non-medical information is used as auxiliary features. Given a patient, the medical history is presented in the form of a hospital visit sequence arranged in chronological order , wherein the visit sequence refers to all visit records of the patient arranged in chronological order, each record containing medical codes and clinical observations; wherein This refers to the total number of patient visits; each visit... It contains a series of medical codes, clinical observations and other information related to the patient.

[0074] S2: Extract historical medical codes and historical non-medical information from the historical medical visit sequence, and merge the historical medical codes and historical non-medical information to obtain a medical visit vector sequence;

[0075] In one embodiment, the medical code representation includes diagnostic codes, procedure codes, and drug codes; the medical code is a key feature of the prediction task; the set of medical codes in the electronic medical record data is represented as... Size is Hospital visit All medical codes that occur at that time are generated by multi-heat vectors. Indicates; where, if ,but The Each element is 1; where the medical code set C contains all possible medical codes; medical codes This represents a specific medical code during a particular medical visit. All medical codes during a medical visit are represented by a multi-hot vector. If a medical code exists during a medical visit, the corresponding position of that medical code in the multi-hot vector has a value of 1.

[0076] Non-medical code information: Patient observations include laboratory tests, vital signs, age, gender, and visit details. Timestamp; observation results are secondary features; let The vector representation of non-medical code information is a concatenation of multi-hot vectors and numerical values.

[0077] S3: Calculate the embedding vector based on the medical visit vector sequence;

[0078] In an embodiment, the sequence of visit vectors is input into an encoder to obtain embedding vectors, the encoder is an encoder of an online network, and the online network belongs to a same-direction self-supervised model, training the encoder includes: using four types of embeddings as input, token embeddings are from all available code or categorical variables of diagnoses, medications, surgeries, tests, blood pressure measurements, alcohol status, smoking status, and body mass index records, obtained from medical codes such as diagnoses, medications, surgeries, tests, etc., and non-medical codes such as blood pressure, alcohol, smoking, BMI, etc.; age embeddings are representations of age, obtained directly from patient age data; segment embeddings alternate between different visits, taking values of 0 and 1, and are obtained according to the order of different visit records; and position embeddings monotonically increase between different visits, and the monotonically increasing values in each visit record represent; the embedding of each visit record is represented by the sum of record, age, segment, and position embeddings, the same-direction self-supervised learning model: the model has similar representations based on different augmentations of the same data; it consists of two neural networks: an online network and a target network; the online network is updated by backpropagation defined, including an encoder , a projector , and a predictor , the weights of the online network are updated by backpropagation;

[0079] The structure of the target network is seen Figure 3 from the online network, the target network has the same architecture as the online network but has a different set of weights ; the target network provides regression targets to train the online network, and its parameters are the exponentially moving average of the online parameters ; given a target decay rate , the following update is performed after each training step;

[0080]

[0081] where is the weight of the target network, and the target network is used to guide the learning of the online network through the weight; is the weight of the online network, and the online network adjusts the weight to adapt to the data and task through learning and adjustment; is a decay factor between 0 and 1, which controls the speed of updating the weight of the target network, if is close to 0, the influence of the new online network weight will be smaller because they are weightedly averaged into the weight of the target network, if is close to 1, the influence of the new online network weight will be greater;

[0082] This equation shows that after each training step, the weights of the target network are updated by taking a weighted average of the weights of the online network and the weights of the target network ; specifically, a fraction of the target network weights is preserved by multiplying by a decay factor and a fraction of the online network weights is obtained by multiplying by , and then they are added to obtain the new target network weights; the goal is to gradually transfer the knowledge of the online network to the target network in order to guide the learning process of the online network more robustly;

[0083] Given a set of access records , uniformly sampled records from and the distribution of electronic health record augmentations and the distribution of segment augmentations ; the co-directional self-supervised learning model generates two augmented models and by respectively applying the electronic health record augmentation and the segment augmentation from ;

[0084] Electronic health record augmentation: its purpose is to enrich electronic health record data and increase data diversity; before the sliding window, apply random cropping to the electronic health record with a probability ; randomly mask each label with a probability ; random cropping means randomly selecting a subset of electronic health records within a period;

[0085] Segment augmentation: only augment the segment representation in the online network; the context can provide enough information for the network to value the representation of the augmented time step; use the online network and the target network to reduce the dissimilarity of the representation of the augmented time step from its original representation, which can be achieved by optimizing the sum of the similarity loss of all augmented time steps; apply segment augmentation to the segment representation, and then input the segment representation into the feature aggregator; mask the segment representation by converting it to 0, or discard the latent representation by adding Gaussian noise;

[0086] In the same-direction self-supervised learning model, an online network and a target network are trained using two augmented data records. The representations output by these two networks are first normalized, and then the mean squared error between the normalized vectors is used as a loss function to measure their similarity. The parameters of the online network are adjusted by optimizing this loss function, while the parameters of the target network are updated using an exponential moving average of the online parameters. The goal of the entire training process is to make the representations generated by the online network as similar as possible to the representations of the target network. Training is complete when the loss function converges to a specified threshold and the training reaches a preset number of epochs. After training, the online network is only used in the inference stage, and only the encoder part is retained to reduce the computational cost during inference, aiming to improve the efficiency and accuracy of prediction.

[0087] like Figure 4 As shown, training the online network and the target network specifically includes: based on the first augmentation record Online network output indicates and projection ;

[0088] The target network from the second augmentation record Output and target projection ;

[0089] Output Prediction and Will and Normalization and ;

[0090] Define the following mean square error between the normalized prediction and the target projection:

[0091]

[0092] in, It is the loss function, and the model parameters are... and The loss; This represents the normalized prediction vector, which is the prediction vector. Divide by norm The result obtained; This represents the target vector for normalization; it is the target vector. Divide it The result obtained from norm; It is the difference between the normalized prediction vector and the normalized target vector. The square of the norm difference is the square of their Euclidean distance; The inner product represents the degree of similarity between two vectors; It consists of two vectors. The norm score normalizes the inner product; this formula quantifies the similarity between the prediction and the target by calculating the inner product between two vectors and then using the difference of the normalized product, and expresses the loss in the form of mean squared error.

[0093] Will Enter online network and Input the target network to proportionally symmetrically represent the loss. To calculate In each training step, a randomized optimization step is performed, targeting only... Instead To minimize The dynamic summary of the same-direction self-supervised learning model is as follows:

[0094]

[0095] in, These are model parameters; It's an optimizer, based on gradients. and learning rate To update parameters ; Represents the loss function Regarding parameters The gradient; this formula updates the model's parameters through the optimizer during training to minimize the loss function. ;

[0096] At the end of training, only the encoder is retained. Only in the final representation The number of inference time weights is considered in the calculation;

[0097] Target parameters of the same direction self-supervised learning model The update is not towards The direction ensures no loss. This makes the dynamics of the same-direction self-supervised learning model in... and upper joint minimization Gradient descent optimizes the predictor of the self-supervised learning model in the same direction, as shown in the following equation:

[0098]

[0099] in, Represents the loss function The optimal solution is the one that minimizes the loss function. ; express Perform a search to find the value that minimizes the loss function. the value of is a loss function that measures the squared Euclidean distance between the representation produced by the online network and the representation produced by the target network The smaller the distance, the higher the similarity between the two representations.

[0100] Let the bad equilibrium be unstable, the update of the co-directional self-supervised learning model on the parameter is expected to follow the gradient of the expected conditional variance:

[0101]

[0102] where, denotes the gradient of the loss function with respect to the online network parameter ; is the optimal predictor that minimizes the loss function; denotes the squared Euclidean distance, where is the vector obtained by concatenating the representation of the online network to the representation of the target network ; denotes the sum of the conditional variances, where denotes the th feature in the vector The conditional variance refers to the variance of the target network representation vector on each feature given the representation of the online network .

[0103] After training is complete, the embedding vector is obtained according to the historical consultation vector sequence, as shown in Figures 5-6 , including: extracting a series of consultation sequences in the form of , where , n is the total number of features; then the sequence reconstructs itself through an encoder-decoder structure.

[0104] Encoder: first map each to a latent space with a learnable embedding matrix ; this expands the original input vector into an array of vectors for Transformer processing;

[0105] Since the Transformer has a quadratic time and space complexity and is computationally expensive, the feature dimension is reduced from n to through a linear transformation ; this reduces the complexity to to improve scalability;

[0106] The embedding of all features occurring at time t is aggregated using a Transformer with one encoder block and one attention head, as follows,

[0107]

[0108] where, , is the output feature representation; is the input feature representation;

[0109] To aggregate the embedding of all features occurring at time t, the is averaged over the reduced feature dimension to obtain a single embedding vector ; denoted as the following relation:

[0110]

[0111] where, denotes the number of features occurring at time t; denotes the embedding vector of the i-th feature;

[0112] The sequence is input into an encoding recurrent neural network layer, which encodes the entire input sequence into a single vector patient embedding ;

[0113]

[0114] where, denotes the hidden state of the recurrent neural network layer, which captures the temporal information in the sequence; denotes the forward propagation function of the recurrent neural network layer, which takes the input embedding at the current time step and the hidden state at the previous time step, and outputs the hidden state at the current time step;

[0115]

[0116] where, is a function that converts the output of the recurrent neural network layer into a patient embedding, which takes the final hidden state of the recurrent neural network layer as input and outputs the patient embedding ; the entire process iterates over the input sequence using the recurrent neural network, and then extracts the patient embedding from the final hidden state; finally, the information of the entire sequence is encoded into a single vector representation of the patient's features;

[0117] Decoder: propagated to the decoding recurrent neural network layer to obtain a decoding sequence ,

[0118]

[0119] wherein, denotes a decoding recurrent neural network layer that receives the patient embedding as input and generates a decoding sequence ;

[0120] The input sequence is then reconstructed from to obtain ;

[0121]

[0122] wherein, is a separate fully connected layer that receives the decoding sequence as input and generates a reconstruction sequence by learning to reconstruct different types of features ;

[0123] Specifically, a separate fully connected layer is added on top of to reconstruct different types of features and obtain a reconstruction sequence ; the loss between the input sequence and the reconstruction sequence is the sum of multiple losses:

[0124]

[0125] wherein, is a loss function used to calculate the loss between the input sequence and the reconstruction sequence ; this equation is used to measure the difference between the input sequence and the reconstruction sequence, optimize the model parameters, and ensure that the model can accurately reconstruct and predict;

[0126] For a vector of multi-hot medical codes and observations, denoted as ; the corresponding true class label, denoted as ; for an input vector , it is input into the softmax function to obtain the predicted class probability distribution , where each element represents the probability that the sample belongs to the th class;

[0127]

[0128] wherein, is the predicted class probability distribution; The function is used to calculate the predicted probability distribution;

[0129] The real class label and the predicted class probability distribution The cross-entropy loss between them is used to measure the difference between the model's prediction and the real situation, the more accurate the prediction of the correct class, the lower the loss;

[0130]

[0131] where, represents the first element of the real class label , represents the first element of the predicted class probability distribution ;

[0132] For the gender vector, it is represented as ; the corresponding real class label is represented as gender, which uses a scalar value to represent gender;

[0133] For the input vector x, it is input into the sigmoid function to get the predicted class probability, for gender, a sigmoid output represents the probability of gender;

[0134]

[0135] where, represents the predicted gender probability;

[0136] For the gender classifier, the cross-entropy loss is:

[0137]

[0138] represents the real gender label;

[0139] The probability is calculated by the sigmoid function, and then the cross-entropy loss is used to measure the accuracy of the prediction, which is used to optimize the classifier;

[0140] For age and timestamp, a linear transformation is used to map them to a new feature space; let the original age feature be a and the timestamp feature be t, then the linear transformation can be represented as follows, the linear transformation is to map age and timestamp to a new feature space:

[0141]

[0142] where, is the weight matrix to be learned, is the bias vector to be learned;

[0143]

[0144] wherein, is the weight matrix to be learned, is the bias vector to be learned;

[0145] the features after linear transformation and the mean square error between the predicted age and the true age and the mean square error between the predicted timestamp and the true timestamp to measure the prediction accuracy of the model; the mean square error is defined as the mean of the square of the difference between the predicted value and the true value:

[0146]

[0147] wherein, N is the number of samples; is the age predicted by the model, is the true age;

[0148]

[0149] wherein, N is the number of samples; is the timestamp predicted by the model, is the true timestamp;

[0150] The accuracy of the model prediction is measured by the mean square error, and the model parameters are optimized to improve the prediction accuracy.

[0151] S4: input the historical medical code into the historical encoder, and output a hidden state sequence;

[0152] In one embodiment, the discrete medical code is input into a continuous latent embedding space as follows:

[0153]

[0154] wherein, represents an input vector; is a learnable weight matrix, which is used to perform linear transformation on the input vector, so as to map the input vector into another space; is the size of the embedding vector; represents an output vector, which is the result obtained after linear transformation of the input vector by the weight matrix ;

[0155] In order to encode the medical code sequence, a recurrent neural network layer is applied on the embedding medical code:

[0156]

[0157] in, It is the time of the recurrent neural network layer The hidden state is used to capture historical information in sequence data; This represents the update function in a recurrent neural network, which updates the data based on the hidden state of the previous time step and the input vector of the current time step. To update the hidden state; This indicates the hidden state of the previous time step; This represents the output vector at time step t, which is typically a feature vector extracted from the input sequence and used to update the hidden state.

[0158] S5: Obtain a feature vector sequence based on the embedding vector and the hidden state sequence, and obtain an attention weight sequence based on the feature vector sequence;

[0159] In one embodiment, the patient is embedded to interact with their medical code history; specifically, computation... and each The interactions between them are as follows:

[0160]

[0161] in, The patient embedding vector is calculated by a pre-trained encoder. The patient embedding vector captures the patient's overall information, including the patient's basic characteristics such as age and gender, as well as medical records such as past visits, diagnoses, and treatments. Indicates at time step The memory state is calculated through a recurrent neural network model and contains historical information learned by the model from sequence data. This represents the new feature vector at time step t, which is obtained by combining the patient embedding vector and the memory state. It contains information about the patient embedding vector and the memory state, as well as the relationship between them.

[0162]

[0163] in, The feature vector at time step t is represented by the patient embedding vector. and memory state The result obtained by combining these elements reflects a comprehensive picture of the patient's current condition and historical information. Indicates the use of transforming feature vectors The weight matrix; Represents the bias vector; denotes the hyperbolic tangent function, which compresses the input value into the range [-1, 1], and helps to extract the nonlinear relationship of the features; u denotes the weight vector used to calculate the score;

[0164]

[0165] S6: input the embedding vector, the sequence of feature vectors and the sequence of attention weights into a co-attention mechanism to obtain a context vector;

[0166] In one embodiment, obtaining the context vector is specifically:

[0167]

[0168] wherein, denotes the weighted sum of the feature vectors at all time steps, where the weights are given by the attention weights ; this part calculates the weighted average according to the attention distribution to capture the important information of each time step in the input sequence; by reconstructing the input sequence through the decoder, it ensures that the model can accurately capture and reconstruct the input information, and provides a complete information background for the model when constructing the context vector;

[0169] S7: obtain a processing result according to the context vector.

[0170] In one embodiment, obtaining the processing result and making a disease prediction according to the processing result is specifically: using the context vector to predict a binary label , as follows:

[0171]

[0172] wherein, is the predicted probability score of the patient; denotes the dot product of the weight vector and the context vector , which is equivalent to the weighted sum of the context vector, where the weights are determined by the weight vector ; is a bias term used to adjust the output of the prediction; is a sigmoid function that maps the input real number to the interval (0, 1), which is used to convert the result of the weighted sum to a probability value representing the probability of the positive example;

[0173] The training target is to minimize the following binary cross-entropy loss using the predicted score and the true label y:

[0174] ​

[0175] where N is the total number of patients; represents, for positive example samples , the first term represents the loss when the model predicts a positive example, while for negative example samples , the second term represents the loss when the model predicts a negative example, the aim being to minimize the difference between the model's predictions and the true labels in order to improve the model's prediction accuracy.

[0176] Embodiment 2

[0177] A computer device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a processing method for a historical visit sequence as in embodiment 1 when executing the computer program.

[0178] The memory and the processor are connected in a bus manner, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers, and power management circuits, etc. together, which are well known in the art, and therefore, further description thereof will not be given herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. Data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0179] In one specific embodiment, a computer readable storage medium is also provided, which stores a computer program, the computer program being executed by a processor to implement a processing method for a historical visit sequence as in embodiment 1.

[0180] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing related hardware, the programs being stored in a storage medium and including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0181] To sum up, the embodiment of the present application provides a kind of historical treatment sequence processing method and application, it is through the treatment vector sequence calculation embedded vector, introduce the multilevel feature extraction and embedding based on Transformer model, can comprehensively analyze the treatment history of patient, better solve the comprehensive analysis of the treatment history of patient, improve the algorithm interpretability simultaneously, to accurately predict whether patient will be diagnosed for the first time as chronic kidney disease at next time, provide strong support for early diagnosis and treatment of chronic kidney disease.

[0182] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and replacements without departing from the technical principles of the present application, and these improvements and replacements should also be considered as the protection scope of the present application.

Claims

1. A method of processing a historical visit sequence, characterized by, The method comprises the following steps: S1: obtaining a historical visit sequence of a subject; S2: extracting historical medical codes and historical non-medical information from the historical visit sequence, and merging the historical medical codes and the historical non-medical information to obtain a visit vector sequence; S3: calculating an embedding vector based on the visit vector sequence, specifically comprising: S3.1: performing mapping processing on the visit vector sequence to obtain a vector array, specifically comprising: wherein, is a learnable embedding matrix, is time is a visit vector, is a total number of features, is a vector array; S3.2: performing linear transformation on the vector array to obtain an input feature representation, specifically comprising: wherein, is an input feature representation; S3.3: inputting the input feature representation into a Transformer to obtain an output feature representation, specifically comprising: wherein, is an output feature representation; S3.4: Aggregate current time t Embedding of all features that occur, and average, to obtain first embedding vector Specifically: wherein, is a time characteristic number, is an embedding vector for the th feature vector; S3.5: inputting the sequence to an encoding recurrent neural network layer, outputting a patient embedding vector , in particular: wherein, is a function that converts the output of a recurrent neural network layer into an embedding vector, and is time is a hidden state of a recurrent neural network layer that captures temporal information in a sequence, and is determined by the following equation: wherein, is a forward propagation function of a recurrent neural network layer, is a time a hidden state of a previous time; S4: inputting the historical medical codes into a historical encoder to output a hidden state sequence; S5: obtaining a feature vector sequence according to the embedding vector and the hidden state sequence, and obtaining an attention weight sequence according to the feature vector sequence; S6: inputting the embedding vector, the feature vector sequence and the attention weight sequence into a collaborative attention mechanism to obtain a context vector; S7: obtaining a processing result according to the context vector.

2. The method of claim 1, wherein, The inputting of the historical medical codes into the historical encoder to output the hidden state sequence specifically comprises: performing mapping processing on the historical medical codes to obtain a mapped output vector, specifically comprising: wherein, is time medical code, is a weight matrix, is time mapped output vector; inputting the mapped output vector into a recurrent neural network layer to obtain a hidden state sequence, specifically comprising: wherein, is an update function in a recurrent neural network for updating a hidden state at a current time from a previous time based on an mth layer hidden state and a mapped output vector at the current time , is an mth layer hidden state of a recurrent neural network layer at a time .

3. The method of claim 1, wherein, obtaining a feature vector sequence according to the embedding vector and the hidden state sequence, and obtaining an attention weight sequence according to the feature vector sequence, specifically comprising: combining the embedding vector and each hidden state in the hidden state sequence to obtain a feature vector sequence, specifically comprising: wherein, embedding a vector for the patient, is a time feature vector; calculating a score corresponding to each feature vector according to the feature vector sequence to obtain a score sequence; calculating an attention weight corresponding to each score according to the score sequence to obtain an attention weight sequence.

4. The method of claim 3, wherein, The calculation of the score corresponding to each feature vector according to the feature vector sequence to obtain a score sequence is specifically as follows: wherein, is a weight matrix for transforming a feature vector ; is a bias vector; is a hyperbolic tangent function for compressing an input value to a range of [0, 1], is a weight vector for computing a score, is a score at time .

5. The method of claim 4, wherein, The calculation of the attention weight corresponding to each score according to the score sequence to obtain an attention weight sequence is specifically as follows: wherein, is the sum of the scores for all time steps after exponentiation, used to calculate the normalization factor.

6. The method of claim 5, wherein, The inputting of the embedding vector, the feature vector sequence and the attention weight sequence into the collaborative attention mechanism to obtain a context vector is specifically calculated as follows: is a weighted sum of the feature vectors for all time steps is a weighted sum of the feature vectors for all time steps is a context vector.

7. The method of claim 6, wherein, The calculation of the processing result according to the context vector is specifically calculated as follows: where, is a weight vector, is a bias term, is a sigmoid function, is a predicted probability.

8. Use of a processing method based on a sequence of historical visits according to any one of claims 1 to 7, characterized in that The application is used for predicting chronic kidney disease, and the obtained processing result can predict chronic kidney disease.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to realize the processing method of the historical visit sequence according to any one of claims 1 to 7.

Images