A medical expense prediction method based on multi-scale features

Abstract

The application discloses a medical expense prediction method based on multi-scale features, comprising the following steps: step S1, collecting historical case data of patients, including numerical data, category data and ICD coding; step S2, based on ICD coding, hierarchical splitting and co-occurrence statistics are carried out to obtain sub-class word tables, fine-class word tables and extended-class word tables, and the co-occurrence matrix of each class word table; step S3, the co-occurrence matrix of the sub-class word table, the fine-class word table and the extended-class word table is subjected to Glove training respectively to obtain three sets of word vectors and splicing; and step S4, a hybrid model based on the GloVe model, the LSTM network and the multilayer perceptron is constructed to realize medical expense prediction. The application is based on multi-scale embedding and a hybrid model, and the precision of medical expense prediction is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of medical cost prediction, and in particular to a medical cost prediction method based on multi-scale features. Background Technology

[0002] Due to a lack of professional knowledge about treatment and a comprehensive understanding of policies, ordinary patients and their families often make medical decisions based only on brief consultations with doctors, making it difficult to estimate treatment costs and make medical decisions that meet their own needs.

[0003] Against this backdrop, using medical record front page data to predict patients' medical expenses during hospitalization can not only provide foundational data for setting medical insurance fund payment standards, but also effectively identify high-cost cases and potential resource waste, supporting the optimization of resource allocation and cost management. The data stored in a standard medical record front page typically includes the patient's basic demographic characteristics, diagnostic information, surgical procedure records, and other standardized medical data.

[0004] Currently, the prediction of treatment costs widely relies on single, traditional machine learning methods. These methods typically require significant manual intervention in feature engineering and are only suitable for small sample models. Furthermore, with at least 60,000 inpatient cases annually in ordinary tertiary hospitals in my country, manual feature processing is prone to errors, is time-consuming and labor-intensive, and leads to prediction bias. Traditional machine learning methods also have relatively weak overall expressive power and cannot incorporate multi-dimensional patient information to comprehensively predict treatment costs. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for predicting medical costs based on multi-scale features.

[0006] The objective of this invention is achieved through the following technical solution: a method for predicting medical costs based on multi-scale features, comprising the following steps:

[0007] Step S1. Collect the patient's historical medical data, including numerical data, categorical data and ICD codes, and use the ICD codes as the raw text corpus. The ICD codes include diagnostic ICD codes and surgical procedure ICD codes.

[0008] Step S2. Based on ICD encoding, perform hierarchical splitting and co-occurrence statistics to obtain subclass vocabulary, fine class vocabulary, and extended class vocabulary, as well as the co-occurrence matrix of each class vocabulary;

[0009] Step S3. Perform GloVe training on the co-occurrence matrices of the subclass vocabulary, the fine class vocabulary, and the extended class vocabulary respectively to obtain three sets of word vectors and then concatenate them.

[0010] Step S4. Construct a hybrid model based on the GloVe model, LSTM network and multilayer perceptron to predict medical costs.

[0011] The beneficial effects of the present invention are: (1) When embedding the ICD encoding in the present invention, the GloVe (Global Vectors for Word Representation) model is used to learn the low-dimensional vector representation through the contextual relationship of the patient's diagnostic sequence, which solves the sparsity problem of the traditional encoding method and the problem that the Word2Vec model does not consider the global text information, and preserves the semantic relationship between diseases;

[0012] (2) This invention uses two different coding layers to process data features and text features respectively (ICD diagnostic coding processing) to better predict related medical data costs;

[0013] (3) The hybrid model of the invention adopts two highly heterogeneous models: one is a combined model that combines cross-attention mechanism and multilayer perceptron (MLP), and the other is the CatBoost model. The two models work together through a data-driven approach and reconstruct the prediction results to obtain the final prediction value. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention;

[0015] Figure 2 This is a schematic diagram of the hierarchical logic in the embodiment;

[0016] Figure 3 This is a schematic diagram illustrating the specific principles of the hybrid model. Detailed Implementation

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0018] like Figure 1 As shown, a medical cost prediction method based on multi-scale features includes the following steps:

[0019] Step S1. Collect the patient's historical medical data, including numerical data, categorical data and ICD codes, and use the ICD codes as the raw text corpus. The ICD codes include diagnostic ICD codes and surgical procedure ICD codes.

[0020] The patient's medical record summary typically includes basic information such as the primary diagnosis, multiple secondary diagnoses, surgical procedures performed during hospitalization, and their corresponding ICD codes. These diagnoses and surgical procedures are often closely related to the patient's hospitalization costs and duration. Therefore, this chapter aims to consider these factors influencing hospitalization costs. However, since neural networks can only process numerical inputs, and ICD codes are usually in a combination of letters and numbers (e.g., the ICD code for a benign breast tumor is "D24.x00x001"), converting text-based ICD codes into numerical formats is a crucial step in model implementation. The numerical data includes the patient's age and length of hospital stay; the categorical data includes the patient's gender and marital status.

[0021] Step S2. Based on ICD encoding, perform hierarchical splitting and co-occurrence statistics to obtain subclass vocabulary, fine class vocabulary, and extended class vocabulary, as well as the co-occurrence matrix of each class vocabulary;

[0022] Traditional one-hot encoding methods suffer from excessive dimensionality when processing large amounts of data, leading to the curse of dimensionality. Furthermore, these methods cannot measure the similarity between words. While word2vec maps words to a vector space and calculates the probabilities between words, the GloVe model calculates the ratio between these probabilities, which is more effective at capturing the correlation and irrelevance between words. The core idea of ​​GloVe (Global Vectors for Word Representation) is to generate dense word vectors by fusing global word co-occurrence statistics with local context windows. This allows the distance in the vector space to reflect the semantic relationships between words, and the objective function directly optimizes the ratio of co-occurrence probabilities. It combines the advantages of global matrix factorization (LSA) and local context window methods (word2vec) while overcoming their shortcomings. The GloVe model first constructs a word-word co-occurrence matrix X, dividing words into center words and context words.

[0023] S201. Standardize the original text corpus (ICD encoding) to a uniform format, remove special characters, including stop words and punctuation marks, to obtain the ICD encoding set. , where n represents the total number of ICD code categories involved in the ICD code set. For each ICD code Each of them has a corresponding ICD code name. ;

[0024] S202. For each ICD code in the unified format ICD code set, analyze the hierarchy to obtain the hierarchical analysis results containing ICD code subclasses, fine classes, and extended classes; for example, automatically split the hierarchy according to the ICD classification system to generate a vocabulary such as ["D24.1", "I10", "E11.9"...], which is used for subsequent encoding and indexing.

[0025] Then, the hierarchical analysis results of all ICD-coded subclasses are used to construct a subclass vocabulary; the hierarchical analysis results of all ICD-coded fine classes are used to construct a fine class vocabulary; and the hierarchical analysis results of all ICD-coded extended classes are used to construct an extended class vocabulary.

[0026] S203. Construct a co-occurrence matrix for each vocabulary class. The method for constructing the co-occurrence matrix for any vocabulary class is as follows:

[0027] Set up a sliding window with a size of 5 to 10; this window is used to scan word sequences in the text.

[0028] Slide a window across the entire vocabulary to scan all word sequences;

[0029] For the central word of a window and the context words of the central word within the window, record the co-occurrence relationship;

[0030] Count the number of times each central word co-occurs with the context words of the central word within the window;

[0031] Based on the statistical co-occurrence frequency, construct a symmetric co-occurrence matrix X, where each element X... ij This represents the number of times the i-th word and the j-th word co-occur in the current word list;

[0032] Example as follows: Table 1: Co-occurrence frequency of the central word and surrounding words

[0033]

[0034] Step S204. Define a function The function performs weighted processing on the elements in the co-occurrence matrices of the three vocabulary lists, and its purpose is to smooth the co-occurrence frequency.

[0035] .

[0036] Step S3. Perform GloVe training on the co-occurrence matrices of the subclass vocabulary, the fine class vocabulary, and the extended class vocabulary respectively to obtain three sets of word vectors and then concatenate them.

[0037] S301. For any vocabulary list obtained in step S202, use the elements X in its co-occurrence matrix... ij and the smoothed result The input feature vector is formed by X, and the corresponding embedded word vectors serve as labels to form training samples; where X ij The co-occurrence counts of the i-th and j-th words in the current word list; the embedded word vectors are... w i ,w j This represents the vector representation of the i-th word and the j-th word in the current word list;

[0038] S302. Design the loss function for the GloVe model:

[0039]

[0040] in, and This is the bias term, where V represents the training batch size;

[0041] S303. The GloVe model is trained using training samples and the designed loss function. During the training process, the gradient descent algorithm is used to update the word vectors and bias terms to minimize the loss function J. Finally, it is checked whether the loss function has converged, that is, whether the loss value has reached a stable state.

[0042] S304. Use the GloVe model to predict the data obtained in step S2 and output ICD-encoded word vectors:

[0043] Using the elements X in the co-occurrence matrix ij and the smoothed result The vectors are input into the trained GloVe model to obtain the predicted embedded word vectors;

[0044] S305. For each vocabulary class, repeat steps S301 to S304 to obtain the GloVe model trained on the three vocabulary classes, and make predictions for the three vocabulary classes respectively;

[0045] Suppose the prediction results obtained from the subclass vocabulary, the fine class vocabulary, and the extended class vocabulary are respectively ; concatenation yields the global embedding vector :

[0046] .

[0047] The embedding representation generated by the above method is defined as a single-granularity embedding of the ICD encoding. Since the ICD encoding itself has a hierarchical structure from broad categories to specific categories, we extend the original features based on this hierarchical logic to more effectively model the inherent hierarchical relationships between ICD encodings. Taking the diagnostic ICD encoding "E11.65x032" as an example, its hierarchical logic is as follows: Figure 2 As shown.

[0048] Step S4. Construct a hybrid model based on the GloVe model, LSTM network and multilayer perceptron to predict medical costs.

[0049] S401. After processing the ICD codes in the patient's historical medical records according to steps S2~S3, the global embedding vector is input into the LSTM network to extract text features, and then inputs them into the cross-attention mechanism module along with numerical and categorical data to obtain sample features. At the same time, the corresponding medical expenses are collected as sample labels.

[0050] Obtain multiple sets of sample features and corresponding sample labels to form a training sample set;

[0051] The Multilayer Perceptron (MLP) and CatBoost models were trained using the training sample set, respectively, to obtain the trained MLP and CatBoost models.

[0052] S402. In actual prediction, after processing the ICD encoding according to steps S2~S3, the global embedding vector is input into the LSTM network to extract text features, and then input into the cross-attention mechanism module along with numerical and categorical data to obtain input features;

[0053] S403. Input the input features into the trained multilayer perceptron (MLP) to predict the total medical cost P1;

[0054] S404. Input the input features into the trained CatBoost model to predict another total medical cost result P2;

[0055] S405. Input P1 and P2 into the fusion layer for weighted fusion to obtain the final predicted medical cost P = αP1 + βP2; where α and β are weight coefficients and satisfy the constraints that α + β = 1 and α, β >= 0.

[0056] To address the differences in various features and enhance its expressive power, this model employs a dual-channel collaborative architecture. First, ICD encoding is input into the GloVe model for encoding, followed by multi-granularity embedding. The embedded vectors are then input into an LSTM network to extract textual features. Simultaneously, numerical data (such as length of hospital stay and age) is processed to extract numerical features. Next, textual, numerical, and categorical features are fused and input into a cross-attention mechanism module to integrate information from different categories. The integrated information is then input into a Multilayer Perceptron (MLP) to predict the total medical cost result P1. Furthermore, textual, numerical, and categorical features are again input into a CatBoost model to predict another total medical cost result P2. Finally, P1 and P2 are reconstructed to further explore the relationships between features, yielding the final predicted medical cost value P. The detailed flowchart is shown below. Figure 3 As shown:

[0057] This study integrates multi-source heterogeneous inpatient data from a tertiary hospital in a certain city of a certain province from 2022 to January 2023, including structured numerical features, categorical variables, and ICD diagnostic / surgical codes.

[0058] First, the ICD encoding (text features) is vectorized using the GloVe model, and its representational power is enhanced through a multi-granularity embedding strategy. Then, an LSTM network is used to process these vectors to extract text features. Simultaneously, numerical data such as length of hospital stay and age are input into an MLP network to extract numerical features. Next, a cross-attention mechanism is used to deeply fuse text features, numerical features, and category features. In this process, the cross-attention mechanism can dynamically capture the interaction relationships between features of different modalities, significantly improving the model's ability to model complex feature associations.

[0059] Simultaneously, structured processing was performed. Numerical features such as length of hospital stay and age were standardized using Z-scores, and missing values ​​were imputed using the median of all values ​​of that type. For medical sub-costs (such as surgery fees), a Box-Cox transformation (λ=0.5) was applied to improve normality.

[0060] In the embodiments of this application, for the discrete characteristic embedding of the input ICD encoded sequence, this study uses the Embedding Layer to process and feeds the embedding vector into the LSTM network for more complex processing in order to perform sequence modeling and capture the temporal and contextual information in the sequence data.

[0061] In the embodiments of this application, the number of ICD codes for different patients is usually variable. Therefore, in model processing, batch padding of the ICD codes is required to ensure that each batch of input has the same length. The input is then processed in the GloVe model to convert it into a numerical format, and finally, the different levels are concatenated before being fed into the LSTM model to extract the final text features.

[0062] The processed features are fused using an attention mechanism, allowing features from different branches to interact and enhancing the model's ability to capture complex relationships between features. Then, a multilayer perceptron (MLP) is used to process the information and predict the first total cost value, labeled P1. Simultaneously, the three processed features are fed into a CatBoost model to predict the second total cost value, labeled P2. Finally, the two total cost predictions are recombined to extract deeper information and predict the final total cost value P.

[0063] During the model evaluation phase, this application can use three evaluation metrics—mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE)—to compare the reconstructed cost predictions with the actual cost values ​​of the original data in order to evaluate the predictive performance of the model.

[0064] The foregoing description illustrates and describes a preferred embodiment of the present invention. However, as previously stated, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for predicting medical costs based on multi-scale features, characterized in that: Includes the following steps: Step S1. Collect the patient's historical medical data, including numerical data, categorical data and ICD codes, and use the ICD codes as the raw text corpus. The ICD codes include diagnostic ICD codes and surgical procedure ICD codes. Step S2. Based on ICD encoding, perform hierarchical splitting and co-occurrence statistics to obtain subclass vocabulary, fine class vocabulary, and extended class vocabulary, as well as the co-occurrence matrix of each class vocabulary; Step S3. Perform GloVe training on the co-occurrence matrices of the subclass vocabulary, the fine class vocabulary, and the extended class vocabulary respectively to obtain three sets of word vectors and then concatenate them. Step S4. Construct a hybrid model based on the GloVe model, LSTM network and multilayer perceptron to predict medical costs.

2. The medical cost prediction method based on multi-scale features according to claim 1, characterized in that: The numerical data includes the patient's age and length of hospital stay; the categorical data includes the patient's gender and marital status.

3. The medical cost prediction method based on multi-scale features according to claim 1, characterized in that: Step S2 includes: S201. Standardize the original text corpus (ICD encoding) to a uniform format, remove special characters, including stop words and punctuation marks, to obtain the ICD encoding set. , where n represents the total number of ICD code categories involved in the ICD code set; for each ICD code Each of them has a corresponding ICD code name. ; S202. For each ICD code in the unified format ICD code set, analyze the hierarchy to obtain the hierarchical analysis results containing ICD code subclasses, fine classes, and extended classes; then construct a subclass vocabulary from the hierarchical analysis results of all ICD code subclasses; construct a fine class vocabulary from the hierarchical analysis results of all ICD code fine classes; and construct an extended class vocabulary from the hierarchical analysis results of all ICD code extended classes. S203. Construct a co-occurrence matrix for each vocabulary class. The method for constructing the co-occurrence matrix for any vocabulary class is as follows: Set up a sliding window with a size of 5 to 10; this window is used to scan word sequences in the text. Slide a window across the entire vocabulary to scan all word sequences; Record the co-occurrence relationship between the central word of the window and the context words of the central word within the window; Count the number of times each central word co-occurs with the context words of the central word within the window; Based on the statistical co-occurrence frequency, construct a symmetric co-occurrence matrix X, where each element X... ij This represents the number of times the i-th word and the j-th word co-occur in the current word list; Step S204. Define a function The function performs weighted processing on the elements in the co-occurrence matrices of the three vocabulary lists, and its purpose is to smooth the co-occurrence frequency. 。 4. The medical cost prediction method based on multi-scale features according to claim 1, characterized in that: Step S3 includes: S301. For any vocabulary list obtained in step S202, use the elements X in its co-occurrence matrix... ij and the smoothed result The input feature vector is formed by X, and the corresponding embedded word vectors serve as labels to form training samples; where X ij The co-occurrence counts of the i-th and j-th words in the current word list; the embedded word vectors are... w i ,w j This represents the vector representation of the i-th word and the j-th word in the current word list; S302. Design the loss function for the GloVe model: ； in, and It is the bias term, and V represents the training batch size; S303. The GloVe model is trained using training samples and the designed loss function. During the training process, the gradient descent algorithm is used to update the word vectors and bias terms to minimize the loss function J. Finally, it is checked whether the loss function has converged, that is, whether the loss value has reached a stable state. S304. Use the GloVe model to predict the data obtained in step S2 and output ICD-encoded word vectors: Using the elements X in the co-occurrence matrix ij and the smoothed result The vectors are input into the trained GloVe model to obtain the predicted embedded word vectors; S305. For each vocabulary class, repeat steps S301 to S304 to obtain the GloVe model trained on the three vocabulary classes, and make predictions for the three vocabulary classes respectively; Suppose the prediction results obtained from the subclass vocabulary, the fine class vocabulary, and the extended class vocabulary are respectively ; concatenation yields the global embedding vector : 。 5. The medical cost prediction method based on multi-scale features according to claim 1, characterized in that: Step S4 includes: S401. After processing the ICD encoding according to steps S2~S3, the global embedding vector is input into the LSTM network to extract the text features, and then input the numerical data and categorical data into the cross-attention mechanism module to obtain the sample features. At the same time, the corresponding medical expenses are collected as sample labels. Obtain multiple sets of sample features and corresponding sample labels to form a training sample set; The Multilayer Perceptron (MLP) and CatBoost models were trained using the training sample set, respectively, to obtain the trained MLP and CatBoost models. S402. In actual prediction, after processing the ICD encoding according to steps S2~S3, the global embedding vector is input into the LSTM network to extract text features, and then input into the cross-attention mechanism module along with numerical and categorical data to obtain input features; S403. Input the input features into the trained multilayer perceptron (MLP) to predict the total medical cost P1; S404. Input the input features into the trained CatBoost model to predict another total medical cost result P2; S405. Input P1 and P2 into the fusion layer for weighted fusion to obtain the final predicted medical cost P = αP1 + βP2; where α and β are weight coefficients and satisfy the constraints that α + β = 1 and α, β >= 0.

Images