A medical insurance fraud detection method and system based on an autoregressive Transformer
By employing an autoregressive Transformer detection method, this approach utilizes pre-trained medical text encoding and multi-head self-attention to capture the relative time dependence of medical insurance settlement items. Combined with self-cleaning training and Z-score normalization, it addresses the challenges faced by traditional methods in medical insurance fraud detection, achieving efficient and accurate fraud identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EAST CHINA UNIVERSITY OF POLITICAL SCIENCE AND LAW
- Filing Date
- 2026-03-04
- Publication Date
- 2026-06-05
AI Technical Summary
Traditional methods are difficult to effectively identify medical insurance fraud, especially when faced with rapidly evolving fraud patterns, imbalanced positive and negative samples, and high-dimensional sparsity of medical data, making it difficult to capture long-term dependencies between medical insurance settlement items.
A medical insurance fraud detection method based on autoregressive Transformer is adopted. Semantic vectors are generated by a pre-trained medical text encoding model, and relative temporal dependencies are captured by a multi-head self-attention layer. The prediction error is processed by a self-cleaning training mechanism and Z-score standardization to construct an account-level anomaly score.
It improves the accuracy and stability of medical insurance fraud detection, reduces training and inference costs, enhances the model's ability to represent unseen items, and improves generalization performance and recall.
Smart Images
Figure CN122155859A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical insurance fraud detection technology, and in particular to a medical insurance fraud detection method and system based on autoregressive Transformer. Background Technology
[0002] Temporal anomaly detection plays a crucial role in protecting complex data-driven systems, especially in high-risk sectors such as finance, cybersecurity, and health insurance. In recent years, with the continuous improvement of healthcare systems and the expansion of health insurance funds, fraudulent activities have become increasingly rampant. These activities not only cause significant economic losses but also jeopardize the sustainability of national insurance programs. Traditional rule-based or supervised learning-based methods face three major challenges: Fraud patterns are evolving rapidly, and the cost of maintaining manual rules is high; The positive and negative samples are extremely imbalanced, and labeled data is scarce. Medical data is characterized by high dimensionality, sparsity, and strong temporal dependence, making it difficult for traditional models to capture long-term dependencies across the entire medical cycle. Summary of the Invention
[0003] According to a first aspect of the present invention, a method for detecting medical insurance fraud based on autoregressive Transformer is provided, comprising the following steps: Obtain the original medical insurance prescription data, and arrange the medical insurance settlement items of the patient's multiple visits in chronological order according to different medical insurance accounts to form the original treatment item sequence for each account; The original medical records for each account are truncated or padded, and all original medical records are processed into standard sequences of the same length. A pre-trained medical text encoding model is used to perform semantic vector encoding on the name text of each standard sequence to obtain the item semantic vector sequence and construct the item embedding matrix; The input sequence is obtained by aligning the sequence of semantic vectors of items to the hidden dimensions of the Transformer encoder through a trainable linear mapping; The input sequence is fed into a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder adds a learnable relative position bias when calculating the attention score to capture the relative time dependency between medical insurance settlement items in the input sequence. For each time position output by the Transformer encoder, a linear prediction layer is used to generate a conditional probability distribution for the next medical insurance settlement item, and the corresponding negative log-likelihood prediction error is calculated. Standardize all prediction errors within each account, select the top preset values with the largest standardized errors and calculate their average, which will be used as the abnormal score for that account. All accounts to be detected are sorted according to their account-level anomaly scores, and the list of accounts with the highest anomaly scores is output as high-risk fraud suspect accounts.
[0004] Furthermore, the pre-trained medical text encoding model is a PULSE-7bv5 model trained on a large-scale medical corpus, and the semantic vector of each medical insurance settlement item is obtained by mean pooling the last hidden state of the PULSE-7bv5 model.
[0005] Furthermore, each attention head of the Transformer encoder incorporates a trainable scalar bias related to the relative distance when calculating the attention score between the query position and the key position, in order to capture the influence of historical behavior at different time intervals.
[0006] Furthermore, the linear mapping used for alignment dimensions includes a trainable weight matrix and biases, thereby transforming the item semantic vector into a vector consistent with the dimensions of the Transformer hidden space.
[0007] Furthermore, when training the Transformer encoder, an autoregressive objective and negative log-likelihood loss are used. After several rounds of initial training, based on the anomaly scores calculated by the current model on the training data, the account samples with the highest proportion of anomaly scores are removed from the training set, and training continues to reduce the interference of anomaly samples.
[0008] Furthermore, the calculation of the account anomaly score is as follows: after standardizing the prediction error of the account at all time steps using Z-score, the largest preset error value is taken, and the average of these values is used as the final anomaly score.
[0009] According to a second aspect of the present invention, a medical insurance fraud detection system based on autoregressive Transformer is provided, comprising: The sequence construction module is used to obtain the original medical insurance prescription data and arrange the medical insurance settlement items of the patient's multiple visits in chronological order according to different medical insurance accounts, forming the original diagnosis and treatment item sequence for each account; The sequence processing module is used to truncate or pad the original medical records of each account, and process all the original medical records into a standard sequence of the same length. The vector encoding module is used to perform semantic vector encoding on the name text of each standard sequence using a pre-trained medical text encoding model, to obtain the item semantic vector sequence and construct the item embedding matrix; The mapping module is used to align the sequence of item semantic vectors to the hidden dimensions of the Transformer encoder through a trainable linear mapping to obtain the input sequence; The computational capture module is used to input the input sequence into a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder incorporates a learnable relative position bias when calculating the attention score to capture the relative temporal dependencies between medical insurance settlement items in the input sequence. The error calculation module is used to generate a conditional probability distribution for the next medical insurance settlement item for each time position output by the Transformer encoder using a linear prediction layer, and to calculate the corresponding negative log-likelihood prediction error. The anomaly scoring module is used to standardize all prediction errors within each account, select the top preset values with the largest standardized errors and calculate the average, which is used as the anomaly score for that account. The results output module is used to sort all the accounts to be detected according to the account-level anomaly score and output the list of accounts with the highest anomaly score as high-risk fraud suspected accounts.
[0010] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes the computer program to perform a medical insurance fraud detection method based on autoregressive Transformer according to the first aspect.
[0011] According to a fourth aspect of the present invention, a readable medium having processor-executable non-volatile program code is provided, the program code causing the processor to run a medical insurance fraud detection method based on an autoregressive Transformer according to the first aspect.
[0012] A method for detecting medical insurance fraud based on autoregressive Transformer according to an embodiment of the present invention has the following beneficial effects: This invention abandons traditional one-hot encoding or random initialization methods, and introduces a text encoding model (such as PULSE-7bv5) pre-trained on medical corpora to generate semantic embeddings for medical insurance items. This approach not only integrates rich medical semantic information such as generic drug names, dosage forms, and treatment categories into the model's prior knowledge, enabling the model to understand higher-order fraud patterns such as "semantic similar item substitution" or "abnormal combinations of similar items," but also avoids the inefficiency of downstream tasks learning semantic relationships from scratch. More importantly, the offline pre-computation and freezing strategy significantly reduces training and inference costs, ensures the stability of the semantic space, enhances the model's ability to represent unseen items, and thus improves overall generalization performance.
[0013] To address the characteristic of medical insurance behavior where "location itself is not important, but the relative distance between actions is more important," this invention employs a multi-head self-attention encoder with learnable relative positional bias. Compared to traditional absolute positional encoding, this design can explicitly model the relative sequence and distance between actions, such as the frequency of repeated medical visits in the short term or fixed combinations over a specific time span. The multi-head mechanism allows the model to simultaneously focus on dependencies at different scales (such as high-frequency repetition and long-term cycles), thereby accurately characterizing the inherent rhythm of medical insurance behavior and providing strong support for identifying structural fraud that deviates from the normal "medical visit-settlement" time pattern.
[0014] Considering that real medical insurance data inevitably contains anomalous or fraudulent samples, this invention designs a self-cleaning training mechanism. By iteratively identifying and filtering out samples with high anomalous scores during training, the training distribution gradually converges towards a pure distribution of normal behavior. This mechanism effectively prevents overfitting caused by the model being forced to "interpret" anomalous transition patterns during training, ensuring that the autoregressive prediction patterns learned by the model can truly reflect the statistical characteristics of normal medical treatment behavior. This significantly improves the stability and accuracy of anomaly detection in weakly supervised or even unsupervised scenarios.
[0015] This invention addresses the issue of vastly different account behavior complexity by employing Z-score standardization to process prediction errors, eliminating misjudgments caused by individual scale differences and allowing the model to focus more on relatively abnormal behaviors within a sequence. Furthermore, considering that fraudulent behavior is often driven by a few key operations, this invention uniquely uses a Top-k aggregation strategy to construct account-level anomaly scores, rather than simple error averaging. This strategy effectively avoids the dilution effect of numerous normal behaviors on anomalous signals, ensuring that the model can focus on the most anomalous segments in the sequence (such as repeated prescriptions or co-occurrence of high-risk items within a short period), thereby significantly improving the recall and accuracy of anomaly detection and truly achieving "point-to-surface" identification of risky accounts.
[0016] This invention employs a fixed-length sequence modeling strategy. By truncating and padding sequences to a uniform length, it solves the problem of large differences in sequence length and long-tailed distribution in the original data while ensuring the maximum receptive field of the model remains controllable. This design not only reduces the complexity of model implementation and ensures the efficiency and feasibility of batch training, but also lays a solid foundation for subsequent Transformer-based modeling, making the model applicable to large-scale, high-concurrency medical insurance fund supervision scenarios.
[0017] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description
[0018] Figure 1 This is a flowchart of a medical insurance fraud detection method based on autoregressive Transformer according to an embodiment of the present invention.
[0019] Figure 2 This is a diagram of the autoregressive model architecture in a medical insurance fraud detection method based on autoregressive Transformer according to an embodiment of the present invention.
[0020] Figure 3 This is a structural diagram of a medical insurance fraud detection system based on autoregressive Transformer according to an embodiment of the present invention.
[0021] Figure 4 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0022] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, further illustrating the present invention.
[0023] like Figure 2 As shown, this application proposes a Transformer-based temporal detection model for medical insurance fraud. For the task of identifying structural anomalies in medical insurance behavior sequences, the overall model architecture follows an encoder-based autoregressive prediction paradigm, and can be divided into four main parts: an input layer, an embedding layer, an encoder layer, and a prediction layer. The input layer receives the original medical insurance settlement sequence (including drugs, treatment items, etc.) and converts it into a uniform length format that the model can process. The embedding layer first maps discrete medical insurance items into semantically rich continuous vectors using a pre-trained medical text encoder, and then aligns them to the model's hidden dimensions through linear projection, laying a semantic foundation for subsequent analysis. The encoder layer is the core of the model, consisting of multiple stacked Transformer encoder blocks. To accurately capture the temporal rhythm of medical behavior, we introduce a learnable positional bias based on relative distance in each multi-head self-attention module, enabling the model to distinguish different temporal patterns such as short-term repetition and long-term dependence. Finally, the prediction layer maps the context representation output by the encoder to the entire medical insurance item space through a linear transformation and uses an autoregressive approach to predict the settlement item at the next time step. By comparing the error between predicted probabilities and actual items, the model can accurately identify anomalous patterns that deviate from normal behavioral paradigms. This end-to-end architecture enables the model to effectively learn the complex mapping from medical insurance behavior sequences to the degree of anomalousness without the need for manual feature engineering.
[0024] The following will combine Figures 1-2 This paper describes a medical insurance fraud detection method based on autoregressive Transformer according to an embodiment of the present invention. It is used for medical insurance fraud detection and has a wide range of applications.
[0025] like Figures 1-2 As shown in the figure, an embodiment of the present invention provides a method for detecting medical insurance fraud based on autoregressive Transformer, which includes the following steps: like Figure 1 As shown, in S1, the original medical insurance prescription data is obtained, and the medical insurance settlement items of the patient's multiple visits are arranged in chronological order according to different medical insurance accounts to form the original treatment item sequence of each account.
[0026] It should be noted that step S1 is the action to be performed by the input layer of the model, responsible for receiving the original medical insurance settlement sequence (including drugs, treatment items, etc.) and converting it into a uniform length format that the model can process. In the scenario of medical insurance fund supervision, abnormal and fraudulent behaviors are often not triggered by a single violation, but rather manifest as long-term, repetitive, and structured patterns of medical treatment and settlement behavior. These behaviors may not be significant at the level of a single record, but they often exhibit highly identifiable sequence structure features over time. Therefore, this paper models medical insurance fraud detection as a sequence-based anomaly detection problem, identifying abnormal accounts by characterizing the predictability and consistency of medical insurance account behavior sequences. Let... This represents a collection of medical insurance accounts (insured individuals or medical insurance cards). This represents the complete set of medical insurance settlement items (medicines, medical treatment items, etc.). For any given account... Their medical treatment and settlement behavior is represented as a time-ordered sequence of items. ,in Indicates account In the Items involved in the next medical visit or settlement The original sequence length for this account.
[0027] like Figure 1 As shown, in S2, the original medical item sequences of each account are truncated or padded, and all the original medical item sequences are uniformly processed into standard sequences of the same length.
[0028] It should be noted that in actual data, the length of medical records varies greatly among different accounts. There are accounts that use medical insurance frequently and for a long time, as well as accounts that use it only occasionally, exhibiting a clear long-tail distribution. Modeling directly on variable-length sequences would significantly increase the complexity of model implementation and training. Therefore, this paper adopts a fixed-length sequence modeling strategy, unifying all sequences to a uniform length through truncation and padding. The truncation operation limits the model's maximum receptive field, avoiding the computational overhead of extremely long sequences, while the padding operation ensures the feasibility of batch training. This results in sequences of uniform length. .
[0029] like Figure 1As shown, in S3, a pre-trained medical text encoding model is used to encode the name text of each standard sequence into semantic vectors, resulting in an item semantic vector sequence and constructing an item embedding matrix.
[0030] It should be noted that step S3 is the action to be performed by the model's embedding layer. The embedding layer first maps discrete medical insurance items into continuous vectors rich in semantic information using a pre-trained medical text encoder, and then aligns them to the model's hidden dimensions through linear projection, laying a semantic foundation for subsequent analysis. Medical insurance items are essentially discrete symbols, but their names contain rich medical semantic information, such as generic drug names, dosage forms, and treatment categories. Many abnormal behaviors are not reflected in a specific item itself, but in the replacement, combination, or abnormal co-occurrence of semantically similar items. If only random initialization or one-hot encoding is used, the model needs to learn the semantic relationships between items from scratch in downstream tasks, which is not only inefficient in training but also detrimental to generalization. Therefore, this paper introduces a text encoding model PULSE-7bv5 pre-trained based on medical corpus to provide semantic-level prior representations for medical insurance items.
[0031] set up Define a text space for medical insurance item names and define a mapping function. Each project The mapping is to its name text. Let the text encoder be... For each project Its semantic vector is calculated in the offline stage. ,in In the implementation, the project name text is input into PULSE-7bv5 to obtain the last hidden state, and mean pooling is used to obtain a fixed-dimensional representation. Since each project can be represented by text, there is no situation where a project cannot be embedded.
[0032] Stack all item vectors into an embedding matrix: The matrix is calculated and persistently saved during the preprocessing stage. During the training and inference of the downstream anomaly detection model, PULSE-7bv5 is no longer invoked; instead, item embeddings are directly obtained via table lookup. The advantage of employing an offline pre-computation and freezing strategy is that it significantly reduces training and inference costs while maintaining stable priors in the item semantic space, allowing the model to focus on learning sequence structure anomalies.
[0033] like Figure 1 As shown, in S4, the sequence of item semantic vectors is aligned to the hidden dimension of the Transformer encoder through a trainable linear mapping to obtain the input sequence.
[0034] It should be noted that step S4 is the action to be performed by the encoder layer of the model. The encoder layer is the core of the model and consists of multiple stacked Transformer encoder blocks. To accurately capture the temporal rhythm of medical behavior, we introduced a learnable positional bias based on relative distance in each multi-head self-attention module, enabling the model to distinguish different temporal patterns such as short-term repetition and long-term dependence.
[0035] Due to the pre-trained embedding dimension Hidden Dimensions of Self-Attention Encoders To address the inconsistency, this paper uses a trainable linear mapping for dimension alignment: ; in , This leads to the account. sequence representation .
[0036] Absolute positional encoding assumes that the t-th position in a sequence has a fixed semantic meaning. However, in medical insurance behavior sequences, the position itself is not important; what matters is the relative distance between behaviors. For example, a short interval between two visits may indicate repeated visits, and the repeated occurrence of item combinations in adjacent time steps may suggest abnormal patterns. Relative positional encoding can explicitly model relative sequence relationships without relying on absolute time indices, thereby improving the model's sensitivity to behavioral rhythms.
[0037] like Figure 1 As shown, in S5, the input sequence is input to a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder adds a learnable relative position bias when calculating the attention score to capture the relative time dependency between medical insurance settlement items in the input sequence.
[0038] It should be noted that the number of attention heads is . The single-head dimension is For the query location AND key position Define relative distance We retain the signed relative distances. Instead of taking the absolute value, we use the directionality of the time dependency in the autoregressive modeling setting, because the directionality of the time dependency is crucial. Based on this, we employ a learnable relative position bias to modulate the attention weights. Specifically, in the... In the attention of the head, the position Position Attention score is defined as: in These represent the query and key representations for this header, respectively. It's about relative distance. The learnable scalar bias explicitly characterizes the different impacts of more recent and more distant historical behaviors on current predictions. This design is particularly suitable for healthcare behavior sequences, where anomalous patterns often manifest as short-term repetition, nearest-neighbor co-occurrence, or fixed combinations within a specific time span. The relative position bias can capture these patterns without relying on absolute position numbers. The two types of masks, along with the relative position bias, work together to ensure that the model allocates attention weights only to valid historical positions and distinguishes the time spans between different behaviors.
[0039] We score attention Perform softmax normalization to obtain attention weights : The summation is performed only at historical positions that are not masked, thus depicting the current position. When making predictions, different historical locations The importance of the current decision. This leads to the attention weight matrix for the entire sequence. Midterm .
[0040] And order Indicates the first The value vector matrix of the n attention head over the entire sequence, then the nth attention head... The output of each attention head over the entire sequence is represented as: in Subsequently, we concatenate the outputs of all attention heads aligned in the sequence dimension along the feature dimension, and then map them back to the hidden space through a linear transformation to obtain the overall output of the self-attention sublayer: in , For learnable parameters, This represents a vector concatenation operation. Through a multi-head mechanism, different attention heads can focus on temporal dependency patterns at different scales on the same sequence: some attention heads focus on characterizing high-frequency repetition and nearest-neighbor co-occurrence behaviors in the short term, while others tend to capture stable dependencies across longer time spans. This multi-scale modeling capability is particularly important for medical insurance behavior sequences, because anomalous patterns may manifest as abnormally dense operations in a short period of time, or as long-term, periodic but structurally anomalous combinations of behaviors.
[0041] Following the self-attention sublayer, we introduce a position-wise feedforward network to enhance the model's nonlinear expressive power. The FFN acts independently and with shared parameters at each position in the sequence, and therefore can be written as an expression for the sequence representation matrix. Line-by-line transformation: in , It is a linear transformation matrix. As the intermediate dimension of the feedforward layer, , For bias terms, This represents an element-wise nonlinear activation function.
[0042] To improve the training stability of deep networks and alleviate the vanishing gradient problem, we employ residual connections and layer normalization operations in both the self-attention sub-layer and the feedforward sub-layer. Let the th... The input of the layer is represented as Then the self-attention sublayer and the feedforward sublayer are represented as follows: ; Layer normalization stabilizes the representation distribution across different accounts and time steps, ensuring good numerical stability even when handling medical insurance data with varying sequence lengths. Through the aforementioned relative position-aware multi-head self-attention and position-by-position feedforward modeling structure, the model can perform refined modeling of temporal dependencies in medical insurance behavior sequences without relying on absolute timestamps or location numbers. This provides a structured, temporally semantic sequence representation for subsequent anomaly detection based on prediction errors.
[0043] like Figure 1 As shown, in S6, for each time position output by the Transformer encoder, a linear prediction layer is used to generate a conditional probability distribution for the next medical insurance settlement item, and the corresponding negative log-likelihood prediction error is calculated.
[0044] It should be noted that, regarding the account sequence After project embedding mapping, linear projection, and a multi-layer relative position-aware Transformer encoder, the final hidden state matrix is obtained. ,in The length of the unified sequence. For the hidden dimensions of the model, Indicates account In the Hidden representations at various time points. Based on the encoder structure described above, this paper uses an autoregressive prediction objective to train the model, in order to learn the inherent generation rules of medical insurance account behavior sequences. Intuitively, if the behavior pattern of an account conforms to the statistical regularity of a large number of normal accounts in history, the model should be able to predict its subsequent items relatively accurately given historical items; conversely, if there are abnormal combinations or atypical transitions in the sequence, the prediction error will increase significantly.
[0045] In the context of medical insurance fund supervision, abnormal behavior often manifests as a shift in the sequence of normal medical treatment and settlement behavior. Therefore, this paper adopts an autoregressive modeling hypothesis to learn the behavior generation mechanism of normal accounts by predicting the next settlement item. Under this hypothesis, the account... The joint probability of the behavioral sequence can be decomposed into a product of a series of conditional probabilities: in Indicates account In position All previous historical projects. This decomposition method has two implications: at the modeling level, it transforms the complex problem of sequence distribution learning into multiple conditional prediction tasks, making it easier to utilize the contextual modeling capabilities of Transformer; at the anomaly detection level, if the probability of a project transfer is extremely low under given historical conditions, then the behavior is more likely to belong to an abnormal or atypical pattern.
[0046] To achieve the aforementioned autoregressive prediction objective, this paper introduces a linear prediction layer based on the Transformer encoder output, mapping the hidden representation to a conditional probability distribution for the next item. Specifically, for position... Hidden representation The conditional probability distribution for the next item is obtained through the softmax function. : ; in This is the prediction layer weight matrix. For bias terms, This refers to the scale of the complete set of medical insurance settlement items. Indicates in a given historical project Under these conditions, the next project is The predicted probability. This prediction process allows the model to explicitly learn, at each time point, what settlement behaviors a normal account is more likely to engage in in the current context.
[0047] The model training uses negative log-likelihood (NLL) as the optimization objective and computes the loss only at the real item locations to avoid noise introduced by the padding operation. The specific loss function is defined as follows: in For sequence mask, Represents a real project. Indicates the fill position. Only when At time, position Only the prediction error is included in the loss. This training objective prompts the model to learn stable item transfer patterns on a large number of account samples, thereby forming a statistical characterization of normal medical insurance behavior sequences.
[0048] Real-world medical insurance data often contains a certain proportion of abnormal / fraudulent accounts. If an autoregressive model is trained directly on the full dataset, the model may be forced to "interpret" abnormal transfer patterns, thus weakening its ability to characterize normal behavior. To address this, this paper introduces a self-cleaning training mechanism: the model first undergoes regular training on the full training set; when the training epoch reaches Estart and the validation set metric shows no improvement within P consecutive epochs, the current model is used to calculate the abnormality score f(su) on the training samples, and the top γ samples with the highest scores are filtered out according to a cleaning ratio γ, resulting in a "cleaner" training subset for further training. This process can be iteratively executed, gradually converging the training distribution towards the normal behavior distribution, thereby suppressing overfitting of abnormal patterns and improving the stability and discriminative power of anomaly detection in weakly supervised / unsupervised scenarios.
[0049] like Figure 1 As shown, in S7, all prediction errors within each account are standardized, and the top preset values with the largest standardized errors are selected and averaged to serve as the abnormal score for that account.
[0050] It should be noted that: regarding the account In position The prediction result is defined as follows: in Indicates the model at position For the real next project The predicted probability. This error term has an intuitive interpretation: if a settlement item is highly predictable under given historical behavior, then its corresponding... The prediction error is relatively small. Conversely, if the probability of this behavior occurring in the normal pattern is extremely low, the prediction error increases significantly, suggesting that it may deviate from the usual medical treatment and settlement path. In actual medical insurance data, different accounts show significant differences in medical treatment frequency, item diversity, and behavioral complexity. High-frequency accounts often involve more complex item combinations, and their overall prediction error level may be systematically higher than that of low-frequency accounts; if the original errors are directly compared, complex but normal accounts are easily misjudged as abnormal. To eliminate this individual-scale difference, this paper performs Z-score normalization on the prediction error within the sequence of a single account. This aligns the error distributions of different accounts on a comparable scale, thus focusing more on relative anomalous behavior within a sequence rather than overall differences between accounts.
[0051] Healthcare fraud typically doesn't occur throughout the entire medical process, but is driven by a few key anomalous actions or combinations. Examples include repeated prescriptions within a short period, unusual co-occurrence of specific items, or high-risk behaviors concentrated in a particular phase. If the error across the entire sequence is simply averaged, these key anomalous signals are easily diluted by a large number of normal behaviors. Based on these considerations, this paper employs a Top-k aggregation strategy, constructing account-level anomalous scores by emphasizing several time steps with the largest prediction errors. The core idea of this strategy is that whether an account is anomalous is often determined by its most anomalous behavioral segment. Let... The proportion of prior anomalies, For account The set of items with the largest prediction errors, accounts The sequence-level anomaly score is defined as: ; This anomaly scoring function will evaluate variable-length behavioral sequences. Mapped to a single real number Furthermore, the higher the score, the more abnormal the account behavior.
[0052] like Figure 1 As shown, in S8, all accounts to be detected are sorted according to the account-level anomaly score, and the list of accounts with the highest anomaly score is output as high-risk fraud suspected accounts.
[0053] Experimental Example: Dataset: Validated on millions of real medical insurance reimbursement records, length of each sequence. .
[0054] Sequence-level metrics: 1. Recall@K (Recall rate of the top K fraudulent sequences): The measure is how many outlier sequences the model can correctly identify in the top K positions across all predictions.
[0055] In healthcare fraud detection, a high Recall@K value means that the model can more effectively identify potential fraudulent behavior.
[0056] 2.AUC (Area Under Curve): The area under the ROC curve measures a model's ability to distinguish between normal and abnormal sequences.
[0057] The closer the AUC value is to 1, the stronger the model's discriminative ability.
[0058] 3. PR-AUC (Area under Precision-Recall Curve): The area under the precision-recall curve is particularly useful in situations where positive and negative samples are imbalanced.
[0059] A high PR-AUC value indicates that the model can maintain both high recall and high precision.
[0060] Decision-level indicators: 1. Precision: The proportion of sequences that are actually abnormal out of all those predicted as abnormal.
[0061] High accuracy means fewer false positives (incorrectly labeling normal sequences as anomalous).
[0062] 2. Recall: The proportion of actual abnormal sequences that are correctly identified by the model.
[0063] A high recall rate means fewer false negatives (failure to identify actual anomalous sequences).
[0064] 3. F1 Score: The harmonic mean of precision and recall takes into account both precision and recall.
[0065] A higher F1 score indicates better overall model performance.
[0066] The results of the Transformer anomaly detection model under multiple different hyperparameter configurations are shown in the table below: By changing the model's hyperparameters (such as the number of heads and layers), we analyzed the impact of these parameters on model performance. Experimental results show that: The optimal configuration of 8 heads and 4 layers indicates that medical sequence dependency patterns are diverse (requiring multi-head capture), but too many layers can easily lead to overfitting (data sparsity).
[0067] Recall@10: Reflects the model's ability to recall anomalous sequences in the first 10 predictions; the 8-head, 4-layer configuration performs best.
[0068] Ndcg@10: Reflects the model's ability to rank outlier sequences in the first 10 predictions; the 8-head, 4-layer configuration also performs optimally.
[0069] AUC and PR-AUC: These measure the model's ability to distinguish between normal and abnormal sequences. The 8-head, 4-layer configuration has the highest AUC and PR-AUC values, indicating that it performs best in distinguishing between normal and abnormal sequences.
[0070] Precision and Recall: These reflect the model's precision and recall at the Top-20% threshold. The 8-head, 4-layer configuration performs best, indicating that it accurately identifies anomalous sequences while also achieving the highest recall. The 8-head, 4-layer configuration is optimal, suggesting that medical sequence dependencies are diverse (requiring multi-head capture), but too many layers can easily lead to overfitting (data sparsity).
[0071] The above combined with the appendix Figures 1-2 A method for detecting medical insurance fraud based on an autoregressive Transformer according to an embodiment of the present invention is described. Furthermore, the present invention can also be applied to a medical insurance fraud detection system based on an autoregressive Transformer.
[0072] like Figure 3 As shown, according to a second aspect of the present invention, a medical insurance fraud detection system based on autoregressive Transformer is provided, comprising: The sequence construction module 100 is used to obtain the original medical insurance prescription data and arrange the medical insurance settlement items of the patient's multiple visits in chronological order according to different medical insurance accounts to form the original diagnosis and treatment item sequence of each account. The sequence processing module 200 is used to truncate or pad the original medical item sequences of each account, and process all the original medical item sequences into standard sequences of the same length. Vector encoding module 300 is used to perform semantic vector encoding on the name text of each standard sequence using a pre-trained medical text encoding model, to obtain the item semantic vector sequence and construct the item embedding matrix; The mapping module 400 is used to align the sequence of item semantic vectors to the hidden dimensions of the Transformer encoder through a trainable linear mapping to obtain the input sequence; The computational capture module 500 is used to input the input sequence into a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder incorporates a learnable relative position bias when calculating the attention score to capture the relative temporal dependencies between medical insurance settlement items in the input sequence. The error calculation module 600 is used to generate a conditional probability distribution for the next medical insurance settlement item using a linear prediction layer for each time position output by the Transformer encoder, and to calculate the corresponding negative log-likelihood prediction error. The anomaly scoring module 700 is used to standardize all prediction errors in each account, select the top preset values with the largest standardized errors and calculate the average, which is used as the anomaly score for that account. The output module 800 is used to sort all the accounts to be detected according to the account-level anomaly score and output the list of accounts with the highest anomaly score as high-risk fraud suspected accounts.
[0073] The above combined with the appendix Figure 3 This paper describes a detection system corresponding to a medical insurance fraud detection method based on an autoregressive Transformer according to an embodiment of the present invention. Therefore, the details are not elaborated further, as they will be fully clear to those skilled in the art. Furthermore, the present invention can also be applied to an electronic device.
[0074] like Figure 4 As shown, according to a third aspect of the present invention, an electronic device is provided, comprising: a memory 1, a processor 2, and a computer program 3, wherein the computer program 3 is stored in the memory 1, and the processor 2 executes the computer program 3 to perform a medical insurance fraud detection method based on autoregressive Transformer according to the first aspect.
[0075] The above combined with the appendix Figure 4 An electronic device according to an embodiment of the present invention is described. Furthermore, the invention can also be applied to a readable medium having processor-executable non-volatile program code.
[0076] According to a fourth aspect of the present invention, a readable medium having processor-executable non-volatile program code is provided, the program code causing the processor to run a medical insurance fraud detection method based on an autoregressive Transformer according to the first aspect.
[0077] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of a computer program from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the ASIC can reside within a device. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. The present invention also provides a program product comprising executable instructions stored in the readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions, causing the device to implement the autoregressive Transformer-based medical insurance fraud detection method provided in the various embodiments described above. In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0078] It should be noted that, in this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0079] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.
Claims
1. A method for detecting medical insurance fraud based on autoregressive Transformer, characterized in that, It includes the following steps: Obtain the original medical insurance prescription data, and arrange the medical insurance settlement items of the patient's multiple visits in chronological order according to different medical insurance accounts to form the original treatment item sequence for each account; The original medical records for each account are truncated or padded, and all original medical records are processed into standard sequences of the same length. A pre-trained medical text encoding model is used to perform semantic vector encoding on the name text of each standard sequence to obtain the item semantic vector sequence and construct the item embedding matrix; The input sequence is obtained by aligning the sequence of semantic vectors of items to the hidden dimensions of the Transformer encoder through a trainable linear mapping; The input sequence is fed into a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder adds a learnable relative position bias when calculating the attention score to capture the relative time dependency between medical insurance settlement items in the input sequence. For each time position output by the Transformer encoder, a linear prediction layer is used to generate a conditional probability distribution for the next medical insurance settlement item, and the corresponding negative log-likelihood prediction error is calculated. Standardize all prediction errors within each account, select the top preset values with the largest standardized errors and calculate their average, which will be used as the abnormal score for that account. All accounts to be detected are sorted according to their account-level anomaly scores, and the list of accounts with the highest anomaly scores is output as high-risk fraud suspect accounts.
2. The medical insurance fraud detection method based on autoregressive Transformer as described in claim 1, characterized in that, The pre-trained medical text encoding model is a PULSE-7bv5 model trained on a large-scale medical corpus. The semantic vector of each medical insurance settlement item is obtained by mean pooling the last hidden state of the PULSE-7bv5 model.
3. The medical insurance fraud detection method based on autoregressive Transformer as described in claim 1, characterized in that, Each attention head of the Transformer encoder incorporates a trainable scalar bias related to the relative distance when calculating the attention score between the query position and the key position, in order to capture the influence of historical behavior at different time intervals.
4. The medical insurance fraud detection method based on autoregressive Transformer as described in claim 1, characterized in that, The linear mapping used for alignment dimensions includes a trainable weight matrix and biases, which transforms the item semantic vector into a vector consistent with the dimensions of the Transformer hidden space.
5. The medical insurance fraud detection method based on autoregressive Transformer as described in claim 1, characterized in that, When training the Transformer encoder, an autoregressive objective and negative log-likelihood loss are used. After several rounds of initial training, based on the anomaly scores calculated by the current model on the training data, the account samples with the highest proportion of anomaly scores are removed from the training set, and training continues to reduce the interference of anomaly samples.
6. The medical insurance fraud detection method based on autoregressive Transformer as described in claim 1, characterized in that, The calculation of the account anomaly score is as follows: After standardizing the prediction error of the account at all time steps using Z-score, the largest preset error value is taken, and the average of these values is used as the final anomaly score.
7. A medical insurance fraud detection system based on autoregressive Transformer, characterized in that, Include: The sequence construction module is used to obtain the original medical insurance prescription data and arrange the medical insurance settlement items of the patient's multiple visits in chronological order according to different medical insurance accounts, forming the original diagnosis and treatment item sequence for each account; The sequence processing module is used to truncate or pad the original medical records of each account, and process all the original medical records into a standard sequence of the same length. The vector encoding module is used to perform semantic vector encoding on the name text of each standard sequence using a pre-trained medical text encoding model, to obtain the item semantic vector sequence and construct the item embedding matrix; The mapping module is used to align the sequence of item semantic vectors to the hidden dimensions of the Transformer encoder through a trainable linear mapping to obtain the input sequence; The computational capture module is used to input the input sequence into a multi-layer Transformer encoder for encoding. The multi-head self-attention layer of the Transformer encoder adds a learnable relative position bias when calculating the attention score to capture the relative time dependency between medical insurance settlement items in the input sequence. The error calculation module is used to generate a conditional probability distribution for the next medical insurance settlement item for each time position output by the Transformer encoder using a linear prediction layer, and calculate the corresponding negative log-likelihood prediction error. The anomaly scoring module is used to standardize all prediction errors within each account, select the top preset values with the largest standardized errors and calculate the average, which is used as the anomaly score for that account. The results output module is used to sort all the accounts to be detected according to the account-level anomaly score and output the list of accounts with the highest anomaly score as high-risk fraud suspected accounts.
8. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program, wherein the computer program is stored in the memory and the processor executes the computer program to perform a medical insurance fraud detection method based on autoregressive Transformer as described in any one of claims 1 to 6.
9. A readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to run the medical insurance fraud detection method based on autoregressive Transformer as described in any one of claims 1-6.