Insurance claim settlement automatic decision optimization method based on reinforcement learning

Abstract

The invention discloses an insurance claim settlement automatic decision optimization method based on reinforcement learning, and the method comprises the following steps: S1, collecting insurance claim settlement data, and carrying out the data preprocessing; s2, defining a state space and an action space, and constructing a reward function for claim strategy training; s3, an EfficientZero reinforcement learning model is constructed, and the EfficientZero reinforcement learning model is constructed; s4, performing off-line training on the OfficientZero reinforcement learning model; s5, deploying the trained model to an insurance claim settlement service system; and S6, constructing an incremental claim settlement sample set, and carrying out periodic parameter updating on the model by adopting an incremental learning mechanism. According to the method, reinforcement learning and claim settlement scene modeling are fused, automatic generation and continuous optimization of the claim settlement strategy are realized, and the method has the advantages of high accuracy, strong adaptability, good deployment and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and particularly relates to an insurance claim settlement automatic decision optimization method based on reinforcement learning. BACKGROUND

[0002] With the acceleration of the digital transformation of insurance business, intelligent claim settlement has become a key direction for improving service efficiency and reducing operating costs in property insurance, personal insurance, and car insurance. In recent years, various insurance companies and technology platforms have actively explored artificial intelligence-based automated claim settlement solutions, trying to improve the automatic identification, rule matching, and case processing capabilities of claim settlement cases through natural language processing, image recognition, expert systems, and process engines. These systems usually rely on artificially designed rule templates, historical claim case samples, and logic tree structures to quickly handle and pay claims for standardized cases, thereby achieving preliminary intelligent process transformation.

[0003] Most existing insurance claim settlement automation solutions are based on a "rule-driven" decision-making model, which involves inputting insurance contract terms, customer information, and accident materials for structured analysis through pre-set claim settlement judgment conditions and processing logic, and then classifying and processing cases through a series of conditional judgment logic. For example, if the insured person's identity is correct, the accident type is within the scope of insurance liability, and the materials are complete, the case will enter the automatic payment process. If there is missing information or the risk label is too high, the case will be transferred to manual review. This approach has some effect in structured and logical basic claim settlement scenarios, but when faced with complex claim settlement cases with high dimensions, unstructured, and multi-variable coupling, it often has poor generalization ability, low rule adaptability, and unclear boundaries.

[0004] To improve the system's processing capability in complex scenarios, some research has begun to introduce machine learning methods, especially supervised learning techniques such as support vector machines, random forests, and XGBoost, for risk prediction, fraud identification, and claim settlement classification. This approach usually involves feature extraction and label modeling on historical claim settlement data to train a classifier model to predict payment behavior. However, supervised learning methods are highly dependent on labels and have difficulty handling fuzzy labels and dynamic strategy changes. Moreover, their prediction results are often single-step static judgments, lacking the closed-loop capability of "action-feedback-adjustment", making them unsuitable for multi-stage decision-making scenarios and difficult to achieve continuous self-optimization of automatic claim settlement strategies.

[0005] In recent years, deep reinforcement learning has shown strong modeling and policy learning capabilities in complex decision-making problems such as game agents, robot control, and autonomous driving. Reinforcement learning interacts with the environment, selects actions in the state space, and obtains rewards, thereby optimizing long-term cumulative returns, and has strong sequence decision-making and self-learning capabilities. In particular, deep reinforcement learning models such as DQN, PPO, and EfficientZero have shown high policy optimization efficiency in unknown state spaces. Among them, EfficientZero is a reinforcement learning framework that combines model learning and policy learning, and through the collaborative optimization of environment modeling networks and prediction networks, it has high sample efficiency and high convergence performance, and has achieved good results in some game and general control scenarios.

[0006] However, applying reinforcement learning to insurance claim automatic decision-making still faces many challenges. First, insurance claim data has the characteristics of multi-source heterogeneity, complex structure, and multi-dimensional semantics. How to construct a reasonable state space so that the model can fully capture key variables such as customer information, contract content, accident scenario, and material completeness is a fundamental difficulty. Second, the claim action space has high risk properties. Payout, partial payout, refusal, and manual review involve multiple factors such as law, compliance, and customer experience in actual application. Therefore, the reinforcement learning model not only needs to focus on model convergence performance, but also needs to model payout reasonableness, fraud probability, and operating costs into the reward function. Third, the claim scenario has the business characteristics of high frequency and small sample updates. The model needs to support real-time inference and have continuous learning ability to adapt to changes in claim types and rule evolution, avoiding the high cost of large-scale retraining of traditional RL models. In addition, existing reinforcement learning solutions generally lack explainability and business collaboration mechanisms, and the output strategy is often a "black box decision", which is not conducive to business personnel understanding and acceptance, and seriously restricts its landing and deployment.

[0007] Based on the above analysis, there is currently a lack of a reinforcement learning optimization method that fully combines insurance claim business semantics, multi-factor decision logic, state-action modeling mechanism, and efficient model structure to achieve end-to-end automatic claim settlement strategy generation and adaptive update. Existing solutions either rely on rule templates and are difficult to adapt to complex scenarios, or use static models and cannot perform closed-loop feedback and continuous optimization, or have insufficient structural generalization and lack dynamic perception and processing of business core fields. SUMMARY

[0008] One purpose of the present application is to provide an insurance claim automatic decision optimization method based on reinforcement learning, which integrates reinforcement learning algorithm and insurance claim business characteristics, constructs an intelligent decision model based on EfficientZero, realizes the automatic generation and continuous optimization of claim settlement strategy, has the advantages of high accuracy, high adaptability and strong deployability, can effectively improve the claim processing efficiency, reduce the operation cost and enhance the risk identification ability of the system, is suitable for various insurance claim scenarios, and has wide application value in the fields of intelligent claim settlement and insurance technology.

[0009] According to an embodiment of the present application, an insurance claim automatic decision optimization method based on reinforcement learning comprises the following steps: S1, collecting insurance claim data, pre-processing the insurance claim data, and constructing a standardized claim sample set; S2, defining a state space and an action space based on the standardized claim sample set, and constructing a reward function for claim strategy training; S3, constructing an EfficientZero reinforcement learning model based on the state space, the action space and the reward function; S4, using the standardized claim sample set to train the EfficientZero reinforcement learning model offline, reusing historical claim trajectories using a sample playback mechanism, introducing a self-supervised consistency loss to optimize the prediction consistency of the environment dynamic modeling network, and updating the parameters of the policy prediction network and the prefix value prediction network by minimizing the policy loss and the cumulative reward error; S5, deploying the trained model to an insurance claim business system, extracting a state vector for new input insurance claim data, and outputting corresponding claim actions; S6, constructing an incremental claim sample set based on actual claim action execution results and customer feedback information, and periodically updating the parameters of the model using an incremental learning mechanism.

[0010] Optionally, the insurance claim data includes customer identity information, policy clause content, accident type, time of accident and claim materials.

[0011] Optionally, the data preprocessing includes format standardization, missing field filling, numerical normalization and outlier removal.

[0012] Optionally, the state space includes customer identity information, policy clause content, accident type, time of accident and claim materials.

[0013] Optionally, the action space is a claim operation, including payment, partial payment, refusal to pay and manual review.

[0014] Optionally, the reward function is a weighted sum of the compensation reasonableness score, fraud risk score, customer satisfaction score, and operating costs.

[0015] Optionally, the EfficientZero reinforcement learning model includes a state encoder, an environment dynamic modeling network, a policy prediction network, and a prefix value prediction network. The environment dynamic modeling network predicts the hidden state and cumulative reward at future time steps, and the policy prediction network outputs the policy probability distribution of each claim operation.

[0016] Optionally, S4 specifically includes: S41. Construct a training trajectory set based on a standardized claims sample set: ; in, For a set of trajectories, For the first The claims status vector at each time step For the execution of claims procedures, The actual instant reward value obtained. The length of the trajectory; S42. Store the training trajectory set into the experience replay buffer, and sample batch trajectories from it in each round of training. And calculate the sample weight for each sample, where Indicates the first Trajectory, Number of sampling trajectories: ; in, For sample weights, It is a natural exponential function. and These are weighting coefficients. For the first Fraud risk score at the start of each trajectory. For a predefined set of claims categories, "belongs to" For the type of claim, For indicator functions, when The value of the exponential function is 1 if it is positive and 0 otherwise. S43. For each sampling trajectory, the initial state is input to the state encoder to calculate the initial hidden state. The encoder uses a multilayer perceptron. ; in, For the first The initial hidden state of the trajectory. Here is the encoder weight matrix. For bias terms, This is the initial state. To modify the activation function of the linear unit; S44. At each time step, the next hidden state and predicted reward are calculated using an environmental dynamic modeling network. The environmental dynamic modeling network employs... network: ; in, For the predicted first The next hidden state of the trajectory. For the predicted first Instant rewards for each trajectory, For the first The current hidden state of the trajectory. For the first Input action of a trajectory, Embed operations for actions; S45. At each time step, a prefix value prediction network is used to predict the value of the hidden state. The prefix value prediction network is a linear fully connected network. ; in, For the predicted first The prefix value of a trajectory For value network weights, For bias terms; Simultaneously, based on the initial hidden state, the policy prediction network generates a policy probability distribution: ; in, For the policy probability distribution, For policy network weights, For bias terms, For normalization; S46. Construct a training loss function by combining sample weights, including immediate reward loss, prefix value loss, and policy loss: ; in, To train the loss function, , , These are the weighted coefficients for immediate reward, value, and strategy loss, respectively. For the first The real rewards for each trajectory For the predicted first Instant rewards for each trajectory, For the first The target prefix value of the trajectory For the predicted first The prefix value of a trajectory For the target policy distribution, For the predicted policy probability distribution, For time steps, The number of sampling trajectories, For the action space, For sample weights; S47. Introduce a self-supervised consistency loss term to constrain the consistency between the hidden state predicted by the model and the true state encoding: ; in, For consistency loss, For the first The actual state of the time step. For the predicted first The next hidden state of the trajectory; S48. Define the total loss function: ; in, For the total loss function, The weighting coefficient for consistency loss; S49. By minimizing the total loss function, the parameters of the state encoder, the environment dynamic modeling network, the prefix value prediction network, and the policy prediction network are updated using the backpropagation algorithm until the model converges, thus completing the offline training process.

[0017] Optionally, S5 specifically includes: S51. Deploy the trained EfficientZero reinforcement learning model to the insurance claims business system to receive new input insurance claims status data and perform real-time inference. S52. Input the standardized state vector of the claims to be processed into the state encoder, extract the semantic feature representation of the current case, and generate the hidden state for action reasoning. S53. Input the hidden state into the policy prediction network and output the corresponding policy probability distribution; S54. Based on the probability distribution of the strategy, select the claim action with the highest probability value as the final decision action, and submit it to the insurance claim business system as the claim processing result. S55. The insurance claims system executes corresponding payment disbursements, partial compensation processing, claim rejection responses, or transfers to manual review processes based on decision actions, thereby achieving automated execution of claims operations. S56. Synchronously generate auxiliary output data related to decision-making, the auxiliary output data including the current case input status, action probability distribution, highest probability value and selected claim action.

[0018] Optionally, S6 specifically includes: S61. Collect status data, execution actions, immediate reward values, customer feedback, and subsequent status data generated during the actual claims process to construct an incremental claims sample set: ; in, As an incremental claims sample, For the first The claims status vector at each time step, representing the claims action performed. The actual instant reward value obtained. This serves as the subsequent claims status vector. Rate the complexity of the claims clauses. Score the risk of fraud. Indicates the length of the incremental trajectory; S62. Merge the incremental claims sample set into the experience replay buffer and record the latest timestamp of each sample; S63. Calculate the sampling probability for each sample, wherein the sampling probability is composed of the immediate reward bias, the clause complexity, and the fraud score: ; in, Indicates sample The sampling probability, Indicates sample Instant reward bias, The power exponent adjustment parameter, This is a weighting factor for the complexity of the clauses. As a weighting factor for fraud rating, and Indicates the complexity score of the claims clause. and Indicates fraud risk score, Indicates sample Instant reward bias; S64. Based on the sampling probability, a sampling subset is constructed by sampling from the experience replay buffer. A random sampling function is used to extract a fixed number of samples from the experience replay buffer to form a training subset. The original EfficientZero reinforcement learning model parameters are loaded and incremental training is performed using a fixed learning rate, keeping the state encoder and the environment dynamic modeling network structure unchanged. S65. Calculate the policy probability distribution under the sampled subset and compare it with the historical model policy probability distribution. Calculate the KL divergence, if meet: ; Wherein, The KL divergence, The strategy probability distribution under the sample subset, The historical model strategy probability distribution, The preset strategy change threshold value; Freeze the parameters of the strategy prediction network, only update the prefix value prediction network and the environment dynamic modeling network; S66, construct an incremental training comprehensive loss function, and add a time decay term: ; Wherein, The comprehensive loss function, The task loss, The consistency loss, The sample time decay factor, And The current and historical model parameters are respectively, And The weighting coefficient, The Euclidean norm; S67, update the model parameters by minimizing the comprehensive loss function, complete the incremental training of the EfficientZero reinforcement learning model.

[0019] The beneficial effects of the application are: Firstly, the insurance claim automatic decision optimization method based on reinforcement learning provided by the application overcomes the limitations of the existing claim automation system, such as relying on artificial rule driving, difficulty in dynamically adjusting strategy, weak model generalization ability, lack of self-learning and feedback mechanism, etc. The method innovatively introduces the EfficientZero model structure in reinforcement learning into the insurance claim decision-making scene in the technical scheme, combines the business semantic features, multi-source structure and evolution features of the claim data, and constructs a complete learning network with state perception, strategy output, prediction modeling and value judgment ability, realizes the end-to-end claim action recommendation mechanism and the strategy continuous optimization process.

[0020] Secondly, by incorporating key influencing factors such as customer identity information, policy terms, accident type, time of incident, and claim materials into the state space, and constructing an action space by combining the business impact dimension of claims behavior, this invention significantly enhances the model's ability to express the real context of claims business. The reward function no longer relies solely on simple claims accuracy, but integrates claims rationality, customer satisfaction, fraud risk probability, and operating cost indicators, achieving comprehensive optimization of decision value. In terms of model structure design, the state encoder, environmental dynamic modeling network, policy prediction network, and prefix value prediction network from EfficientZero are introduced, enabling the model to have strong training efficiency and policy expression ability even under limited sample conditions, significantly improving its adaptability in complex decision-making scenarios.

[0021] Furthermore, this invention introduces sample replay, self-supervised consistency loss optimization, and sample weighting mechanisms into the training mechanism. This allows the model to reuse high-value historical data during offline training while improving its internal time consistency modeling capabilities, further enhancing policy stability and robustness. Especially after model deployment, an online update mechanism is designed to construct an incremental sample set based on actual claims execution results and customer feedback. The sample sampling probability is jointly constructed using reward bias, clause complexity, and fraud scoring, achieving incremental data-first learning. Simultaneously, a freezing mechanism is introduced into the policy network using KL divergence calculation between the policy output distribution and historical model policies, effectively controlling the stability risk of policy migration. A regularization term constructed using a time decay factor ensures the convergence speed and historical memory retention capability of the model parameters during continuous optimization.

[0022] Finally, compared with traditional rule template systems or classification models based on shallow supervised learning, this invention not only supports adaptation to complex, unstructured, and dynamic claims scenarios, but also has the advantages of continuous strategy evolution, interpretable decision output, multi-factor control, and semantic fusion of insurance business. The method can be widely deployed in various claims systems such as life insurance, auto insurance, and health insurance to improve claims efficiency, reduce processing costs, and enhance risk identification capabilities. Furthermore, it continuously improves the system's automation level and business intelligence capabilities through a dynamic learning mechanism, and has broad application prospects and promotional value in the field of insurance technology. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0024] Fig. 1 This is a flowchart of an automatic decision-making optimization method for insurance claims based on reinforcement learning proposed in this invention; Fig. 2This is a schematic diagram of the EfficientZero reinforcement learning model structure, which is a reinforcement learning-based automatic decision optimization method for insurance claims proposed in this invention. Fig. 3 This diagram illustrates the claims decision output and strategy freezing mechanism of an automatic insurance claims decision optimization method based on reinforcement learning proposed in this invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0026] refer to Figs. 1-3 An automatic decision-making optimization method for insurance claims based on reinforcement learning includes the following steps: S1. Collect insurance claim data, perform data preprocessing on the insurance claim data, and construct a standardized claim sample set; S2. Define the state space and action space based on the standardized claims sample set, and construct the reward function for training the claims strategy; S3. Construct the EfficientZero reinforcement learning model based on the state space, action space, and reward function; S4. The EfficientZero reinforcement learning model is trained offline using the standardized claims sample set. The historical claims trajectory is reused using the sample replay mechanism. Self-supervised consistency loss is introduced to optimize the prediction consistency of the dynamic environment modeling network. The parameters of the policy prediction network and the prefix value prediction network are updated by minimizing the policy loss and cumulative reward error. S5. Deploy the trained model to the insurance claims business system, extract the state vector for the newly input insurance claims data, and output the corresponding claims action; S6. Based on the actual claims action execution results and customer feedback information, construct an incremental claims sample set, and use an incremental learning mechanism to periodically update the parameters of the model.

[0027] The automatic decision optimization method for insurance claims based on reinforcement learning proposed in this invention constructs a full-process intelligent decision-making system from data preprocessing, state and action modeling, strategy learning to deployment and updates, which significantly improves the automation and intelligence level of claims processing, has the ability to learn continuously, optimize autonomously and implement strategies, and is applicable to various claims business scenarios.

[0028] In this embodiment, the insurance claims data includes customer identity information, policy terms, accident type, time of incident, and claim materials.

[0029] This invention clarifies the key components of insurance claims data, including customer identity, policy terms, accident information, and claim materials, making the state modeling process more semantically complete and industry-adaptable. This helps reinforcement learning models accurately perceive the context of claims and improves the authenticity and generalization of state representation.

[0030] In this embodiment, the data preprocessing includes format standardization, missing field filling, numerical normalization, and outlier removal.

[0031] This invention introduces preprocessing steps such as format standardization, missing field filling, numerical normalization, and outlier removal to ensure the consistency and stability of input data, reduce noise interference during model training, improve sample effectiveness, and lay a data quality foundation for subsequent state modeling and policy learning.

[0032] In this embodiment, the state space includes customer identity information, policy terms, accident type, time of incident, and claim materials.

[0033] This invention enhances the model's ability to perceive the context of claims, temporal characteristics, and business rules by incorporating customer identity, policy terms, accident type, time of incident, and claim materials into the state space, thereby improving the relevance and rationality of the reinforcement learning strategy output.

[0034] In this embodiment, the action space refers to claims processing, including payment, partial payment, claim rejection, and manual review.

[0035] This invention defines compensation, partial compensation, claim rejection, and manual review as an action space, enabling the model to optimize strategies in multiple processing paths, adapt to actual claims business processes, support flexible switching between automatic decision-making and manual intervention, and improve the system's flexibility and robustness in handling complex claims.

[0036] In this embodiment, the reward function is a weighted sum of the compensation reasonableness score, fraud risk score, customer satisfaction score, and operating cost.

[0037] This invention constructs a composite reward function that integrates the reasonableness of compensation, the probability of fraud risk, customer satisfaction, and operating costs. It effectively balances the multi-objective optimization goals of the model between compliance, security, and service experience, and guides the learning strategy to achieve maximum benefit while meeting business rules.

[0038] In this embodiment, the EfficientZero reinforcement learning model includes a state encoder, an environment dynamic modeling network, a policy prediction network, and a prefix value prediction network. The environment dynamic modeling network predicts the hidden state and cumulative reward at future time steps, and the policy prediction network outputs the policy probability distribution of each claim operation.

[0039] This invention constructs the EfficientZero reinforcement learning model, which integrates a state encoder, an environment dynamic modeling network, a policy prediction network, and a prefix value prediction network. This enables efficient policy learning and future state prediction in a high-dimensional state space, thereby improving the model's inference depth and policy accuracy in complex claims scenarios.

[0040] In this embodiment, S4 specifically includes: S41. Construct a training trajectory set based on a standardized claims sample set: ; in, For a set of trajectories, For the first The claims status vector at each time step For the execution of claims procedures, The actual instant reward value obtained. The length of the trajectory; S42. Store the training trajectory set into the experience replay buffer, and sample batch trajectories from it in each round of training. And calculate the sample weight for each sample, where Indicates the first Trajectory, Number of sampling trajectories: ; in, For sample weights, It is a natural exponential function. and These are weighting coefficients. For the first Fraud risk score at the start of each trajectory. For a predefined set of claims categories, "belongs to" For the type of claim, For indicator functions, when The value of the exponential function is 1 if it is positive and 0 otherwise. S43. For each sampling trajectory, the initial state is input to the state encoder to calculate the initial hidden state. The encoder uses a multilayer perceptron. ; in, For the first The initial hidden state of the trajectory. Here is the encoder weight matrix. For bias terms, This is the initial state. To modify the activation function of the linear unit; S44. At each time step, the next hidden state and predicted reward are calculated using an environmental dynamic modeling network. The environmental dynamic modeling network employs... network: ; in, For the predicted first The next hidden state of the trajectory. For the predicted first Instant rewards for each trajectory, For the first The current hidden state of the trajectory. For the first Input action of a trajectory, Embed operations for actions; S45. At each time step, a prefix value prediction network is used to predict the value of the hidden state. The prefix value prediction network is a linear fully connected network. ; in, For the predicted first The prefix value of a trajectory For value network weights, For bias terms; Simultaneously, based on the initial hidden state, the policy prediction network generates a policy probability distribution: ; in, For the policy probability distribution, For policy network weights, For bias terms, For normalization; S46. Construct a training loss function by combining sample weights, including immediate reward loss, prefix value loss, and policy loss: ; in, To train the loss function, , , These are the weighted coefficients for immediate reward, value, and strategy loss, respectively. For the first The real rewards for each trajectory For the predicted first Instant rewards for each trajectory, For the first The target prefix value of the trajectory For the predicted first The prefix value of a trajectory For the target policy distribution, For the predicted policy probability distribution, For time steps, The number of sampling trajectories, For the action space, For sample weights; S47. Introduce a self-supervised consistency loss term to constrain the consistency between the hidden state predicted by the model and the true state encoding: ; in, For consistency loss, For the first The actual state of the time step. For the predicted first The next hidden state of the trajectory; S48. Define the total loss function: ; in, For the total loss function, The weighting coefficient for consistency loss; S49. By minimizing the total loss function, the parameters of the state encoder, the environment dynamic modeling network, the prefix value prediction network, and the policy prediction network are updated using the backpropagation algorithm until the model converges, thus completing the offline training process.

[0041] This invention proposes a complete offline training process, including a sample replay mechanism, self-supervised consistency loss, sample weight construction, and multi-network joint optimization, which significantly improves the training efficiency and stability of the model, ensures the accuracy and convergence of policy generation, and has engineering reproducibility.

[0042] In this embodiment, S5 specifically includes: S51. Deploy the trained EfficientZero reinforcement learning model to the insurance claims business system to receive new input insurance claims status data and perform real-time inference. S52. Input the standardized state vector of the claims to be processed into the state encoder, extract the semantic feature representation of the current case, and generate the hidden state for action reasoning. S53. Input the hidden state into the policy prediction network and output the corresponding policy probability distribution; S54. Based on the probability distribution of the strategy, select the claim action with the highest probability value as the final decision action, and submit it to the insurance claim business system as the claim processing result. S55. The insurance claims system executes corresponding payment disbursements, partial compensation processing, claim rejection responses, or transfers to manual review processes based on decision actions, thereby achieving automated execution of claims operations. S56. Synchronously generate auxiliary output data related to decision-making, the auxiliary output data including the current case input status, action probability distribution, highest probability value and selected claim action.

[0043] This invention deploys the trained model to the insurance claims system, enabling real-time case status extraction, strategy reasoning, and action output, forming a closed-loop process from intelligent analysis to business execution. This improves the automation level of claims processing and enhances the system's availability and response efficiency in actual business operations.

[0044] In this embodiment, S6 specifically includes: S61. Collect status data, execution actions, immediate reward values, customer feedback, and subsequent status data generated during the actual claims process to construct an incremental claims sample set: ; in, As an incremental claims sample, For the first The claims status vector at each time step, representing the claims action performed. The actual instant reward value obtained. This serves as the subsequent claims status vector. Rate the complexity of the claims clauses. Score the risk of fraud. Indicates the length of the incremental trajectory; S62. Merge the incremental claims sample set into the experience replay buffer and record the latest timestamp of each sample; S63. Calculate the sampling probability for each sample, wherein the sampling probability is composed of the immediate reward bias, the clause complexity, and the fraud score: ; in, Indicates sample The sampling probability, Indicates sample Instant reward bias, The power exponent adjustment parameter, This is a weighting factor for the complexity of the clauses. As a weighting factor for fraud rating, and Indicates the complexity score of the claims clause. and Indicates fraud risk score, Indicates sample Instant reward bias; S64. Based on the sampling probability, a sampling subset is constructed by sampling from the experience replay buffer. A random sampling function is used to extract a fixed number of samples from the experience replay buffer to form a training subset. The original EfficientZero reinforcement learning model parameters are loaded and incremental training is performed using a fixed learning rate, keeping the state encoder and the environment dynamic modeling network structure unchanged. S65. Calculate the policy probability distribution under the sampled subset and compare it with the historical model policy probability distribution. Calculate the KL divergence if the following conditions are met: ; in, Let KL divergence be the KL divergence. The policy probability distribution under the sampling subset, For the historical model strategy probability distribution, The preset policy change threshold; Freeze the parameters of the policy prediction network and only update the prefix value prediction network and the environment dynamics modeling network; S66. Construct an incremental training loss function and add a time decay term: ; in, For the comprehensive loss function, For mission losses, For consistency loss, The sample time decay factor, and These are the parameters for the current and historical models, respectively. and These are weighting coefficients. It is the Euclidean norm; S67. Update the model parameters by minimizing the comprehensive loss function to complete the incremental training of the EfficientZero reinforcement learning model.

[0045] This invention designs an incremental learning mechanism oriented towards real business feedback. It constructs a sample sampling strategy by using reward bias, clause complexity, and fraud scoring. Combined with strategy freezing and time decay control, it achieves stable evolution and dynamic adaptation of the model, thereby enhancing the system's self-learning ability and reliability in long-term operation.

[0046] Example 1: To verify the feasibility of this invention in practice, it was applied to the auto insurance and property insurance claims system of a large insurance group, covering the pilot project of intelligent claims process reconstruction in its branches in Beijing, Shanghai, and Guangdong. The project implementation period was from May 2024 to March 2025, covering a wide range of claims including traffic accidents, theft, natural disasters, and personal accidents. The main goal of the system is to replace the original claims preliminary review mechanism that relied on manual rule matching and business experience judgment with a reinforcement learning optimization model, so as to achieve automatic generation of claims strategies, autonomous learning, and accurate compensation.

[0047] In this embodiment, the insurance company's original system adopts a claims decision-making method based on a process engine and a static rule tree, which has the following problems: the processing results are highly dependent on preset rules, making it difficult to accurately classify complex claims, especially in fraud identification and handling cases with high clause complexity, which are prone to misjudgment or high rejection rates. In addition, the original system lacks a feedback mechanism and cannot adaptively correct the strategy in the stages of customer complaints and post-payment satisfaction recovery, resulting in poor customer experience and significant fluctuations in operating costs.

[0048] After deployment, this invention first collects multi-dimensional claims data, including customer identity information, insurance liability terms, accident information, time of incident, and claim materials. It then constructs a standardized claims sample set by combining historical claims judgments, and completes data cleaning, normalization, and field completion. During the training phase, the system uses the EfficientZero reinforcement learning framework, which integrates a state encoder, an environment dynamic modeling network, a policy prediction network, and a prefix value prediction network to model the claims trajectory and train the strategy. The reward function simultaneously considers factors such as the reasonableness of compensation, the probability of fraud, customer satisfaction, and operating costs to improve the model's real-world adaptability in business scenarios.

[0049] After deployment, the model begins to reason and make decisions on real claims. For each new case, a state vector is generated in real time and entered into the model, automatically outputting claims action suggestions, including "payment", "partial payment", "rejection" or "manual review". The system will simultaneously generate a strategy probability distribution and confidence level for back-office staff to review. In actual business, if the customer submits supplementary materials or feedback complaint information later, this information will be re-injected into the model as feedback incremental samples. The model performs periodic incremental learning through priority sampling, self-supervised consistency mechanism and strategy freeze control to adapt to the evolution of data distribution.

[0050] Table 1 Comparison of experimental data between the present invention system and the traditional claims system

[0051] Firstly, regarding the average claims processing time, traditional systems generally have a long processing cycle, averaging 9.4 days. However, this invention, through a reinforcement learning-driven intelligent strategy recommendation mechanism, shortens the processing time to 5.7 days, improving efficiency by nearly 40% on average. This significantly accelerates the settlement of customer claims and enhances the user experience. In terms of customer satisfaction, traditional systems lack an effective mechanism for absorbing user feedback and updating strategies, making it difficult to optimize and adjust for complaints or non-standard cases. However, after introducing incremental learning and feedback sampling mechanisms, this invention improved customer satisfaction by 12.4%, indicating that the strategy is closer to customer expectations and fairness.

[0052] Regarding the control of false claim rates, this invention effectively reduces the probability of false claims and incorrect denials by fraud scoring, clause complexity modeling, and dynamic strategy training, resulting in an overall decrease in the false claim rate of 18.9%. In terms of automation rate, traditional systems operate at approximately 52%, meaning that nearly half of the cases still require manual review and intervention; while the automatic decision-making rate of this invention's system reaches 81%, demonstrating the model's stability and scalability across various complex scenarios.

[0053] To identify high-risk fraud cases, this invention integrates multi-dimensional state features and a risk scoring mechanism, achieving an 89% identification rate, significantly higher than the approximately 61% of traditional systems. This improves risk control capabilities and reduces potential compensation losses. Furthermore, this invention features a fixed-cycle model update mechanism (every 7 days), while traditional systems typically lack automatic strategy optimization and the ability to adapt to evolving compensation patterns. Regarding the reasonableness of compensation, this invention improves the strategy output score by 17.8%, reflecting the model's accurate judgment ability to balance compliance and customer rights.

[0054] Furthermore, the proportion of cases requiring manual review has decreased from the traditional 27% to 14%, significantly alleviating the pressure on claims review and improving human resource efficiency. The response time for supplementary claims documentation has also decreased from an average of 48 hours to 12 hours, improving the speed of customer data return and workflow. Regarding system stability, this invention demonstrates high availability and robustness during continuous operation, with no significant performance degradation even under concurrent processing by multiple branches.

[0055] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatic decision optimization in insurance claims based on reinforcement learning, characterized in that, Includes the following steps: S1. Collect insurance claim data, perform data preprocessing on the insurance claim data, and construct a standardized claim sample set; S2. Define the state space and action space based on the standardized claims sample set, and construct the reward function for training the claims strategy; S3. Construct the EfficientZero reinforcement learning model based on the state space, action space, and reward function; S4. The EfficientZero reinforcement learning model is trained offline using the standardized claims sample set. The historical claims trajectory is reused using the sample replay mechanism. Self-supervised consistency loss is introduced to optimize the prediction consistency of the dynamic environment modeling network. The parameters of the policy prediction network and the prefix value prediction network are updated by minimizing the policy loss and cumulative reward error. S5. Deploy the trained model to the insurance claims business system, extract the state vector for the newly input insurance claims data, and output the corresponding claims action; S6. Based on the actual claims action execution results and customer feedback information, construct an incremental claims sample set, and use an incremental learning mechanism to periodically update the parameters of the model.

2. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The insurance claims data includes customer identity information, policy terms, accident type, time of incident, and claim materials.

3. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The data preprocessing includes format standardization, missing field filling, numerical normalization, and outlier removal.

4. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The state space includes customer identity information, policy terms, accident type, time of incident, and claim materials.

5. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The action space refers to claims processing, including payment, partial payment, claim rejection, and manual review.

6. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The reward function is a weighted sum of the compensation reasonableness score, fraud risk score, customer satisfaction score, and operating costs.

7. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, The EfficientZero reinforcement learning model includes a state encoder, an environment dynamic modeling network, a policy prediction network, and a prefix value prediction network. The environment dynamic modeling network predicts the hidden state and cumulative reward for future time steps, and the policy prediction network outputs the policy probability distribution for each claim operation.

8. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Construct a training trajectory set based on a standardized claims sample set; S42. Store the training trajectory set into the experience replay buffer, and sample batch trajectories from it in each round of training. And calculate the sample weight for each sample, where Indicates the first Trajectory, Number of sampling trajectories: ； in, For sample weights, It is a natural exponential function. and These are weighting coefficients. For the first Fraud risk score at the start of each trajectory. For a predefined set of claims categories, "belongs to" For the type of claim, For indicator functions, when The value of the exponential function is 1 if it is positive and 0 otherwise. S43. For each sampling trajectory, the initial state is input to the state encoder to calculate the initial hidden state. The encoder is a multilayer perceptron. S44. At each time step, the next hidden state and predicted reward are calculated using an environmental dynamic modeling network. The environmental dynamic modeling network employs... network; S45. At each time step, a prefix value prediction network is used to predict the value of the hidden state. The prefix value prediction network is a linear fully connected network. Simultaneously, based on the initial hidden state, the policy prediction network generates a policy probability distribution; S46. Construct a training loss function by combining sample weights, including immediate reward loss, prefix value loss, and policy loss: ； in, To train the loss function, , , These are the weighted coefficients for immediate reward, value, and strategy loss, respectively. For the first The real rewards for each trajectory For the predicted first Instant rewards for each trajectory, For the first The target prefix value of the trajectory For the predicted first The prefix value of a trajectory For the target policy distribution, For the predicted policy probability distribution, For time steps, The number of sampling trajectories, For the action space, For sample weights; S47. Introduce a self-supervised consistency loss term to constrain the consistency between the hidden state predicted by the model and the true state encoding: ； in, For consistency loss, For the first The actual state of the time step. For the predicted first The next hidden state of the trajectory; S48. Define the total loss function; S49. By minimizing the total loss function, the parameters of the state encoder, the environment dynamic modeling network, the prefix value prediction network, and the policy prediction network are updated using the backpropagation algorithm until the model converges, thus completing the offline training process.

9. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. Deploy the trained EfficientZero reinforcement learning model to the insurance claims business system to receive new input insurance claims status data and perform real-time inference. S52. Input the standardized state vector of the claims to be processed into the state encoder, extract the semantic feature representation of the current case, and generate the hidden state for action reasoning. S53. Input the hidden state into the policy prediction network and output the corresponding policy probability distribution; S54. Based on the probability distribution of the strategy, select the claim action with the highest probability value as the final decision action, and submit it to the insurance claim business system as the claim processing result. S55. The insurance claims system executes corresponding payment disbursements, partial compensation processing, claim rejection responses, or transfers to manual review processes based on decision actions, thereby achieving automated execution of claims operations. S56. Synchronously generate auxiliary output data related to decision-making, the auxiliary output data including the current case input status, action probability distribution, highest probability value and selected claim action.

10. The automatic decision-making optimization method for insurance claims based on reinforcement learning according to claim 1, characterized in that, S6 specifically includes: S61. Collect status data, execution actions, instant reward values, customer feedback and subsequent status data generated during the actual execution of claims actions, and construct an incremental claims sample set; S62. Merge the incremental claims sample set into the experience replay buffer and record the latest timestamp of each sample; S63. Calculate the sampling probability for each sample, wherein the sampling probability is composed of the immediate reward bias, the clause complexity, and the fraud score: ； in, Indicates sample The sampling probability, Indicates sample Instant reward bias, The power exponent adjustment parameter, This is a weighting factor for the complexity of the clauses. As a weighting factor for fraud rating, and Indicates the complexity score of the claims clause. and Indicates fraud risk score, Indicates sample Instant reward bias; S64. Based on the sampling probability, a sampling subset is constructed by sampling from the experience replay buffer. A random sampling function is used to extract a fixed number of samples from the experience replay buffer to form a training subset. The original EfficientZero reinforcement learning model parameters are loaded and incremental training is performed using a fixed learning rate, keeping the state encoder and the environment dynamic modeling network structure unchanged. S65. Calculate the policy probability distribution under the sampled subset and compare it with the historical model policy probability distribution. Calculate the KL divergence. If the KL divergence is less than the preset policy change threshold, freeze the parameters of the policy prediction network and only update the prefix value prediction network and the environmental dynamic modeling network. S66. Construct an incremental training loss function and add a time decay term: ； in, For the comprehensive loss function, For mission losses, For consistency loss, The sample time decay factor, and These are the parameters for the current and historical models, respectively. and These are weighting coefficients. It is the Euclidean norm; S67. Update the model parameters by minimizing the comprehensive loss function to complete the incremental training of the EfficientZero reinforcement learning model.

Images