Claim settlement auditing agent training method and device, equipment and storage medium

Abstract

The invention discloses a training method and device of a claim settlement auditing agent, equipment and a storage medium. According to the scheme, based on a training data set containing historical claim settlement data and auditing result labels, a multi-task training environment containing a claim settlement auditing main task and at least one auxiliary task is constructed, and the auxiliary task is used for strengthening the information processing basic capacity of an intelligent agent. And configuring a multi-dimensional reward signal for each task, wherein the multi-dimensional reward signal at least comprises a signal for evaluating the reasonability of the reasoning process. And based on a reinforcement learning algorithm, completing preliminary training by using the training environment and the reward signal. And when it is detected that an abnormal reasoning behavior exists in the preliminary training, the reward signal is adjusted to suppress the behavior. And iteratively optimizing the intelligent agent based on the adjusted reward signal. Through multi-task training, multi-dimensional reward configuration and closed-loop iterative optimization, the behavior of neglecting the reasoning process is reduced, and the audit compliance is enhanced, so that synchronous improvement of the agent audit accuracy, the process compliance and the result reliability is realized.

Description

Technical Field

[0001] This application relates to the fields of insurance and artificial intelligence technology, and in particular to a training method, apparatus, device and storage medium for a claims review intelligent agent. Background Technology

[0002] With the deep integration of artificial intelligence and the financial insurance industry, intelligent claims processing agents, with their efficient automated processing capabilities, have become a key technological means to improve the efficiency of claims processing and reduce labor costs. Currently, the training of existing intelligent claims processing agents mostly relies on a single-task training mode, that is, taking the correctness of the claims review results as the core training objective and optimizing model parameters through reinforcement learning algorithms.

[0003] However, this type of training method has significant technical flaws: On the one hand, single-task training focuses only on the final review result, lacking targeted training for the basic information processing capabilities of the claims review agent. This results in weaknesses in fundamental capabilities such as logical reasoning coherence, contextual information fit, and the accuracy of key information extraction, leading to problems such as imprecise reasoning processes and insufficient evidence, ultimately affecting the reliability of the review results. On the other hand, the reward mechanisms used in existing training are mostly single-dimensional, result-oriented rewards, only rewarding or punishing based on whether the review result is correct, without evaluating or constraining the rationality of the claims review agent's reasoning process. This allows some claims review agents to obtain "correct results" that contradict the true review intent by circumventing normal reasoning processes, fabricating reasoning basis, and abusing rule loopholes—a phenomenon known as "reward hacking." Such abnormal reasoning behavior not only fails to guarantee the compliance of claims reviews but may also lead to inaccurate review results due to hidden logical flaws, making it difficult to meet the high requirements of accuracy and process standardization in actual claims business, thus limiting the widespread application of claims review agents in large-scale business scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a training method, apparatus, device, and storage medium for a claims review intelligent agent, with the aim of improving the review accuracy, process compliance, and result reliability of the claims review intelligent agent.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] The first aspect of this application provides a method for training a claims review intelligent agent, the method comprising:

[0007] Based on a training dataset containing historical claims data and its audit results annotations, a multi-task training environment is constructed, which includes a main claims audit task and at least one auxiliary task; the auxiliary task is used to train the basic information processing capabilities of the claims audit agent.

[0008] In the multi-task training environment, multi-dimensional reward signals are configured for the main task and the at least one auxiliary task, and the multi-dimensional reward signals include at least a reward signal for evaluating the rationality of the reasoning process of the claims review agent.

[0009] Based on the reinforcement learning algorithm, the claims review agent is initially trained using the multi-task training environment and the multi-dimensional reward signal.

[0010] In response to the detection that the claims review agent exhibits abnormal reasoning behavior in the reasoning process data generated during the initial training process, the multi-dimensional reward signal is adjusted to suppress the abnormal reasoning behavior.

[0011] Based on the adjusted multi-dimensional reward signal, the claims review agent is iteratively optimized.

[0012] In an optional implementation, the auxiliary task includes at least one of the following:

[0013] Recall tasks are used to train information association and filtering capabilities, while extraction tasks are used to train precise information location capabilities.

[0014] In an optional implementation, the recall task is used to train the claims review agent to recall relevant items from a preset set based on claims case information; the extraction task is used to train the claims review agent to extract field information of a specified type from the claims case text.

[0015] In an optional implementation, the multi-dimensional reward signal configured for the recall task includes recall accuracy and / or whether irrelevant output is generated; the multi-dimensional reward signal configured for the extraction task includes extraction accuracy and / or semantic consistency between the extracted content and the context.

[0016] In an optional implementation, the preliminary training of the claims review agent based on the reinforcement learning algorithm, utilizing the multi-task training environment and the multi-dimensional reward signal, includes:

[0017] The multi-dimensional reward signals of the main task and each of the auxiliary tasks are weighted and fused to obtain the total reward signal; wherein each dimension of the reward signal is associated with an adjustable weight coefficient.

[0018] Based on the total reward signal, the model parameters of the claims review agent are updated using a reinforcement learning algorithm.

[0019] In an optional implementation, adjusting the multi-dimensional reward signal to suppress the aberrant reasoning behavior includes:

[0020] Apply a negative reward value to the reward signal dimension corresponding to the abnormal reasoning behavior; and / or increase the weight coefficient associated with the reward signal dimension used to evaluate the rationality of the reasoning process of the claims review agent.

[0021] In an optional implementation, the abnormal reasoning behavior includes:

[0022] The claims review agent obtains high rewards through fabricating information, misusing context, or violating logical rules.

[0023] A second aspect of this application provides a training device for a claims review intelligent agent, the device comprising:

[0024] The construction module is used to build a multi-task training environment based on a training dataset containing historical claims data and its audit results annotations. This environment includes a main claims audit task and at least one auxiliary task. The auxiliary task is used to train the basic information processing capabilities of the claims audit agent.

[0025] The configuration module is used to configure multi-dimensional reward signals for the main task and the at least one auxiliary task in the multi-task training environment, wherein the multi-dimensional reward signals include at least a reward signal for evaluating the rationality of the reasoning process of the claims review agent.

[0026] The training module is used to perform preliminary training on the claims review agent based on the reinforcement learning algorithm, using the multi-task training environment and the multi-dimensional reward signal;

[0027] An adjustment module is used to adjust the multi-dimensional reward signal to suppress abnormal reasoning behavior in response to the detection that the claims review agent has abnormal reasoning behavior in the reasoning process data generated during the initial training process.

[0028] The optimization module is used to iteratively optimize the claims review agent based on the adjusted multi-dimensional reward signal.

[0029] A third aspect of this application provides a training device for a claims review intelligent agent, the device comprising: a processor and a memory.

[0030] The memory is used to store program code and transmit the program code to the processor;

[0031] The processor is configured to execute, according to the instructions in the program code, the steps of the training method for the claims review agent described in any implementation of the first aspect.

[0032] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the steps of the training method for the claims review agent described in any implementation of the first aspect.

[0033] Compared with the prior art, this application has the following beneficial effects:

[0034] In this application's technical solution, firstly, a multi-task training environment is constructed based on a training dataset containing historical claims data and its audit result annotations. This environment includes a main claims audit task and at least one auxiliary task, where the auxiliary task is used to train the basic information processing capabilities of the claims audit agent. Secondly, in the multi-task training environment, multi-dimensional reward signals are configured for the main task and the at least one auxiliary task, respectively. These multi-dimensional reward signals include at least a reward signal used to evaluate the rationality of the claims audit agent's reasoning process. Thirdly, based on a reinforcement learning algorithm, the claims audit agent is initially trained using the multi-task training environment and the multi-dimensional reward signals. Then, in response to the detection of abnormal reasoning behavior in the reasoning process data generated during the initial training, the multi-dimensional reward signals are adjusted to suppress this abnormal reasoning behavior. Finally, the claims audit agent is iteratively optimized based on the adjusted multi-dimensional reward signals. Therefore, this application, through a multi-task training environment and auxiliary tasks, strengthens the basic information processing capabilities of the claims audit agent, compensating for the lack of basic capability development in existing training methods and laying the foundation for improving the reliability of audit results. By configuring multi-dimensional reward signals and incorporating the rationality assessment of the reasoning process, the training results and process of the claims review agent are dually guided, reducing the agent's tendency to ignore the reasoning process. An iterative optimization model precisely responds to training issues, dynamically adjusting reward signals to suppress abnormal reasoning behavior, forming a closed-loop optimization and enhancing the compliance of the claims review process. Ultimately, this achieves a simultaneous improvement in the claims review agent's accuracy, process compliance, and result reliability. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating a training method for a claims review intelligent agent provided in this application embodiment;

[0037] Figure 2This is a schematic diagram of the structure of a training device for a claims review intelligent agent provided in an embodiment of this application. Detailed Implementation

[0038] As described earlier, with the deep integration of artificial intelligence and the financial insurance industry, intelligent claims processing agents, with their efficient automated processing capabilities, have become a key technological means to improve the efficiency of claims processing and reduce labor costs. Currently, the training of existing intelligent claims processing agents mostly relies on a single-task training mode, that is, taking the correctness of the claims review results as the core training objective and optimizing model parameters through reinforcement learning algorithms.

[0039] However, this type of training method has significant technical flaws: On the one hand, single-task training focuses only on the final review result, lacking targeted training for the basic information processing capabilities of the claims review agent. This results in weaknesses in fundamental capabilities such as logical reasoning coherence, contextual information fit, and the accuracy of key information extraction, leading to problems such as imprecise reasoning processes and insufficient evidence, ultimately affecting the reliability of the review results. On the other hand, the reward mechanisms used in existing training are mostly single-dimensional, result-oriented rewards, only rewarding or punishing based on whether the review result is correct, without evaluating or constraining the rationality of the claims review agent's reasoning process. This allows some claims review agents to obtain "correct results" that contradict the true review intent by circumventing normal reasoning processes, fabricating reasoning basis, and abusing rule loopholes—a phenomenon known as "reward hacking." Such abnormal reasoning behavior not only fails to guarantee the compliance of claims reviews but may also lead to inaccurate review results due to hidden logical flaws, making it difficult to meet the high requirements of accuracy and process standardization in actual claims business, thus limiting the widespread application of claims review agents in large-scale business scenarios.

[0040] To address the aforementioned problems, the inventors have proposed a training method, apparatus, equipment, and storage medium for a claims review intelligent agent.

[0041] First, based on a training dataset containing historical claims data and its labeled review results, a multi-task training environment is constructed, comprising a main claims review task and at least one auxiliary task. The auxiliary task is used to train the basic information processing capabilities of the claims review agent. Second, in the multi-task training environment, multi-dimensional reward signals are configured for the main task and the at least one auxiliary task, respectively. These multi-dimensional reward signals include at least a reward signal used to evaluate the rationality of the claims review agent's reasoning process. Third, based on a reinforcement learning algorithm, the claims review agent is initially trained using the multi-task training environment and the multi-dimensional reward signals. Then, in response to the detection of abnormal reasoning behavior in the reasoning process data generated during the initial training, the multi-dimensional reward signals are adjusted to suppress this abnormal reasoning behavior. Finally, the claims review agent is iteratively optimized based on the adjusted multi-dimensional reward signals. Therefore, this application, through a multi-task training environment and auxiliary tasks, strengthens the basic information processing capabilities of the claims review agent, compensating for the lack of basic capability development in existing training methods and laying the foundation for improving the reliability of review results. By configuring multi-dimensional reward signals and incorporating the rationality assessment of the reasoning process, the training results and process of the claims review agent are dually guided, reducing the agent's tendency to ignore the reasoning process. An iterative optimization model precisely responds to training issues, dynamically adjusting reward signals to suppress abnormal reasoning behavior, forming a closed-loop optimization and enhancing the compliance of the claims review process. Ultimately, this achieves a simultaneous improvement in the claims review agent's accuracy, process compliance, and result reliability.

[0042] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0043] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] See Figure 1 This figure is a flowchart of a training method for a claims review intelligent agent provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0045] S101. Based on a training dataset containing historical claims data and its audit results annotations, construct a multi-task training environment that includes the main task of claims audit and at least one auxiliary task.

[0046] In this embodiment, the main task is to allow the claims review agent to input claims data and output a review decision (approve or reject). The core objective is to ensure that the review results meet business compliance requirements and the true review intent. The auxiliary task is used to train the claims review agent's basic information processing capabilities, providing support for accurately and efficiently completing the main task.

[0047] In this embodiment of the application, the training dataset is constructed by collecting historical claims data, which includes customer medical records, policy information, past correct review results and corresponding reasoning processes. Each data sample is labeled with a clear result label to ensure the integrity and accuracy of the training data.

[0048] In one example implementation, historical claims data can be deduplicated and standardized in format, and invalid data can be removed before labeling, thereby further improving the quality of the training dataset and laying the foundation for subsequent training results.

[0049] In one alternative implementation, the auxiliary task includes at least one of the following:

[0050] Recall tasks are used to train information association and filtering capabilities, while extraction tasks are used to train precise information location capabilities.

[0051] Optionally, the recall task is used to train the claims review agent, recalling relevant items from a preset set based on claims case information. The extraction task is used to train the claims review agent, extracting specified types of field information from the claims case text.

[0052] In this embodiment, the preset set can be configured according to the needs of the claims business scenario, and includes commonly used medical terms, insurance terms, and other related phrases. Specified types of field information may include key information for claims review such as drug specifications, treatment costs, consultation time, and scope of coverage.

[0053] In one example implementation, phrases in the preset set can be stored according to business categories, and the specified field types can be dynamically adjusted according to the review focus of different insurance types to adapt to diverse claims scenarios.

[0054] This application embodiment strengthens the basic information processing capabilities of the claims review agent through targeted auxiliary tasks, making up for the lack of basic capability development in existing training, and laying the foundation for improving the reliability of review results in the future.

[0055] S102. In a multi-task training environment, configure multi-dimensional reward signals for the main task and at least one auxiliary task respectively.

[0056] In this embodiment, the multi-dimensional reward signal includes at least a reward signal for evaluating the rationality of the reasoning process of the claims review agent, thereby achieving dual guidance on the training results and process of the claims review agent and avoiding the one-sidedness of judging rewards and punishments solely based on results.

[0057] In this embodiment, the reward signal dimension for each task is set according to the core objective of the task, and the reward signals of different dimensions are associated with adjustable weight coefficients, which can be dynamically optimized according to the training focus.

[0058] In one example implementation, the initial weights of each reward signal dimension can be determined based on the compliance requirements and audit priorities of the claims business, and then iteratively adjusted based on the training results.

[0059] In one alternative implementation, the multi-dimensional reward signals configured for the recall task include recall accuracy and / or whether irrelevant output is generated. The multi-dimensional reward signals configured for the extraction task include extraction accuracy and / or the semantic consistency of the extracted content with the context.

[0060] In this embodiment, recall accuracy is used to evaluate the accuracy of the claims review agent in recalling relevant items from a preset set; whether irrelevant output is generated is used to determine whether the claims review agent generates non-specified phrases outside the preset set, thus avoiding error awareness; extraction accuracy is used to measure the matching degree between the extracted field information and the actual situation; semantic consistency is used to ensure that the extracted content is consistent with the context logic, thus avoiding erroneous extraction out of context.

[0061] In one example implementation, quantitative scoring standards can be set for each reward signal dimension, such as positive reward points for accurate recall and negative reward points for irrelevant output, to ensure the operability of the reward signals.

[0062] This application embodiment effectively constrains the reasoning behavior of the claims review agent by configuring multi-dimensional reward signals that fit the training objectives of different tasks, especially by incorporating the dimension of reasoning process rationality assessment, thereby reducing the tendency to ignore the process and pursue superficially correct results.

[0063] S103. Based on reinforcement learning algorithm, the claims review agent is initially trained using a multi-task training environment and multi-dimensional reward signals.

[0064] In this embodiment, the reinforcement learning algorithm adopts an algorithm adapted for multi-task training, which can efficiently update model parameters based on reward signals.

[0065] In this embodiment, the initial training process is executed on the server side. The server builds a multi-task parallel training framework to support the synchronous advancement of the main task and auxiliary tasks, thereby improving training efficiency.

[0066] In one example implementation, training batch and iteration round thresholds can be set, and the accuracy of the claims review agent can be calculated after each round of training, with real-time monitoring of training progress and effectiveness.

[0067] In one alternative implementation, step S103 includes:

[0068] Step 1: Weight and fuse the multi-dimensional reward signals of the main task and each auxiliary task to obtain the total reward signal.

[0069] In the embodiments of this application, each dimension of the reward signal is associated with an adjustable weight coefficient.

[0070] In the embodiments of this application, the setting of weight coefficients needs to be combined with the importance of the task. The weight coefficient of the main task can be higher than that of the auxiliary task, while reserving space for dynamic adjustment to adapt to the needs of different training stages.

[0071] In one example implementation, the formula for calculating the total reward signal is:

[0072] Total reward = w1 × Main quest reward + w2 × Side quest 1 reward + ... + w n ×Auxiliary task n rewards, where n≥1, w1, w2, ..., w n This represents the weighting coefficient for each task reward, and the sum of these weighting coefficients is 1.

[0073] Step 2: Based on the total reward signal, update the model parameters of the claims review agent using a reinforcement learning algorithm.

[0074] In this embodiment of the application, the reinforcement learning algorithm judges the quality of the claims review agent's behavior based on the level of the total reward signal. If the total reward signal meets expectations, the model parameters of the corresponding behavior are strengthened; if the expected result is not met, the parameters are adjusted to optimize subsequent behaviors.

[0075] In one example implementation, the reinforcement learning algorithm uses the GRPO algorithm, which can efficiently handle multi-task reward fusion scenarios and ensure the stability and effectiveness of model parameter updates.

[0076] This application embodiment integrates multi-task and multi-dimensional reward signals using reinforcement learning algorithms to achieve initial training of the claims review agent, allowing the claims review agent to gradually strengthen its basic information processing capabilities while learning the core capabilities of claims review.

[0077] S104. In response to the detection that the claims review agent exhibits abnormal reasoning behavior in the reasoning process data generated during the initial training process, adjust the multi-dimensional reward signal to suppress the abnormal reasoning behavior.

[0078] In this embodiment of the application, the reasoning process data refers to the detailed information generated by the claims review intelligent agent when performing various tasks, such as the decision basis, intermediate reasoning steps, and data call records, which can comprehensively reflect the behavioral logic of the claims review intelligent agent.

[0079] Abnormal reasoning behavior refers to actions by the claims review agent that do not follow the normal reasoning process and contradict the true review intent, which can lead to non-compliance in the review process.

[0080] In this embodiment of the application, the reasoning process logs of the claims review agent are collected periodically, and the logs are analyzed in a structured manner using natural language processing technology to accurately identify abnormal reasoning behavior.

[0081] In one example implementation, a log collection cycle can be set, collecting inference process data once every 20 rounds of training to ensure timely detection of problems during training.

[0082] In one alternative implementation, anomalous reasoning behavior includes actions by which a claims review agent obtains high rewards by fabricating information, misusing context, or violating logical rules.

[0083] In this application embodiment, such abnormal reasoning behavior does not follow the real business logic and compliance requirements of claims review, but only aims to obtain high rewards. Its reasoning process lacks valid basis. Even if it accidentally obtains the correct review result, there are potential business risks and the stability of review quality cannot be guaranteed.

[0084] For example, when reviewing claims for excessive drug dosage, the claims review AI may not base its reasoning on the drug specifications and dosage clearly stated on the prescription submitted by the customer. Instead, it may fabricate unrecorded drug specification parameters to make the calculated dosage comply with regulations, thereby obtaining a reward. Or, when processing claims involving multiple treatment items, the claims review AI may misuse contextual information from other cases, applying irrelevant treatment fee reduction rules to the current case to circumvent the normal cost reasonableness verification process and obtain a high reward. Or, when determining whether a claim falls within the scope of coverage, the claims review AI may violate the logical rule of "first verifying the insurance period, then confirming whether the treatment item is on the coverage list," directly skipping the insurance period verification step and only relying on the treatment item matching result to draw a conclusion of approval, thereby fraudulently obtaining a reward.

[0085] In one optional implementation, to effectively suppress anomalous reasoning behavior and strengthen guidance for compliant reasoning, adjustments are made to multi-dimensional reward signals to suppress such anomalous reasoning behavior, including:

[0086] Apply a negative reward value to the reward signal dimension corresponding to the abnormal reasoning behavior; and / or increase the weight coefficient associated with the reward signal dimension used to assess the reasonableness of the reasoning process of the claims review agent.

[0087] In the embodiments of this application, applying negative reward values ​​can directly punish abnormal behavior, and increasing the weight of relevant reward dimensions can strengthen the constraint on the reasoning process. The two methods can be used alone or in combination to improve the suppression effect.

[0088] In one example implementation, a negative reward score is applied to the error awareness-related reward dimension for the abnormal behavior of fabricating drug specifications. Simultaneously, the weight of the reasoning process rationality dimension is increased from 0.2 to 0.3, further strengthening the guidance for compliant reasoning.

[0089] This application embodiment identifies and suppresses abnormal reasoning behavior in a timely manner, preventing the claims review agent from developing illegal reasoning habits and ensuring the compliance of the claims review process.

[0090] S105. Based on the adjusted multi-dimensional reward signal, iteratively optimize the claims review agent.

[0091] In this embodiment, the iterative optimization process repeatedly executes the logic of multi-task training and reward signal adjustment. After each iteration, the accuracy and compliance of the claims review agent are detected until the preset training target is reached.

[0092] In one example implementation, the preset training objective is that the accuracy of the claims review agent is no less than 0.99. If the accuracy reaches the target for multiple consecutive rounds and there is no obvious abnormal reasoning behavior during the iterative optimization process, the training is considered complete; if the target is not met, the reward signal is adjusted and training is continued.

[0093] This application embodiment constructs a complete training system for a claims review intelligent agent through the coordinated operation of the above steps. The multi-task training environment strengthens the basic capabilities of the claims review intelligent agent, multi-dimensional reward signals implement dual constraints on results and processes, and abnormal behavior detection and iterative optimization ensure the accuracy and continuity of training. Ultimately, it achieves a simultaneous improvement in the accuracy, process compliance, and result reliability of the claims review intelligent agent, effectively addressing core shortcomings such as insufficient basic capabilities and reward hacking in existing training models, and meeting the application needs of large-scale promotion in actual claims business.

[0094] Based on the training method for the claims review intelligent agent provided in the foregoing embodiments, this application also provides a training device for the claims review intelligent agent. Figure 2 This is a schematic diagram of the structure of a training device for a claims review intelligent agent provided in an embodiment of this application. Figure 2 As shown, the training device for the claims review intelligent agent includes: a construction module 201, a configuration module 202, a training module 203, an adjustment module 204, and an optimization module 205.

[0095] Module 201 is used to build a multi-task training environment based on a training dataset containing historical claims data and its audit results annotations. The training environment includes a main claims audit task and at least one auxiliary task. The auxiliary task is used to train the basic information processing capabilities of the claims audit agent.

[0096] Configuration module 202 is used to configure multi-dimensional reward signals for the main task and at least one auxiliary task in a multi-task training environment. The multi-dimensional reward signals include at least a reward signal for evaluating the rationality of the reasoning process of the claims review agent.

[0097] Training module 203 is used to perform preliminary training on the claims review agent based on reinforcement learning algorithm, using a multi-task training environment and multi-dimensional reward signals;

[0098] The adjustment module 204 is used to adjust the multi-dimensional reward signal to suppress the abnormal reasoning behavior in response to the detection that the claim review agent has abnormal reasoning behavior in the reasoning process data generated during the initial training process.

[0099] The optimization module 205 is used to iteratively optimize the claims review agent based on the adjusted multi-dimensional reward signal.

[0100] This application embodiment improves the accuracy, compliance, and reliability of claims review by the intelligent agent through the coordinated operation of the construction module 201, configuration module 202, training module 203, adjustment module 204, and optimization module 205.

[0101] In the optional implementations, the auxiliary task includes at least one of the following:

[0102] Recall tasks are used to train information association and filtering capabilities, while extraction tasks are used to train precise information location capabilities.

[0103] In the optional implementation, the recall task is used to train the claims review agent, which recalls relevant items from a preset set based on the claims case information; the extraction task is used to train the claims review agent, which extracts field information of a specified type from the claims case text.

[0104] In an optional implementation, the configuration module 202 configures multi-dimensional reward signals for the recall task, including recall accuracy and / or whether irrelevant output is generated; and configures multi-dimensional reward signals for the extraction task, including extraction accuracy and / or semantic consistency between the extracted content and the context.

[0105] In the optional implementation, training module 203 is specifically used for:

[0106] The multi-dimensional reward signals of the main task and each auxiliary task are weighted and fused to obtain the total reward signal; among them, each dimension of the reward signal is associated with an adjustable weight coefficient.

[0107] Based on the total reward signal, the model parameters of the claims review agent are updated using a reinforcement learning algorithm.

[0108] In the optional implementation, module 204 is adjusted specifically for:

[0109] Apply a negative reward value to the reward signal dimension corresponding to the abnormal reasoning behavior; and / or increase the weight coefficient associated with the reward signal dimension used to assess the reasonableness of the reasoning process of the claims review agent.

[0110] In the optional implementation, the abnormal reasoning behavior includes:

[0111] Claims review agents may receive high rewards by fabricating information, misusing context, or violating logical rules.

[0112] Furthermore, this application embodiment also provides a training device for a claims review intelligent agent, the training device for the claims review intelligent agent including a processor and a memory.

[0113] The memory is used to store program code and transmit the program code to the processor;

[0114] The processor is used to execute the steps of the training method for the claims review agent described in any of the above method embodiments according to the instructions in the program code.

[0115] Furthermore, embodiments of this application also provide a computer-readable storage medium for storing program code, which is used to execute the steps of the training method for the claims review agent described in any of the above method embodiments.

[0116] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0117] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an intelligent agent for claim settlement auditing, characterized in that, The method comprises the steps of: Based on the training data set containing historical claim data and its audit result annotation, a multi-task training environment containing claim audit main task and at least one auxiliary task is constructed; The auxiliary task is used to train the information processing basic ability of the claim audit intelligent agent; In the multi-task training environment, multi-dimensional reward signals are configured for the main task and the at least one auxiliary task respectively, and the multi-dimensional reward signals at least include a reward signal for evaluating the rationality of the reasoning process of the claim audit intelligent agent; Based on the reinforcement learning algorithm, the multi-task training environment and the multi-dimensional reward signals are used to preliminarily train the claim audit intelligent agent; In response to detecting that the claim audit intelligent agent has abnormal reasoning behavior in the reasoning process data generated in the preliminary training process, the multi-dimensional reward signals are adjusted to suppress the abnormal reasoning behavior; Based on the adjusted multi-dimensional reward signals, the claim audit intelligent agent is iteratively optimized.

2. The method of claim 1, wherein, The auxiliary task includes at least one of the following: The recall task for training information association and screening ability, and the extraction task for training accurate information positioning ability.

3. The method of claim 2, wherein, The recall task is used to train the claim audit intelligent agent to recall relevant items from a preset set according to claim case information; the extraction task is used to train the claim audit intelligent agent to extract field information of a specified type from claim case text.

4. The method according to claim 2 or 3, characterized in that, The multi-dimensional reward signal configured for the recall task includes recall accuracy and / or whether irrelevant output is generated; the multi-dimensional reward signal configured for the extraction task includes extraction accuracy and / or semantic consistency of the extracted content with the context.

5. The method of claim 1, wherein, The preliminary training of the claim audit intelligent agent based on the reinforcement learning algorithm, the multi-task training environment and the multi-dimensional reward signals comprises: The multi-dimensional reward signals of the main task and each auxiliary task are weighted and fused to obtain a total reward signal; wherein each dimension reward signal is associated with an adjustable weight coefficient; Based on the total reward signal, the model parameters of the claim audit intelligent agent are updated through the reinforcement learning algorithm.

6. The method of claim 5, wherein, The adjustment of the multi-dimensional reward signals to suppress the abnormal reasoning behavior comprises: A negative reward value is applied to the reward signal dimension corresponding to the abnormal reasoning behavior; and / or, the weight coefficient associated with the reward signal dimension for evaluating the rationality of the reasoning process of the claim audit intelligent agent is increased.

7. The method of claim 1, wherein, The abnormal reasoning behavior includes: The behavior of the claim audit intelligent agent obtaining high reward by fabricating information, misusing context or violating logical rules.

8. A training device of a claim auditing agent, characterized by, The method comprises the steps of: A construction module is configured to construct a multi-task training environment containing claim audit main task and at least one auxiliary task based on a training data set containing historical claim data and its audit result annotation; the auxiliary task is used to train the information processing basic ability of the claim audit intelligent agent; A configuration module is configured to configure multi-dimensional reward signals for the main task and the at least one auxiliary task in the multi-task training environment, and the multi-dimensional reward signals at least include a reward signal for evaluating the rationality of the reasoning process of the claim audit intelligent agent; a training module configured to preliminarily train the claim auditing agent based on a reinforcement learning algorithm, using the multi-task training environment and the multi-dimensional reward signal; an adjusting module configured to, in response to detecting that the claim auditing agent has abnormal reasoning behavior in the reasoning process data generated in the preliminary training process, adjust the multi-dimensional reward signal to suppress the abnormal reasoning behavior; an optimizing module configured to iteratively optimize the claim auditing agent based on the adjusted multi-dimensional reward signal.

9. A training device of a claim auditing agent, characterized by, comprising: a processor and a memory: the memory is configured to store program code and transmit the program code to the processor; the processor is configured to execute the steps of the training method of the claim auditing agent according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, the computer readable storage medium is configured to store program code, and the program code is configured to execute the steps of the training method of the claim auditing agent. the computer readable storage medium is configured to store program code, and the program code is configured to execute the steps of the training method of the claim auditing agent.

Images