Automatic generation method of rejection defense document based on multi-agent collaboration

Abstract

The invention discloses a method for automatically generating a rejection defense document based on multi-agent collaboration, and belongs to the technical field of artificial intelligence. The method comprises the steps of collecting and preprocessing original multi-modal interaction data related to a payment refusing case, processing the original multi-modal interaction data into a text form to obtain an original interaction text, and storing the original interaction text in a database; performing cleaning processing on the original interaction text in the database by adopting Non-Agent to obtain semantic intermediate representation with consistent format; and based on the structured input, generating an anti-distinguishing reason by a multi-agent collaborative anti-distinguishing framework to obtain an anti-distinguishing document of the current payment refusing case. According to the invention, based on structured layering, responsibility constraint and parallel scheduling, the key defects of'hallusion caused by input noise ', 'single LLM responsibility overload' and'end-to-end time delay 'in the prior art are overcome together, so that the automatic defense signal generation system realized based on the method is remarkably improved in the aspects of compliance and fact accuracy; and practical advantages are embodied in business generalizability and engineering efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a method for automatically generating a refusal defense document based on multi-agent cooperation. BACKGROUND

[0002] With the popularity of e-commerce and third-party payment, refusal has become a common problem in platform operation. Processing refusal usually requires manual retrieval of orders, interaction records, policy basis and writing of defense materials. The manual process has defects such as low efficiency, high cost, difficulty in experience accumulation, and limited coverage. Existing automation solutions are mostly based on single LLM or rule engine to directly generate defense documents, but in actual scenarios, the following problems often occur:

[0003] 1. The input data is multi-source and multi-modal (chat records, telephone summaries, emails, logistics screenshots, etc.), and the original unstructured text contains noise, redundancy and multiple languages. Direct input of LLM will result in high reasoning cost and amplify generation errors or hallucinations;

[0004] 2. Single Agent architecture is prone to perspective misalignment, fictional statements based on assumptions, and abnormal information leakage in tool calls under complex business constraints (channel differentiation, policy details);

[0005] 3. High latency and low throughput in end-to-end processing, which restricts engineering landing and large-scale coverage;

[0006] 4. Current evaluation methods mainly rely on bank decisions, which have large time lags and are affected by external factors, making it difficult to achieve fine-grained online monitoring and continuous optimization of generation quality.

[0007] Therefore, an automated defense letter generation method that takes into account semantic quality, compliance, and engineering efficiency is needed, which can improve coverage and reduce labor costs while ensuring factual accuracy. SUMMARY

[0008] The purpose of the present application is to provide a refusal defense letter automatic generation system and method based on a hybrid workflow, which combines Non-Agentic data cleaning pipeline and Agentic multi-role cooperation to realize the standardization of multi-source and multi-modal interaction data, rule-based policy analysis, templated routing selection, evidence-based defense document generation, and automatic quality inspection, thereby improving the success rate of defense and system throughput efficiency while ensuring compliance and factual accuracy.

[0009] The technical solution adopted by the present application is: a refusal defense document automatic generation method based on multi-agent cooperation, which comprises the following steps:

[0010] Step 1, collect and pre-process the original multi-modal interaction data related to the case of refusal, process into text form to obtain the original interaction text and store in the database;

[0011] Step 2, adopt the Non-Agentic data cleaning pipeline to clean the original interaction text in the database, and obtain the consistent semantic intermediate representation, the processing nodes of cleaning processing include: rewriting, translation, meaning judgment and key information extraction based on large language model;

[0012] Step 3, generate the defense reason based on the structured input by the multi-agent Agent collaborative defense framework to obtain the defense document of the current case of refusal;

[0013] Among them, the multi-agent Agent collaborative defense framework includes:

[0014] Router Agent, for matching defense templates based on structured input, determining the generation of highlights and outputting template number and prompts; wherein the structured input is the semantic intermediate representation obtained in step 2;

[0015] Policy Agent, for retrieving and analyzing the business policy applicable to the current case of refusal, and generating a policy position summary as a rule basis;

[0016] Defense Agent, then in the first person perspective of the merchant, according to the template number and prompts output by Router Agent, the rule basis output by Policy Agent, the semantic intermediate representation obtained by cleaning processing, and the original interaction text, generates a complete defense reason text and evidence list;

[0017] Judge Agent, for fact and compliance double-checking of the outputs of Router Agent and Defense Agent, to form a closed-loop quality inspection process of "generation-proofing-regeneration".

[0018] Further, the original multi-modal interaction data includes: online customer service chat, telephone call summary, email correspondence, log and structured order information.

[0019] Further, during cleaning processing, the output format is checked for legality and the back-off retry strategy is adopted at each processing node.

[0020] Further, step 2 specifically includes:

[0021] Step 201, the original interactive text input is preprocessed based on a large language model, and a uniform natural language form interactive text is output; wherein the content rewriting of the preprocessing model includes: identifying the dialogue object, deleting the mood word and the pause, and processing the dialogue content based on the format unification;

[0022] Step 202, the output of the preprocessing model is uniformly translated by a language recognition and translation model to obtain interactive text in a unified language environment;

[0023] Step 203, based on the large language model, the interactive text obtained in step 202 is event-level segmented and intent-recognized to generate an event sequence representation organized by time axis to obtain a structured event sequence;

[0024] Step 204, based on the large language model, the structured event sequence is subjected to key field extraction to form a structured intermediate representation conforming to a predefined JSON mode, and an input of a multi-agent Agent collaborative argumentation framework is obtained.

[0025] Further, step 203 includes: dividing the interactive text into a plurality of semantic event segments, labeling the time sequence, interactive role and semantic intent of each event segment, and generating an event sequence representation organized by time axis.

[0026] Further, in step 204, the objects corresponding to the key fields include: order time, amount, commodity / service state, user behavior segment, evidence index and verification behavior.

[0027] Further, in step 3, the Router Agent outputs the template number and the prompt of the routing strategy, which is set as: first checking whether there is interactive data in the current refusal case; if yes, directly checking the order state of the order corresponding to the current refusal case; if no, checking the order state of the order corresponding to the current refusal case based on the corresponding template without interaction; if the order state is canceled, checking whether to refund, and respectively matching the corresponding argumentation template based on whether to refund; if the order state is not canceled, checking the use state, and respectively checking the refund situation under the used and unused states, and then respectively matching the corresponding argumentation template based on whether to refund.

[0028] Further, in step 3, the double verification of the Router Agent by the Judge Agent includes: checking the format and legality of the routing result of the output template number and prompt; based on the preset verification key field, the verification key field of the order corresponding to the current refusal case is checked by calling the retrieval or query tool again; when the routing result check fails, triggering the Router Agent to output the template number and the prompt based on the routing strategy; or marking the current refusal case as manual review.

[0029] Further, in step 3, the double-check of the Judge Agent to the Defense Agent includes: checking whether the generated defense reason text appears the problem exposed in the existing bad case; reviewing the defense reason text and the evidence list from the structure and style level; when the Judge Agent finds an exception, outputting modification suggestions for the exception or rejecting the current defense reason text and the evidence list, triggering the Defense Agent to regenerate or supplement the content; when the exception reaches the preset manual review condition, returning the current rejected case to manual review.

[0030] Further, the problem exposed in the existing bad case includes: narrative perspective dislocation, assumed fictional evidence, tool call error information, and inconsistent statement content and transaction facts.

[0031] Further, reviewing the defense reason text and the evidence list from the structure and style level includes: whether the key controversial points and key evidence are completely covered, whether the business compliance requirements are followed, and whether the language maintains a unified merchant first-person perspective.

[0032] The technical solution provided by the present application at least brings the following beneficial effects:

[0033] The present application significantly improves the processing efficiency of the defense document automatic generation, improves the system throughput efficiency, and improves the defense success rate of the generated defense document. Based on the structured layering, responsibility constraint and parallel scheduling, the present application overcomes the key defects of "input noise leading to illusion", "single LLM responsibility overload" and "high end-to-end delay" in the prior art, so that the defense document automatic generation system based on the method of the present application not only achieves significant improvement in compliance and fact accuracy, but also embodies practical advantages in business generalizability and engineering efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0035] Figure 1 It is a whole system architecture diagram, showing the data and control flow between the Non-Agentic data cleaning pipeline and the Agentic multi-Agent collaborative module.

[0036] Figure 2Figure 1 is a schematic diagram of a data cleaning pipeline process for a non-agentic workflow.

[0037] Figure 3 Figure 2 is a schematic diagram of a multi-agent collaborative framework, showing the interaction between four types of agents, namely, Router, Policy, Defense, and Judge, and the interaction between the dispatcher and the tool system.

[0038] Figure 4 Figure 3 is a schematic diagram of a path matching process for a Router Agent.

[0039] Figure 5 Figure 4 is a comparison chart of the total time consumption of single-thread and multi-thread under different materials.

[0040] Figure 6 Figure 5 is a comparison chart of the time consumption of each step of impulse_clean.

[0041] Figure 7 Figure 6 is a comparison chart of the time consumption of each step of Email_clean. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not only to limit the scope of the claimed present application, but only to represent selected embodiments of the present application.

[0043] The embodiments of the present application propose a method for generating a refusal defense document based on multi-agent collaboration, which is a defense document generation scheme combining non-agentic workflow and multi-agent collaborative framework, as shown in Figure 1 Figure 1. Among them, the non-agentic workflow is oriented to the data processing stage, and automatically completes the rewriting, translation, intention extraction and key information extraction of interactive records in a pipeline form. The Agentic Workflow is oriented to the defense reason generation stage, and through the high degree of collaboration of four or more functional Agents, namely, Router, Judge, Policy, and Defense, completes the selection of defense templates, the retrieval of evidence, the interpretation of policies, and the generation of final documents.

[0044] In the method of the embodiment of the present application, the technical solutions adopted include: a data cleaning pipeline (Non-Agentic Workflow): sequentially performing data processing steps such as content rewriting, multilingual translation, event-level segmentation, and key information extraction on original interaction information (including but not limited to online customer service chat, telephone call summary, email correspondence, log, and structured order information), outputting a unified structured intermediate representation, and performing legality verification and backtracking retry strategy on the output format at each node, as shown in Figure 2 A multi-agent Agent collaborative framework (Agentic Workflow): after data cleaning, the system is collaboratively worked by at least four types of functional Agents: the role of Router Agent is to match the most suitable defense template based on the structured input, determine the generated key points, and output the template number and prompt; Policy Agent is used to retrieve and analyze the business policy (refund and change rules, channel differentiation rules) applicable to the order and channel, generate a policy stance summary as a rule basis; Defense Agent generates a complete defense reason text and evidence list in the first person perspective of the merchant based on the template skeleton provided by Router, the rule basis provided by Policy, and the structured evidence related to the order; Judge Agent checks the output of Router and Defense for factual and compliance, determines whether there is perspective dislocation, illusion statement or tool abnormal information, and triggers re-routing, output rejection or backtracking manual review when necessary, forming a closed-loop quality inspection process of "generation-inspection-re-generation", as shown in Figure 3 .

[0045] In addition, the method of the embodiment of the present application also adopts parallel and batch scheduling to improve efficiency. In the data cleaning stage, multi-thread / small batch concurrent calling strategy and order-level pipeline parallel scheduling are adopted to improve resource utilization of model calling, realize batch parallel processing of time-consuming steps such as dialogue rewriting, and significantly reduce end-to-end latency.

[0046] In the evaluation method, the method of the embodiment of the present application adopts an automated semantic evaluation system. An offline evaluation set is constructed based on artificial successful defense samples, a special large language model LLM scorer is used to score the generated defense reason and the artificial reason in three dimensions of "core claim coincidence degree", "fact description completeness" and "conclusion consistency" in a specified score range (for example, 1-5 points), and output a structured gap analysis, and form a semi-automatic evaluation loop with manual sampling.

[0047] In one embodiment, the method for generating a refusal defense document based on multi-agent collaboration proposed by the embodiments of the present application adopts a Non-Agentic data cleaning pipeline combined with a multi-agent collaboration framework. The overall operation process includes the following steps:

[0048] Step 1: Collect and preprocess the original multi-modal interaction data related to the refusal case, and process it into text form stored in the database.

[0049] Step 2: Cleanse using Non-Agentic data cleaning pipeline. This step aims to uniformly cleanse the multi-source, multi-modal, and format-uniform refusal-related data, so that the subsequent generation module can receive consistent format and clear semantic intermediate representation. Referring to Figure 2 , this step includes the following sub-steps:

[0050] Step 201. Rewrite the content of the original interaction text. Input the multi-source data from customer service dialogues, emails, call summaries, etc. into the preprocessing model to normalize the text, annotate the speaker, filter noise, and format the text, so that the original unstructured text is converted into a unified natural language form.

[0051] Step 202. Translate the multi-language data uniformly. For detected multi-language mixed text, call the language recognition and translation model to uniformly translate non-target language segments into the target language, so that the subsequent steps can complete event recognition and evidence information extraction in a unified language environment.

[0052] Step 203. Perform event-level segmentation and intent recognition on the interaction text. Divide the long dialogue into several semantic event segments, annotate the time sequence, interaction role, and semantic intent of each event segment, and generate an event sequence representation organized by time axis, i.e. obtain a structured event sequence.

[0053] Step 204. Extract key fields from the structured event sequence. According to the field definition required by the task, extract the order time, amount, product / service status, user behavior segment, evidence index, verification behavior, etc. core information from the event segment, and form a structured intermediate representation conforming to the pre-defined JSON schema.

[0054] After the above sub-steps, the original refusal task is converted into a structured, language-unified, and semantically clear data object, which serves as the input for the subsequent Agent collaboration module.

[0055] Step 3: Generate defense reasons based on structured input by multi-Agent collaboration defense framework. This step includes the following sub-steps:

[0056] Step 301. The task scheduler serves as the unified entrance of the system, responsible for receiving the chargeback cases from the platform risk control system, and performing basic field parsing, case standardization, and task distribution. That is, after receiving the chargeback case, the task scheduler first calls the response processing module / processing device to perform steps 1 and 2, so as to obtain the consistent semantic intermediate representation of the original interaction text related to the current chargeback case, and obtain the structured input. The scheduler does not participate in decision-making, but pushes the case as an independent task to the routing and strategy layer, so that the entire generation process realizes "input unification and execution loose coupling".

[0057] Step 302. In the routing and strategy layer, the Router Agent and the Policy Agent jointly determine "what defense path should the system take, what argument template should be used, and which policy rules should be called". The Router Agent receives the standardized case input from the task scheduler, including order source, business line (such as subway / train / air ticket), dispute information, transaction amount, user interaction summary, etc. It completes two core tasks by calling tools:

[0058] (1) Match the defense structure template most suitable for the current chargeback type.

[0059] (2) Determine the generation focus and clearly whether to emphasize user responsibility, platform compliance, service delivery certificate, etc. The Agent calls internal rules and retrieval tools according to the routing strategy shown in Figure 4 , compares with the pre-deposited defense template library, and outputs the template number and related prompts. That is, first check if there is interaction, if yes, directly check the order status; if not, check the order status based on the corresponding template without interaction; if the order status is canceled, check if it is refunded, and perform corresponding operation processing based on whether it is refunded; if the order status is not canceled, check the use status, and check the refund situation under the used and unused states respectively. The template encodes the common argument structure and key concerns of different types of chargeback cases, providing a "skeleton" for subsequent defense reason generation. The Policy Agent focuses on the explanation of business policies, refund and change rules, and channel differentiation rules. For each order, it needs to identify user demands and dispute focuses from structured interaction information, and clearly define platform responsibility boundaries and user obligations in combination with applicable refund and change policies at the time. Refund and change policies usually contain complex time conditions, ticket restrictions, and channel differences in actual business, so processing them by a dedicated Agent helps to reduce illusion and improve consistency. The output of the Policy Agent can be regarded as a "policy position summary", providing a rule basis for Defense Agent to write defense reasons. The Router Agent completes the defense template matching and generation task routing

[0060] Step 303. In the generation layer, the Defense Agent is the final argument generation producer. It relies on the template framework of the Router Agent, the policy stance summary of the Policy Agent, the structured interactive materials provided by the data cleaning pipeline, and the evidence and data returned by tool calls, to write the argument as required. Among them, the evidence and data returned by the tool call are embodied as follows: for a certain business process, when the evidence involved needs corresponding data to further characterize, the corresponding data processing tool can be called to obtain the value of the data. For example, for a ticket refund, when the passenger's time and the flight departure time are involved in the argument, a function needs to be called to calculate the data of the passenger's time and the flight departure time, which will be used as argument proof materials. In the present application, the Defense Agent is only responsible for "writing" and not for all reasoning, making the generation process more stable and controllable.

[0061] Step 304. In the quality inspection layer, the Judge Agent mainly plays the role of "quality inspection and supervision", and checks the facts and compliance of the outputs of the Router Agent and the Defense Agent. For the Router Agent, on the one hand, it checks the format and legality of the routing results, such as whether the template number exists and whether the order type matches the order attributes; on the other hand, it can check the key information of the order by calling the retrieval or query tool again, to avoid the deviation of the overall argument direction caused by routing errors. Once the Judge Agent considers that the routing result is unreliable, it can trigger a re-routing process or mark the case for manual review. For the Defense Agent, on the one hand, it focuses on checking whether the generated argument contains the problems exposed in the existing bad cases, such as misplacement of narrative perspective (using third person or "neutral bystander" perspective), fictional evidence based on assumptions, direct exposure of tool call errors, and statements that are obviously inconsistent with transaction facts; on the other hand, it reviews the text from the structure and style aspects, checks whether it covers the core dispute points and key evidence completely, whether it follows the requirements of the card issuer and internal compliance, and whether the language maintains a unified merchant first-person perspective. If the Judge Agent finds abnormalities, it will give specific modification suggestions or reject the current result, requiring the Defense Agent to regenerate or supplement the necessary content, and if necessary, it can also return the case to manual review, thereby forming a closed-loop quality inspection mechanism of "generation - review - regeneration".

[0062] The generation effect of the method of the present application is further verified by the following examples.

[0063] In the system evaluation stage, a representative high-value subset is first identified from the overall chargeback data. This subset contains two typical scenarios: one is based on PayPal fraud reasons, historically always by manual defense; the other is the abandonment of defense cases in the business process, which is basically not invested in resources under the historical strategy of manual. The two types of cases account for about 23% of the total chargeback transactions during the evaluation period, and have high business value and evaluation significance.

[0064] The online experiment period is 7 consecutive days. During this period, the defense system implemented by the multi-agent framework based on the method of the embodiment of the application automatically generates 121 defense letters, of which 119 pass the internal operation audit and are successfully delivered to the card issuer, with a delivery pass rate of about 98%. From the order dimension, these 119 orders account for 16.5% of the total 721 chargeback orders during the evaluation period.

[0065] In the experiment of "defense success rate and its relative improvement", as shown in Table 1, the above high-value subset is further divided into two parts according to the type of chargeback and channel: one is the defense case of PayPal channel, accounting for about 20% of the subset; the other is the case covering Adyen, PayPal (acquiring side) and Worldpay channels, and is characterized by fraud and abandonment of defense by manual, accounting for about 80%. In these two scenarios, the performance of the manual historical baseline, single-agent version and multi-agent version is compared respectively. The results show that in the PayPal fraud scenario, the success rate of the manual historical baseline is 100%, the multi-agent version also reaches 100% on the current sample, while the early single-agent prototype degrades slightly, only 95%. In the "multi-channel + original abandonment of defense" subset, the success rate of the manual historical baseline (estimated based on a small amount of manual attempt samples) is 12.1%, the single-agent version is only 5.6%, and the multi-agent version restores and approaches the manual level, reaching 11.5%. If the overall average success rate is calculated according to the weight of the two scenarios 2:8, the manual historical baseline is about 29.68%, the single-agent version is 23.48%, and the multi-agent version is 29.2%. It can be seen that the single-agent architecture has obvious performance degradation under complex business constraints, and the multi-agent through the division of roles such as Router, Policy, Defense and Judge, the overall success rate reapproaches or even partially surpasses the manual baseline, about 5.7 percentage points higher than the single-agent (about 24% relative improvement), effectively verifying the value of the multi-agent design.

[0066] Table 1 Defense success rate and its relative improvement

[0067]

[0068] In the experiment of "funds recovery amount and recovery rate", as shown in Table 2, the present embodiment further quantifies the business benefits of different schemes from the perspective of funds. During the evaluation period, the total amount of the 721 refused orders corresponds to 1191.27 million yuan; among them, the high-value subset selected by the system and entered into the experiment, the refused amount is about 196.56 million yuan, accounting for 16.5% of the total amount. Based on the "coverage rate" and "average success rate" obtained in the previous experiment, the funds recovery rate is defined as:

[0069] Funds recovery rate = coverage rate x average success rate

[0070] Among them, the coverage rate represents the proportion of the actual initiation of the defense of the refused amount to the total refused amount during the evaluation period under the given scheme; the average success rate is the success rate of the defense within the coverage of the scheme. Compared with the historical baseline of manual, the coverage rate is 60%, and the success rate remains its historical average level of 29.68%, which corresponds to about 212.14 million yuan that can be recovered within 7 days.

[0071] In the actual online experiment, the automatic scheme only intervenes in 16.5% of the refused amount (i.e. the high-value subset described above). Under this premise, the coverage rate of the single-Agent version is 16.5%, the average success rate is 23.48%, and the funds recovery rate is about 3.87%, which corresponds to a recovery amount of about 46.10 million yuan within 7 days on the total amount of 1191.27 million yuan of refused amount. The multi-Agent version also covers 16.5% of the refused amount, but relies on a higher success rate (29.2%), and the funds recovery rate increases to 4.82%, which corresponds to a recovery amount of about 57.42 million yuan, which is about 11.01 million yuan more than the single-Agent version, with an increase of about 24.56%.

[0072] It should be emphasized that the current multi-Agent system only pilots on the high-value subset, and the coverage rate is controlled by human strategy; if the same success rate level is extended to a larger range, the theoretical funds recovery capacity will increase linearly with the coverage range. Even under the existing limited coverage of 16.5%, the multi-Agent version can already bring significant incremental funds recovery without increasing manpower, and has more prominent business benefits and promotion potential than the single-Agent scheme.

[0073] Table 2 Funds recovery amount and recovery rate

[0074]

[0075] In the "Large Model-based Automated Semantic Evaluation System" experiment, for each sample with a human work answer, the present embodiment first lets the argument system generate several candidate argument reasons in a multi-agent configuration. Then, a dedicated LLM scorer is called to analyze the semantic differences between "model-generated reasons (GENERATED_REASON)" and "manual reasons (MANUAL_REASON)". The core idea of the scoring prompt is to treat manual reasons as expert standards and require the model to score the generated content from three dimensions: first, core argument overlap, that is, whether the generated reason grasps the defense core and argument direction consistent with the manual reason; second, fact description completeness, focusing on whether the generated reason covers the key information related to the core argument, such as order placement and travel time, device fingerprint, commonly used delivery address, consistent email / phone number, payment channel, and verification path; third, conclusion consistency, that is, whether the judgment of responsibility attribution and treatment suggestion in the generated reason is logically consistent with the manual reason.

[0076] From the 60 randomly selected human successful argument samples, senior operations and automated evaluation LLMs scored them on the "core argument overlap, fact description completeness, and conclusion consistency" dimensions on a scale of 1-5, as shown in Table 3. The results (as shown in Tables 3 and 4) show that the average human scores on the three dimensions were 4.14, 3.99, and 4.42, respectively, indicating that the reference samples had high overall quality; the automated scores were 3.33, 2.59, and 3.77, respectively, which were slightly lower overall, especially in the fact completeness dimension. In terms of correlation, the Pearson correlation coefficients of the three dimensions were 0.73, 0.61, and 0.72, respectively, indicating that there was a significant positive correlation between automated scoring and human subjective scoring, especially in the core argument and conclusion consistency dimensions, where automated evaluation could better reproduce human judgment. Overall, this automated evaluation system can provide stable and consistent semantic quality scores for large-scale argument samples without the need for extensive human annotation, and through structured "missing points / redundant points" feedback, it can help quickly locate systematic problems in evidence selection, argument chain, and expression style. The high consistency between senior operations and human scoring indicates that this LLM-based scoring mechanism can serve as a "trusted approximation" of human evaluation.

[0077] Table 3 Average scores of three dimensions and human-machine correlation

[0078]

[0079] Table 4 Mean comparison chart (average scores of three dimensions)

[0080]

[0081] In the “multi-threading efficiency improvement” experiment, the time consumption of the pre- and post-optimization was recorded on two typical materials (impulse_clean dialogue class and Email_clean email class), as shown in Figure 5 For impulse_clean, as shown in Figure 6 , the total time consumption of the serial version was 4.50 hours, of which 3.06 hours (68.0%) was for dialogue rewriting, 0.98 hours (21.8%) was for content translation, 0.36 hours (7.9%) was for meaning judgment, and 0.10 hours (2.2%) was for key information extraction. After introducing multi-threading and batch scheduling, the total time consumption decreased to 1.26 hours, of which 0.79 hours (62.8%) was for dialogue rewriting, 0.24 hours (18.8%) was for content translation, and 0.15 hours (11.7%) was for meaning judgment. The overall speedup was about 3.57 times.

[0082] On the Email_clean email class material, as shown in Figure 7 , the total time consumption of the serial version was 3.58 hours, of which 1.79 hours (50.0%) was for content translation, 0.32 hours (8.9%) was for email rewriting, and 1.00 hours (27.8%) was for meaning judgment. The multi-threading version compressed the total time consumption to 0.82 hours, of which 0.42 hours (51.4%) was for content translation, 0.05 hours (6.1%) was for email rewriting, and 0.12 hours (15.2%) was for meaning judgment, with an end-to-end speedup of about 4.37 times. Combined with the 6.28-hour baseline of full data processing, these results show that by introducing multi-threading and batch scheduling in the data cleaning stage without changing the model size and parameter configuration, an end-to-end throughput improvement of 3-5 times can be achieved, laying a necessary engineering foundation for subsequent multi-Agent argument generation.

[0083] After reconstruction, the end-to-end time consumption is measured again under the same order scale and the same model configuration. The experimental results show that the overall pipeline time consumption is reduced from the original 22618.90 seconds to about one fifth of the original, achieving about 5.3 times efficiency improvement, while the functions and output quality of each stage remain unchanged. In terms of time distribution, the dialogue rewriting stage still dominates, but its absolute time consumption is greatly shortened, and the relative proportions of other stages such as translation, meaning judgment and key information extraction are slightly improved, and the resource utilization of the overall pipeline is more balanced. From an engineering perspective, this result shows that: under the premise of keeping the model architecture and prompt design unchanged, only by optimizing the calling method and task arrangement, the system delay can be significantly reduced, and the automated argumentation and semantic evaluation module can have the processing capacity required for landing in a production environment. Future optimization directions can further focus on the dialogue rewriting stage itself, for example, under the premise of keeping the key semantics intact, introducing a more lightweight rewriting model, or sampling and chunking long dialogues to reduce the computational load of single rewriting from the source, complementing the current parallel scheduling strategy, thereby continuing to compress the end-to-end processing time without sacrificing the quality of evaluation and argumentation.

[0084] In a real business 7-day online test, the method proposed by the embodiment of the application achieves an argumentation success rate of about 29.2% on the selected high-value subset, which has basically approached the historical baseline of experienced manual work (29.68%), and the overall success rate has increased by about 5.7 percentage points and relatively increased by about 24% compared with the single Agent version, and about 574,200 yuan has been recovered in 7 days, which is about 110,100 yuan more than the single Agent; for engineering performance, by introducing multi-threading and small batch parallel scheduling in the Non-Agentic pipeline, the end-to-end processing time is reduced from about 6.3 hours to about 1.2 hours, with an overall speedup of about 5.3 times, thereby balancing the semantic quality and engineering throughput without changing the model size and prompt design. These structured layers, duty constraints and parallel scheduling together overcome the key defects of "input noise leading to illusion", "single LLM responsibility overload" and "high end-to-end latency" in the background technology, so that the argumentation text generation system of the present application not only achieves significant improvement in compliance and factual accuracy, but also embodies practical advantages in business generalizability and engineering efficiency.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatically generating non-payment defense documents based on multi-agent collaboration, characterized in that, Includes the following steps: Step 1: Collect and preprocess the original multimodal interaction data related to the non-payment case, process it into text form to obtain the original interaction text and store it in the database; Step 2: The original interactive text in the database is cleaned using a non-Agentic data cleaning pipeline to obtain a semantic intermediate representation with consistent format. The cleaning process includes the following nodes in sequence: rewriting based on a large language model, translation, meaning judgment and key information extraction. Step 3: Based on the structured input, the multi-agent collaborative defense framework generates defense arguments to obtain the defense document for the current non-payment case; The multi-agent collaborative defense framework includes: The Router Agent is used to match the defense template based on the structured input, determine the key points of the generation, and output the template number and prompts; wherein, the structured input is the semantic intermediate representation obtained in step 2; Policy Agent is used to retrieve and parse business policies applicable to the current chargeback case, and generate a summary of policy positions as the basis for rules. The Defense Agent, from the merchant's first-person perspective, generates a complete defense text and evidence list based on the template number and prompts output by the Router Agent, the rules output by the Policy Agent, the semantic intermediate representation obtained through cleaning and processing, and the original interactive text. Judge Agent is used to perform factual and compliance verification on the outputs of Router Agent and Defense Agent to form a closed-loop quality inspection process of "generation-review-regeneration".

2. The method as described in claim 1, characterized in that, The raw multimodal interaction data includes: online customer service chat, telephone call summaries, email correspondence, logs, and structured order information.

3. The method as described in claim 1, characterized in that, During the cleaning process, the output format is validated and a rollback and retry strategy is implemented at each processing node.

4. The method as described in claim 1, characterized in that, Step 2 specifically includes: Step 201: Rewrite the content of the original interactive text input based on the preprocessing model of the large language model, and output the interactive text in a unified natural language form; wherein, the content rewriting of the preprocessing model includes: identifying the dialogue object, deleting interjections and pauses, and processing the simplified dialogue content based on the unified format. Step 202: Using a language recognition and translation model, the output of the preprocessing model is translated into a unified language data to obtain interactive text in a unified language environment. Step 203: Based on the large language model, perform event-level segmentation and intent recognition on the interactive text obtained in step 202 to generate an event sequence representation organized by the time axis, so as to obtain a structured event sequence. Step 204: Based on the large language model, key fields are extracted from the structured event sequence to form a structured intermediate representation that conforms to the predefined JSON pattern, thus obtaining the input of the multi-agent collaborative defense framework.

5. The method as described in claim 4, characterized in that, Step 203 includes: segmenting the interactive text into several semantic event fragments, labeling the time sequence, interactive roles and semantic intent of each event fragment, and generating an event sequence representation organized by the timeline.

6. The method as described in claim 4, characterized in that, In step 204, the objects corresponding to the key fields include: order time, amount, product / service status, user behavior fragments, evidence index, and verification behavior.

7. The method as described in claim 1, characterized in that, In step 3, the routing strategy for the Router Agent output template number and prompts is set as follows: First, check if there is any interaction data for the current chargeback case; if so, directly check the order status of the order corresponding to the current chargeback case. If not, the order status of the order corresponding to the current chargeback case will be viewed based on the corresponding template without interaction. If the order status is cancelled, check whether a refund has been issued, and match the corresponding defense template based on whether a refund has been issued; If the order status is not cancelled, check the usage status, and then check the refund status for both used and unused statuses. Finally, match the corresponding defense template based on whether a refund has been issued.

8. The method as described in claim 1, characterized in that, In step 3, the Judge Agent's dual verification of the Router Agent includes: checking the format and validity of the output template number and the routing result prompt; based on the preset key verification fields, sampling and verifying the key verification fields of the order corresponding to the current chargeback case by calling the retrieval or query tool again; when the routing result check fails, triggering the Router Agent to re-output the template number and prompt based on the routing policy; or marking the current chargeback case as requiring manual review.

9. The method as described in claim 1, characterized in that, In step 3, the Judge Agent's dual verification of the Defense Agent includes: checking whether the generated defense text contains problems exposed in existing bad cases; reviewing the defense text and evidence list from a structural and stylistic perspective; when the Judge Agent finds an anomaly, it outputs modification suggestions for the anomaly or rejects the current defense text and evidence list, triggering the Defense Agent to regenerate or supplement the content; when the anomaly meets the preset conditions for manual review, the current non-payment case is rolled back to manual review.

10. The method as described in claim 9, characterized in that, Problems exposed in existing bad cases include: misaligned narrative perspective, fabricated hypothetical evidence, incorrect information in the use of tools, and discrepancies between the statements and the facts of the transaction.

Images