Financial text abstract optimization method and device based on multi-dimensional structured feedback
By constructing a multi-dimensional structured feedback mechanism and jointly generating fine-tuning a large language model, the problem of weak structured understanding ability in financial text summarization is solved, achieving efficient and interpretable financial text summarization optimization, which is applicable to scenarios such as re-guarantee assessment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUHAN UNIV OF SCI & TECH
- Filing Date
- 2026-02-13
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies for generating financial text summaries suffer from weak structured understanding capabilities and a lack of interpretable optimization paths, resulting in low generation efficiency and unstable quality. This makes it difficult to meet the financial sector's requirements for factual accuracy, rigorous business logic, and complete risk coverage.
A multi-dimensional structured feedback mechanism is constructed. By performing structured analysis on financial documents, a quality evaluation system is designed that includes factual accuracy, business logic, risk coverage, and information conciseness. Structured feedback is generated, and joint generative fine-tuning is performed using a large language model to optimize the quality of the summary.
It significantly improves the accuracy, completeness, and interpretability of financial text summaries, reduces the cost of manual annotation, and enhances the efficiency and consistency of financial document processing. It is suitable for business scenarios such as reinsurance assessment and risk review.
Smart Images
Figure CN121901415A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of natural language processing and artificial intelligence, and more specifically, to a method and apparatus for optimizing financial text summarization based on multi-dimensional structured feedback. Background Technology
[0002] With the widespread application of large language models in the financial industry, the demand for automated processing of professional texts such as reinsurance assessment reports, risk assessment documents, and financial review materials is constantly increasing. However, financial institutions currently still mainly rely on manual methods or traditional supervised fine-tuning models for summary generation, which suffers from problems such as low summarization efficiency, strong reliance on professional knowledge, and difficulty in ensuring output quality. In particular, reinsurance documents are generally complex in structure, involving key information such as the division of guarantee responsibilities, risk exposure structure, asset status, project background, and key points of credit decision-making. Manually generating summaries is not only time-consuming and labor-intensive, but also difficult to maintain consistency.
[0003] Existing summarization algorithms mostly employ supervised fine-tuning (SFT) or preference optimization methods (such as RLHF and DPO), typically relying on static labeled data or single-dimensional preference signals. However, financial scenarios have extremely high requirements for factual accuracy, rigorous business logic, and complete risk coverage. Traditional methods struggle to capture the cross-paragraph logical relationships in reinsurance assessment reports, easily leading to factual errors, omissions of risk information, and improper citation of indicators. Furthermore, the number of professionals in the financial field is limited, and constructing high-quality training data is extremely costly, making it difficult to meet the needs of large-scale annotation.
[0004] In terms of text quality assessment, existing automatic assessment methods (such as BLEU, ROUGE, etc.) can only measure the degree of surface text matching and cannot identify professional issues such as business logic errors, insufficient risk explanations, or inaccurate summaries of key clauses. Although some automatic assessment methods based on large models have certain capabilities, they lack structured and interpretable assessment dimensions, and the assessment results often lack traceability or error correction guidance, making it difficult to use them directly for model optimization.
[0005] It is evident that existing summary generation methods suffer from technical problems such as weak structured understanding capabilities and a lack of interpretable optimization paths. Summary of the Invention
[0006] Analysis of existing technologies reveals a lack of a method for automatically generating high-quality, interpretable, and multi-dimensional feedback data to improve the reliability of large language models in summarizing reinsurance and related financial documents at low cost. Therefore, a technical solution is needed that adapts to the characteristics of such financial texts, reduces manual annotation costs, clearly identifies summarization errors and deficiencies, and provides the model with a directly learnable, structured improvement path to achieve intelligent, structured, and high-precision optimization of financial summarization tasks.
[0007] The purpose of this invention is to propose an intelligent solution capable of automatically generating structured, multi-dimensional feedback and optimizing summary quality based on that feedback. This solution aims to meet the summary generation needs of professional financial documents such as reinsurance assessment reports, improving the accuracy, completeness, and business credibility of the summaries. Specifically, this invention first performs structured analysis on the financial documents, constructing a quality evaluation system encompassing dimensions such as factual accuracy, business logic, risk coverage, information conciseness, and decision-making value, and generating corresponding structured feedback formats. Then, the original text is obtained and an initial summary is generated. Semantic alignment analysis, multi-dimensional quality assessment, and chain-reasoning interpretation are performed through a joint input of original text and summary, resulting in structured feedback including scores, error location, and modification suggestions. Based on this, an enhanced summary is generated, and a training sample of "original text—initial summary—structured feedback—enhanced summary" is constructed. A joint generative fine-tuning of the large language model is then performed to improve the accuracy, consistency, and interpretability of the financial text summaries. The method of this invention can automatically identify summary quality issues and perform targeted optimization, making it suitable for summarizing professional financial documents such as reinsurance assessment reports.
[0008] To achieve the above objectives, the first aspect of the present invention provides a financial text summarization optimization method based on multi-dimensional structured feedback, comprising: Conduct structured analysis on financial documents, construct a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness, and decision value, and generate corresponding structured feedback as a summary evaluation template; Obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary; The original text and initial summary of the reinsurance assessment report are input into the automatic evaluator to perform a multi-dimensional quality assessment of the initial summary and generate multi-dimensional structured feedback. The automatic evaluator is built based on the summary evaluation template and the basic language model. Based on the generated multi-dimensional structured feedback, the initial summary is modified in a targeted manner to generate an enhanced summary, and a training sample with the structure of: original text - initial summary - structured feedback - enhanced summary is constructed. The basic language model is fine-tuned by executing instructions using training samples, enabling the model to learn the summary quality diagnostic logic and the corresponding error correction path; The optimized summary of the re-guarantee assessment report text is generated using the fine-tuned model.
[0009] In one implementation, financial documents undergo structured analysis to construct a quality evaluation system encompassing dimensions such as factual accuracy, business logic, risk coverage, information conciseness, and decision-making value, and to generate corresponding structured feedback formats, including: Perform structured analysis on financial documents; The evaluation dimensions include factual accuracy, business logic, risk coverage, information conciseness, and decision value. Set dimensional scoring rules; For each dimension, a chain-like reasoning template is designed. The template format includes: reading the key content of the original text, comparing with the initial summary, pointing out inconsistencies, judging whether the issue affects financial decisions, and providing reasons and improvement suggestions. The corresponding structured feedback is generated, and the fields of the structured feedback include evaluation dimensions, scores, chain reasoning, improvement suggestions, and the relevant original text excerpts.
[0010] In one implementation, the original text of the reinsurance assessment report and the initial summary are input into an automated assessor to perform a multi-dimensional quality assessment of the initial summary and generate multi-dimensional structured feedback, including: The original text and initial summary of the reinsurance assessment report are organized into an assessment input text in a structured format, which includes the original text content field, the initial summary field, and the task description field. Semantic alignment analysis was performed on the original text and initial summary of the reinsurance assessment report; Based on a multi-dimensional quality evaluation system, the initial summary is analyzed and scored from multiple business-related dimensions, and structured evaluation results are output. Generate chain reasoning to illustrate the thought process that led to the score.
[0011] In one implementation, semantic alignment analysis is performed on the original text and initial summary of the reinsurance assessment report, including: Perform overall semantic alignment between the original text and the initial summary to identify the correspondence between the two in terms of main information, guarantee liability structure, key financial indicators, risk descriptions and review conclusions; By reading the original text to extract key business points and comparing them with the initial summary to determine whether the summary omits key information, contains factual errors, or has deviations in relational descriptions.
[0012] In one implementation, the initial summary is specifically revised based on the generated multi-dimensional structured feedback to generate an enhanced summary, including: Based on the generated multi-dimensional structured feedback, generate one or more modification suggestions for the initial summary for each dimension, including risk information that needs to be added, erroneous data that needs to be corrected, or redundant content that needs to be deleted. Based on all the suggested revisions, an enhanced summary was generated from the original text.
[0013] In one implementation, after enhancing the summary, the method further includes: The structured feedback includes information on the scope of the original text citations, which identifies the original text paragraphs on which the current score and recommendations are based.
[0014] In one implementation, fine-tuning the basic language model using training samples includes: The instruction text used for model fine-tuning is standardized. The instruction template includes task description, original text, initial summary content, and structured feedback objectives. A position labeling mechanism is introduced into the instruction template so that the model can clearly refer to each dimension in the feedback and its corresponding improvement objectives. The base model is fine-tuned by executing instructions, and a loss function for joint generation of structured feedback is introduced. The base language model is then fine-tuned by minimizing this joint generation loss function, enabling the model to simultaneously generate structured feedback and enhanced summaries. The loss function is as follows:
[0015] in, Indicates model parameters, For input documents, The initial summary is, The structured feedback generated by the self-evaluator Enhanced summary This is used to measure the model's ability to generate structured feedback and enhanced summaries under given input conditions.
[0016] Based on the same inventive concept, a second aspect of the present invention provides a financial text summarization optimization device based on multi-dimensional structured feedback, comprising: The quality evaluation system construction module is used to perform structured analysis on financial documents, build a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness, and decision value, and generate corresponding structured feedback as a summary evaluation template. The initial summary generation module is used to obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary; A multi-dimensional structured feedback generation module is used to input the original text and initial summary of the reinsurance assessment report into an automatic evaluator, perform multi-dimensional quality assessment on the initial summary, and generate multi-dimensional structured feedback. The automatic evaluator is built based on the summary evaluation template and the basic language model. The training sample construction module is used to make targeted corrections to the initial summary based on the generated multi-dimensional structured feedback, generate enhanced summaries, and construct training samples with the structure: original text - initial summary - structured feedback - enhanced summary. The model fine-tuning module is used to fine-tune the basic language model by executing instructions using training samples, enabling the model to learn the summary quality diagnostic logic and the corresponding error correction path. The summary generation module has been optimized to generate an optimized summary of the re-guarantee assessment report text using the fine-tuned model.
[0017] Based on the same inventive concept, a third aspect of the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, provides the financial text summarization optimization method based on multi-dimensional structured feedback described in the first aspect.
[0018] Based on the same inventive concept, a fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the financial text summarization optimization method based on multi-dimensional structured feedback described in the first aspect.
[0019] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows: In the intelligent processing of financial text summarization, this invention combines a complete multi-dimensional quality evaluation template with an automated structured feedback generation mechanism. This allows for the systematic identification of key information in reinsurance assessment reports, including but not limited to the scope of guarantee liability, risk exposure structure, basic project and asset information, key business points related to credit granting or decision-making, and potential risk information. Specifically, in step S1, this invention constructs a quality evaluation system encompassing dimensions such as factual accuracy, business logic, risk coverage, information conciseness, and decision value, and defines corresponding scoring rules and reasoning templates for each evaluation dimension. In step S3, the automatic evaluator performs dimension-by-dimensional analysis and reasoning on the original text and initial summary based on the evaluation template, thereby achieving fine-grained quality analysis and structured feedback generation of the summary content. Through these steps, this invention significantly improves the information extraction capability and professionalism in financial summarization tasks, reducing common content omissions and logical errors in traditional methods.
[0020] Regarding summary quality optimization, this invention utilizes a high-performance large language model to generate structured feedback with chained reasoning explanations, making summary quality assessment interpretable and traceable. Compared to methods relying on manual review or static supervised data, this invention constructs high-quality training samples through a path of "original text—initial summary—structured feedback—enhanced summary," enabling the model to learn explicit rewriting logic and optimization methods, thereby improving the model's stability and generalization ability in complex financial scenarios.
[0021] In practical applications, this invention significantly reduces the costs of manual annotation and review, and improves the efficiency and consistency of professional financial document processing. This method can be applied to various business scenarios such as re-guarantee assessment, risk assessment, compliance checks, and project due diligence, helping financial institutions improve the automation level of their text processing workflows, reduce workload, and mitigate business risks. It has significant engineering application value and promising prospects for widespread adoption.
[0022] In summary, by introducing a configurable, multi-dimensional structured feedback mechanism, this invention enables the model not only to generate summaries but also to clearly identify errors, provide reasoning basis, and complete self-correction during the generation process. Furthermore, it can flexibly adjust the evaluation and feedback methods according to different financial document types or business scenarios, thereby significantly improving the accuracy, completeness, interpretability, and business applicability of the summaries, achieving technical effects that are difficult to achieve with existing technologies. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a flowchart of a financial text summarization optimization method based on multi-dimensional structured feedback in an embodiment of the present invention; Figure 2 This is a schematic diagram of the financial document summary quality evaluation system architecture in an embodiment of the present invention; Figure 3 This is a schematic diagram of the automated structured feedback generation process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the abstract enhancement and model fine-tuning technology structure in an embodiment of the present invention; Figure 5 This is a performance comparison chart of different fine-tuning strategies on the test set in the embodiments of the present invention; Figure 6 This is a structural block diagram of the financial text summarization optimization device based on multi-dimensional structured feedback in an embodiment of the present invention. Detailed Implementation
[0025] This invention provides a method for optimizing financial text summarization based on multi-dimensional structured feedback. The method first constructs a multi-dimensional summary quality evaluation system suitable for financial documents. This system employs a configurable, templated structure, including evaluation dimensions such as factual accuracy, business logic, risk coverage, information conciseness, and decision value. Scoring rules, chain-reasoning templates, and structured feedback formats are defined for each dimension. The evaluation dimensions and their corresponding scoring rules and feedback fields are not fixed but can be selected, adjusted, or expanded according to different financial document types or business application scenarios. Next, the original reinsurance assessment report is obtained, and a basic large language model is used to generate the corresponding initial summary. Then, a more powerful large language model is used as an automatic evaluator to perform semantic alignment analysis between the original text and the initial summary, generating structured feedback with chain-reasoning explanations according to the aforementioned evaluation system. Subsequently, the original text, initial summary, and structured feedback are combined to construct training samples, and the basic language model is trained using a fine-tuning method, enabling the model to simultaneously possess summary quality evaluation and summary enhancement capabilities. Finally, the fine-tuned target model is used to generate higher-quality financial document summaries.
[0026] Example 1 This embodiment provides a financial text summarization optimization method based on multi-dimensional structured feedback. Please refer to [link to relevant documentation]. Figure 1 ,include: S1: Conduct structured analysis on financial documents, construct a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness, and decision value, and generate corresponding structured feedback as a summary evaluation template.
[0027] Specifically, the financial text summarization optimization method based on structured feedback provided in this embodiment of the invention mainly includes the following three aspects: (1) Design and construction of a quality evaluation system for financial document summaries (S1); (2) Automated structured feedback generation method based on large language model (S2 and S3); (3) Summarization enhancement and model fine-tuning techniques based on structured feedback (S4, S5 and S6).
[0028] Before executing the method of the present invention, it is first necessary to obtain the original text of the re-guarantee assessment report to be processed and generate the corresponding initial summary using the basic language model. The initial summary serves as one of the inputs to the subsequent automated structured feedback generation module and is used to trigger the summary quality diagnosis and structured error correction process.
[0029] Through the synergy of the above three parts, this invention realizes the automated generation, quality assessment and quality optimization of professional financial documents such as reinsurance assessment reports, forming an end-to-end, highly interpretable and low-manual-cost financial document summarization processing method.
[0030] S1 involves constructing a multi-dimensional quality evaluation system. After building this system, corresponding scoring rules and reasoning templates are defined for each evaluation dimension. The evaluation dimensions and their corresponding scoring rules are defined in a configurable manner, allowing for selection, adjustment, or expansion based on different financial document types or business application scenarios. The evaluation template serves as a constraint framework for subsequent summary quality assessment and structured feedback generation. The structured evaluation template (summary evaluation template) is applicable not only to reinsurance assessment reports but also to risk control reports, financial analysis reports, credit approval materials, due diligence documents, and other professional financial documents involving complex structured information.
[0031] Specifically, S1 can be implemented in the following way: S1.1: Perform structured analysis on financial documents; S1.2: Set evaluation dimensions, including factual accuracy, business logic, risk coverage, information simplicity, and decision value; S1.3: Set the dimensional scoring rules; S1.4: Design a chain reasoning template for each dimension. The template format includes: reading the key content of the original text, comparing with the initial summary, pointing out inconsistencies, judging whether the problem affects financial decision-making, and providing reasons and improvement suggestions. S1.5: Form corresponding structured feedback. The fields of the structured feedback include evaluation dimensions, scores, chain reasoning, improvement suggestions, and relevant original text excerpts.
[0032] like Figure 2As shown, this invention first conducts a structured analysis of professional documents such as reinsurance assessment reports, credit approval documents, and risk inspection materials that are frequently used within financial institutions. Combining the key review points summarized by business personnel with the relevant requirements of financial regulatory authorities regarding text standardization and risk disclosure, a multi-dimensional summary quality evaluation system for financial scenarios is constructed. In this embodiment, the summary quality evaluation system includes a document structured analysis module, a summary quality evaluation dimension design module, a scoring rule definition module, a chain reasoning template module, and a structured feedback format design module. The evaluation dimensions are used to comprehensively analyze summary quality from multiple business-related perspectives. The specific number and content of evaluation dimensions can be configured according to different financial document types and application scenarios. Through the synergistic effect of the above modules, this invention can quantitatively and qualitatively evaluate summary quality in a structured and multi-faceted manner, enabling the large language model to learn summary generation logic that better meets the requirements of professional financial scenarios during training, thereby improving the accuracy, completeness, and business applicability of the summary results.
[0033] Step S1.1 Structured Analysis of Financial Documents Financial documents typically contain the following structure, which this invention has solidified in the evaluation system through statistical analysis of a large number of samples: Basic background information: including the project entity, financing background, guarantee structure, and basic project information; Asset status and financial data: including asset and liability status, cash flow status, solvency indicators, etc.; Risk analysis section: includes historical defaults, industry risks, project risk points, guarantee liability structure, etc. Re-guarantee arrangements or risk control measures: including risk-sharing methods, collateral details, and recovery arrangements; Review conclusions include whether the review is approved and the risk assessment level.
[0034] To ensure the universality of this invention, the structure is represented in an extensible manner in the evaluation system, which can be adapted to financial documents with different structures.
[0035] Step S1.2 Design of a multi-dimensional evaluation system This invention designs a multi-dimensional evaluation system covering key quality requirements for professional financial document summaries. In this embodiment, the evaluation system includes evaluation dimensions such as factual accuracy, business logic, completeness of risk coverage, information conciseness, and decision value. (1) Accuracy of facts All data, subjects, responsibility structures, and risk points in the abstract must be consistent with the original text.
[0036] (2) Business logic The abstract should follow the common financial business logic structure and avoid logical jumps and contradictions.
[0037] (3) Completeness of risk coverage The summary should cover the core risk points in the report, such as guarantee liability, debt risk, and counter-guarantee measures.
[0038] (4) Information conciseness The abstract should remove redundant descriptions and highlight key information.
[0039] (5) Decision value The abstract should be of practical reference value to business personnel or reviewers.
[0040] Each evaluation dimension is designed in conjunction with the standards for verifying the authenticity of financial documents, making the evaluation system highly applicable to various industries. In other implementations, the evaluation dimensions can be added, removed, or adjusted according to different financial business scenarios or document types.
[0041] Step S1.3 Dimensional Scoring Rules To achieve quantitative evaluation, this invention establishes the following five-level scoring system (1–5 points), as shown in the example below: Table 1: Scoring System
[0042] Step S1.4 Design of Chain Reasoning Template To enhance interpretability, this invention designs a chain-like reasoning template for each dimension, with the following format: 1) Read the key content of the original text; 2) Compare with the initial abstract; 3) Point out the inconsistencies; 4) Determine whether the problem will affect financial decisions; 5) Provide reasons and suggestions for improvement.
[0043] This template ensures that the reasoning path output by the evaluator is traceable.
[0044] Step S1.5 Design of Structured Feedback Format To achieve standardization and trainability of the automatic evaluator output, this implementation clearly defines the constituent fields of the structured feedback, and organizes the analysis results of each evaluation dimension into a unified structured data format. Preferably, the structured feedback consists of the following fields: Table 2: Description of Structured Feedback Fields
[0045] Field meaning explanation ① Dimension (Evaluation Dimension) This field identifies the summary quality evaluation dimension corresponding to this structured feedback. In one embodiment, the evaluation dimensions include factual accuracy, business logic, risk coverage completeness, information conciseness, and decision value. Each structured feedback corresponds to only one evaluation dimension, enabling the model to distinguish and learn different types of summary questions and error correction patterns during training.
[0046] ② score (rating value) This field is used to numerically reflect the quality of the abstract in the current dimension, with a score range of 1–5. This field is primarily used for: 1) Weight calculation during the training phase 2) Quantitative basis for subsequent quality assessment 3) Determine severity when generating enhanced summaries In this embodiment, the generation strategy can be automatically adjusted based on the score.
[0047] ③ cot_reasoning (chain reasoning explanation) Used to record the complete reasoning process of the automatic evaluator when evaluating this dimension, including: 1) Comparative analysis of the original text and the abstract 2) Error source identification 3) Logical inference path 4) Key evidence affecting the judgment This field ensures that the feedback is interpretable, which is one of the key innovations that distinguishes this invention from existing technologies.
[0048] ④ Suggestion (Revision Recommendation) It provides specific suggestions that can be directly used to optimize the summary and serves as a direct supervisory signal for training the model on "how to improve the summary".
[0049] For example, "supplementary explanation of the division of guarantee liability", "delete redundant descriptions unrelated to risk", etc.
[0050] ⑤ reference_span (scope of reference in the original text) Used to record the specific locations in the original text involved in this assessment, such as "paragraph 3, paragraph 4".
[0051] This field helps the model pinpoint the problem and facilitates manual sampling and auditing.
[0052] S2: Obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary.
[0053] Specifically, S2 is the generation of the initial summary. In this step, the original text of the target re-guarantee assessment report is first obtained, and the basic large language model is called to generate the initial summary.
[0054] S3: Input the original text and initial summary of the reinsurance assessment report into the automatic evaluator to conduct a multi-dimensional quality assessment of the initial summary and generate multi-dimensional structured feedback. The automatic evaluator is built based on the summary evaluation template and the basic language model.
[0055] Specifically, S3 generates multi-dimensional structured feedback. The automatic evaluator uses a chain-of-thought approach to output scoring reasons, which is conducive to explanatory review and error localization, and improves the transparency and professionalism of the evaluation results.
[0056] This invention designs an automated structured feedback generation mechanism based on a large language model, used to assess the quality of initial summaries and generate interpretable structured feedback results. The overall process is as follows: Figure 3 As shown, this mechanism takes the original text of the reinsurance assessment report and the initial summary generated by the basic language model as input, automatically outputs multi-dimensional scores, reasoning processes, and modification suggestions according to a preset financial text evaluation template, and generates an enhanced summary based on this, providing high-quality supervision signals for subsequent model training.
[0057] Specifically, S3 can be implemented in the following way: S3.1: Organize the original text and initial summary of the reinsurance assessment report into an assessment input text in a structured format, wherein the input text includes the original text content field, the initial summary field, and the task description field; S3.2: Perform semantic alignment analysis on the original text and initial summary of the reinsurance assessment report; S3.3: Based on a multi-dimensional quality evaluation system, the initial summary is analyzed and scored from multiple business-related dimensions, and structured evaluation results are output. S3.4: Generate chain reasoning to present the thought process that led to the score.
[0058] In the specific implementation process, S3.1 constructs the evaluation input text. The original text and initial summary of the reinsurance assessment report are organized into assessment input text using a unified and structured format. Preferably, the input text includes fields for the original document content, initial summary, and task description, which clearly inform the automated assessor to perform quality analysis on the initial summary and output structured assessment results based on the multi-dimensional summary evaluation system defined in this embodiment. By unifying the input format and field definitions, the processing logic of different documents during the assessment stage can be ensured to be consistent, and a standardized basis can be provided for the subsequent generation of structured feedback.
[0059] Step S3.2: Perform semantic alignment analysis between the original text and the initial summary, specifically including: S3.2.1: Perform overall semantic alignment between the original text and the initial summary to identify the correspondence between the two in terms of main information, guarantee liability structure, key financial indicators, risk description and review conclusion; S3.2.2: Extract key business points by reading the original text and compare them with the initial summary to determine whether the summary omits key information, contains factual errors, or has deviations in relational descriptions.
[0060] In practice, the automated evaluator performs overall semantic alignment between the original text and the initial summary, identifying the correspondence between the two in terms of subject information, guarantee liability structure, key financial indicators, risk descriptions, and review conclusions. The evaluator extracts key business points from the original text and compares them with the initial summary to determine whether the summary omits key information, contains factual errors, or has deviated from the intended descriptions. This semantic alignment process provides the foundation for subsequent multi-dimensional quality assessment, ensuring that the analysis of each evaluation dimension is based on the original text-summary comparison.
[0061] Step S3.3: Conduct quality assessment according to the multi-dimensional template. After semantic alignment, the automatic evaluator, based on the financial summary quality evaluation system constructed according to this invention, analyzes and scores the initial summary from multiple business-related dimensions. In this embodiment, the evaluation dimensions include factual accuracy, business logic, completeness of risk coverage, information conciseness, and decision-making value; these dimensions can be added, removed, or adjusted according to different financial document types or business scenarios. Under each dimension, the evaluator combines the original text content and the corresponding fragment of the initial summary to determine whether the summary expression is consistent with the original text, whether it follows reasonable business logic, whether it fully covers major risk points, and whether it has decision-making reference value. The evaluation results are output in a structured form, including dimension names, scoring results, detailed analysis processes, and improvement suggestions, forming a complete evaluation record.
[0062] Step S3.4: Generate chain-reasoning explanation information In this step, the automated evaluator not only provides scores for each dimension but also generates chain-of-thought explanations to fully present the thought process that led to the score. Chain-of-thought typically includes summarizing key information from the original text, comparing relevant statements in the summary line by line, identifying errors or omissions, and analyzing potential business judgment biases caused by these issues. By recording these reasoning steps, this invention ensures that each score and suggestion has clear basis and traceability, which is beneficial for subsequent manual sampling and interpretable training of the model.
[0063] S4: Based on the generated multi-dimensional structured feedback, the initial summary is modified in a targeted manner to generate an enhanced summary, and a training sample with the structure of: original text - initial summary - structured feedback - enhanced summary is constructed.
[0064] Specifically, S4 is the construction of training samples. The constructed training samples adopt a unified instruction-input-output format, enabling the model to learn the improvement path from the initial summary to the enhanced summary, thereby achieving a summary quality improvement that is significantly better than traditional supervised methods.
[0065] Specifically, S4 includes: S4.1: Based on the generated multi-dimensional structured feedback, generate one or more modification suggestions for the initial summary for each dimension, including risk information that needs to be added, erroneous data that needs to be corrected, or redundant content that needs to be deleted; S4.2: Based on all the modification suggestions, generate an enhanced summary from the original text. S4.3: The structure is as follows: original text - initial summary - structured feedback - training samples for enhanced summary.
[0066] Specifically, the core objective of this step is to construct an "error-analysis-correction" sample system, enabling the model to observe errors in the initial summary, the logical inference process of the automatic evaluator, and the generation method of the enhanced summary during training. Through a large number of such samples, this invention allows the model to understand common quality problems in summaries of different types of financial documents, improving its ability to automatically identify and correct errors during the inference stage.
[0067] In practice, the automated evaluator, based on inference and analysis, generates one or more modification suggestions for each dimension of the initial summary. These suggestions may include identifying which risk information needs to be added, which erroneous data needs to be corrected, or which redundant content needs to be removed. Subsequently, the evaluator integrates the modification suggestions from all dimensions and regenerates an enhanced summary based on the original text. The enhanced summary improves upon the initial summary in terms of factual accuracy, logical structure, and risk coverage, more closely resembling the actual needs of financial institutions for professional summaries. This enhanced summary will serve as an important reference output for subsequent model training phases.
[0068] This invention utilizes the structured feedback results generated by an automatic evaluator to organize the original text, initial summary, and enhanced summary into training samples according to a unified format. Preferably, the training samples adopt an instruction-input-output structure, where the instruction field indicates to the model that it needs to generate structured feedback and optimized summaries based on the original text and initial summary; the input field contains the complete original text and the initial summary generated by the base model; and the output field includes the structured feedback output by the automatic evaluator and its corresponding enhanced summary. Through this unified data format, this invention enables the model to fully understand the input purpose and output target of each sample during training, thereby learning the complete path from identifying the summarization problem to performing summary correction.
[0069] In one implementation, after enhancing the summary, the method further includes: The structured feedback includes information on the scope of the original text citations, which identifies the original text paragraphs on which the current score and recommendations are based.
[0070] Specifically, to improve the auditability of the evaluation results, the automatic evaluator in this step includes original text citation range information in the structured feedback. This information identifies the original text paragraphs upon which the current score and suggestions are based, such as "paragraph 2 and paragraph 4 of the original text." The final output structured feedback includes records of multiple evaluation dimensions, corresponding chained reasoning content, modification suggestions, and enhanced summaries, all organized into a predefined data structure and used as input for the training data construction module. This invention can also include an optional manual review process to manually verify some of the structured feedback, further improving the reliability and stability of the feedback quality.
[0071] Through steps S3 and S4, the automated structured feedback generation mechanism of the present invention can automatically complete the quality assessment and correction guidance of financial document summaries without manual annotation, providing high-quality supervision signals for subsequent feedback-based summary optimization and model fine-tuning.
[0072] S5: Fine-tune the basic language model by executing instructions using training samples, enabling the model to learn the summary quality diagnostic logic and the corresponding error correction path.
[0073] Specifically, S5 is the model's instruction fine-tuning. After the automatic evaluator generates structured feedback, this invention further utilizes this feedback to construct training data and trains the basic large language model based on instruction fine-tuning technology, enabling it to automatically identify summarization defects and generate enhanced summaries. The summarization enhancement and training method of this invention is as follows: Figure 4 As shown.
[0074] Specifically, S5 includes: S5.1: Standardize the instruction text used for model fine-tuning. The instruction template includes task description, original text content, initial summary content, and structured feedback objectives. Introduce a position labeling mechanism in the instruction template so that the model can clearly refer to each dimension in the feedback and its corresponding improvement objectives. S5.2: Fine-tune the base model using instructions and introduce a loss function for joint generation of structured feedback. Fine-tune the base language model by minimizing the joint generation loss function, enabling the model to simultaneously generate structured feedback and enhanced summaries. The loss function is:
[0075] in, Indicates model parameters, For input documents, The initial summary is, The structured feedback generated by the self-evaluator Enhanced summary This is used to measure the model's ability to generate structured feedback and enhanced summaries under given input conditions.
[0076] In the specific implementation process, S5.1: a unified instruction template for fine-tuning the design model. In this step, the invention standardizes the instruction text used for model fine-tuning, ensuring consistency in input structure across different training samples. Preferably, the instruction template includes a task description, original text content, initial summary content, and structured feedback objectives, explicitly requiring the model to output a complete multi-dimensional quality assessment result after reading the original text and initial summary, and to generate an enhanced summary based on the assessment. This template design not only ensures the standardization of the model training process but also strengthens the model's adherence to predetermined analytical logic during the inference phase, improving the structure and interpretability of the generated content.
[0077] This invention further introduces a positional annotation mechanism into the instruction template, allowing the model to explicitly reference each dimension in the feedback and its corresponding improvement goals. This enables the model to learn during training how to simultaneously complete quality assessment and summary rewriting tasks under the same instruction. Since financial documents are typically complex in structure and cover multiple business points, the unified structure of this instruction template helps the model establish a universal understanding template across different documents, thereby improving the model's generalization ability in cross-scenario financial text summarization tasks.
[0078] Step S5.2: Fine-tune the basic model by executing instructions and introduce the loss function jointly generated by structured feedback. In this step, the present invention fine-tunes the basic language model by minimizing the joint generation loss function, enabling the model to simultaneously generate structured feedback and enhanced summaries. Let the input document be x, the initial summary be y, the structured feedback generated by the automatic evaluator be e, and the enhanced summary be s. The training objective is to maximize the loss function with respect to the parameters. The representation of the joint probability of the underlying language model generating structured feedback and enhanced summarization given the input conditions. The corresponding training loss function is shown below:
[0079] in, Indicates model parameters, This is used to measure the model's ability to generate structured feedback and enhanced summaries under given input conditions. Through continuous iterative training, the model can gradually learn typical error types, error sources, structured analysis methods, and feedback-based correction methods in financial summaries, enabling it to have a stronger comprehensive understanding and error correction ability during the inference stage.
[0080] After training, this invention yields a large language model for the financial field capable of automatic summary generation, automatic quality assessment, and automatic summary error correction. Upon receiving the original text, the model proactively identifies logical flaws, factual errors, or omissions of risk information in the summary and automatically adjusts the summary content based on built-in structured assessment logic, thereby generating more accurate, complete, and professional summaries that meet business review requirements. Compared to traditional supervised fine-tuning methods, the model obtained in this invention shows significant improvements in factual accuracy, risk coverage, and logical consistency, and is applicable to various financial text scenarios such as reinsurance assessment, risk review, due diligence analysis, and compliance review.
[0081] S6: Use the fine-tuned model to generate an optimized summary of the re-guarantee assessment report text.
[0082] Specifically, S6 is a concrete application of the model.
[0083] To illustrate the technical effectiveness of the financial text summarization optimization method based on multi-dimensional structured feedback proposed in this invention in practical applications, several reinsurance assessment reports are selected as test samples in this embodiment, and summaries generated directly using a basic language model and summaries generated using the method of this invention are compared and analyzed.
[0084] In the comparison process, the summary generated by the basic language model is generated only once based on the original text, while the model using the method of this invention introduces a multi-dimensional summary quality evaluation template and an automated structured feedback mechanism after generating the initial summary to diagnose and optimize the summary content.
[0085] Comparative results show that the summaries generated using the method of this invention exhibit better performance in terms of factual consistency, completeness of risk information coverage, and rationality of business logic. They reduce the omission of key information and factual errors, and the generated results better meet the review and decision-making needs of financial professionals. Therefore, the method of this invention has good practical effects in financial document summarization tasks.
[0086] This invention employs classic text generation evaluation metrics, such as BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L, to measure the lexical matching, coverage, and conciseness of the generated summary.
[0087] Zero-shot: The base model is used directly without any downstream task training.
[0088] SFT (Supervised Fine-Tuning): The model is trained to reproduce high-quality human-written summaries, which are typically completed by domain experts and strong LLMs to ensure quality and efficiency.
[0089] CFT (Critical Fine-Tuning): The model learns to critique noisy outputs instead of mimicking them. In recent CFT work, CFT, trained on a critical basis with GPT-4, delivers performance improvements over SFT on multiple mathematical benchmarks.
[0090] DPO (Direct Preference Optimization): A lightweight alternative to RLHF that directly aligns the model with human preferences using a classification loss—without requiring a reward model or reinforcement learning. In application, it matches or outperforms PPO-based RLHF in summarizing and dialogue tasks.
[0091] RLHF (Reinforcement Learning Based on Human Feedback): This method trains a reward model from human preference rankings and optimizes it using PPO (Personal Feedback Objective). It is very effective in text generation tasks, but requires a large amount of manually labeled data and computational costs.
[0092] Figure 5 Experimental results demonstrate that, in the financial reinsurance summarization task, the proposed method significantly outperforms traditional supervised fine-tuning (SFT) on multiple metrics. Notably, the ROUGE-L score increased from 24.99 in the zero-sample setting to 31.78, and the BLEU-4 score increased from 17.09 to 23.69. These results strongly validate the effectiveness of multi-dimensional feedback mechanisms in improving the quality and accuracy of language model-generated summaries.
[0093] Example 2 Based on the same inventive concept, this embodiment discloses a financial text summarization optimization device based on multi-dimensional structured feedback. Please refer to [link to relevant documentation]. Figure 6,include: The quality evaluation system construction module 101 is used to perform structured analysis on financial documents, construct a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness and decision value, and generate corresponding structured feedback as a summary evaluation template. The initial summary generation module 102 is used to obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary; The multi-dimensional structured feedback generation module 103 is used to input the original text and initial summary of the reinsurance assessment report into the automatic evaluator, perform multi-dimensional quality assessment on the initial summary, and generate multi-dimensional structured feedback. The automatic evaluator is constructed based on the summary evaluation template and the basic language model. The training sample construction module 104 is used to make targeted corrections to the initial summary based on the generated multi-dimensional structured feedback, generate an enhanced summary, and construct a training sample with the structure of: original text - initial summary - structured feedback - enhanced summary. The model fine-tuning module 105 is used to fine-tune the basic language model by executing instructions using training samples, so that the model learns the summary quality diagnosis logic and the corresponding error correction path. The summary generation module 106 is optimized to generate an optimized summary of the re-guarantee assessment report text using the fine-tuned model.
[0094] Since the apparatus in Embodiment 2 of this invention is the same apparatus used in the financial text summarization optimization method based on multi-dimensional structured feedback in Embodiment 1, those skilled in the art can understand the specific structure and variations of this apparatus based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All apparatuses used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0095] Example 3 Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a financial text summarization optimization method based on multi-dimensional structured feedback as described in Embodiment 1.
[0096] Since the computer-readable storage medium described in Embodiment 3 of this invention is the same computer-readable storage medium used in implementing the financial text summarization optimization method based on multi-dimensional structured feedback in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer-readable storage medium based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer-readable storage media used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0097] Example 4 Based on the same inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in Embodiment 1.
[0098] Since the computer device described in Embodiment 4 of this invention is the same computer device used to implement the financial text summarization optimization method based on multi-dimensional structured feedback in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer device based on the method described in Embodiment 1 of this invention, and therefore will not be described again here. All computer devices used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0099] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0100] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0101] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.
Claims
1. A financial text summarization optimization method based on multi-dimensional structured feedback, characterized in that, include: Conduct structured analysis on financial documents, construct a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness, and decision value, and generate corresponding structured feedback as a summary evaluation template; Obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary; The original text and initial summary of the reinsurance assessment report are input into the automatic evaluator to perform a multi-dimensional quality assessment of the initial summary and generate multi-dimensional structured feedback. The automatic evaluator is built based on the summary evaluation template and the basic language model. Based on the generated multi-dimensional structured feedback, the initial summary is modified in a targeted manner to generate an enhanced summary, and a training sample with the structure of: original text - initial summary - structured feedback - enhanced summary is constructed. The basic language model is fine-tuned by executing instructions using training samples, enabling the model to learn the summary quality diagnostic logic and the corresponding error correction path; The optimized summary of the re-guarantee assessment report text is generated using the fine-tuned model.
2. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 1, characterized in that, Structured analysis of financial documents is conducted to construct a quality evaluation system encompassing dimensions such as factual accuracy, business logic, risk coverage, information conciseness, and decision-making value. Corresponding structured feedback formats are then generated, including: Perform structured analysis on financial documents; The evaluation dimensions include factual accuracy, business logic, risk coverage, information conciseness, and decision value. Set dimensional scoring rules; For each dimension, a chain-reasoning template is designed. The template format includes: reading the key content of the original text, comparing with the initial summary, pointing out inconsistencies, judging whether the issue affects financial decisions, and providing reasons and improvement suggestions. The corresponding structured feedback is generated, and the fields of the structured feedback include evaluation dimensions, scores, chain reasoning, improvement suggestions, and the relevant original text excerpts.
3. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 1, characterized in that, The original text and initial summary of the reinsurance assessment report are input into the automated assessor, which performs a multi-dimensional quality assessment of the initial summary and generates multi-dimensional structured feedback, including: The original text and initial summary of the reinsurance assessment report are organized into assessment input text in a structured format, which includes the original text content field, the initial summary field, and the task description field. Semantic alignment analysis was performed on the original text and initial summary of the reinsurance assessment report; Based on a multi-dimensional quality evaluation system, the initial summary is analyzed and scored from multiple business-related dimensions, and structured evaluation results are output. Generate chain reasoning to illustrate the thought process that led to the score.
4. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 3, characterized in that, A semantic alignment analysis was performed on the original text and initial summary of the reinsurance assessment report, including: Perform overall semantic alignment between the original text and the initial summary to identify the correspondence between the two in terms of main information, guarantee liability structure, key financial indicators, risk descriptions and review conclusions; By reading the original text to extract key business points and comparing them with the initial summary to determine whether the summary omits key information, contains factual errors, or has deviations in relational descriptions.
5. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 1, characterized in that, Based on the generated multi-dimensional structured feedback, the initial summary is revised in a targeted manner to generate an enhanced summary, including: Based on the generated multi-dimensional structured feedback, generate one or more modification suggestions for the initial summary for each dimension, including risk information that needs to be added, erroneous data that needs to be corrected, or redundant content that needs to be deleted. Based on all the suggested revisions, an enhanced summary was generated from the original text.
6. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 1, characterized in that, Following the enhanced summary, the method further includes: The structured feedback includes information on the scope of the original text citations, which identifies the original text paragraphs on which the current score and recommendations are based.
7. The financial text summarization optimization method based on multi-dimensional structured feedback as described in claim 1, characterized in that, Fine-tuning the basic language model using training samples includes: The instruction text used for model fine-tuning is designed in a standardized manner. The instruction template includes task description, original text content, initial summary content and structured feedback objectives. A position labeling mechanism is introduced into the instruction template so that the model can clearly refer to each dimension in the feedback and its corresponding improvement objectives. The base model is fine-tuned by executing instructions, and a loss function for joint generation of structured feedback is introduced. The base language model is then fine-tuned by minimizing this joint generation loss function, enabling the model to simultaneously generate structured feedback and enhanced summaries. The loss function is as follows: in, Indicates model parameters, For input documents, The initial summary is, The structured feedback generated by the self-evaluator Enhanced summary This is used to measure the model's ability to generate structured feedback and enhanced summaries under given input conditions.
8. A financial text summarization optimization device based on multi-dimensional structured feedback, characterized in that, include: The quality evaluation system construction module is used to perform structured analysis on financial documents, build a multi-dimensional quality evaluation system that includes factual accuracy, business logic, risk coverage, information conciseness, and decision value, and generate corresponding structured feedback as a summary evaluation template. The initial summary generation module is used to obtain the original text of the reinsurance assessment report and call the basic language model to generate the corresponding initial summary; A multi-dimensional structured feedback generation module is used to input the original text and initial summary of the reinsurance assessment report into an automatic evaluator, perform multi-dimensional quality assessment on the initial summary, and generate multi-dimensional structured feedback. The automatic evaluator is built based on the summary evaluation template and the basic language model. The training sample construction module is used to make targeted corrections to the initial summary based on the generated multi-dimensional structured feedback, generate enhanced summaries, and construct training samples with the structure: original text - initial summary - structured feedback - enhanced summary; The model fine-tuning module is used to fine-tune the basic language model by executing instructions using training samples, enabling the model to learn the summary quality diagnostic logic and the corresponding error correction path. The summary generation module has been optimized to generate an optimized summary of the re-guarantee assessment report text using the fine-tuned model.
9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements a financial text summarization optimization method based on multi-dimensional structured feedback as described in any one of claims 1 to 7.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a financial text summarization optimization method based on multi-dimensional structured feedback as described in any one of claims 1 to 7.