Government affair service-oriented long text generation method and system

Abstract

The invention provides a long text generation method and system oriented to government affair service. The method comprises the following steps: constructing a dynamic government term knowledge graph; performing domain adaptation adjustment on the long text generation model according to the dynamic government affair term knowledge graph to obtain a government affair special long text generation model; analyzing a long text generation requirement of a user to obtain an item type, a core element and a format requirement; according to the obtained demand analysis result, calling a long text generation model special for government affairs to generate an initial long text; performing format specification detection, content compliance detection and business logic detection on the initial long text to obtain a quality value of the initial long text; under the condition that the quality value of the initial long text is lower than a preset quality threshold value, taking the quality value of the generated long text as a reward value, and performing reinforcement learning training on a long text generation model special for government affairs until the quality value of the currently generated long text is not lower than the quality threshold value; and outputting the currently generated long text. The system comprises each function module for correspondingly realizing each step. According to the method, the high-quality government affair long text can be generated.

Description

Technical Field

[0001] The present invention belongs to the intersection of natural language processing and government informationization. More specifically, it relates to a long text generation method and system for government services based on domain knowledge enhancement and reinforcement learning. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and natural language processing technologies, GPT models have been increasingly used for intelligent writing in government scenarios, such as writing official documents for local government administration departments, interpretations of government-oriented regulations, and administrative approval documents. However, existing GPT models often have the following prominent issues when used to write government-related documents:

[0003] Compliance risks: According to relevant statistics, the existing GPT model has an 18.7% format error rate (such as irregular document number marking) and a 12.3% content deviation rate (such as citing invalid government-oriented regulations) when generating government documents.

[0004] Lack of terminology consistency: The discrepancy rate in the use of professional terminology in cross-departmental documents reached 24.5% (2023 government service evaluation report of a certain province);

[0005] Logical breaks in long texts: When generating government-oriented regulation interpretation documents of more than 2,000 words, the rate of logical inconsistencies is as high as 31%.

[0006] Update lag problem: The existing GPT model fine-tuning method takes 3-6 months to adapt to newly issued government-oriented regulations. Summary of the Invention

[0007] In view of this, the present invention provides a long text generation method and system for government services.

[0008] According to a first aspect of the present invention, a method for generating long text for government services is provided, the method comprising the following steps:

[0009] Build a dynamic knowledge graph of government terminology;

[0010] Performing domain adaptation adjustment on the pre-built long text generation model according to the dynamic government terminology knowledge graph to obtain a government-specific long text generation model;

[0011] Parse the user's long text generation requirements to obtain the matter type, core elements and format requirements;

[0012] Based on the acquired matter type, core elements and format requirements, the government affairs-specific long text generation model is called to generate an initial long text;

[0013] Perform format specification detection, content compliance detection, and business logic detection on the initial long text, and determine the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score;

[0014] In the case where the quality value of the initial long text is lower than a predetermined quality threshold, use the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs dedicated long text generation model until the quality value of the currently generated long text is not lower than the quality threshold;

[0015] Output the currently generated long text.

[0016] Optionally, the construction of the dynamic government affairs term knowledge graph includes:

[0017] Perform dynamic data collection on a predetermined government affairs data source;

[0018] Extract government affairs entity terms from government affairs related documents obtained through dynamic data collection based on a term extraction model;

[0019] Construct a government affairs term knowledge graph based on the extracted government affairs entity terms, and the government affairs term knowledge graph is used to represent the inclusion relationship, conflict relationship, and reference relationship between government affairs entity terms.

[0020] Optionally, the domain adaptation adjustment of the pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model includes:

[0021] Use knowledge distillation technology to transform the dynamic government affairs term knowledge graph into a vector recognizable by the model;

[0022] Inject a BERT variant and a lightweight version of the GPT model into the vectorized dynamic government affairs term knowledge graph;

[0023] Use the BERT variant as the teacher model, the GPT model as the student model, and adopt a teacher-student training method to train the GPT model into the government affairs dedicated long text generation model.

[0024] Optionally, the detection content of the format specification detection includes the detection of the matching degree of the red-headed document format template, the compliance detection of the document number generation rule, and the integrity detection of the official document structure, and the format specification detection score is the sum of the scores of the corresponding detection contents;

[0025] The detection content of the content compliance detection includes the verification of the timeliness of government affairs oriented regulations and articles, the legal risk prediction, and the sensitive information filtering detection, and the content compliance detection score is the sum of the scores of the corresponding detection contents;

[0026] The detection content of the business logic detection includes the compliance verification of the approval process between departments, the logic check of the matter handling time limit, and the verification of the electronic signature permission chain. The business logic detection score is the sum of the scores of the corresponding detection contents.

[0027] Optionally, determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score includes:

[0028] Obtaining the product of the format specification detection score and the predetermined format weight;

[0029] Obtaining the product of the content compliance detection score and the predetermined content weight;

[0030] Obtaining the product of the business logic detection score and the predetermined business logic weight;

[0031] Taking the sum of the three obtained products as the quality value of the initial long text.

[0032] Optionally, the types of government affairs entity terms extracted based on the term extraction model include: legal and regulatory reference articles, descriptions of the authorities of local government affairs management departments, administrative approval item codes, and specific expressions of local dialects.

[0033] Optionally, constructing the government affairs term knowledge graph based on the extracted government affairs entity terms is implemented based on the following rules:

[0034] Term conflict detection rule;

[0035] Conflict marking rule for superior and subordinate government affairs guiding regulations;

[0036] Departmental power and responsibility cross-warning rule;

[0037] Timeliness automatic verification rule.

[0038] Optionally, in the process of performing reinforcement learning training on the government affairs dedicated long text generation model, the PPO algorithm based on Transformer is used to update the policy network parameters of the government affairs dedicated long text generation model.

[0039] Optionally, the long text generation method further includes:

[0040] Obtaining the optimized content of the currently generated long text relative to the initial long text, and updating the dynamic government affairs term knowledge graph according to the optimized content.

[0041] According to the second aspect of the present invention, there is provided a long text generation system for government affairs services, and the system includes the following functional modules:

[0042] A knowledge graph construction module for constructing a dynamic government affairs term knowledge graph;

[0043] A dedicated model acquisition module for performing domain adaptation adjustment on a pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model;

[0044] A generation requirement analysis module for analyzing the long text generation requirements of a user to obtain an item type, core elements, and format requirements;

[0045] An initial text acquisition module for calling the government affairs dedicated long text generation model to generate an initial long text according to the obtained item type, core elements, and format requirements;

[0046] A text quality value acquisition module for performing format specification detection, content compliance detection, and business logic detection on the initial long text, and determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score;

[0047] A reinforcement learning training module for, when the quality value of the initial long text is lower than a predetermined quality threshold, using the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs dedicated long text generation model until the quality value of the currently generated long text is not lower than the quality threshold;

[0048] A final text output module for outputting the currently generated long text.

[0049] The beneficial effects of the present invention are as follows:

[0050] The long text generation method for government affairs services of the present invention includes the following steps: first, constructing a dynamic government affairs term knowledge graph; second, performing domain adaptation adjustment on a pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model; third, analyzing the long text generation requirements of a user to obtain an item type, core elements, and format requirements; fourth, calling the government affairs dedicated long text generation model to generate an initial long text according to the obtained item type, core elements, and format requirements; fifth, performing format specification detection, content compliance detection, and business logic detection on the initial long text, and determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score; sixth, when the quality value of the initial long text is lower than a predetermined quality threshold, using the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs dedicated long text generation model until the quality value of the currently generated long text is not lower than the quality threshold; finally, outputting the currently generated long text.

[0051] The long text generation method for government affairs services of the present invention, on the one hand, the government affairs dedicated long text generation model for outputting long texts is a GPT model adapted to the field based on a dynamic government affairs term knowledge graph; on the other hand, the quality value of the finally output long text is not lower than a predetermined quality threshold, and the quality value of the long text is jointly determined based on the format specification detection score, content compliance detection score, and business logic detection score of the long text. Based on the above two aspects, it can be seen that adopting the long text generation method for government affairs services of the present invention can effectively improve the compliance risks, insufficient term consistency, long text logical breakage, and update lag problems existing in the existing GPT model when writing government affairs related documents.

[0052] The long text generation system for government affairs services of the present invention and the above-mentioned long text generation method for government affairs services belong to a general inventive concept, and at least have the same beneficial effects as the above-mentioned long text generation method for government affairs services. The beneficial effects will not be elaborated here.

[0053] Other features and advantages of the present invention will be described in detail in the subsequent specific implementation part. Brief Description of the Drawings

[0054] The present invention can be better understood by referring to the descriptions made in conjunction with the drawings in the following text, where the same or similar reference numerals are used in all the drawings to represent the same or similar components.

[0055] Figure 1 Shows the implementation flowchart of the long text generation method for government affairs services according to an embodiment of the present invention;

[0056] Figure 2 Shows the structural block diagram of the long text generation system for government affairs services according to an embodiment of the present invention. Detailed Description of the Invention

[0057] In order to enable those skilled in the art to more fully understand the technical solutions of the present invention, the exemplary embodiments of the present invention will be described more comprehensively and in detail in the following text with reference to the drawings. Obviously, one or more of the embodiments of the present invention described below are only one or more of the specific ways to implement the technical solutions of the present invention, and are not exhaustive. It should be understood that other ways belonging to a general inventive concept can be used to implement the technical solutions of the present invention, and should not be limited by the exemplary embodiments described. Based on one or more embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0058] Embodiment: Figure 1The figure shows the implementation flowchart of the long text generation method for government affairs services according to an embodiment of the present invention. Referring to Figure 1 , the long text generation method for government affairs services according to an embodiment of the present invention includes the following steps:

[0059] Step S100: Construct a dynamic government affairs term knowledge graph;

[0060] Step S200: Perform domain adaptation adjustment on the pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs-specific long text generation model;

[0061] Step S300: Analyze the user's long text generation requirements to obtain the matter type, core elements, and format requirements;

[0062] Step S400: According to the obtained matter type, core elements, and format requirements, call the government affairs-specific long text generation model to generate an initial long text;

[0063] Step S500: Perform format specification detection, content compliance detection, and business logic detection on the initial long text, and determine the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score;

[0064] Step S600: In the case where the quality value of the initial long text is lower than a predetermined quality threshold, use the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs-specific long text generation model until the quality value of the currently generated long text is not lower than the quality threshold;

[0065] Step S700: Output the currently generated long text.

[0066] Further, in step S100 of the embodiment of the present invention, constructing a dynamic government affairs term knowledge graph includes:

[0067] Perform dynamic data collection on a predetermined government affairs data source;

[0068] Extract government affairs entity terms from the government affairs-related documents obtained through dynamic data collection based on a term extraction model;

[0069] Construct a government affairs term knowledge graph based on the extracted government affairs entity terms, and the government affairs term knowledge graph is used to represent the inclusion relationship, conflict relationship, and reference relationship between government affairs entity terms.

[0070] Specifically, in the embodiments of the present invention, the government affairs data source includes a government affairs guiding regulation document library of government affairs management departments, a list of local government affairs service items, a historical approval document library, a legal provision database, a department regulation library, and a real-time government affairs guiding regulation update stream; the term extraction model is implemented using a BiLSTM-CRF model. After the government affairs related documents are recognized by OCR, they are input into the BiLSTM-CRF model, and the BiLSTM-CRF model correspondingly outputs the government affairs entity terms with annotations.

[0071] Specifically, in the embodiments of the present invention, a basic term set is constructed based on the government affairs entity terms output by the BiLSTM-CRF model; the relationship establishment module identifies the inclusion relationships of the government affairs entity terms in the basic term set to obtain a hierarchical relationship graph; the relationship establishment module detects the conflict relationships of the government affairs entity terms in the basic term set to obtain a conflict relationship matrix; the relationship establishment module performs timeliness association on the government affairs entity terms in the basic term set to obtain a timeline mapping; the hierarchical relationship graph, the conflict relationship matrix, and the timeline mapping are input into the vector mapping engine to obtain a dynamic government affairs term knowledge graph. The vector mapping engine includes a term importance calculation layer, and the term importance calculation layer determines the importance of each government affairs entity term based on the proportion of the significance score of each government affairs entity term in the sum of the significance scores of all government affairs entity terms.

[0072] Furthermore, in step S200 of the embodiments of the present invention, the domain adaptation adjustment of the pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model includes:

[0073] Adopt the knowledge distillation technology to transform the dynamic government affairs term knowledge graph into a vector recognizable by the model;

[0074] Inject the BERT variant and the lightweight version of the GPT model into the vectorized dynamic government affairs term knowledge graph;

[0075] Take the BERT variant as the teacher model, take the GPT model as the student model, and adopt the teacher-student training method to train the GPT model into a government affairs dedicated long text generation model.

[0076] Furthermore, in step S500 of the embodiments of the present invention, the detection content of the format specification detection includes the detection of the matching degree of the red-headed document format template, the detection of the compliance of the document number generation rule, and the detection of the integrity of the official document structure. The format specification detection score is the sum of the scores of the corresponding detection contents;

[0077] The detection content of the content compliance detection includes the verification of the timeliness of the government affairs guiding regulation provisions, the legal risk prediction, and the sensitive information filtering detection. The content compliance detection score is the sum of the scores of the corresponding detection contents;

[0078] The detection content of business logic detection includes the compliance verification of the approval process between departments, the logical check of the handling time limit of matters, and the verification of the electronic signature permission chain. The business logic detection score is the sum of the scores of the corresponding detection contents.

[0079] Furthermore, in step S500 of the embodiment of the present invention, determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score includes:

[0080] Obtaining the product of the format specification detection score and the predetermined format weight;

[0081] Obtaining the product of the content compliance detection score and the predetermined content weight;

[0082] Obtaining the product of the business logic detection score and the predetermined business logic weight;

[0083] Taking the sum of the three obtained products as the quality value of the initial long text.

[0084] Specifically, in the embodiment of the present invention, the format weight is 0.4, the content weight is 0.35, and the business logic weight is 0.25.

[0085] Furthermore, in the embodiment of the present invention, the types of government affairs entity terms extracted based on the term extraction model include:

[0086] Cited articles of laws and regulations, descriptions of the authorities of local government affairs management departments, administrative approval item codes, and specific expressions of local dialects.

[0087] Furthermore, in the embodiment of the present invention, constructing a government affairs term knowledge graph based on the extracted government affairs entity terms is realized based on the following rules:

[0088] Term conflict detection rule;

[0089] Superior and subordinate government affairs orientation regulation conflict marking rule;

[0090] Department power and responsibility cross-warning rule;

[0091] Timeliness automatic verification rule.

[0092] Furthermore, in step S600 of the embodiment of the present invention, in the process of performing reinforcement learning training on the government affairs dedicated long text generation model, the PPO algorithm based on Transformer is used to update the policy network parameters of the government affairs dedicated long text generation model.

[0093] Furthermore, the long text generation method for government affairs services in the embodiment of the present invention further includes:

[0094] Obtain the optimized content of the current generated long text relative to the initial long text, and update the dynamic government affairs term knowledge graph according to the optimized content.

[0095] Specifically, the long text generation method for government affairs services in the embodiments of the present invention is based on the measured data of a provincial government affairs service platform:

[0096] Format compliance rate: increased from 82.3% to 99.1%;

[0097] Content accuracy rate: the correct rate of citing government affairs-oriented regulations and articles increased from 76.8% to 98.4%;

[0098] Generation efficiency: the generation time of a 5000-word comprehensive report was shortened from 4.2 hours to 18 minutes;

[0099] Update agility: the adaptation cycle for new government affairs-oriented regulations was compressed from 3 months to 72 hours.

[0100] Correspondingly, based on the long text generation method for government affairs services in the embodiments of the present invention, the embodiments of the present invention also propose a long text generation system for government affairs services.

[0101] Figure 2 The structural block diagram of the long text generation system for government affairs services in the embodiments of the present invention is shown. Refer to Figure 2 , the long text generation system for government affairs services in the embodiments of the present invention includes the following functional modules:

[0102] Knowledge graph construction module, used to construct a dynamic government affairs term knowledge graph;

[0103] Dedicated model acquisition module, used to perform domain adaptation adjustment on the pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model;

[0104] Generation requirement analysis module, used to analyze the long text generation requirements of users to obtain the matter type, core elements, and format requirements;

[0105] Initial text acquisition module, used to call the government affairs dedicated long text generation model to generate an initial long text according to the obtained matter type, core elements, and format requirements;

[0106] Text quality value acquisition module, used to perform format specification detection, content compliance detection, and business logic detection on the initial long text, and determine the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score;

[0107] A reinforcement learning training module, which is used to, when the quality value of the initial long text is lower than a predetermined quality threshold, use the quality value of the generated long text as a reward value to perform reinforcement learning training on the long text generation model for government affairs until the quality value of the currently generated long text is not lower than the quality threshold;

[0108] A final text output module, which is used to output the currently generated long text.

[0109] Although the above describes one or more embodiments of the present invention, those of ordinary skill in the art should be aware that the present invention can be implemented in any other form without departing from its gist and scope. Therefore, the above-described embodiments are illustrative rather than restrictive, and many modifications and substitutions are obvious to those of ordinary skill in the art without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A long text generation method for government services, characterized in that, including: constructing a dynamic government affairs term knowledge graph; performing domain adaptation adjustment on a pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs-specific long text generation model; analyzing the user's long text generation requirement to obtain the matter type, core elements, and format requirements; invoking the government affairs-specific long text generation model to generate an initial long text according to the obtained matter type, core elements, and format requirements; performing format specification detection, content compliance detection, and business logic detection on the initial long text, and determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score; in the case where the quality value of the initial long text is lower than a predetermined quality threshold, using the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs-specific long text generation model until the quality value of the currently generated long text is not lower than the quality threshold; outputting the currently generated long text.

2. The long text generation method for government affairs services according to claim 1, wherein The constructing of the dynamic government affairs term knowledge graph includes: performing dynamic data collection on a predetermined government affairs data source; extracting government affairs entity terms from government affairs-related documents obtained through dynamic data collection based on a term extraction model; constructing a government affairs term knowledge graph based on the extracted government affairs entity terms, where the government affairs term knowledge graph is used to represent the inclusion relationship, conflict relationship, and reference relationship between government affairs entity terms.

3. The long text generation method for government affairs services according to claim 2, wherein The performing of domain adaptation adjustment on a pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs-specific long text generation model includes: using knowledge distillation technology to transform the dynamic government affairs term knowledge graph into a vector recognizable by the model; injecting a BERT variant and a lightweight version of the GPT model into the vectorized dynamic government affairs term knowledge graph; using the BERT variant as the teacher model, using the GPT model as the student model, and adopting a teacher-student training method to train the GPT model into the government affairs-specific long text generation model.

4. The long text generation method for government affairs services according to claim 3, wherein The detection content of the format specification detection includes the matching degree detection of the red-headed document format template, the compliance detection of the document number generation rule, and the integrity detection of the official document structure, and the format specification detection score is the sum of the scores of the corresponding detection contents; The detection content of the content compliance detection includes the timeliness verification of government affairs-oriented regulatory provisions, the legal risk prediction, and the sensitive information filtering detection, and the content compliance detection score is the sum of the scores of the corresponding detection contents; The detection content of the business logic detection includes the compliance verification of the inter-department approval process, the logic check of the matter handling time limit, and the verification of the electronic signature permission chain, and the business logic detection score is the sum of the scores of the corresponding detection contents.

5. The long text generation method for government affairs services according to claim 4, wherein The determining of the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score includes: obtaining the product of the format specification detection score and a predetermined format weight; obtaining the product of the content compliance detection score and a predetermined content weight; obtaining the product of the business logic detection score and a predetermined business logic weight; Use the sum of the three obtained products as the quality value of the initial long text.

6. The method for generating long texts for government affairs services according to claim 5, wherein The types of government affairs entity terms extracted based on the term extraction model include: legal and regulatory reference articles, descriptions of the authorities of local government affairs management departments, administrative approval item codes, and specific expressions of local dialects.

7. The long text generation method for government affairs services according to claim 6, wherein The construction of the government affairs term knowledge graph based on the extracted government affairs entity terms is realized based on the following rules: Term conflict detection rules; Superior and subordinate government affairs-oriented regulation conflict marking rules; Departmental power and responsibility cross-warning rules; Timeliness automatic verification rules.

8. The long text generation method for government affairs services according to claim 7, characterized in that During the process of performing reinforcement learning training on the government affairs dedicated long text generation model, use the PPO algorithm based on Transformer to update the policy network parameters of the government affairs dedicated long text generation model.

9. The long text generation method for government affairs services according to claim 8, wherein It also includes: Obtain the optimized content of the currently generated long text relative to the initial long text, and update the dynamic government affairs term knowledge graph according to the optimized content.

10. A long text generation system for government services, characterized in that, It includes: A knowledge graph construction module for constructing a dynamic government affairs term knowledge graph; A dedicated model acquisition module for performing domain adaptation adjustment on a pre-constructed long text generation model according to the dynamic government affairs term knowledge graph to obtain a government affairs dedicated long text generation model; A generation requirement analysis module for analyzing the long text generation requirements of users to obtain the matter type, core elements, and format requirements; An initial text acquisition module for calling the government affairs dedicated long text generation model to generate an initial long text according to the obtained matter type, core elements, and format requirements; A text quality value acquisition module for performing format specification detection, content compliance detection, and business logic detection on the initial long text, and determining the quality value of the initial long text according to the obtained format specification detection score, content compliance detection score, and business logic detection score; A reinforcement learning training module for, when the quality value of the initial long text is lower than a predetermined quality threshold, using the quality value of the generated long text as a reward value to perform reinforcement learning training on the government affairs dedicated long text generation model until the quality value of the currently generated long text is not lower than the quality threshold; A final text output module for outputting the currently generated long text.

Images