Intelligent government affairs processing method and system based on big model and process robot

Abstract

The present application provides a method and system for intelligent government affairs processing based on a large model and process robot technology. By obtaining the original government affairs text to be processed; loading the original government affairs text into a large natural language model, obtaining key information detection results, the key information detection results include the key information coverage window identified in the original government affairs text, the key information core characters, the information classification corresponding to the key information characters, and the classification confidence corresponding to the information classification. The debugging process of the large natural language model includes debugging each branch component separately, and then further multi-task learning; generating a target government affairs text based on the key information detection results and the original government affairs text. The present application can achieve the effect of key information detection using only one large natural language model, while ensuring the accuracy of detection, reducing the model's dependence on computing power.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and system for intelligent government affairs processing based on large models and process robot technology. Background Art

[0002] Government affairs processing is the core part of public management and services, involving a huge amount of documents, information and processes. With the development of society and the growth of public demand, the workload and complexity of government affairs processing are constantly increasing. The traditional government affairs processing model mainly relies on manual operations, which faces many severe challenges. When manually handling government affairs, it takes a lot of time and energy to process various government documents, such as approval documents and application materials. For example, in the administrative approval process, staff need to find key information from lengthy application documents. This process is not only extremely inefficient, but also prone to errors due to human negligence.

[0003] Natural language processing technology has certain application potential in the exploration of intelligent government affairs, but there are still many limitations. Existing natural language processing models often have difficulty in accurately locating and parsing key information when processing government texts. At the same time, existing natural language processing technologies have problems with efficiency and resource utilization in model construction and training. Building models that can accurately process government texts often requires a large amount of computing resources, including computing power support from hardware devices. This not only increases costs, but also limits the application of technology in some resource-constrained environments. Summary of the invention

[0004] In view of this, the embodiment of the present application at least provides a method and system for intelligent government affairs processing based on large models and process robot technology. The technical solution of the present application is implemented as follows:

[0005] On the one hand, the present application provides a method for intelligent government affairs processing based on a big model and process robot technology, including: obtaining an original government affairs text to be processed; loading the original government affairs text into a natural language big model to obtain a key information detection result, wherein the key information detection result includes a key information coverage window, key information core characters, information classification corresponding to the key information characters, and classification confidence corresponding to the information classification identified in the original government affairs text; the natural language big model is obtained by debugging a basic language model, and the basic language model includes a basic recognition branch component, a basic core character recognition branch component, and a basic segmentation branch component. After the debugging of the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component is completed, multi-task learning is performed on the basic language model to obtain the natural language large model; the target government text is generated by combining the key information detection result and the original government text, wherein the target government text is annotated with the key information coverage window, the key information core characters, the information classification and the confidence corresponding to the information classification; the preset process robot is called to extract the key information that meets the confidence requirement from the target government text, and the extracted key information is filled into the preset order table to obtain the key information entry result.

[0006] On the other hand, the present application provides a computer system, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above method when executing the program.

[0007] The beneficial effects of the present application include: the present application provides a method and system for intelligent government affairs processing based on a big model and process robot technology, by obtaining the original government affairs text to be processed; loading the original government affairs text into a natural language big model to obtain key information detection results, wherein the key information detection results include the key information coverage window identified in the original government affairs text, the key information core characters, the information classification corresponding to the key information characters, and the classification confidence corresponding to the information classification. The natural language big model is a network model obtained by debugging a basic language model, and the basic language model includes a basic recognition branch component, a basic core character recognition branch component, and a basic segmentation branch component. After the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component are debugged, the basic language model is multi-task learned to obtain a large natural language model; the target government text is generated by combining the key information detection results and the original government text, wherein the target government text is annotated with key information coverage windows, key information core characters, information classification and the confidence corresponding to the information classification. Based on the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component in the basic language model, the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component are debugged separately, and then the three different components in the basic language model are multi-task learned to obtain a large natural language model. After the original government text is loaded into the large natural language model, the key information detection result is obtained, and the target government text is generated by combining the original government text, so as to achieve the effect of using only one large natural language model to detect key information based on the original government text, and reduce the computing power dependence of the model while ensuring the accuracy of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of the implementation process of an intelligent government affairs processing method based on a large model and process robot technology provided in an embodiment of the present application.

[0009] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] The embodiment of the present application provides a method for intelligent government affairs processing based on large models and process robot technology, which can be executed by a processor of a computer system. The computer system can refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, a desktop computer, a mobile device, etc.

[0011] Figure 1 A schematic diagram of the implementation process of a method for intelligent government affairs processing based on a large model and process robot technology provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0012] Step S100: Obtain the original government affairs text to be processed.

[0013] In the process of intelligent government affairs processing, the computer system first obtains the original government affairs text to be processed. The original government affairs text refers to the unprocessed text data containing various government affairs related information. This government affairs information covers many aspects, such as the declaration materials related to administrative approval matters, the description of government affairs service requests, etc.

[0014] Specifically, the computer system can obtain the original government text in a variety of ways. One feasible technical means is to read the relevant text records from the database of the government information system. For example, in an administrative approval system, when an enterprise or individual submits an application for approval of a project construction, all the text content in the application constitutes the original government text. This text may contain basic information about the project, such as the project name, construction location, investment scale, etc., and may also contain some project-related document descriptions, technical solutions, etc. This information is stored in the database in text form. The computer system can use database query statements and extract the original government text to be processed from the database according to pre-set rules. Another way to obtain it can be to receive government text data from the outside through a network interface. For example, some government service platforms support online declaration. After the applicant fills in the declaration information on the web page or mobile terminal and submits it, the information will be transmitted to the government processing system through the network. The computer system sets up corresponding network communication protocols and data parsing modules at the receiving end so that it can correctly receive and identify the original government text transmitted. Taking online tax declaration as an example, the text content such as corporate financial information, tax declaration category, and basis for calculating taxable amount filled in by taxpayers on the web page will be obtained by the computer system as the original government text after submission. When obtaining the original government text, the computer system ensures the integrity and accuracy of the text. Integrity requires that the obtained text contains all necessary information related to government affairs. For example, in the original government text of an enterprise registration application, key information such as the enterprise name, legal representative information, and business scope must be included without omission. Accuracy requires that there are no errors in the text during transmission and storage, for example, the character encoding is correct, there is no garbled code, and the text content has not been tampered with. In order to ensure integrity and accuracy, the computer system may use data verification technology. For example, for text read from a database, the text checksum (such as CRC checksum) can be calculated and saved when storing, and the checksum can be calculated again when reading and compared with the saved value. If it is inconsistent, it means that there may be errors in the data. For texts transmitted over the network, security protocols such as SSL / TLS can be used to ensure the integrity and accuracy of data transmission. At the same time, the data format is checked at the receiving end to ensure that the received text meets the expected format requirements. In addition, the format of the original government text may also vary. It may be a plain text format, such as a normal text file encoded in UTF-8; it may also be a text with certain format tags, such as a government document in XML format, in which tags can be used to identify different categories of government information. Computer systems are able to recognize and process original government texts in different formats. For plain text format, the system can directly read and process the text content.For text in XML format, the system can use the XML parser to parse the values ​​of each element and attribute according to the XML grammar rules, and convert them into an internal data structure that is convenient for subsequent processing.

[0015] Step S200: Load the original government text into the natural language big model to obtain key information detection results, wherein the key information detection results include the key information coverage window identified in the original government text, the key information core characters, the information classification corresponding to the key information characters, and the classification confidence corresponding to the information classification. The natural language big model is obtained by debugging the basic language model. The basic language model includes a basic recognition branch component, a basic core character recognition branch component, and a basic segmentation branch component. After the basic recognition branch component, the basic core character recognition branch component, and the basic segmentation branch component are debugged, multi-task learning is performed on the basic language model to obtain the natural language big model.

[0016] The natural language big model is a specially constructed and trained model, which is debugged on the basis of the basic language model. The basic language model includes the basic recognition branch component, the basic core character recognition branch component, and the basic segmentation branch component. These components each have different functions in natural language processing. For example, the basic recognition branch component may be responsible for identifying certain specific elements or structures in the text. Assuming that the original government text is an application for environmental protection approval of an enterprise, the basic recognition branch component may identify the approximate location and related expressions of specific elements such as the production scale of the enterprise and the type of pollution emission in the text, which is similar to marking the area where specific information is located in a complex document. The basic core character recognition branch component focuses on identifying characters that play a core role in expressing key information. For example, when describing the type of pollution emission, "COD" in "chemical oxygen demand (COD)" is the core character, and this component can accurately find these characters that are crucial to understanding government information. The basic segmentation branch component mainly performs operations related to text segmentation and classification, such as segmenting a description of the production process of an enterprise according to different links or different information categories, and marking possible categories for each part.

[0017] When building a large natural language model, during the debugging process of these basic components, each component is optimized based on specific algorithms and training data to improve its accuracy and effectiveness in processing government texts. For example, when debugging the basic recognition branch component, a large number of government text samples with key element positions and expressions already annotated are used for training. The computer system compares the differences between the component output results and the annotated results, and uses some error measurement method (such as mean square error, the formula is: ,in is the annotation result, is the component output result, n is the number of samples) to adjust the parameters of the component so that the output result is constantly close to the annotation result. When the debugging of the basic recognition branch component, the basic core character recognition branch component, and the basic segmentation branch component has reached a certain standard, the computer system will perform multi-task learning on the basic language model. Multi-task learning is a machine learning technique that allows the model to learn knowledge from multiple related tasks at the same time, thereby improving the performance of the model on each task. For example, in this scenario, the model can simultaneously learn how to identify key elements, find core characters, and segment and classify text, and improve the overall processing capability by sharing some features and information between tasks.

[0018] When the natural language model is built, the computer system loads the original government text into the model. The model will conduct a comprehensive analysis and processing of the original government text to obtain the key information detection results. The key information coverage window indicates the scope of the key information in the original government text. Taking a government text of land use approval as an example, the description of the land use nature (such as commercial land, industrial land, etc.) is a key information coverage window. This window may be composed of several sentences or paragraphs, which defines the text scope of information related to the land use nature. The key information core characters are the characters that best reflect the key information in this coverage window. For example, when describing the land use nature as commercial land, the two words "commercial" are the core characters. The information classification corresponding to the key information characters refers to the classification of these key information according to certain categories. In the text of land use approval, the land use nature belongs to land planning information, while information about land area, geographical location, etc. belongs to other different categories. The classification confidence indicates the model's credibility assessment of this classification result. For example, the model determines that a certain key information belongs to land planning information and gives a classification confidence of 0.8, which means that the model is 80% sure that this classification is correct.

[0019] Specifically, when the computer system loads the original government text into the natural language model, it needs to preprocess the text appropriately to meet the input requirements of the model. This may include operations such as encoding the text (such as converting the text into a vector representation that the model can handle), removing noise (such as some irrelevant punctuation or formatting marks), etc. Inside the model, the neural network structure in the deep learning algorithm will be used to process the text. For example, structures such as convolutional neural networks (CNN) or recurrent neural networks (RNN) and their variants (such as long short-term memory networks, LSTM) may be used. These neural network structures contain multiple neurons and layers, calculate the representation of the input text in the network through forward propagation, and then adjust the network's weight parameters according to the difference between the output result and the expected result through the back-propagation algorithm, thereby continuously optimizing the performance of the model and ultimately obtaining accurate key information detection results.

[0020] Step S300: Generate a target government text by combining the key information detection result and the original government text, wherein the target government text is annotated with a key information coverage window, key information core characters, information classification, and confidence level corresponding to the information classification.

[0021] When the computer system executes step S300, the key information detection result obtained in step S200 is integrated with the original government text to generate the target government text. This process involves the effective integration of the key information obtained previously and the annotation based on the original text, so that the subsequent processing can use this information more efficiently and accurately.

[0022] First, the key information detection result includes the key information coverage window identified in the original government text, the key information core characters, the information classification corresponding to the key information characters, and the classification confidence corresponding to the information classification. The key information coverage window determines the scope of the key information-related content in the original government text. For example, in a government text about an application for a subsidy for technological innovation of an enterprise, if the key information is a description of the enterprise's R&D investment, then the key information coverage window may be the scope of several paragraphs or sentences containing relevant information such as the amount of R&D expenses, the proportion of R&D investment, and the name of the R&D project. The key information core characters are the most representative and key characters in this coverage window. For content related to R&D investment, "R&D" in the expression "R&D expenses" may be the core character, which highly summarizes the theme of this part of information. Information classification is to divide these key information into specific categories. For example, in an application for a subsidy for technological innovation of an enterprise, R&D investment information can be classified as enterprise operation and development information. Classification confidence refers to the computer system's assessment of the accuracy of the classification result. For example, if R&D investment information is classified as business operation and development information, the system gives a classification confidence of 0.9, indicating that the system is 90% sure that the classification is correct.

[0023] When the computer system generates the target government text, it needs to organically combine these key information with the original government text. A feasible technical means is through marking and embedding. Marking refers to marking the key information overlay window at the corresponding position of the original government text. For example, specific symbols or markup languages ​​can be used to mark the scope of information related to R&D investment in the original text, such as adding "<Key information starts-R&D investment>" at the starting position of the overlay window and "<Key information ends-R&D investment>" at the end position. For the core characters of key information, they can be reflected in the original text by highlighting or special encoding. For example, the core characters "R&D" are displayed in bold or different colors (if the text supports color display). If it is reflected in coded form, specific coding tags can be added before and after the core characters, such as "<Core characters-R&D>".

[0024] The annotation of information classification and its confidence can be added to the original government text in the form of annotations or associated with the original text in the form of specific metadata. For example, a comment column is opened next to the original government text, in which it is noted that the R&D investment information belongs to the category of business operation and development information, and the classification confidence is 0.9. Specifically, if the original government text is regarded as a string data structure (assuming the string variable originalText), the marking and embedding operations can be implemented through string operation functions. For example, in the Python language, if you want to add a key information overlay window mark, you can use string slicing and splicing operations. Assuming that the starting index of the key information overlay window in the original text is startIndex and the ending index is endIndex, the marked original text can be expressed as: markedOriginalText=originalText[:startIndex]+"<key information start-R&D investment>"+originalText[startIndex:endIndex]+"<key information end-R&D investment>"+originalText[endIndex:]. The highlight encoding operation for the core characters of key information is similar. Here, it is assumed that the index of the core character in the original text is coreCharIndex, and the core character is coreChar. The text after the operation is: markedOriginalText=markedOriginalText[:coreCharIndex]+"<core character-"+coreChar+">"+markedOriginalText[coreCharIndex:coreCharIndex+len(coreChar)]+"<core character-"+coreChar+">"+markedOriginalText[coreCharIndex+len(coreChar):].

[0025] The original government text after this marking and embedding operation becomes the target government text. In this way, the target government text integrates all the contents of the original government text and the key information detection results, so that the subsequent processing flow (such as the operation in step S400) can easily identify and utilize these key information. At the same time, these annotations and marks in the target government text also facilitate manual review and verification. For example, when reviewing an application for a company's technological innovation subsidy, government staff can quickly locate key information, such as R&D investment and its classification, through the clear marks and annotations in the target government text, and make a preliminary judgment on the analysis results of the computer system based on the classification confidence. If the classification confidence is low, the staff can further carefully review this part of the information to ensure the accuracy of government processing.

[0026] The generation of target government affairs text is of great significance for improving the efficiency and accuracy of government affairs processing. Taking the planning and approval of urban construction projects as an example, the original government affairs text contains a large amount of various information about the project, such as the project location, project scale, and assessment of the impact of the project on the surrounding environment. After the target government affairs text is generated through step S300, under the key information overlay window mark, the information about the project location (such as the specific address, the regional planning, etc.) is clearly marked, and its core characters (such as specific place names, regional names) are also highlighted, and the information classification (such as geographic information category) and its confidence are also clearly marked. This enables the subsequent computer system to call the process robot for further processing, or manual review and decision-making, to obtain key information more quickly and accurately, thereby improving the efficiency and quality of the entire government affairs processing.

[0027] Step S400: Call the preset process robot to extract key information that meets the confidence requirement from the target government text, and fill the extracted key information into the preset order form to obtain the key information entry result.

[0028] A pre-set process robot is an automated program designed to execute a specific task process. In the context of intelligent government affairs processing, it is pre-defined with relevant rules and operation logic to process government affairs-related information. For example, in the process of enterprise registration, the process robot may be set as a program that can recognize key information such as enterprise name, registered address, and registered capital.

[0029] The target government text is the annotated text after processing in step S300, which contains important elements such as the key information coverage window, the key information core characters, the information classification, and the confidence corresponding to the information classification. The confidence requirement is a pre-set threshold used to measure the credibility of the key information. For example, if the confidence requirement is set to 0.8, only the key information with a classification confidence greater than or equal to 0.8 will be considered reliable and extracted.

[0030] The computer system calls the process robot to process the target government text. Specifically, the process robot scans the target government text according to predefined algorithms and rules. It identifies key information tags and annotations in the text, and then screens them according to the confidence requirements. For example, in a target government text of a corporate tax return, there is information about the basis for calculating the amount of tax payable, which is classified as tax calculation information with a confidence of 0.9. If the confidence requirement is 0.8, then this key information meets the requirement.

[0031] For the operation of extracting key information, the process robot may use pattern matching technology. Assuming that the target government text is stored in a certain format (such as XML format, in which key information is marked in a specific tag), the process robot can locate the key information by parsing the XML tag. Taking the basis for calculating the amount of tax payable as an example, if it is marked as <basis for calculating the amount of tax payable> specific basis for calculating the amount of tax payable content <basis for calculating the amount of tax payable> in XML, the process robot can extract the content by identifying this tag. The preset form is a pre-designed structured form used to store the results of key information entry. The structure of this form is determined according to the needs of government affairs. For example, in the government affairs process of enterprise registration, this form may have column titles such as "Enterprise Name", "Registered Address", "Registered Capital", and "Legal Representative". The computer system fills the key information extracted by the process robot into the corresponding position of this form. Taking corporate tax declaration as an example, if the basis for calculating the tax payable is "calculated based on the company's operating income and tax rate for the year, the operating income is 1 million yuan and the tax rate is 25%", then this information may be filled in the cell named "Basis for calculating the tax payable" in the order form.

[0032] In actual operation, in order to ensure the accuracy and completeness of information, the computer system may verify the filled form. One verification method can be data format verification. For example, if the "registered capital" column in the form is required to be in digital format, and the key information extracted by the process robot is a text description (such as "five hundred thousand yuan"), the computer system may convert it into a digital format (500,000) before filling it. Another verification method is logical relationship verification. For example, in the enterprise registration process, the "registered address" must be within a certain geographical area. If the address extracted by the process robot does not meet this range, the computer system will issue a warning prompt. From the perspective of data management, the key information entry results are stored in the form table, which is convenient for subsequent government processing processes to query, count and analyze. For example, when the government department conducts statistics on the tax payment of enterprises, it can directly obtain data from the form table that stores key information such as the basis for calculating the taxable amount. At the same time, these key information entry results can also be used as the basis for government decision-making. If it is found that there is an abnormality in the basis for calculating the taxable amount of a certain enterprise, the government department can further investigate and verify it, so as to make a reasonable decision. In different government affairs scenarios, the specific operation of step S400 will be different, but the overall purpose is to extract reliable key information from the target government affairs text and organize it into a preset form to facilitate the standardization and automation of government affairs processing. For example, in the government affairs of construction project approval, the target government affairs text contains information such as project planning and design, environmental impact assessment, and source of funds. After the process robot extracts this key information according to the confidence requirements, it fills it into the corresponding project approval form, such as filling the project planning and design information into the "project planning" column, the environmental impact assessment information into the "environmental impact" column, and the source of funds information into the "source of funds" column. This helps to improve the efficiency and accuracy of construction project approval and reduce errors and delays that may be caused by manual operations.

[0033] As an implementation method, step S200, loading the original government text into the natural language big model to obtain key information detection results, specifically includes:

[0034] Step S210: loading the original government text into the target feature extraction branch component, performing semantic feature extraction on the original government text, and obtaining text semantic features, wherein the text semantic features are text features after the semantic features of the original government text are extracted, and the natural language large model includes the target feature extraction branch component;

[0035] Step S220: loading the text semantic features into the target recognition branch component, detecting the key semantics in the text semantic features, and obtaining a key information coverage window, wherein the natural language large model includes the target recognition branch component;

[0036] Step S230: loading the text semantic features into the key information core character recognition branch component, detecting the core characters in the text semantic features, and obtaining the key information core characters, wherein the natural language large model includes the key information core character recognition branch component;

[0037] Step S240: loading the text semantic features into the target segmentation branch component to obtain an information classification set, the information classification set including the classifications and corresponding confidences corresponding to the key information characters, and the natural language large model including the target segmentation branch component;

[0038] Step S250: Generate a target government text by combining the key information coverage window, the key information core characters, the classification and confidence level corresponding to the key information characters, and the original government text.

[0039] When the computer system executes step S210, the target feature extraction branch component aims to mine the semantic features from the original government text, thereby obtaining the text semantic features. The semantic feature is an abstract representation of the meaning of the text, which can reflect the relationship between various concepts in the text and the subject information of the text.

[0040] For example, in a government document about the environmental impact assessment of an enterprise, the original text may contain information about the enterprise's production process, the raw materials used, the types and quantities of waste generated, the surrounding environment, etc. The target feature extraction branch component analyzes this information and identifies the semantic connections between related concepts. For example, certain links in the production process may have a causal relationship with the generation of specific waste, and these semantic relationships are one of the features to be extracted.

[0041] Specifically, the computer system can use a word vector model to implement semantic feature extraction. The word vector model maps each word in the text to a low-dimensional vector space, so that words with similar semantics are closer in the vector space. For example, the commonly used Word2Vec model is an effective word vector model. For each word in the original government text, its corresponding word vector can be obtained through the pre-trained Word2Vec model. These word vectors contain the semantic information of the word. Then, the computer system can combine and analyze all the word vectors in the text, such as by weighted summation or averaging, to obtain the text semantic feature vector of the entire original government text.

[0042] Assume that the original government text is ,in represents the i-th word in the text, and the pre-trained Word2Vec model is f, then the word vector corresponding to each word is . A computer system can calculate To obtain the text semantic feature vector V. This text semantic feature vector can represent the semantic features of the original government text to a certain extent.

[0043] In step S220, the text semantic features obtained in step S210 are loaded into the target recognition branch component. This component focuses on identifying key semantics in text semantic features. Key semantics refer to those semantic elements in government texts that are very important for determining the location and scope of key information.

[0044] Take a government document on land planning approval as an example. The text semantic features may contain semantic information about land use (such as residential land, commercial land, etc.), land area, land geographical location, etc. The object recognition branch component will detect that the semantics related to land use are key semantics, because land use is a key information in land planning approval.

[0045] The key information coverage window represents the location range of the key information in the original government text. In order to determine this coverage window, the target recognition branch component may use some algorithms to analyze the text semantic features. For example, an algorithm based on the attention mechanism can be used. The attention mechanism enables the model to focus on the part of the text that is relevant to the current task. In this example, when it is detected that the semantics related to land use is the key semantics, the attention mechanism will find the part containing the land use description in the original government text, which constitutes the key information coverage window.

[0046] Specifically, assuming that the text semantic feature vector is V, and the algorithm based on the attention mechanism adopted by the target recognition branch component is A, a weight vector W can be obtained by calculating W=A(V), and this weight vector represents the importance of each part of the text semantic feature for determining the key semantics. Then, according to this weight vector, the key information coverage window can be located in the original government text. For example, if the weight of the part corresponding to the land use semantics in the weight vector is high, then the sentence or paragraph range containing the land use description can be found in the original government text according to this weight. When the computer system executes step S230, the key information core character recognition branch component detects the core characters from the text semantic features. The core characters are the most representative and key characters in the key information. For example, in a government text about an application for corporate tax incentives, the key information may be the industry type to which the enterprise belongs (such as high-tech industry). In this key information, the words "high-tech" are the core characters. For the text semantic feature, it contains the semantic information of the entire text about the application for corporate tax incentives, and the key information core character recognition branch component needs to find the core characters that can represent the industry type.

[0047] Specifically, this component can adopt a character-level semantic analysis method. For example, a character-semantic mapping table can be constructed, which is obtained by pre-analyzing a large number of government texts. For each character in the text semantic features, its corresponding semantic weight is found according to this mapping table. The semantic weight indicates the importance of this character in expressing specific semantics. Then, by setting a threshold, characters with semantic weights higher than the threshold are screened out as key information core characters.

[0048] Assume that the text semantic feature vector is V, which contains the following characters: , the character-semantic mapping table is M, then for each character , whose semantic weight is . Set the threshold to t, then the key information core character set is .

[0049] In step S240, the computer system loads the text semantic features into the target segmentation branch component. This component is mainly responsible for segmenting and classifying the text semantic features to obtain an information classification set. The information classification set contains the classification corresponding to the key information characters and the corresponding confidence.

[0050] Take a government funding application from an education department as an example. The semantic features of the text include information such as the school size, education project type, and number of students. The target segmentation branch component will classify this information, for example, classifying information related to school size into school infrastructure, education project type into education development planning, and number of students into school resources.

[0051] For the calculation of confidence, the target segmentation branch component may be based on existing classification models and training data. For example, a Bayesian classifier is used for classification and confidence is calculated. The Bayesian classifier determines the probability that a data point belongs to a certain category based on the prior probability and conditional probability. Assume that for the information on the number of students, based on the historical government text data, the prior probability that the number of students belongs to the school resource category is P(C), and the conditional probability that the number of students belongs to the school resource category given the current text semantic features is P(X|C). According to the Bayesian formula , where P(X) is the evidence factor, which can calculate the confidence that the number of students belongs to the school resource class.

[0052] In actual operation, the target segmentation branch component may perform multi-level segmentation and classification of text semantic features. For example, it may first be segmented according to large subject areas, and then more detailed classification is performed within each subject area. This can improve the accuracy of classification and the detail of information.

[0053] Finally, in step S250, the computer system combines the key information coverage window, key information core characters, classifications corresponding to the key information characters and corresponding confidences obtained in the previous steps with the original government text to generate a target government text.

[0054] For example, in a government document on the allocation of medical and health resources, the original government document describes in detail the basic situation of each medical institution, the number of people served, the distribution of disease types, etc. Through the previous steps, the coverage window of the key information about the number of people served (such as the paragraph containing the description of the number of people served), the core characters (such as the words "number of people served"), the classification (medical and health resource demand category), and the confidence level (assuming it is 0.9) are determined.

[0055] The computer system will integrate these key information elements into the original government text. One way is to add annotations or marks on the basis of the original government text. An annotation can be added next to the paragraph describing the number of people served in the original government text, indicating that this is a key information overlay window, in which the core characters are "number of people served", classified as medical and health resource demand, and the confidence level is 0.9. In this way, the generated target government text not only contains the entire content of the original government text, but also clearly marks the key information and its related attributes, which facilitates the subsequent government processing process (such as the information extraction and table filling operations in step S400).

[0056] Specifically, this integration can be achieved by expanding or modifying the data structure of the original government text. If the original government text is a text field stored in the database, a new field can be added next to this field to store the relevant attributes of the key information (coverage window, core characters, classification and confidence). If the government text is in the form of a file, a markup language (such as XML) can be used to repackage the original text and embed the relevant attributes of the key information into the original text in the form of XML tags.

[0057] As an implementation mode, in step S210, the original government text is loaded into the target feature extraction branch component, and semantic features are extracted from the original government text to obtain text semantic features, which specifically includes:

[0058] Step S211: loading the original government text into the first feature extraction branch component, extracting keywords from the original government text, and obtaining a first text feature, wherein the dimension of the first text feature is smaller than that of the original government text;

[0059] Step S212: Load the first text feature into the second feature extraction branch component, upgrade the dimension of the first text feature, and obtain a text semantic feature, wherein the dimension of the text semantic feature is smaller than that of the original government text and larger than that of the first text feature.

[0060] When the computer system executes step S211, the main task of the first feature extraction branch component is to extract keywords from the original government text. Keywords are words that can summarize the main content and have important meanings in the government text. These keywords can reflect the core theme, key affairs and related elements of the government text to a certain extent.

[0061] For example, in a government document about the approval of an urban transportation infrastructure construction project, the original text may contain a large amount of information such as the detailed planning of the project, such as the construction site (such as a specific area in the city center), the project scale (including road length, number of bridges, etc.), the project budget (specific amount of funds), the project duration (start and end time), the expected benefits (such as relieving traffic congestion, promoting regional economic development, etc.) and related approval requirements. The first feature extraction branch component extracts keywords from this complex original government text. For example, words such as "transportation infrastructure construction", "city center", "project budget", and "project duration" may be extracted as keywords.

[0062] Specifically, the computer system can use a variety of methods to extract keywords. One feasible method is based on statistics. For example, the term frequency-inverse document frequency (TF-IDF) algorithm. TF (Term Frequency) indicates the frequency of a word appearing in a document (here, the original government text). The calculation method is: ,in represents the number of times word i appears in document j (i.e., the original government text), Represents the sum of the occurrences of all words in document j.

[0063] IDF (Inverse Document Frequency) is the inverse document frequency, which reflects the prevalence of a word in the entire document collection. The calculation method is: , where |D| represents the total number of documents in the document collection, Represents the number of documents containing word i.

[0064] Finally, the value of TF-IDF is The computer system may sort the words in the original government text according to the TF-IDF value, and select words with higher TF-IDF values ​​as keywords, thereby obtaining the first text feature.

[0065] In addition to statistical methods, rule-based methods can also be used. For example, if a government document has a specific format or a fixed expression in a field, the computer system can extract keywords based on pre-set rules. In some government documents for administrative approval, there may be specific headers or fixed terminology expressions, such as "application unit", "approval matters", "acceptance number", etc. These words determined according to the rules can also be extracted as keywords.

[0066] These extracted keywords form the first text feature. Since the number of keywords is relatively small compared to the total vocabulary in the original government text, the dimension of the first text feature is smaller than that of the original government text. This reduction in dimension helps to extract semantic features more efficiently in the future because it focuses on the key information parts in the original text and reduces unnecessary information interference.

[0067] In step S212, the computer system loads the first text feature obtained in step S211 into the second feature extraction branch component to perform a dimensionality increase operation to obtain a text semantic feature.

[0068] Taking the government text of the approval of the urban transportation infrastructure construction project as an example, although the keywords in the first text feature, such as "transportation infrastructure construction", "urban center", "project budget", etc., have summarized some key contents of the original government text, these keywords themselves lack a deeper semantic connection representation. The dimensionality increase operation of the second feature extraction branch component aims to explore the richer semantic relationship between these keywords and between them and the original government text, thereby constructing a text semantic feature with a higher dimension than the first text feature.

[0069] Specifically, a feasible method is to use semantic embedding technology. For example, use pre-trained word vector models (such as Word2Vec or GloVe, etc.). These word vector models can map each keyword into a low-dimensional vector space, and the vectors in this vector space can reflect the semantic similarity between keywords. For example, the distance between "transportation infrastructure construction" and "road construction" in the vector space may be close, indicating that they have a high semantic similarity.

[0070] The computer system can combine and transform the word vectors corresponding to each keyword in the first text feature to achieve dimensionality increase. Assume that the word vector corresponding to the keyword in the first text feature is , vector addition, multiplication, or more complex neural network layer operations can be used to construct new vector representations. For example, a new vector can be obtained through a simple weighted sum operation ,in is the weight corresponding to each keyword, which can be determined based on the importance of the keyword or its relevance in the original government text. This new vector V contains more information about the semantic relationship between keywords, thereby achieving a dimensionality increase operation, and the obtained V is the text semantic feature.

[0071] Another method is to use the autoencoder structure in deep learning. The autoencoder consists of two parts: an encoder and a decoder. The encoder compresses the input (the first text feature) into a low-dimensional representation (the encoded representation), and then the decoder restores this low-dimensional representation to an output similar to the input. In this process, the intermediate low-dimensional encoded representation actually learns some potential features of the input data. The computer system can adjust the autoencoder so that its intermediate encoded representation serves as the text semantic feature after dimensionality increase. For example, the first text feature is input into the encoder part of the autoencoder to obtain an intermediate encoded vector, the dimension of which can be set to be larger than the dimension of the first text feature according to pre-set parameters, and then this encoded vector is used as the text semantic feature.

[0072] The dimension of the text semantic feature is smaller than that of the original government text because it is constructed based on the first text feature (keywords), which has compressed and focused the original government text to a certain extent; at the same time, it is larger than the first text feature because it incorporates more semantic relationship information through dimensionality increase operations and is no longer just isolated keyword information.

[0073] As an implementation mode, in step S200, before the original government text is loaded into the natural language big model and the key information detection result is obtained, the method may further include a training process of the natural language big model, specifically including the following steps:

[0074] Step S10: obtaining government training texts, wherein the government training texts include a first government training text annotated with a training overlay window, a second government training text annotated with training core characters, and a third government training text annotated with training categories corresponding to training text characters and training confidences corresponding to the training categories;

[0075] Step S20: loading the government training text into the basic feature extraction branch component to obtain training text features, wherein the training text features are text features after the semantic features of the government training text are extracted;

[0076] Step S30: loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component respectively for separate debugging to obtain a transitional large model, wherein the transitional large model includes a transitional recognition branch component, a transitional core character recognition branch component and a transitional segmentation branch component, the transitional recognition branch component, the transitional core character recognition branch component and the transitional segmentation branch component are debugged independently according to the corresponding debugging branches respectively, and the basic language model includes a basic feature extraction branch component;

[0077] Step S40: Perform multi-task learning on the transition large model to obtain a natural language large model.

[0078] When the computer system executes step S10, obtaining the government training text is the basis for building a natural language large model. The government training text is data specifically used for training models, which covers various government-related information content. The first government training text annotation has a training coverage window. For example, in a government training text about enterprise registration and approval, the training coverage window may be a mark for the text range where the key information of the enterprise name is located. Assume that the entire government training text is a paragraph describing the detailed information of the enterprise registration, which contains the enterprise name, registered address, business scope, etc., and the sentence or part of the paragraph where the enterprise name is located is marked as a training coverage window. This helps the model learn how to locate the position range of key information in the text. The second government training text annotation has training core characters. Taking a land planning government training text as an example, for the key information of land use, if "commercial land" is the key content, then the two words "commercial" may be marked as training core characters. In this way, the model can understand which characters have core significance for the expression of key information in different government contexts. The third government training text annotation has training categories corresponding to training text characters and training confidence corresponding to training categories. For example, in a tax administration training text, the part about the tax amount may be labeled as "finance-tax amount" and given a training confidence, assuming it is 0.9, to evaluate the accuracy of this classification. This helps the model learn how to classify the information in the government text and judge the credibility of the classification results. The computer system can collect these texts from multiple data sources. On the one hand, historical government documents can be extracted from existing government databases as training texts. These government databases store a large number of past government records and contain rich government information. On the other hand, some typical government cases can also be manually sorted and annotated to form government training texts. For example, staff of government departments can select some representative cases of enterprise registration, project approval, tax declaration, etc., annotate the key information in detail, and then provide these annotated cases as government training texts to the computer system.

[0079] When the computer system executes step S20, the function of the basic feature extraction branch component is to extract semantic features from the government training text, thereby obtaining training text features.

[0080] For example, when faced with a training text containing complex government information (such as an environmental impact assessment report for a city construction project), the basic feature extraction branch component needs to dig out the hidden semantic information. If a word vector model (such as Word2Vec) is used as a technical means, the computer system will map each word in the government training text to a vector. For words such as "environmental impact", "pollutant emissions", and "ecological protection measures" in the text, they will all be converted into corresponding vector representations. These vectors are not just simple numerical combinations, but contain semantic information of the words. For example, words with similar semantics are closer in the vector space. In addition to the word vector model, feature extraction methods based on deep learning can also be used, such as convolutional neural networks (CNNs). CNNs can automatically learn local features in government training texts. For example, for a text describing environmental impact assessment indicators, CNNs can identify the association patterns between different indicators as semantic features. Through these technical means, the computer system converts government training texts into training text features. These training text features are an abstract representation of the semantics of government training texts. They can more effectively reflect the key information and semantic relationships in government training texts, laying the foundation for subsequent training in different branch components.

[0081] In step S30, the computer system separately debugs different branch components in the basic language model to obtain a transitional large model.

[0082] First, load the training text features into the basic recognition branch component. Taking the processing of corporate financial auditing and government affairs training text as an example, the goal of the basic recognition branch component is to identify key elements in the training text features, such as specific items in financial statements, key statements in audit opinions, etc. During the debugging process, the computer system will evaluate the output results of the basic recognition branch component according to pre-set standards. Assuming that for a training text containing financial data and audit opinions, the basic recognition branch component should be able to accurately identify the location and related statements of key financial indicators such as "operating income" and "net profit" in the training text features. If there is a deviation between the recognition result and the correct result of the annotation, the computer system will adjust the parameters of the basic recognition branch component, such as the weights and biases in the neural network, to improve the accuracy of recognition.

[0083] Next, the training text features are loaded into the basic core character recognition branch component. For example, when processing the content about the names of policies and regulations in the government training text, the basic core character recognition branch component must find the core characters that best represent the names of policies and regulations. For example, for the policy and regulatory name "Environmental Protection Law", the two words "environment" and "protection" may be core characters. The computer system will judge the performance of the component by comparing the recognition results of the component with the annotated training core characters. If the recognized core characters are inconsistent with the annotations, the parameters of the basic core character recognition branch component will be adjusted so that it can recognize the core characters more accurately.

[0084] Finally, the training text features are loaded into the basic segmentation branch component. Taking the training text of government news reports as an example, the basic segmentation branch component needs to segment and classify the training text features. For example, the event subject, event process, event impact and other parts in the news report are segmented and classified into different categories, such as "government subject category", "government event category", "government impact category", etc. The computer system will evaluate the performance of the basic segmentation branch component based on the consistency of the classification results and the labeled training classification. If there is a classification error or inaccuracy, the parameters of the component will be adjusted.

[0085] By debugging these three branch components separately, the computer system finally obtains a transitional large model, which includes a transitional recognition branch component, a transitional core character recognition branch component, and a transitional segmentation branch component. These transitional components are obtained through preliminary debugging based on the basic components, and their performance has been improved to a certain extent in processing government training texts compared to the basic components.

[0086] When the computer system executes step S40, multi-task learning is a learning method that can make full use of the correlation between different tasks to improve model performance. In the context of government text processing, the transition recognition branch component, the transition core character recognition branch component, and the transition segmentation branch component in the transition model each undertake different but related tasks. For example, when processing a government text about a government public service project, the transition recognition branch component may be responsible for identifying the name of the project, the service object, and other information; the transition core character recognition branch component is responsible for finding the core characters in this information, such as the key words in the project name; the transition segmentation branch component is responsible for classifying this information, such as classifying the project name into the basic information class of the project, and the service object into the service target class. Multi-task learning improves the overall performance of the model by sharing information between these tasks. Specifically, the computer system can use a neural network structure with shared parameters to implement multi-task learning. For example, in a multi-layer neural network, the bottom network layer can be shared by multiple tasks, while the top network layer is specifically set according to different tasks. In this way, during the training process, different tasks can influence and promote each other. To explain with a simple mathematical model, assuming that the loss function of the transition recognition branch component is , the loss function of the transition core character recognition branch component is , the loss function of the transition split branch component is In multi-task learning, a total loss function can be constructed ,in and is a weight parameter that indicates the relative importance of each task in multi-task learning. The computer system adjusts the parameters of the transition model by minimizing this total loss function, so that the performance of the model on each task can be improved.

[0087] In the process of multi-task learning, as training progresses, the transitional large model will gradually learn the intrinsic connections and rules between different tasks in government text processing. For example, it will understand the relationship between the recognition of certain key information and specific core characters, as well as the commonalities and differences in the classification of this information. Finally, after multi-task learning, the transitional large model is transformed into a natural language large model, which can process government texts more effectively and accurately identify key elements such as key information coverage windows, key information core characters, information classifications corresponding to key information characters, and classification confidences corresponding to information classifications.

[0088] As an implementation method, the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component are debugged separately based on the basic feature extraction branch component to obtain a transitional large model, which specifically includes:

[0089] Step SA: debug the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component one by one based on the basic feature extraction branch component based on the collaborative training error metric to obtain a transitional large model, wherein the collaborative training error metric is obtained by fusing the first training error metric, the second training error metric and the third training error metric. When debugging to obtain the transitional recognition branch component, the second training error metric and the third training error metric are ignored. When debugging to obtain the transitional core character recognition branch component, the first training error metric and the third training error metric are ignored. When debugging to obtain the transitional segmentation branch component, the first training error metric and the second training error metric are ignored.

[0090] Based on this, step S40 performs multi-task learning on the transition large model to obtain a natural language large model, which specifically includes:

[0091] Step S41: multi-task learning is performed on the transition large model based on the collaborative training error metric to obtain a natural language large model, wherein during the multi-task learning, the first training error metric, the second training error metric and the third training error metric are not ignored.

[0092] When the computer system executes step SA, it debugs the three important branch components (basic recognition branch component, basic core character recognition branch component and basic segmentation branch component) in the basic language model based on the collaborative training error metric to obtain a transitional large model.

[0093] First, the collaborative training error metric is a comprehensive evaluation indicator, which is a fusion of the first training error metric, the second training error metric, and the third training error metric. These different training error metrics correspond to different debugging goals and tasks.

[0094] Taking the training of a large natural language model for processing government documents as an example, it is assumed that the first training error metric is mainly used to evaluate the accuracy of the basic recognition branch component in identifying key elements in government documents. For example, in a government document of a corporate tax return, key elements may include the company name, tax amount, tax period, etc. After the computer system inputs the training text into the basic recognition branch component, it obtains the recognition results of these key elements, and then compares them with the pre-annotated correct results to calculate the first training error metric. This metric can be calculated in a variety of ways, such as mean square error (MSE). The second training error metric may focus on the accuracy of the basic core character recognition branch component in recognizing core characters. For example, in the recognition of the names of policies and regulations, for the "Environmental Protection Law", the accurate recognition of the core characters "environment" and "protection" is very important. The computer system will compare the recognition results of the core characters with the correctly annotated core characters to calculate the second training error metric. The third training error metric is mainly aimed at the accuracy of the basic segmentation branch component in segmenting and classifying government texts. For example, in a government news report, the text needs to be segmented into parts such as the event subject, the event process, and the event impact, and accurately classified. The computer system will calculate the third training error metric based on the comparison of the segmentation and classification results with the correctly labeled results.

[0095] When debugging the basic recognition branch component, the computer system only focuses on the first training error metric, while ignoring the second training error metric and the third training error metric. This is because at this stage, the main goal is to optimize the recognition ability of the basic recognition branch component for key elements. For example, in the processing of the government documents of corporate tax declaration, focus on enabling the basic recognition branch component to accurately identify key elements such as the company name and tax amount without being disturbed by the results of core character recognition and text segmentation and classification. The computer system adjusts the parameters of the basic recognition branch component, such as weights and biases in the neural network, according to the value of the first training error metric. If the first training error metric is large, it means that the recognition result deviates greatly from the correct result, and the parameters need to be adjusted to reduce the error. When debugging the basic core character recognition branch component, the computer system only relies on the second training error metric, ignoring the first training error metric and the third training error metric. This is because the focus at this time is on improving the accuracy of core character recognition. For example, for government texts related to the names of policies and regulations, only whether the core characters are accurately recognized is considered, without considering other factors. By adjusting the parameters of the basic core character recognition branch component according to the second training error metric, it can more accurately recognize core characters such as "environment" and "protection".

[0096] Similarly, when debugging the basic segmentation branch component, the computer system only considers the third training error metric and excludes the influence of the first training error metric and the second training error metric. For example, when processing government news reports and government texts, the computer system focuses entirely on the accuracy of segmentation and classification, and adjusts the parameters of the basic segmentation branch component according to the third training error metric to ensure that the event subject, event process, event impact and other parts can be correctly segmented and classified.

[0097] By extracting branch components based on basic features and debugging the three branch components one by one according to different debugging focuses (i.e., the corresponding training error metrics), the computer system finally obtains a transitional large model. The transitional recognition branch component, transitional core character recognition branch component, and transitional segmentation branch component in this transitional large model have achieved certain performance improvements in their respective tasks compared to the basic components, laying the foundation for subsequent multi-task learning.

[0098] When the computer system executes step S41, multi-task learning is performed on the transition large model based on the collaborative training error metric to obtain a natural language large model.

[0099] Multi-task learning is a learning strategy that uses the relationship between multiple related tasks to improve model performance. In this context, the tasks undertaken by the transition recognition branch component, the transition core character recognition branch component, and the transition segmentation branch component in the transition large model are interrelated.

[0100] Taking government document processing as an example, in the multi-task learning process, the computer system must simultaneously consider the performance of these three components when processing government documents. For example, for a complex government document containing enterprise information and government policy associations, the transition recognition branch component identifies key information related to the enterprise (such as enterprise name, business scope, etc.), the transition core character recognition branch component finds the core characters in these key information (such as "enterprise", "operation", etc.), and the transition segmentation branch component reasonably classifies this information (such as enterprise information category, government policy association category, etc.). These three tasks affect each other.

[0101] The collaborative training error metric at this time comprehensively considers the first training error metric, the second training error metric, and the third training error metric. In the multi-task learning process, the computer system will construct a total loss function, which contains the weighted sum of the three training error metrics. The computer system adjusts the parameters of the transition large model by minimizing this total loss function. In this process, since all three training error metrics are taken into account, the model will take into account the performance improvement of the three components at the same time during the optimization process. For example, if the first training error metric (related to the transition recognition branch component) is large, it means that there are more errors in identifying key information, and the model will adjust the parameters to improve performance in this regard; at the same time, if the second training error metric (related to the transition core character recognition branch component) is also large, that is, the core character recognition is inaccurate, the model will also adjust the relevant parameters, and this adjustment will take into account the mutual influence with other tasks (such as identifying key information and segmentation and classification tasks); the same is true for the third training error metric (related to the transition segmentation branch component).

[0102] This multi-task learning approach enables the transitional large model to fully utilize the correlation and complementarity between different tasks. For example, accurate identification of key information helps to more accurately determine the core characters, and accurate identification of core characters may have a positive impact on the segmentation and classification of information, and vice versa. By continuously adjusting parameters to minimize the total loss function, the transitional large model gradually learns more complex rules and features of government text processing during the multi-task learning process, and is eventually transformed into a natural language large model. This natural language large model can process government texts more effectively, accurately identify key information in government texts, find core characters, perform reasonable segmentation and classification, and give accurate classification confidence and other results.

[0103] As an implementation mode, step S30, respectively loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model, including:

[0104] Step S31: when the training text feature is related to the first government training text, the training text feature is loaded into the basic recognition branch component to generate an inference coverage window;

[0105] Step S32: determining a first training error metric based on the inference coverage window and the training coverage window;

[0106] Step S33: if the first training error metric does not meet the first convergence requirement, modifying the network parameters of the basic feature extraction branch component and the basic recognition branch component;

[0107] Step S34: If the first training error metric reaches the first convergence requirement, the basic recognition branch component is determined as a transition recognition branch component.

[0108] When the computer system executes step S31, it first determines whether the training text feature is related to the first government training text. The relevance here refers to the correlation between the semantic information contained in the training text feature and the government theme, key information, etc. involved in the first government training text. For example, in the field of government affairs, if the first government training text is about enterprise registration, then the training text features related thereto may include relevant semantic information such as enterprise name, registered address, and registered capital.

[0109] After determining the relevance, the computer system loads the training text features into the basic recognition branch component. The basic recognition branch component is a part of the basic language model. Its main function is to perform recognition operations on the input text features to determine the relevant features of the key information. In this process, generating an inference coverage window is an important operation. The inference coverage window represents the range of possible key information in the input training text features inferred by the computer system based on the algorithm of the basic recognition branch component.

[0110] Taking the government affairs scenario of enterprise registration as an example, assume that the training text features contain a series of vector representations describing the information related to enterprise registration. The basic recognition branch component may determine a range in this vector representation space based on predefined rules or learned patterns, and the part of the original text corresponding to this range constitutes the reasoning coverage window. For example, if the training text features are obtained by converting a piece of enterprise registration text through a word vector model, the basic recognition branch component may identify the range of word vectors related to the enterprise name, and the sentence or paragraph in the original text corresponding to this range is considered to be the reasoning coverage window, because this range may contain the key information of the enterprise name.

[0111] Specifically, the computer system can use a pattern matching-based algorithm to implement this operation. For example, for known key information types (such as company names, registered addresses, etc.), the computer system searches for matching patterns in the training text features. Assuming that company names usually follow a certain format (such as containing specific industry vocabulary, organizational form vocabulary, etc.), the computer system searches for parts that match this format in the training text features, determines its position and range, and thus generates an inference coverage window.

[0112] In step S32, the computer system determines the first training error metric based on the obtained reasoning coverage window and the pre-marked training coverage window. The training coverage window is the range where the key information that has been accurately marked in the first government training text is located, and it is a standard reference. The reasoning coverage window is the range where the key information that the computer system infers through the basic recognition branch component may be located.

[0113] For example, in the business registration scenario mentioned above, if the training coverage window accurately marks the specific sentence range of the business name in the original text, and the inference coverage window is the range where the computer system infers that the business name may be located. The computer system compares the difference between the two windows to determine the first training error metric. One method for calculating the first training error metric can be to calculate the overlap ratio and non-overlap ratio of the two windows. Assume that the length of the training coverage window (which can be the number of characters or vectors contained, etc.) is , the length of the inferred coverage window is , the length of their overlapping part is . Then the first training error metric It can be calculated by the following formula: ; This formula indicates that when the inference coverage window completely overlaps with the training coverage window (i.e. ), the first training error metric is 0, indicating a perfect match; and when the two windows have no overlap (i.e. =0), the first training error metric is 1, indicating a complete mismatch. In this way, the first training error metric can quantitatively reflect the degree of difference between the inference coverage window and the training coverage window, thereby evaluating the accuracy of the basic recognition branch component in identifying the key information range.

[0114] When the computer system finds in step S33 that the first training error metric does not meet the first convergence requirement, this means that the basic recognition branch component has not achieved satisfactory results in identifying the key information range (i.e., the degree of matching between the inference coverage window and the training coverage window), and the relevant components need to be adjusted.

[0115] The first convergence requirement is a pre-set threshold value used to determine whether the training of the basic recognition branch component has reached an acceptable level. For example, this threshold value can be set to 0.1. If the first training error metric is greater than 0.1, it is considered that the convergence requirement has not been met.

[0116] When both the basic feature extraction branch component and the basic recognition branch component are constructed based on a neural network, network parameters (such as weights and biases in the neural network) determine their functions and performances. The purpose of the computer system correcting these network parameters is to enable the basic recognition branch component to better identify the key information range, thereby reducing the first training error metric.

[0117] Specifically, the computer system can use a back-propagation algorithm to correct the network parameters. The back-propagation algorithm back-propagates the error signal from the output layer to the input layer based on the calculated error (here, the first training error metric), and adjusts the network parameters of each layer according to the error signal. For example, for a basic recognition branch component of a multi-layer neural network structure, assuming that there is an error between the predicted result of the output layer (i.e., the inference coverage window) and the true result (i.e., the training coverage window), the back-propagation algorithm will calculate the gradient of each layer based on this error, and then update the network parameters according to a certain learning rate. By continuously adjusting the network parameters according to the error, the performance of the basic recognition branch component will gradually improve, making the inference coverage window closer to the training coverage window.

[0118] In step S34, when the computer system determines that the first training error metric has reached the first convergence requirement, this indicates that the basic recognition branch component has reached an acceptable level in recognizing the key information range. At this time, the computer system determines the basic recognition branch component as a transition recognition branch component.

[0119] The transition recognition branch component is a component that has been trained and optimized to a certain extent. Compared with the basic recognition branch component, it has better performance in identifying the scope of key information. For example, in the processing of government texts for enterprise registration, if the basic recognition branch component has been trained and the first training error metric in identifying the scope of key information such as enterprise name and registered address reaches the pre-set first convergence requirement (such as less than 0.1), then it will be upgraded to a transition recognition branch component, which will be used in subsequent more complex training processes (such as multi-task learning, etc.).

[0120] This process ensures that the basic recognition branch components are incorporated into the component system for building more advanced models (such as transitional large models) only after they reach a certain accuracy standard. Through this gradual training and screening process, the accuracy and effectiveness of the entire natural language large model in processing government texts can be improved, allowing the model to more accurately identify the location range of key information in the text, providing a more reliable foundation for subsequent key information core character recognition, information classification and other operations.

[0121] As an implementation mode, step S31, when the training text feature is related to the first government training text, loading the training text feature into the basic recognition branch component to generate an inference coverage window specifically includes:

[0122] Step S311: Load the training text features into the basic recognition branch component, determine the inference core character, the inference core character context range, and the number of inference characters, wherein the inference core character and the inference core character context range represent the position of the inference coverage window, and the number of inference characters represents the dimension of the inference coverage window;

[0123] Step S312: Generate a reasoning coverage window by combining the reasoning core character, the reasoning core character context range and the number of reasoning characters.

[0124] When the computer system executes step S311, after loading the training text features into the basic recognition branch component, it starts to determine the inference core characters, the inference core character context range, and the number of inference characters.

[0125] The training text features are obtained after feature extraction of the government training text, and they contain the semantic information of the government training text. The basic recognition branch component aims to identify elements related to key information from these features. Taking a government training text about corporate tax declaration as an example, the training text features may be vector representations obtained through word vector conversion and other methods. These vectors contain semantic information related to tax declaration, such as company name, tax amount, and tax deadline.

[0126] Inference core characters refer to characters that are closely related to key information and representative in the training text features and are recognized by the basic recognition branch component. In the example of corporate tax declaration, if the key information is the tax amount, then specific characters related to the tax amount (such as characters representing the amount, currency unit characters, etc.) may be determined as inference core characters. For example, "10000" and "yuan" in "10000 yuan" may be inference core characters.

[0127] The context range of the core character of inference is a certain range of text features around the core character of inference. It helps to determine the positional relationship of the core character of inference in the entire text. Taking the tax amount as an example, the context range of the core character of inference may include some words related to the amount description before and after "10,000 yuan", such as "tax payable amount" and "total". This context range can be determined by setting certain rules or based on the patterns learned by the model. For example, the range of several word vectors before and after the core character of inference can be set as the context range.

[0128] The number of inference characters indicates the dimension of the inference coverage window, which is a quantitative representation of the size of the inference coverage window. After determining the core inference character and its context range, the number of inference characters is the number of characters or vectors contained in this range. For example, if the core inference character is "10,000 yuan" and its context range "tax payable amount" and "total" corresponding to the vector representation, which contains a total of 10 vectors, then the number of inference characters is 10.

[0129] Specifically, the computer system may use an algorithm based on semantic analysis to determine the core characters of reasoning. For example, for the vector representation in the training text features, the similarity between each vector and a predefined key information semantic template (such as a semantic template of the tax amount) may be calculated. Assuming that the vector represents an element in the training text feature, the tax amount semantic template is T, and the similarity calculation can use the cosine similarity formula: ;in, is the vector dot product, They are vectors and the modulus of the semantic template T. The computer system can select the character corresponding to the vector with the highest similarity as the core character for reasoning.

[0130] To determine the context range of the core characters of inference, a window sliding method can be used. Set a window size (e.g., the window size in units of word vectors is 5), and slide the window forward and backward with the vector corresponding to the core characters of inference as the center. The vector range within the window constitutes the context range of the core characters of inference.

[0131] In step S312 , the computer system generates an inference coverage window according to the inference core character, the inference core character context range, and the number of inference characters determined in step S311 .

[0132] The inference coverage window is a range in the training text features that is considered to contain key information. It is determined by the inference core character and its context range, and the number of inference characters determines the size of this range. Taking the previous example of corporate tax declaration, assume that the inference core character is "10,000 yuan", the inference core character context range is "tax payable" and "total", and the number of inference characters is 10 (here it is assumed that each character or word corresponds to a unit of inference characters). The computer system will combine these elements to form an inference coverage window. This inference coverage window corresponds to a specific range in the training text features, which contains key information related to the tax amount and its related context information.

[0133] Specifically, if the training text features are regarded as an ordered vector sequence (assuming V=[v1,V2,…,V n]), the computer system first determines the vector position corresponding to the inference core character (assuming that the vector corresponding to the inference core character "10000 yuan" is v i ), and then determine the starting vector (assuming it is v) based on the inference core character context range. i-m , where m is the forward offset determined by the context scope) and the end vector (assumed to be v i+k , where k is the backward offset determined by the context range), and the number of vectors in this range is guaranteed to be equal to the number of inference characters. i-m to v i+k The vector range constitutes the reasoning coverage window. The generation of this reasoning coverage window is very important for subsequent steps (such as determining the first training error metric based on the reasoning coverage window and the training coverage window in step S32). It provides a basis for evaluating the accuracy of the basic recognition branch component in identifying the key information range. By comparing the reasoning coverage window and the pre-labeled training coverage window, the computer system can judge the performance of the basic recognition branch component and make adjustments based on the performance (such as correcting the network parameters of the relevant components if the performance does not meet the requirements in step S33), thereby continuously improving the ability of the basic recognition branch component in identifying the key information range, and ultimately achieving the goal of building an accurate and effective natural language large model.

[0134] As another implementation, step S30, loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model, specifically includes:

[0135] Step S301: when the training text feature is related to the second government training text, the training text feature is loaded into the basic core character recognition branch component to generate the inference core character;

[0136] Step S302: determining a second training error metric based on the inference core character and the training core character;

[0137] Step S303: if the second training error metric does not meet the second convergence requirement, modifying the network parameters of the basic feature extraction branch component and the basic core character recognition branch component;

[0138] Step S304: If the second training error metric reaches the second convergence requirement, the basic core character recognition branch component is determined as a transitional core character recognition branch component.

[0139] When the computer system executes step S301, it must first determine whether the training text features are related to the second government training text. The relevance here refers to the correlation between the semantic content contained in the training text features and the government content targeted by the second government training text. For example, if the second government training text is about land planning approval, then the training text features related to land use, land area, planning area, etc. are related to it. After determining the relevance, the computer system loads the training text features into the basic core character recognition branch component. The function of this component is to identify characters in the text features that have core significance for key information. Taking the government text of land planning approval as an example, for the part describing the land use as "commercial land", the basic core character recognition branch component needs to find the core characters therein.

[0140] In this process, generating inference core characters is a key operation. Inference core characters are characters that may be core characters inferred from the training text features based on the algorithms and logic of the computer system to identify branch components based on the basic core characters. For example, in the text representation of the land use as "commercial land", the two words "commercial" may be identified as inference core characters.

[0141] Specifically, the computer system can use a feature weight-based method to generate the core characters of reasoning. Assume that the training text features are vector representations obtained through a certain feature extraction method, and each vector element corresponds to a feature in the text (which may be a semantic representation of a word or phrase, etc.). The computer system can determine the importance of each feature based on a predefined weight matrix. For example, for lexical features related to land use, those words that are often used as key description parts in historical government texts may have higher corresponding weights. If the weight of a feature exceeds a preset threshold, then the character corresponding to this feature can be determined as the core character of reasoning.

[0142] In step S302, the computer system determines the second training error metric based on the obtained inference core characters and the pre-annotated training core characters. The training core characters are characters with core significance to key information that are accurately annotated in the second government training text and are a standard reference. The inference core characters are possible core characters inferred by the computer system through the basic core character recognition branch component.

[0143] For example, in the case of land planning approval, if the training core characters are marked as "commercial" and the inference core characters are also "commercial", this indicates a high accuracy in core character recognition. On the contrary, if the inference core characters are different from the training core characters, there is an error.

[0144] One way to calculate the second training error metric can be based on the accuracy of character matching. Assume that the training core character set is , the inference core character set is . You can define a matching function ,when hour, ,otherwise .

[0145] Then the second training error metric It can be calculated by the following formula:

[0146] ;

[0147] This formula indicates that if all training core characters can find a match in the inference core characters (i.e., the numerator is equal to n), then the second training error metric is 0, indicating a complete match; if no training core character can find a match in the inference core characters (i.e., the numerator is 0), then the second training error metric is 1, indicating a complete mismatch. In this way, the second training error metric can quantitatively reflect the degree of difference between the inference core characters and the training core characters, thereby evaluating the accuracy of the basic core character recognition branch component in recognizing core characters.

[0148] When the computer system finds in step S303 that the second training error metric does not meet the second convergence requirement, this means that the basic core character recognition branch component has not achieved satisfactory results in recognizing core characters, and the relevant components need to be adjusted.

[0149] The second convergence requirement is a pre-set threshold value used to determine whether the training of the basic core character recognition branch component has reached an acceptable level. For example, this threshold value can be set to 0.1. If the second training error metric is greater than 0.1, it is considered that the convergence requirement has not been met.

[0150] When both the basic feature extraction branch component and the basic core character recognition branch component are constructed based on a neural network, network parameters (such as weights and bias parameters in the neural network) determine their functions and performances. The purpose of the computer system correcting these network parameters is to enable the basic core character recognition branch component to more accurately recognize the core characters, thereby reducing the second training error metric.

[0151] Specifically, the computer system can use the stochastic gradient descent (SGD) algorithm to correct the network parameters. The stochastic gradient descent algorithm gradually adjusts the network parameters in small batches of data based on the calculated error (here, the second training error metric). Assume that the network parameters are (including the parameters of the basic feature extraction branch component and the basic core character recognition branch component), the loss function is L( ) (here the second training error metric can be used as the loss function), and the learning rate is , for a small batch of data x and the corresponding label y (here the label is the core character of the training), the update formula of the network parameter is:

[0152] ;

[0153] in, is the loss function L( ) About network parameters By continuously adjusting the network parameters according to the error, the performance of the basic core character recognition branch component will gradually improve, making the inference core characters closer to the training core characters.

[0154] In step S304, when the computer system determines that the second training error metric has reached the second convergence requirement, this indicates that the basic core character recognition branch component has reached an acceptable level in recognizing core characters. At this point, the computer system determines the basic core character recognition branch component as a transitional core character recognition branch component.

[0155] The transitional core character recognition branch component is a component that has been trained and optimized to a certain extent. Compared with the basic core character recognition branch component, it has better performance in recognizing core characters. For example, in the processing of government texts for land planning approval, if the basic core character recognition branch component has been trained and the second training error metric in recognizing core characters of key information such as land use and land area reaches the pre-set second convergence requirement (such as less than 0.1), then it will be upgraded to the transitional core character recognition branch component, which will be used in subsequent more complex training processes (such as multi-task learning, etc.).

[0156] This process ensures that the basic core character recognition branch components are incorporated into the component system for building more advanced models (such as transitional large models) only after they reach a certain accuracy standard. Through this gradual training and screening process, the accuracy and effectiveness of the entire natural language large model in processing government texts can be improved, allowing the model to more accurately identify the core characters in key information, providing a more reliable foundation for subsequent information classification and other operations.

[0157] As an implementation mode, step S301, when the training text feature is related to the second government training text, loading the training text feature into the basic core character recognition branch component to generate the inference core character, specifically includes:

[0158] Step S3011: loading the training text features into the basic core character recognition branch component, determining the number of inference core characters and the distribution order and confidence of each inference core character;

[0159] Step S3012: The inference core characters are obtained by combining the number of inference core characters and the distribution order and confidence of each inference core character.

[0160] When the computer system executes step S3011, after loading the training text features into the basic core character recognition branch component, it begins to determine the number, distribution order and confidence of the inference core characters.

[0161] The training text features are the results obtained after feature extraction of the government training text, which contains the semantic information in the government training text. The basic core character recognition branch component aims to mine the information related to the characters that have core significance for key information from these features, that is, to infer the number, distribution order and confidence of the core characters.

[0162] Taking a government training text about the application of corporate scientific and technological innovation projects as an example, the training text features may be vector representations obtained through conversion of word vector models. These vectors contain semantic information related to scientific and technological innovation projects, such as project name, R&D team, innovative technology, and expected results.

[0163] Determining the number of inference core characters refers to the number of possible core characters determined by the computer system in the training text features based on the algorithm of the branch components identified by the basic core characters. For example, in the part describing the innovative technology, if "artificial intelligence algorithm optimization" is mentioned, the computer system may determine that both "artificial intelligence" and "algorithm optimization" are closely related to the key information, thereby determining that the number of inference core characters is 2. The distribution order of each inference core character reflects their positional relationship in the training text features. Continuing with the example of "artificial intelligence algorithm optimization", "artificial intelligence" may be in the front position in the text features, and "algorithm optimization" is after it, and this order is their distribution order. The computer system determines this order relationship by analyzing the vector sequence or semantic structure in the training text features. Specifically, the training text features can be regarded as an ordered sequence of semantic units. By marking the position index of each semantic unit, the computer system can determine the order of semantic units corresponding to different inference core characters.

[0164] Confidence is the computer system's credibility assessment of each core character of reasoning as a true core character. For each possible core character of reasoning, the computer system will assign a confidence value according to predefined rules or based on learned patterns. For example, in the application text of an enterprise's scientific and technological innovation project, if the term "artificial intelligence" often appears as a key description part in a large number of similar successful application projects, then the confidence level when it is determined as a core character of reasoning may be higher, assuming 0.8; and if "algorithm optimization" appears relatively less frequently, its confidence level may be 0.6.

[0165] Specifically, the number, distribution order and confidence of the core characters for inference can be determined by using a method based on semantic relevance. Assume that the training text feature vector is , for each vector , the computer system calculates its semantic relevance with a predefined key information semantic template (e.g., a key information template for a scientific and technological innovation project). Semantic relevance can be calculated in a variety of ways, such as based on the cosine similarity formula: ;in, is the vector dot product, They are vectors and the modulus of the semantic template T. For the characters or character combinations corresponding to the vectors whose semantic relevance exceeds a certain threshold, the computer system regards them as potential inference core characters. By further analyzing the distribution of these potential inference core characters in the vector sequence, their number and distribution order can be determined. The calculation of confidence can be based on the frequency statistics of occurrence in historical government training texts. For example, suppose the number of occurrences of a potential inference core character in historical related texts is , the total number of samples is N, then the confidence can be simply expressed as: .

[0166] In step S3012, the computer system obtains the inference core characters according to the number of inference core characters, the distribution order of each inference core character and the confidence level determined in step S3011.

[0167] The inference core characters are characters with core significance for key information determined by combining multiple factors after the basic core character recognition branch component analyzes the training text features.

[0168] Taking the previous example of the enterprise science and technology innovation project application, it is assumed that the number of inference core characters determined in step S3011 is 2, namely "artificial intelligence" and "algorithm optimization", "artificial intelligence" is distributed in the first order with a confidence of 0.8, and "algorithm optimization" is distributed in the last order with a confidence of 0.6. The computer system will combine this information to finally determine the inference core characters.

[0169] Specifically, the computer system can set a screening threshold based on the confidence level. For example, if the threshold is set to 0.5, if the confidence level of a core character of reasoning is lower than this threshold, it may be excluded. In this example, the confidence levels of "artificial intelligence" and "algorithm optimization" are both higher than 0.5, so both will be retained as core characters of reasoning.

[0170] At the same time, the distribution order also plays a role in determining the core characters of reasoning. In actual government text processing, different character orders may represent different semantic relationships. For example, although "artificial intelligence algorithm optimization" and "algorithm optimization artificial intelligence" contain the same vocabulary, their semantics may be different. By considering the distribution order, the computer system can more accurately determine the combination of the core characters of reasoning.

[0171] This method of obtaining the inference core characters by comprehensively considering the number, distribution order and confidence of the inference core characters helps to improve the accuracy of the basic core character recognition branch component in recognizing core characters. In subsequent steps (such as determining the second training error metric based on the inference core characters and the training core characters in step S302), these accurately determined inference core characters will serve as an important basis for evaluating the performance of the basic core character recognition branch component. If the inference core characters are not accurately determined, it may cause the second training error metric to be large, thereby triggering the correction of the network parameters of the basic feature extraction branch component and the basic core character recognition branch component (such as step S303). Only by accurately determining the inference core characters can the basic core character recognition branch component be continuously optimized during the training process, and ultimately achieve better performance, thereby contributing to the construction of an effective natural language large model.

[0172] As another implementation, step S30, respectively loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model, specifically includes:

[0173] Step S30a: when the training text feature is related to the third government training text, the training text feature is loaded into the basic segmentation branch component, and an inference classification corresponding to the inference text character and an inference confidence corresponding to the inference classification are generated;

[0174] Step S30b: determining a third training error metric based on the inference classification, the inference confidence and the training classification and the training confidence;

[0175] Step S30c: when the third training error metric does not meet the third convergence requirement, modifying network parameters of the basic feature extraction branch component and the basic segmentation branch component;

[0176] Step S30d: When the third training error metric reaches the third convergence requirement, the basic split branch component is used as a transition split branch component.

[0177] When the computer system executes step S30a, it first determines whether the training text feature is related to the third government training text. The correlation here means that the semantic content contained in the training text feature is related to the government information involved in the third government training text. For example, if the third government training text is about emergency response to public health events, then the training text features related to the location of the event, the people affected by the event, the response measures, etc. are related to it.

[0178] After determining the relevance, the computer system loads the training text features into the basic segmentation branch component. The main function of the basic segmentation branch component is to segment the text and classify the segmented parts, thereby generating the inference classification corresponding to the inference text characters and the inference confidence corresponding to the inference classification. Taking a government training text for emergency response to public health events as an example, the training text features may be vector representations after feature extraction, containing information about various aspects of the event, such as "suspected infectious disease cases have been found in a certain city, and the surrounding communities have been isolated and controlled, and the medical team is conducting testing and treatment work" and other relevant semantic information. The basic segmentation branch component will analyze these training text features to determine the category to which each part should belong. For example, "a certain city" may be classified as the event location class, "suspected infectious disease cases" may be classified as the event subject class, "surrounding communities are isolated and controlled" may be classified as the response measures class, and "medical teams are conducting testing and treatment work" may be classified as the event handling class, etc. These are inference classifications.

[0179] For each inference classification, the computer system also determines an inference confidence. The inference confidence indicates the degree of confidence of the computer system in the correctness of this classification. For example, for "a certain city" classified as the location of the event, since the description of the location usually has clear classification characteristics in most public health event-related texts, the inference confidence of this inference classification may be relatively high, assuming it is 0.9; while for some more complex or semantically ambiguous parts, the inference confidence may be relatively low.

[0180] Specifically, the computer system can use rule-based and statistical methods to generate inference classifications and inference confidence. For rule-based methods, some classification rules are predefined. For example, the definition rule for the place class may be that the part containing words such as city names, place names, or geographic area identifiers is classified as a place class. For statistical-based methods, a large amount of government training texts can be used to count the frequency of correlation between each part and each category. Assuming that a word or phrase is associated with a certain category in 80% of the public health emergency response texts, then when it appears in the current training text features, the inference confidence of being classified into this category can be set to 0.8.

[0181] In step S30b, the computer system determines a third training error metric based on the obtained inference classification, inference confidence, and pre-labeled training classification and training confidence.

[0182] The training classification is the correct classification accurately marked in the third-party training text and is a standard reference. The inference classification is the classification result inferred by the computer system through the basic segmentation branch component. The training confidence and inference confidence are the confidence values ​​corresponding to the training classification and inference classification respectively.

[0183] For example, in a government training text on emergency response to public health incidents, if the training classification labels "a certain city" as the location of the incident and the training confidence is 0.95, the computer system's reasoning classification also classifies it as the location of the incident with a reasoning confidence of 0.9. This indicates that there is a high degree of consistency in this classification, but there are certain differences in confidence.

[0184] There are many ways to calculate the third training error metric. A simpler way is to comprehensively consider whether the classification is correct and the difference in confidence. Assume that the training classification set is , the corresponding training confidence set is ; The inference classification set is , the corresponding inference confidence set is .

[0185] Define a classification matching function ,when hour, ,otherwise . Then the third training error metric It can be calculated by the following formula:

[0186]

[0187] in, Representation and The corresponding reasoning classification, Representation and The corresponding reasoning confidence. This formula indicates that if the classifications match perfectly and the confidence is the same, this item is 0, indicating no error; if the classifications do not match or there is a difference in confidence, a corresponding error value will be generated. The third training error metric is obtained by comprehensive calculation of all training classifications, which can quantitatively reflect the overall difference between the reasoning classification and the training classification, including the difference in classification accuracy and confidence, thereby evaluating the accuracy of the basic segmentation branch component in text segmentation and classification.

[0188] When the computer system finds in step S30c that the third training error metric does not meet the third convergence requirement, this means that the basic segmentation branch component has not achieved satisfactory results in text segmentation and classification, and the relevant components need to be adjusted.

[0189] The third convergence requirement is a pre-set threshold used to determine whether the training of the basic split branch component has reached an acceptable level. For example, this threshold can be set to 0.1. If the third training error metric is greater than 0.1, it is considered that the convergence requirement has not been met.

[0190] When both the basic feature extraction branch component and the basic segmentation branch component are constructed based on neural networks, network parameters (such as weights and biases in the neural network) determine their functions and performances. The purpose of the computer system correcting these network parameters is to enable the basic segmentation branch component to perform text segmentation and classification more accurately, thereby reducing the third training error metric.

[0191] Specifically, the computer system can use an optimization algorithm, such as the Adaptive Moment Estimation (Adam) algorithm, to correct the network parameters. The Adam algorithm combines the ideas of the momentum method and the adaptive learning rate. Since it is a prior art, it will not be described here. By continuously adjusting the network parameters according to the third training error metric, the performance of the basic segmentation branch component will gradually improve, making the inference classification closer to the training classification, and the inference confidence closer to the training confidence.

[0192] In step S30d, when the computer system determines that the third training error metric has reached the third convergence requirement, this indicates that the basic segmentation branch component has reached an acceptable level in terms of text segmentation and classification. At this point, the computer system determines the basic segmentation branch component as a transitional segmentation branch component.

[0193] The transitional segmentation branch component is a component that has been trained and optimized to a certain extent. Compared with the basic segmentation branch component, it has better performance in text segmentation and classification. For example, in the processing of government texts for public health emergency response, if the basic segmentation branch component has been trained and the third training error metric reaches the pre-set third convergence requirement (such as less than 0.1) when segmenting and classifying key information such as the location of the incident, the subject of the incident, and the response measures, then it will be upgraded to a transitional segmentation branch component, which will be used in subsequent more complex training processes (such as multi-task learning, etc.).

[0194] This process ensures that the basic segmentation branch components are incorporated into the component system for building more advanced models (such as transitional large models) only after they reach a certain accuracy standard. Through this gradual training and screening process, the accuracy and effectiveness of the entire natural language large model in processing government texts can be improved, allowing the model to more accurately segment and classify information in government texts, providing a more reliable foundation for subsequent operations (such as information integration, key information extraction, etc.).

[0195] As an implementation mode, step S30a, when the training text feature is related to the third government training text, loads the training text feature into the basic segmentation branch component, generates an inference classification corresponding to the inference text character and an inference confidence corresponding to the inference classification, specifically including:

[0196] Step S30a1: Loading the training text features into the basic segmentation branch component to generate a text semantic vector, wherein the text semantic vector includes a preset number of classification labels, and each vector element in the text semantic vector represents the classification of the corresponding text character and the corresponding confidence level;

[0197] Step S30a2: Generate inference classification and inference confidence by combining text semantic vectors.

[0198] When the computer system executes step S30a1, after loading the training text features into the basic segmentation branch component, it starts to generate text semantic vectors. The training text features are the results obtained after feature extraction of the government training text, which contain the semantic information in the government training text.

[0199] The basic segmentation branch component is designed to process the training text features in a certain way to generate text semantic vectors that can reflect the text character classification information and its confidence. Taking a government training text on the allocation of educational resources as an example, the training text features may be vector representations extracted through technical means such as word vector models and syntactic analysis. These vectors contain semantic information related to the allocation of educational resources, such as the number of schools, the number of students, teacher resources, and the distribution of educational facilities.

[0200] The number of pre-set classification labels is predetermined according to the needs of the government text processing task. For example, in the government text processing of educational resource allocation, classification labels such as "school basic information category", "student related category", "teacher related category", "educational facilities category" may be set. Assume that a total of n classification labels are set. When the computer system processes the training text features in the basic segmentation branch component, it will generate a corresponding classification and confidence for each text character (the text characters here can be semantic units such as words and phrases). For example, for the text character "a certain school" in the description "a certain school has 1,000 students", it may be classified as "school basic information category", and the computer system will assign this classification a confidence value, assuming it is 0.8; for the text character "1,000 students", it may be classified as "student related category" with a confidence of 0.9.

[0201] These classification and confidence information are combined into a text semantic vector. If each classification label is regarded as a dimension, then the dimension of the text semantic vector is the number of classification labels n set in advance. Each vector element corresponds to the classification and confidence information of a text character. Specifically, the computer system can use a neural network-based classifier to generate a text semantic vector. For example, a multi-layer perceptron (MLP) is used. Assuming that the training text feature vector is x, after the forward propagation calculation of the multi-layer perceptron, the number of neurons in the output layer is equal to the number of classification labels n set in advance. For each output neuron j, its output value It can be converted into a probability value through some activation function (such as Softmax function), indicating the confidence of the corresponding classification. The formula of Softmax function is: ;in, is the confidence of the jth classification, is the value of the jth output neuron, and n is the number of classification labels. In this way, the classification and confidence corresponding to each text character are determined and combined into a text semantic vector.

[0202] In step S30a2 , the computer system generates an inference classification and an inference confidence according to the text semantic vector generated in step S30a1 .

[0203] Continuing with the example of the government training text on education resource allocation, assume that the text semantic vector has been obtained, in which each vector element represents the classification and confidence of the corresponding text character.

[0204] For inference classification, the computer system will determine the inference classification of the entire text based on the classification information of each vector element in the text semantic vector. For example, if there are multiple vector elements in the text semantic vector that are classified as "student-related classes", the computer system can determine that the part of the entire text that is related to students is the "student-related class" part in the inference classification. Specifically, if the vector elements corresponding to multiple words or phrases in a sentence are all classified as "student-related classes", then the sentence as a whole is classified as "student-related classes". For inference confidence, the computer system comprehensively considers the confidence of the vector elements in the text semantic vector that are related to the inference classification. A simple method is to take the average. Assume that when determining the inference classification of "student-related classes", there are m vector elements related to it, and their confidences are respectively , then the reasoning confidence P of this reasoning classification can be calculated by the following formula: ; For example, if the confidences of the vector elements corresponding to the text parts describing the number of students and the distribution of student scores are 0.9, 0.85, and 0.92 respectively, then the inference confidence of the inference classification of "student-related class" is .

[0205] This method of generating inference classification and inference confidence by combining text semantic vectors enables the computer system to perform preliminary classification and confidence assessment on the government training text according to the processing results of the basic segmentation branch component on the training text features. This is very important for subsequent steps (such as determining the third training error metric based on inference classification, inference confidence and training classification and training confidence in step S30b). If the inference classification and inference confidence are inaccurate, it may cause the third training error metric to be large, thereby affecting the training effect of the basic segmentation branch component (such as the need to correct the network parameters of the relevant components in step S30c). Only by accurately generating inference classification and inference confidence can the basic segmentation branch component be continuously optimized during the training process, and finally achieve better performance, so as to contribute to the construction of an effective natural language large model.

[0206] As an implementation mode, step S41, performing multi-task learning on the transition large model based on the collaborative training error metric to obtain the natural language large model, specifically includes:

[0207] Step S411: loading the government training text into the transition feature extraction branch component, performing feature extraction on the government training text, and generating transition text features, wherein the transition text features represent text features after the semantic features of the government training text are extracted by the transition feature extraction branch component;

[0208] Step S412: Loading the transition text features into the transition large model, and determining the collaborative training error metric;

[0209] Step S413: when the collaborative training error metric does not meet the collaborative convergence requirement, correct the network parameters of the transition large model;

[0210] Step S414: When the collaborative training error metric reaches the collaborative convergence requirement, the transitional large model is used as the natural language large model.

[0211] When the computer system executes step S411, the transition feature extraction branch component starts to operate on the government training text. The government training text is text data containing various government-related information, such as text content in scenarios such as government approval and policy interpretation.

[0212] The transition feature extraction branch component is part of the multi-task learning process. Its task is to extract more effective semantic features from the government training text and generate transition text features. The feature extraction in the previous basic model training may be different. The transition feature extraction branch component may have been optimized or adjusted to better meet the needs of multi-task learning.

[0213] Take a government training text on corporate tax policies as an example, which contains information on corporate types, turnover, tax types, tax incentives, and other aspects. The transition feature extraction branch component may use some advanced semantic analysis techniques to extract features. For example, it may use convolutional neural networks (CNN) or recurrent neural networks (RNN) and their variants (such as long short-term memory networks, LSTM) in deep learning. If CNN is used, it can automatically capture local semantic features in the text. For each word or phrase in the government training text, CNN can perform convolution operations by sliding the convolution kernel on the text to extract semantic features at different levels. Assuming that the government training text is represented as a word vector matrix X, the convolution kernel is K, and the feature map obtained after the convolution operation is C, the calculation process can be simply expressed as: ;

[0214] Among them, h and w are the height and width of the convolution kernel respectively. represents the convolution operation, b is the bias term, and f is the activation function (such as the ReLU function: f(x)=max(0,x)).

[0215] If RNN (LSTM for example) is used, it can handle long-distance dependencies in text. LSTM controls the flow of information through special gating mechanisms (input gate, forget gate, and output gate) to better capture the semantic information of the text. For each time step in the government training text (which can be the time step corresponding to each word), LSTM calculates based on the current input, the hidden state and cell state at the previous moment, and updates the hidden state and cell state.

[0216] Through these technical means, the transition feature extraction branch component extracts transition text features from the government affairs training text. These transition text features can better reflect the semantic information of the government affairs training text in the multi-task learning scenario compared to the text features obtained by the previous basic feature extraction, providing a more suitable data basis for the multi-task learning of the subsequent transition large model.

[0217] In step S412, the computer system loads the transition text features obtained in step S411 into the transition large model. The transition large model includes a transition recognition branch component, a transition core character recognition branch component, and a transition segmentation branch component, and these components have been preliminarily optimized during the previous separate debugging process.

[0218] The co-training error metric is a comprehensive evaluation metric used to measure the performance of the transition large model in multi-task learning. It is composed of a first training error metric, a second training error metric, and a third training error metric fused together, and these three training error metrics are respectively related to the transition recognition branch component, the transition core character recognition branch component, and the transition segmentation branch component.

[0219] For example, when processing a government affairs training text about the approval of a urban construction project, the transition recognition branch component is responsible for identifying key information such as the project name, construction location, and project scale; the transition core character recognition branch component is responsible for finding the core characters in this key information, such as the keywords in the project name; the transition segmentation branch component is responsible for classifying this information, such as classifying the construction location as geographical information, etc.

[0220] For the first training error metric, assume that there is a difference between the project name identified by the transition recognition branch component and the accurately labeled project name in the government affairs training text. This difference can be quantified by a certain distance metric (such as the edit distance) or a probability-based metric (such as cross-entropy). If the identified project name is , and the accurately labeled project name is y, the first training error metric can be calculated using the cross-entropy formula : ; for the second training error metric, taking the transition core character recognition branch component as an example, if the accurately labeled core character is "construction" and the identified core character is "building", the error can be calculated according to the accuracy of character matching. Assume is the matching function, when , , otherwise . Let the training core character set be , and the inference core character set be , then the second training error metric L_2 can be calculated by the following formula:

[0221] ;

[0222] For the third training error metric, taking the transition split branch component as an example, if an information is misclassified, such as classifying information that should belong to the project scale as project purpose, this will result in a classification error. The third training error metric can be calculated based on the difference between the classification result and the correct labeling result For example, we use the complement of classification accuracy to represent the number of correctly classified samples. , the total number of samples is N, then the third training error metric .

[0223] The collaborative training error metric L can be the weighted sum of these three training error metrics, with the weights being ,but By calculating the co-training error metric, the computer system can comprehensively evaluate the performance of the transition large model in multi-task learning and provide a basis for subsequent model parameter adjustments.

[0224] When the computer system finds in step S413 that the collaborative training error metric does not meet the collaborative convergence requirement, this means that the transitional large model has not achieved satisfactory performance in the multi-task learning process and its network parameters need to be corrected.

[0225] The collaborative convergence requirement is a pre-set threshold used to determine whether the multi-task learning of the transition large model has reached an acceptable level. For example, this threshold can be set to 0.1. If the collaborative training error metric is greater than 0.1, it is considered that the collaborative convergence requirement has not been met.

[0226] The network parameters of the transitional large model (such as weights and biases in the neural network) determine the function and performance of the model. The purpose of the computer system to correct these network parameters is to make the transitional large model better adaptable to multi-task learning, thereby reducing the collaborative training error metric.

[0227] Specifically, the computer system can use a gradient descent algorithm to correct the network parameters. In the context of multi-task learning, since the collaborative training error metric is a fusion of multiple training error metrics, the impact of each task needs to be considered when calculating the gradient.

[0228] By continuously adjusting the network parameters according to the collaborative training error metric, the performance of the transitional large model will gradually improve, so that the performance of each task (recognizing key information, recognizing core characters, information classification, etc.) can be optimized, thereby gradually reducing the collaborative training error metric.

[0229] In step S414, when the computer system determines that the collaborative training error metric has reached the collaborative convergence requirement, this indicates that the transitional large model has reached an acceptable level in the multi-task learning process. At this time, the computer system determines the transitional large model as the natural language large model.

[0230] The natural language big model is a model optimized through multi-task learning. Compared with the transitional big model, it has better performance in processing government texts. For example, when processing texts in various government scenarios, such as administrative approval, policy implementation supervision, etc., the natural language big model can more accurately identify key information, find core characters and perform reasonable information classification.

[0231] Taking the online service guide text in government services as an example, the natural language model can more accurately identify the conditions (key information) and core requirements (core characters) required to handle a certain matter, and correctly classify this information (such as classifying identity verification requirements as pre-service requirements, etc.). This makes government services more automated and intelligent, and improves the efficiency and accuracy of government processing.

[0232] This process ensures that the transitional large model is determined as a natural language large model only after it has undergone multi-task learning and reached a certain accuracy standard. Through this gradual training and optimization process, the effectiveness and accuracy of the entire natural language large model in processing government texts can be improved, providing strong technical support for intelligent government affairs processing.

[0233] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

Claims

1. A method for intelligent government affairs processing based on large models and process robot technology, characterized in that: include: Obtain the original government affairs text to be processed; The original government text is loaded into the natural language big model to obtain key information detection results, wherein the key information detection results include the key information coverage window, key information core characters, information classifications corresponding to the key information characters, and classification confidences corresponding to the information classifications identified in the original government text. The natural language big model is obtained by debugging a basic language model, and the basic language model includes a basic recognition branch component, a basic core character recognition branch component, and a basic segmentation branch component. After the basic recognition branch component, the basic core character recognition branch component, and the basic segmentation branch component are debugged, multi-task learning is performed on the basic language model to obtain the natural language big model. Combining the key information detection result and the original government text to generate a target government text, wherein the target government text is annotated with the key information coverage window, the key information core characters, the information classification, and the confidence level corresponding to the information classification; Calling a preset process robot to extract key information that meets the confidence requirement from the target government affairs text, and filling the extracted key information into a preset order form to obtain a key information entry result; The step of loading the original government text into a large natural language model to obtain key information detection results includes: The original government text is loaded into the target feature extraction branch component, and semantic features are extracted from the original government text to obtain text semantic features, wherein the text semantic features are text features after the semantic features of the original government text are extracted, and the natural language large model includes the target feature extraction branch component; Loading the text semantic features into the target recognition branch component, detecting the key semantics in the text semantic features, and obtaining the key information coverage window, wherein the natural language large model includes the target recognition branch component; Loading the text semantic features into a key information core character recognition branch component, detecting core characters in the text semantic features, and obtaining the key information core characters, wherein the natural language large model includes the key information core character recognition branch component; Loading the text semantic features into a target segmentation branch component to obtain an information classification set, wherein the information classification set includes classifications and corresponding confidences corresponding to the key information characters, and the natural language large model includes the target segmentation branch component; Generate the target government text by combining the key information overlay window, the key information core characters, the classification and confidence corresponding to the key information characters, and the original government text; The step of loading the original government text into the target feature extraction branch component, extracting semantic features from the original government text, and obtaining text semantic features includes: Loading the original government text into a first feature extraction branch component, extracting keywords from the original government text, and obtaining a first text feature, wherein the dimension of the first text feature is smaller than that of the original government text; The first text feature is loaded into the second feature extraction branch component, and the first text feature is dimensionally upgraded to obtain the text semantic feature, wherein the dimension of the text semantic feature is smaller than that of the original government text and larger than that of the first text feature.

2. The method according to claim 1, characterized in that Before loading the original government affairs text into the natural language big model to obtain the key information detection result, the method further includes: Acquire government affairs training texts, wherein the government affairs training texts include a first government affairs training text annotated with a training overlay window, a second government affairs training text annotated with training core characters, and a third government affairs training text annotated with training categories corresponding to training text characters and training confidences corresponding to the training categories; Loading the government training text into the basic feature extraction branch component to obtain training text features, wherein the training text features are text features after the semantic features of the government training text are extracted; The training text features are loaded into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model, wherein the transitional large model includes a transitional recognition branch component, a transitional core character recognition branch component and a transitional segmentation branch component, the transitional recognition branch component, the transitional core character recognition branch component and the transitional segmentation branch component are debugged independently according to the corresponding debugging branches, and the basic language model includes the basic feature extraction branch component; Multi-task learning is performed on the transition large model to obtain the natural language large model.

3. The method according to claim 2, characterized in that The basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component are debugged separately based on the basic feature extraction branch component to obtain a transitional large model, including: the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component are debugged separately based on the basic feature extraction branch component based on the collaborative training error metric to obtain the transitional large model, wherein the collaborative training error metric is obtained by fusing the first training error metric, the second training error metric and the third training error metric, when debugging to obtain the transitional recognition branch component, the second training error metric and the third training error metric are ignored, when debugging to obtain the transitional core character recognition branch component, the first training error metric and the third training error metric are ignored, and when debugging to obtain the transitional segmentation branch component, the first training error metric and the second training error metric are ignored; The performing multi-task learning on the transition large model to obtain the natural language large model includes: performing multi-task learning on the transition large model based on the collaborative training error metric to obtain the natural language large model, wherein during the multi-task learning, the first training error metric, the second training error metric and the third training error metric are not ignored.

4. The method according to claim 3, characterized in that The step of loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model includes: When the training text feature is related to the first government training text, loading the training text feature into the basic recognition branch component to generate an inference coverage window; determining the first training error metric based on the inference coverage window and the training coverage window; If the first training error metric does not meet the first convergence requirement, modifying network parameters of the basic feature extraction branch component and the basic recognition branch component; If the first training error metric meets the first convergence requirement, determining the basic recognition branch component as the transition recognition branch component; Wherein, when the training text feature is related to the first government training text, loading the training text feature into the basic recognition branch component to generate an inference coverage window includes: Loading the training text features into the basic recognition branch component, determining the inference core character, the inference core character context range, and the number of inference characters, wherein the inference core character and the inference core character context range represent the position of the inference coverage window, and the number of inference characters represents the dimension of the inference coverage window; The reasoning coverage window is generated by combining the reasoning core character, the reasoning core character context range and the number of reasoning characters.

5. The method according to claim 3, characterized in that: The step of loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model includes: When the training text feature is related to the second government training text, loading the training text feature into the basic core character recognition branch component to generate an inference core character; Determining the second training error metric based on the reasoning core character and the training core character; If the second training error metric does not meet the second convergence requirement, modifying the network parameters of the basic feature extraction branch component and the basic core character recognition branch component; If the second training error metric meets the second convergence requirement, determining the basic core character recognition branch component as the transitional core character recognition branch component; Wherein, when the training text feature is related to the second government training text, loading the training text feature into the basic core character recognition branch component to generate the inference core character includes: Loading the training text features into the basic core character recognition branch component, determining the number of inference core characters and the distribution order and confidence of each inference core character; The inference core characters are obtained by combining the number of the inference core characters and the distribution order and confidence of each of the inference core characters.

6. The method according to claim 3, characterized in that The step of loading the training text features into the basic recognition branch component, the basic core character recognition branch component and the basic segmentation branch component for separate debugging to obtain a transitional large model includes: When the training text feature is related to the third government affairs training text, the training text feature is loaded into the basic segmentation branch component to generate an inference classification corresponding to an inference text character and an inference confidence corresponding to the inference classification; Determining the third training error metric based on the inference classification, the inference confidence and the training classification, the training confidence; When the third training error metric does not meet the third convergence requirement, modifying network parameters of the basic feature extraction branch component and the basic segmentation branch component; When the third training error metric reaches the third convergence requirement, using the basic split branch component as the transition split branch component; Wherein, when the training text feature is related to the third government training text, the training text feature is loaded into the basic segmentation branch component, and the reasoning classification corresponding to the reasoning text character and the reasoning confidence corresponding to the reasoning classification are generated, including: Loading the training text features into the basic segmentation branch component to generate a text semantic vector, wherein the text semantic vector includes a preset number of classification labels, and each vector element in the text semantic vector represents the classification of the corresponding text character and the corresponding confidence; The inference classification and the inference confidence are generated in combination with the text semantic vector.

7. The method according to claim 3, characterized in that The performing multi-task learning on the transition large model based on the collaborative training error metric to obtain the natural language large model includes: Loading the government training text into a transition feature extraction branch component, performing feature extraction on the government training text, and generating transition text features, wherein the transition text features represent text features after the semantic features of the government training text are extracted based on the transition feature extraction branch component; Loading the transition text features into the transition large model, and determining the collaborative training error metric; When the collaborative training error metric does not meet the collaborative convergence requirement, modifying the network parameter variables of the transitional large model; When the collaborative training error metric reaches the collaborative convergence requirement, the transitional large model is used as the natural language large model.

8. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 7 are implemented.

Images