Text processing method and apparatus

Abstract

This disclosure relates to the field of computer technology, and more particularly to a text processing method and apparatus. The method includes: identifying input text using a knowledge engineering model and a target rule base to obtain a first output element set corresponding to the input text; identifying the input text using a large model to obtain a second output element set corresponding to the input text; obtaining a first overlap degree between the first output element set and the second output element set; if the first overlap degree is greater than a first overlap degree threshold, obtaining a target element set corresponding to the input text, and obtaining output text corresponding to the input text based on the target element set. Using this disclosure can improve the efficiency and accuracy of element extraction.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a text processing method and apparatus. Background Technology

[0002] With the development of science and technology, there are more and more types of text. However, when it is necessary to convert text to a preset format or to obtain parameters from a specific text during text review, manual operation is often required. For example, manual sentence-by-sentence screening can lead to low accuracy and efficiency in text extraction. Summary of the Invention

[0003] This disclosure provides a text processing method and apparatus that can improve the efficiency and accuracy of feature extraction. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a text processing method is provided, comprising: The input text is identified using a knowledge engineering model and a target rule base to obtain the first set of output elements corresponding to the input text; A large model is used to identify the input text, and a second set of output elements corresponding to the input text is obtained; Obtain the first overlap degree corresponding to the first output element set and the second output element set; If the first overlap is greater than the first overlap threshold, obtain the target element set corresponding to the input text, and obtain the output text corresponding to the input text based on the target element set.

[0004] According to some embodiments, the method further includes: The knowledge engineering model is used to segment unstructured historical text to obtain the word set corresponding to the unstructured historical text. Each word in the word segmentation set is labeled using a domain dictionary corresponding to the unstructured historical text, and the labeling information corresponding to each word is obtained. Obtain the first text embedding vector corresponding to the unstructured historical text; Based on the knowledge weight matrix and feature type bias term corresponding to the domain information of the unstructured historical text, the first text embedding vector is modified to obtain the second text embedding vector corresponding to the unstructured historical text. The annotation information corresponding to each word segment and the second text embedding vector corresponding to the unstructured historical text are associated and stored in the target rule base.

[0005] According to some embodiments, the method further includes: If the first overlap is less than the first overlap threshold, the knowledge engineering model and the target rule base are used to re-identify the input text to obtain the third output element set corresponding to the input text. Obtain the second overlap degree corresponding to the third output element set and the second output element set; If the second overlap degree is greater than the second overlap threshold, obtain the target element set corresponding to the input text; If the second overlap degree is less than the second overlap threshold, a prompt message is output. The prompt message includes the third output element set, the second output element set, and the difference information between the third output element set and the second output element set.

[0006] According to some embodiments, the method further includes: Obtain a first text and a second text corresponding to the first text, wherein the second text is the updated text of the first text; A similarity algorithm is used to obtain the similarity between the first text and the second text; Based on the similarity, obtain the update information of the target rule base; When the update information indicates that the target rule base needs to be updated, the knowledge engineering model is used to identify the second text, obtain the update prompt information corresponding to the target rule base, and send the update prompt information to the target account corresponding to the update prompt information according to the risk level information corresponding to the update prompt information.

[0007] According to some embodiments, the method further includes: Obtain the knowledge graph corresponding to the target element set; The large model and chain reasoning technique are used to complete the node relationships of the knowledge graph to obtain the processed knowledge graph; The processed knowledge graph is subjected to integrity verification, a verification report is generated, and the verification report is pushed to the target device.

[0008] According to some embodiments, the method further includes: Obtain the display format information corresponding to the selection instruction, wherein the selection instruction is used to select the business area, job information and risk level from the multi-dimensional tag set; The output text is displayed using the aforementioned display format information.

[0009] According to some embodiments, obtaining the output text corresponding to the input text based on the target element set includes: Obtain the input information corresponding to the display format information, and generate the display format information based on the input information; Receive processing instructions for the input text through the Application Programming Interface (API); Based on the processing instructions and the display format information, the target element set is formatted to obtain structured data corresponding to the input text.

[0010] According to a second aspect of the present disclosure, a text processing apparatus is provided, comprising: The set acquisition unit is used to identify the input text using a knowledge engineering model and a target rule base, and to acquire the first set of output elements corresponding to the input text; The set acquisition unit is also used to identify the input text using a large model and obtain the second output element set corresponding to the input text; The overlap degree acquisition unit is used to acquire the first overlap degree corresponding to the first output element set and the second output element set; The text processing unit is configured to, when the first overlap degree is greater than the first overlap degree threshold, obtain the target element set corresponding to the input text, and obtain the output text corresponding to the input text based on the target element set.

[0011] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the text processing method described in any one of the preceding aspects.

[0012] According to a fourth aspect of the present disclosure, a storage medium is provided that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the text processing method described in any one of the preceding aspects.

[0013] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in any one of the preceding aspects.

[0014] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: In some or related embodiments, the input text is identified using a knowledge engineering model and a target rule base to obtain a first set of output elements corresponding to the input text; a large model is used to identify the input text to obtain a second set of output elements corresponding to the input text; a first overlap degree is obtained between the first set of output elements and the second set of output elements; if the first overlap degree is greater than a first overlap degree threshold, a target element set corresponding to the input text is obtained, and output text corresponding to the input text is obtained based on the target element set. Therefore, target elements can be obtained through the association between elements and different output element acquisition methods without manual intervention, reducing the time required for manual intervention and minimizing the inaccuracies of manual screening. Furthermore, cross-validation of element extraction results from multiple models can improve the association between elements and the accuracy of element set acquisition, thereby improving element extraction efficiency and accuracy, and ultimately improving the accuracy of the output text.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0017] Figure 1 This is a flowchart of the first text processing method provided in the embodiments of this disclosure; Figure 2 This is a flowchart of the second text processing method provided in this disclosure embodiment; Figure 3 A schematic diagram illustrating an example of a system framework according to an embodiment of this disclosure is shown; Figure 4 A schematic diagram illustrating an example of a text processing method according to an embodiment of this disclosure is shown. Figure 5 A schematic diagram illustrating an example of the vector correction process according to an embodiment of this disclosure is provided. Figure 6 A schematic diagram illustrating an example of rule base updates according to an embodiment of this disclosure is provided. Figure 7 A schematic diagram illustrating an example of cross-validation in an embodiment of this disclosure is shown; Figure 8 This is a block diagram illustrating a text processing apparatus according to an exemplary embodiment; Figure 9 This is an example schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0019] This disclosure provides text processing methods, apparatus, electronic devices, and storage media. In some embodiments, the terms "text processing method" and "information processing method," "communication method," etc., can be used interchangeably; the terms "text processing apparatus" and "information processing apparatus," "communication apparatus," etc., can be used interchangeably; and the terms "information processing system," "communication system," etc., can be used interchangeably.

[0020] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0021] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0022] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure.

[0023] In this disclosure, unless otherwise stated, elements expressed in the singular form, such as "a," "an," "the," "the," "the," "the," "the," "the," "this," etc., can mean "one and only one," or "one or more," "at least one," etc. For example, when using articles such as "a," "an," "the," etc. in translation, the noun following the article can be understood as either a singular or a plural expression.

[0024] In the embodiments disclosed herein, "multiple" refers to two or more.

[0025] In some embodiments, the terms “at least one of,” “one or more,” “a plurality of,” and “multiple” may be used interchangeably.

[0026] The prefixes "first," "second," etc., used in the embodiments of this disclosure are merely for distinguishing different descriptive objects and do not impose restrictions on the position, order, priority, quantity, or content of the descriptive objects. The description of the descriptive objects is found in the claims or the context of the embodiments, and the use of prefixes should not constitute unnecessary restrictions. For example, if the descriptive object is a "field," the ordinal numbers preceding "field" in "first field" and "second field" do not restrict the position or order of the "fields." "First" and "second" do not restrict whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the descriptive object is a "level," the ordinal numbers preceding "level" in "first level" and "second level" do not restrict the priority between "levels." Furthermore, the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in "first device," the number of "devices" can be one or more. Furthermore, the objects modified by different prefixes can be the same or different. For example, if the object being described is "device", then "first device" and "second device" can be the same device or different devices, and their types can be the same or different. Similarly, if the object being described is "information", then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0027] In some embodiments, "terminal" or "terminal device" may be referred to as "user equipment (UE)," "user terminal," "mobile station (MS)," "mobile terminal (MT)," "subscriber station," "mobile unit," "subscriber unit," "wireless unit," "remote unit," "mobile device," "wireless device," "wireless communication device," "remote device," "mobile subscriber station," "access terminal," "mobile terminal," "wireless terminal," "remote terminal," "handset," "user agent," "mobile client," "client," etc.

[0028] In some embodiments, data, information, etc., may be obtained with the user's consent.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] Figure 1 This is a flowchart of the first text processing method provided in the embodiments of this disclosure, such as... Figure 1 As shown, this text processing method can be used in scenarios involving text detection and automated text generation, and includes the following steps: In step S11, the input text is identified using a knowledge engineering model and a target rule base to obtain the first set of output elements corresponding to the input text; In some embodiments, the implementing entity of this disclosure may be, for example, an electronic device. This electronic device does not specifically refer to a particular fixed electronic device. For example, when the device identifier changes, the electronic device may also change accordingly. For example, when the structure of the electronic device changes, the electronic device may also change accordingly. The name of the electronic device is not limited. The electronic device may, for example, be a processing device, a terminal, etc.

[0031] In some embodiments, the technical solutions of this disclosure can be used, for example, in scenarios involving the generation of enterprise compliance risk lists.

[0032] According to some embodiments, the knowledge engineering model can be, for example, a model obtained after training a large model. This knowledge engineering model can also be called a text recognition model, and it can be used to obtain output elements corresponding to input text. These output elements can be called key elements or necessary elements. The knowledge engineering model does not specifically refer to a fixed model. For example, when the training method or the large model changes, the knowledge engineering model can also change accordingly.

[0033] In some embodiments, the target rule base may be a rule base obtained by pre-processing historical text. This target rule base is not specifically defined by a single fixed rule base. For example, the target rule base may change accordingly when the method of generating the rule base changes. Similarly, the target rule base may change accordingly when the historical text changes.

[0034] In some embodiments, the input text may be, for example, the text to be processed. The input text may also be referred to as a list, table, etc. The input text does not specifically refer to any fixed text. For example, the input text may change when the time of its acquisition changes. Similarly, the input text may change when its content changes.

[0035] In some embodiments, the output elements may be elements obtained by recognizing the input text. The first set of output elements may be a collection of at least one output element obtained by the knowledge engineering model from the input text. The first set of output elements does not specifically refer to a fixed set. For example, when the number of output elements corresponding to the first set of output elements changes, the first set of output elements may also change accordingly. For example, when the knowledge engineering model or the target rule base changes, the first set of output elements may also change accordingly.

[0036] In some embodiments, a knowledge engineering model and a target rule base can be used to identify the input text and obtain the first set of output elements corresponding to the input text.

[0037] In step S12, a large model is used to identify the input text and obtain the second output element set corresponding to the input text; In some embodiments, the large model may be a trained model that can be used in the text analysis process. This large model may perform multiple operations, specifically different sub-models within the large model. Alternatively, there may be multiple large models, each performing different operations. This application does not limit the scope of these embodiments.

[0038] In some embodiments, the second output element set may be, for example, the output element set obtained by recognizing the input text using a large model. The "second" in this second output element set is used to distinguish it from other output element sets and does not specifically refer to a fixed output element set. For example, the second output element set may change accordingly when the number of output elements it includes changes. Similarly, the second output element set may change accordingly when a particular element included in it changes.

[0039] In some embodiments, a large model can be used to identify the input text and obtain a second set of output elements corresponding to the input text.

[0040] In step S13, the first overlap degree corresponding to the first output feature set and the second output feature set is obtained; In some embodiments, the overlap ratio can be used to indicate the proportion of identical elements among different sets of output elements. The first overlap ratio can be used to indicate the proportion of identical elements among a first set of output elements and a second set of output elements. This first overlap ratio does not specifically refer to a fixed overlap ratio. For example, the first overlap ratio can change when the first set of output elements or the second set of output elements changes. For example, the first overlap ratio can also change when the method of obtaining the overlap ratio changes.

[0041] In some embodiments, a first degree of overlap can be obtained between the first set of output elements and the second set of output elements.

[0042] Specifically, a large model can be used to recognize the input text, obtain a second set of output elements corresponding to the input text, and input it into the large model. The large model can receive the first set of output elements and compare it with the second set of output elements recognized by the large model to obtain a first degree of overlap. This disclosure does not limit the method of obtaining the first degree of overlap.

[0043] In step S14, if the first overlap degree is greater than the first overlap degree threshold, the target feature set corresponding to the input text is obtained, and the output text corresponding to the input text is obtained based on the target feature set.

[0044] According to some embodiments, the overlap threshold may be, for example, a threshold used to determine whether the overlap meets the requirements and whether the output element set should be used to output the text. This overlap threshold is not specifically a fixed threshold. For example, when a modification instruction for the overlap threshold is received, the overlap threshold may also change accordingly. For example, when obtaining the corresponding overlap threshold based on the text type of the input text, if the text type of the input text changes, the overlap threshold may also change accordingly.

[0045] According to some embodiments, the target feature set may be, for example, the features corresponding to the current input text. This target feature set may be, for example, a set of features used for information extraction or format conversion of the input text. The target feature set does not specifically refer to a fixed set of features. For example, when the features included in the target feature set change, the target feature set may also change accordingly.

[0046] In some embodiments, the output text may be the text obtained after extracting information from the input text. This output text does not specifically refer to a fixed text. For example, the output text may change accordingly when the input text or the target feature set changes.

[0047] According to some embodiments, when the first overlap degree is greater than the first overlap degree threshold, the target element set corresponding to the input text is obtained, and the output text corresponding to the input text is obtained based on the target element set.

[0048] In some or related embodiments, the input text is identified using a knowledge engineering model and a target rule base to obtain a first set of output elements corresponding to the input text; a large model is used to identify the input text to obtain a second set of output elements corresponding to the input text; a first overlap degree is obtained between the first and second output element sets; if the first overlap degree is greater than a first overlap degree threshold, a target element set corresponding to the input text is obtained, and output text corresponding to the input text is obtained based on the target element set. Therefore, target elements can be obtained through the association between elements and different output element acquisition methods without manual intervention, reducing the time required for manual intervention and minimizing the inaccuracies of manual screening. Furthermore, cross-validation of element extraction results from multiple models can improve the association between elements and the accuracy of element set acquisition, thereby improving element extraction efficiency and accuracy, and increasing the accuracy of output text output. In addition, it can improve vector retrieval accuracy and the recall rate of elements after vector correction, improving recognition accuracy compared to a single pre-trained model, providing a standardized data foundation for subsequent element extraction, and reducing extraction errors caused by data noise.

[0049] Figure 2 This is a flowchart of the second text processing method provided in the embodiments of this disclosure, such as... Figure 2 As shown, it includes the following steps: In step S21, the input text is identified using a knowledge engineering model and a target rule base to obtain the first set of output elements corresponding to the input text; The relevant descriptions can be as described above, and will not be repeated here.

[0050] According to some embodiments, Figure 3 An example schematic diagram of a system framework according to an embodiment of this disclosure is shown, such as... Figure 3 As shown, it can include a data layer, platform layer, application layer, and components of each layer. The core components and data interaction relationships of the data layer, platform layer, and application layer can demonstrate a logical closed loop of "data support - capability empowerment - application implementation." Among them: Data layer: Responsible for the "input-preprocessing-standardized storage" of compliance texts. It achieves text segmentation and vector correction through the compliance knowledge preprocessing module, and finally stores the texts in the form of a dual library of "vector library + original text library", providing high-quality data support for the platform layer. Platform layer: namely the "compliance management brain", which can integrate knowledge engineering models, large models and collaborative engines to realize core functions such as element extraction, dynamic updates and knowledge association. Among them, modules such as dual-engine cross-validation and knowledge graph construction directly solve the defects of existing technologies such as "low accuracy and logical break". Application layer: Output functions for actual business scenarios. The risk list generation module enables general / customized list output, the system integration module solves the "integration difficulty" problem, and the manual assistance module is retained to deal with medium and high risk scenarios, thus balancing automation and security.

[0051] According to some embodiments, Figure 4 An example schematic diagram illustrating a text processing method according to an embodiment of this disclosure is shown, such as... Figure 4 As shown, the process can include: Start point: The data layer provides standardized compliance text to ensure noise-free input data, laying the foundation for subsequent extraction accuracy; Dual-engine parallel processing: The Knowledge Engineering Model (KEM) relies on a pre-set compliance concept library to achieve "rule-based extraction," ensuring the accuracy of explicit elements; The Large Language Model (LLM) achieves "implicit element recognition" through few-shot learning, covering implicit information that traditional rule engines cannot capture (such as the time node implied by "should be filed"); Cross-validation mechanism: The consistency of the dual-engine results is judged by the overlap degree calculation. If there is inconsistency, a secondary matching of the Knowledge Engineering Model is triggered, effectively reducing the risk of "illusion" in the large model. This step directly solves the defect of "high error rate in single-model extraction" in existing technologies, ultimately outputting a high-accuracy element set. The specific description of each step can be as follows.

[0052] According to some embodiments, the method further includes: A knowledge engineering model is used to segment unstructured historical text to obtain the word set corresponding to the unstructured historical text; A domain dictionary corresponding to the unstructured historical text is used to annotate each word in the word segmentation set, and the annotation information corresponding to each word is obtained. Obtain the first text embedding vector corresponding to the unstructured historical text; Based on the knowledge weight matrix and feature type bias term corresponding to the domain information of the unstructured historical text, the first text embedding vector is modified to obtain the second text embedding vector corresponding to the unstructured historical text. The annotation information corresponding to each word segment and the second text embedding vector corresponding to the unstructured historical text are associated and stored in the target rule base. Therefore, by reducing the coarseness of text processing and improving the accuracy of text segmentation, key semantic breaks can be reduced, and the accuracy of subsequent output element acquisition can be improved.

[0053] In some embodiments, the word segmentation set may be, for example, a set obtained after segmenting unstructured text. This word segmentation set does not specifically refer to a fixed set. For example, the word segmentation set may change when the number of words included in it changes. Similarly, the word segmentation set may change when the knowledge engineering model changes.

[0054] In some embodiments, the first text embedding vector may be, for example, an embedding vector corresponding to unstructured historical text, and different unstructured historical texts may correspond to different embedding vectors.

[0055] According to some embodiments, Figure 5 An example schematic diagram illustrating the vector correction process of an embodiment of this disclosure is shown, such as... Figure 5 As shown, it can include: using a knowledge engineering model to process unstructured compliance text in layers, for example, it can first be dynamically segmented according to "chapter-clause-semantic segment", specifically by using a sliding window plus natural semantic boundary recognition, and then completing word segmentation and entity annotation through a domain dictionary; then, based on the compliance concept system built into the knowledge engineering model, i.e., the compliance concept library of the knowledge engineering model, the text embedding vector (generated by a pre-trained language model) can be numerically corrected, as shown in formula (1): \(V_{corrected}=V_{original}×W_k+B_i\)(1) Wherein, \(W_k\) is the compliance domain knowledge weight matrix (obtained through training with historical compliance data), and \(B_i\) is the element type bias term, which can realize the embedding of background policy content and the pruning of irrelevant information.

[0056] According to some implementations, the processed data can be tagged and stored according to "compliance basis type - issuing agency - effective time - applicable field", supporting the linkage between the vector library (such as FAISS) and the original text library.

[0057] According to some embodiments, the method further includes: Get the first text and the corresponding second text, where the second text is the updated text of the first text; A similarity algorithm is used to obtain the similarity between the first text and the second text; Based on similarity, obtain updated information about the target rule base; When an update message indicates an update to the target rule base, a knowledge engineering model is used to identify the second text, obtain the update prompt information corresponding to the target rule base, and send the update prompt information to the target account corresponding to the update prompt information based on the risk level information. Therefore, this reduces the likelihood of delays in updating the target rule base, shortens the update time, improves update efficiency, reduces the need for manual review, shortens the manual review time, and reduces manual intervention, thus improving the efficiency of automated updates and audits.

[0058] According to some embodiments, Figure 6 This illustration shows an example diagram of rule base updates according to an embodiment of the present disclosure, such as... Figure 6 As shown, it may include: Monitoring: Real-time capture of compliance data updates via multi-source interfaces solves the problem of "lagging manual monitoring"; Automatic revision process: The knowledge engineering model accurately locates the affected elements, and the large model generates specific revision suggestions, realizing full automation of the "update-locate-revise" process, which improves efficiency by more than 90% compared with traditional manual revision. Tiered approval process: Differentiated processing based on risk level. Low-risk cases are automatically approved to reduce human intervention, while medium- and high-risk cases are subject to human review to ensure security. This balances "automation efficiency" with "compliance rigor" and addresses the shortcomings of "rigid update processes".

[0059] In some embodiments, regulatory / system updates can be captured from regulatory information release interfaces and public databases using a compliance control brain's "knowledge crawler module." Based on text similarity algorithms (such as cosine similarity, with a threshold of 0.7), it's determined whether a list update should be triggered. If the similarity exceeds the threshold, an update is triggered. After an update is triggered, the knowledge engineering model automatically identifies the affected risk factors, and the large model generates revision suggestions (including "add / delete / modify" type annotations), forming revision information. For example, a tiered approval mechanism can also be implemented. Specifically, low-risk revisions (such as wording optimization) take effect automatically, while medium- to high-risk revisions (such as risk level changes) are pushed to compliance specialists for review. After approval, the list version automatically iterates (version number rule: V[year].[month].[revision number]).

[0060] According to some embodiments, the first output element set may, for example, output element S1.

[0061] In step S22, a large model is used to identify the input text and obtain the second set of output elements corresponding to the input text; The relevant descriptions can be as described above, and will not be repeated here.

[0062] According to some embodiments, the second set of output elements may be, for example, output element S2.

[0063] In step S23, the first overlap degree corresponding to the first output feature set and the second output feature set is obtained; The relevant descriptions can be as described above, and will not be repeated here.

[0064] According to some embodiments, the first degree of overlap may be, for example, overlap = (S1∩S2) / (S1∪S2).

[0065] According to some embodiments, Figure 7 An example schematic diagram illustrating cross-validation in an embodiment of this disclosure is shown, such as... Figure 7 As shown, the method may include: if the first overlap degree is greater than the first overlap degree threshold, obtaining the target element set corresponding to the input text, and obtaining the output text corresponding to the input text based on the target element set; if the first overlap degree is less than the first overlap degree threshold, using a knowledge engineering model and a target rule base to re-identify the input text and obtain the third output element set corresponding to the input text; obtaining the second overlap degree corresponding to the third output element set and the second output element set; if the second overlap degree is greater than the second overlap degree threshold, obtaining the target element set corresponding to the input text; if the second overlap degree is less than the second overlap degree threshold, outputting prompt information, the prompt information including the third output element set, the second output element set, and the difference information between the third output element set and the second output element set.

[0066] In step 24, if the first overlap degree is greater than the first overlap threshold, the target element set corresponding to the input text is obtained; The relevant descriptions can be as described above, and will not be repeated here.

[0067] According to some embodiments, the first overlap threshold can be, for example, 90%. For example, when the first overlap is greater than or equal to 90%, the target element set can be (S1∩S2), where the confidence level of each element can be the average value.

[0068] According to some embodiments, the first overlap threshold can be, for example, 90%. For instance, when the first overlap is less than or equal to 90%, a second matching of the rule base can be triggered. Specifically: If the first overlap degree is less than the first overlap threshold, the input text is re-identified using a knowledge engineering model and a target rule base to obtain the third output element set corresponding to the input text. Obtain the second degree of overlap between the third output element set and the second output element set; If the second overlap degree is greater than the second overlap degree threshold, the target feature set corresponding to the input text is obtained; if the second overlap degree is less than the second overlap degree threshold, a prompt message is output, which includes the third output feature set, the second output feature set, and the difference information between the third output feature set and the second output feature set. Therefore, this approach can reduce situations where feature extraction efficiency is low, improve the accuracy of feature acquisition, reduce the coverage of latent feature recognition compared to a single rule engine, mitigate the risk of large model illusions, lower the feature error rate after cross-validation, and improve the accuracy of feature acquisition.

[0069] In step S25, if the first overlap degree is less than the first overlap threshold, the input text is re-identified using a knowledge engineering model and a target rule base to obtain the third output element set corresponding to the input text. The relevant descriptions can be as described above, and will not be repeated here.

[0070] In some embodiments, the third output element set may be, for example, the element set obtained by re-identifying the input text using a knowledge engineering model and a target rule base. The "third" in this third output element set is used to distinguish it from the other output element sets and does not specifically refer to a particular fixed output element set.

[0071] In step S26, the second overlap degree corresponding to the third output feature set and the second output feature set is obtained; The relevant descriptions can be as described above, and will not be repeated here.

[0072] According to some embodiments, the second overlap degree may be, for example, the overlap degree between the third output feature set and the second output feature set. According to some embodiments, the second overlap degree may be, for example, overlap degree = (S3∩S2) / (S3∪S2).

[0073] In step S27, if the second overlap degree is greater than the second overlap threshold, the target element set corresponding to the input text is obtained; The relevant descriptions can be as described above, and will not be repeated here.

[0074] In some embodiments, the second overlap threshold may be the same as or different from the first overlap threshold, and this disclosure does not limit this.

[0075] According to some implementations, a dual-engine architecture of "knowledge engineering model + large model" can be constructed: The knowledge engineering model is responsible for "rule-based element screening": based on a preset compliance element framework (including four core elements: "prohibited behavior - control node - violation consequences - related evidence"), it extracts structured information through concept matching (such as "prohibited behavior" corresponding to the term "must not / prohibited"); the large model (based on a language model fine-tuned in the compliance domain) is responsible for "implicit element identification": using Few-Shot learning (inputting 10-20 labeled samples), it identifies implicit time nodes (such as "within 10 days after signing"), responsible entities (such as "approval department"), and other information in the text, and outputs element confidence scores (with a threshold set to 0.85); a cross-validation mechanism can also be introduced: the output results of the two engines are compared, and if the element overlap is ≥90%, it is directly adopted; if <90%, a secondary matching of the concept library of the knowledge engineering model is triggered to correct the identification bias of the large model.

[0076] According to some embodiments, when the second overlap degree is greater than the second overlap degree threshold, the difference elements between the third output element set and the second output element set can be obtained, and the difference elements can be added to the first output element set, and the added first output element set can be used as the target element set.

[0077] In step S28, if the second overlap degree is less than the second overlap threshold, a prompt message is output. The prompt message includes the third output element set, the second output element set, and the difference information between the third output element set and the second output element set. The relevant descriptions can be as described above, and will not be repeated here.

[0078] In step S29, the output text corresponding to the input text is obtained based on the target element set.

[0079] The relevant descriptions can be as described above, and will not be repeated here.

[0080] According to some embodiments, the method further includes: Obtain the knowledge graph corresponding to the target element set; Using large models and chain reasoning techniques, the node relationships of the knowledge graph are completed to obtain the processed knowledge graph; The processed knowledge graph undergoes integrity verification, generating a verification report which is then pushed to the target device. This improves the correlation between elements, reduces instances of poor element correlation due to lack of correlation, enhances the accuracy of element acquisition, provides end-to-end compliance guidance, reduces control vulnerabilities arising from logical breaks between elements, and further improves the accuracy of element acquisition.

[0081] According to some implementations, a compliance risk knowledge graph can be constructed: using "risk elements" as nodes and "semantic associations" (such as "prohibited behavior → consequences of violation") and "basis associations" (such as "risk point → regulatory clauses") as edges, scattered elements are structurally associated; large models can be used for reasoning enhancement: using chain of reasoning technology, the large model can complete the closed-loop logic of "risk point - control measures - associated basis" based on the node relationships of the knowledge graph (e.g., "signing a contract without compliance review → consequences of violation: dispute → control measures: pre-signing review → basis: Article X of XX regulation"); the completeness of association can be verified: the knowledge graph automatically detects missing association edges (such as "risk point has no associated basis"), generates a verification report, and pushes it to manual supplementation.

[0082] According to some embodiments, the method further includes: Obtain the display format information corresponding to the selection command, whereby the selection command is used to select the business area, job information, and risk level from the multi-dimensional tag set; By using a display format to present the output text, we can reduce issues with poor adaptability, improve the efficiency of scenario-based customization, eliminate the need for manual intervention, enhance the accuracy of the output text display, reduce the presence of unnecessary elements in the output text, and improve the convenience of the output text display.

[0083] According to some embodiments, based on a set of target elements, the output text corresponding to the input text is obtained, including: Obtain the input information corresponding to the display format information, and generate the display format information based on the input information; Receive processing instructions for the input text through the API interface; Based on the processing instructions and display format information, the target element set is formatted to obtain structured data corresponding to the input text.

[0084] According to some implementation examples, a multi-dimensional tag system can be constructed, which can cover three core tags: "business domain" (such as procurement, contracts, and finance), "job role" (such as compliance specialist, procurement personnel, and finance personnel), and "risk level" (high / medium / low). It supports user-defined tag expansion. Based on the user-selected tag combination (such as "procurement domain + procurement personnel + high risk"), it can match relevant risk elements through vector retrieval, and the large model can automatically adjust the display of list fields (such as hiding finance domain-specific fields in the procurement personnel list) to generate scenario-based lists. It can provide a standardized template library (including three formats: spreadsheet (Excel), portable document format (PDF), and JavaScript object notation (JSON)) and support one-click export of scenario-based lists.

[0085] According to some implementation examples, compliance data standards are defined: a JSON schema for risk list data is established, specifying required fields and format specifications such as "risk ID, risk category, risk point, control node, association basis, and risk level"; a general API interface can also be provided: supporting RESTful API integration with enterprise internal control systems (such as contract management systems and procurement approval systems), with the interface including three core functions: "list query, element synchronization, and early warning triggering," compatible with HyperText Transfer Protocol Secure (HTTPS) and JSON Web Token (JWT) authentication; data format conversion is possible: a built-in format conversion tool supports automatically converting scenario-based lists into structured data (such as extensible markup language (XML) and comma-separated values ​​(CSV)) required by the internal control system, without requiring manual secondary processing.

[0086] In one or related embodiments, if the first overlap degree is less than a first overlap degree threshold, a knowledge engineering model and a target rule base can be used to re-identify the input text to obtain a third output element set corresponding to the input text; a second overlap degree corresponding to the third output element set and the second output element set can be obtained; if the second overlap degree is greater than the second overlap threshold, a target element set corresponding to the input text can be obtained; if the second overlap degree is less than the second overlap threshold, a prompt message can be output, which includes the third output element set, the second output element set, and the difference information between the third output element set and the second output element set. Therefore, the accuracy of obtaining the target element set and the accuracy of obtaining the output text can be improved.

[0087] A block diagram of a text processing apparatus according to an exemplary embodiment is shown. (Refer to...) Figure 8 The device 800 includes: The set acquisition unit 801 is used to identify the input text using a knowledge engineering model and a target rule base, and to acquire the first output element set corresponding to the input text; The set acquisition unit 801 is also used to identify the input text using a large model and obtain the second output element set corresponding to the input text; The overlap degree acquisition unit 802 is used to acquire the first overlap degree corresponding to the first output element set and the second output element set; The text processing unit 803 is used to obtain the target feature set corresponding to the input text when the first overlap degree is greater than the first overlap degree threshold, and to obtain the output text corresponding to the input text based on the target feature set.

[0088] According to some embodiments, the collection acquisition unit 801 is further configured to: A knowledge engineering model is used to segment unstructured historical text to obtain the word set corresponding to the unstructured historical text; A domain dictionary corresponding to the unstructured historical text is used to annotate each word in the word segmentation set, and the annotation information corresponding to each word is obtained. Obtain the first text embedding vector corresponding to the unstructured historical text; Based on the knowledge weight matrix and feature type bias term corresponding to the domain information of the unstructured historical text, the first text embedding vector is modified to obtain the second text embedding vector corresponding to the unstructured historical text. The annotation information corresponding to each word segment and the second text embedding vector corresponding to the unstructured historical text are associated and stored in the target rule base.

[0089] According to some embodiments, the overlap degree acquisition unit 802 is further configured to: If the first overlap degree is less than the first overlap threshold, the input text is re-identified using a knowledge engineering model and a target rule base to obtain the third output element set corresponding to the input text. Obtain the second degree of overlap between the third output element set and the second output element set; If the second overlap degree is greater than the second overlap threshold, obtain the target element set corresponding to the input text; If the second overlap is less than the second overlap threshold, output a prompt message. The prompt message includes the third output feature set, the second output feature set, and the difference information between the third output feature set and the second output feature set.

[0090] According to some embodiments, the collection acquisition unit 801 is further configured to: Get the first text and the corresponding second text, where the second text is the updated text of the first text; A similarity algorithm is used to obtain the similarity between the first text and the second text; Based on similarity, obtain updated information about the target rule base; When the update information indicates that the target rule base needs to be updated, a knowledge engineering model is used to identify the second text, obtain the update prompt information corresponding to the target rule base, and send the update prompt information to the target account corresponding to the update prompt information based on the risk level information corresponding to the update prompt information.

[0091] According to some embodiments, the text processing unit 803 is also used for: Obtain the knowledge graph corresponding to the target element set; Using large models and chain reasoning techniques, the node relationships of the knowledge graph are completed to obtain the processed knowledge graph; The processed knowledge graph is subjected to integrity verification, a verification report is generated, and the verification report is pushed to the target device.

[0092] According to some embodiments, the text processing unit 803 is also used for: Obtain the display format information corresponding to the selection command, whereby the selection command is used to select the business area, job information, and risk level from the multi-dimensional tag set; The output text is displayed using a specific format.

[0093] According to some embodiments, when the text processing unit 803 obtains the output text corresponding to the input text based on the target element set, it is specifically used for: Obtain the input information corresponding to the display format information, and generate the display format information based on the input information; Receive processing instructions for the input text through the API interface; Based on the processing instructions and display format information, the target element set is formatted to obtain structured data corresponding to the input text.

[0094] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0095] In some or related embodiments, a set acquisition unit is used to identify the input text using a knowledge engineering model and a target rule base to obtain a first output element set corresponding to the input text; the set acquisition unit is also used to identify the input text using a large model to obtain a second output element set corresponding to the input text; an overlap degree acquisition unit is used to obtain a first overlap degree corresponding to the first output element set and the second output element set; a text processing unit is used to obtain a target element set corresponding to the input text when the first overlap degree is greater than a first overlap degree threshold, and to obtain the output text corresponding to the input text based on the target element set. Therefore, target elements can be obtained through the association between elements and different output element acquisition methods without manual intervention, which can reduce the time required for manual intervention and reduce the inaccuracy of manual screening. Furthermore, cross-validation can be performed on the element extraction results of multiple models to improve the association between elements and the accuracy of element set acquisition, thereby improving the efficiency and accuracy of element extraction and the accuracy of output text output.

[0096] Figure 9A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Figure 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded into a random access memory (RAM) 903 from a storage unit 908. The RAM 903 may also store various programs and data required for the operation of the electronic device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0098] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above. For example, in some embodiments, the above methods can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the above methods by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0105] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0106] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A text processing method, characterized in that, include: The input text is identified using a knowledge engineering model and a target rule base to obtain the first set of output elements corresponding to the input text; A large model is used to identify the input text, and a second set of output elements corresponding to the input text is obtained; Obtain the first overlap degree corresponding to the first output element set and the second output element set; If the first overlap is greater than the first overlap threshold, obtain the target element set corresponding to the input text, and obtain the output text corresponding to the input text based on the target element set.

2. The method according to claim 1, characterized in that, The method further includes: The knowledge engineering model is used to segment unstructured historical text to obtain the word set corresponding to the unstructured historical text. Each word in the word segmentation set is labeled using a domain dictionary corresponding to the unstructured historical text, and the labeling information corresponding to each word is obtained. Obtain the first text embedding vector corresponding to the unstructured historical text; Based on the knowledge weight matrix and feature type bias term corresponding to the domain information of the unstructured historical text, the first text embedding vector is modified to obtain the second text embedding vector corresponding to the unstructured historical text. The annotation information corresponding to each word segment and the second text embedding vector corresponding to the unstructured historical text are associated and stored in the target rule base.

3. The method according to claim 1, characterized in that, The method further includes: If the first overlap is less than the first overlap threshold, the knowledge engineering model and the target rule base are used to re-identify the input text to obtain the third output element set corresponding to the input text. Obtain the second overlap degree corresponding to the third output element set and the second output element set; If the second overlap degree is greater than the second overlap threshold, obtain the target element set corresponding to the input text; If the second overlap degree is less than the second overlap threshold, a prompt message is output. The prompt message includes the third output element set, the second output element set, and the difference information between the third output element set and the second output element set.

4. The method according to claim 1, characterized in that, The method further includes: Obtain a first text and a second text corresponding to the first text, wherein the second text is the updated text of the first text; A similarity algorithm is used to obtain the similarity between the first text and the second text; Based on the similarity, obtain the update information of the target rule base; When the update information indicates that the target rule base needs to be updated, the knowledge engineering model is used to identify the second text, obtain the update prompt information corresponding to the target rule base, and send the update prompt information to the target account corresponding to the update prompt information according to the risk level information corresponding to the update prompt information.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the knowledge graph corresponding to the target element set; The large model and chain reasoning technique are used to complete the node relationships of the knowledge graph to obtain the processed knowledge graph; The processed knowledge graph is subjected to integrity verification, a verification report is generated, and the verification report is pushed to the target device.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the display format information corresponding to the selection instruction, wherein the selection instruction is used to select the business area, job information and risk level from the multi-dimensional tag set; The output text is displayed using the aforementioned display format information.

7. The method according to claim 6, characterized in that, The step of obtaining the output text corresponding to the input text based on the target element set includes: Obtain the input information corresponding to the display format information, and generate the display format information based on the input information; Receive processing instructions for the input text through the API interface; Based on the processing instructions and the display format information, the target element set is formatted to obtain structured data corresponding to the input text.

8. A text processing device, characterized in that, include: The set acquisition unit is used to identify the input text using a knowledge engineering model and a target rule base, and to acquire the first set of output elements corresponding to the input text; The set acquisition unit is also used to identify the input text using a large model and obtain the second output element set corresponding to the input text; The overlap degree acquisition unit is used to acquire the first overlap degree corresponding to the first output element set and the second output element set; The text processing unit is configured to, when the first overlap degree is greater than the first overlap degree threshold, obtain the target element set corresponding to the input text, and obtain the output text corresponding to the input text based on the target element set.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the text processing method as described in any one of claims 1 to 7.

10. A storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the text processing method as described in any one of claims 1 to 7.

Images