Text processing method, device, electronic device, storage medium and program product

Abstract

The present application discloses a text processing method, device, electronic device, storage medium and program product for improving the rigor and accuracy of question and answer based on a language model. The method comprises: searching for a first knowledge item similar to a first question text and a first text corresponding to the first knowledge item in a knowledge base; searching for a target entity related to the first text and the relationship between the target entities from a knowledge graph, wherein the knowledge graph stores multiple entities and the relationship between the multiple entities; and generating a response text corresponding to the first question text based on the first text, the target entity and the relationship between the target entities through a language model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text processing method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of artificial intelligence (AI) technology, knowledge question answering technology based on large language models (LLM) has emerged. One of the core output logics of large language models is reasoning. The reasoning output of trained large language models is usually different from the original data, so it cannot be applied to scenarios such as legal consulting that require high rigor of reasoning output. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a text processing method, device, electronic device, storage medium and program product for improving the rigor and accuracy of question and answer based on language models.

[0004] In order to achieve the above objectives, the present application embodiment adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a text processing method, comprising:

[0006] Searching for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item in the knowledge base;

[0007] Searching for a target entity related to the first text and a relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities;

[0008] Through the language model, based on the first text, the target entity and the relationship between the target entities, an answer text corresponding to the first question text is generated.

[0009] In a second aspect, an embodiment of the present application provides a text processing device, including:

[0010] A first search module, used to search the knowledge base for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item;

[0011] A second search module is used to search for a target entity related to the first text and a relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities;

[0012] A generation module is used to generate a response text corresponding to the first question text based on the first text, the target entity and the relationship between the target entities through a language model.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0014] processor;

[0015] a memory for storing instructions executable by the processor;

[0016] The processor is configured to execute the instructions to implement the text processing method provided in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the text processing method provided in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the text processing method provided in the first aspect.

[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0020] Combine the knowledge graph and the knowledge base, use the knowledge base to find the first knowledge item similar to the first question text to be answered, and track the first text corresponding to the first knowledge item. The first text can describe the content of the first knowledge item in detail, and can reflect the text structure, pattern, etc. in the scenario to which the first question text belongs, which can assist the language model to learn this information; then, search for the target entities related to the first text and the relationship between the target entities in the knowledge graph, which can assist the language model to learn the relationship between the knowledge related to the first question text; further, the language model combines the first text, the target entity and the relationship between the target entities, and the first question text for professional integrated reasoning, which can enhance language comprehension, reasoning, interpretation and polishing capabilities, and generate a more accurate and rigorous answer text. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1A flowchart of a text processing method provided for an embodiment of the present application;

[0023] Figure 2 A flowchart of a text processing method provided for another embodiment of the present application;

[0024] Figure 3 A flowchart of a method for pre-training a language model provided for one embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of a text processing device provided for one embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0028] The terms "first", "second", etc. in this specification and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in this specification and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0029] As mentioned earlier, a core output logic of a large language model is reasoning. The reasoning output of a trained large language model is usually different from the original data, and therefore cannot be applied to scenarios such as legal consulting that require high rigor in reasoning output.

[0030] The inventors have found through extensive research that, for scenarios that require high rigor in reasoning output, when using a large language model to answer a question text, in addition to the question text, the large language model is also provided with text data related to the question text as a reference, which is conducive to enhancing the large language model's ability to understand the question text and the rigor of reasoning, thereby improving the rigor of the answer text. For example, in legal consulting scenarios, providing the large language model with text data such as legal provisions and case files is conducive to the large language model learning the text structure and patterns of the legal industry, and enhancing the language comprehension and reasoning capabilities of the large language model.

[0031] In addition, text data related to question text is more suitable for storage in knowledge graphs and knowledge bases, so that the relationship between the rich knowledge implicit in the text data can be intuitively and accurately reflected. For example, in the legal consultation scenario, a certain legal clause applies to a certain case, a certain legal document cites another legal document, and a certain legal event leads to the occurrence of another legal event. Based on this, by obtaining this type of text data and the relationship between the knowledge in these text data from the knowledge graph and knowledge base, and providing them to the large language model for reasoning, the language understanding, reasoning, interpretation, polishing and other capabilities of the large language model can be further enhanced, and a more accurate and rigorous response text can be output.

[0032] Taking the above aspects into consideration, an embodiment of the present application proposes a text processing method, which combines a knowledge graph and a knowledge base, uses the knowledge base to search for a first knowledge item similar to a first question text to be answered, and tracks the first text corresponding to the first knowledge item. The first text can describe the content of the first knowledge item in detail, and can reflect the text structure, pattern, etc. in the scenario to which the first question text belongs, which can assist the language model in learning this information; then, searching the knowledge graph for target entities related to the first text and the relationship between target entities can assist the language model in learning the relationship between knowledge related to the first question text; further, the language model combines the first text, the target entity and the relationship between the target entities, and the first question text for professional integrated reasoning, which can enhance language understanding, reasoning, interpretation and polishing capabilities, and generate a more accurate and rigorous answer text.

[0033] It should be understood that the text processing method provided in the embodiments of the present application can be executed by an electronic device. The electronic device referred to here may include a terminal device, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a smart watch, a vehicle terminal, an aircraft, etc.; or, the electronic device may also include a server, such as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0034] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0035] Please refer to Figure 1 , is a flowchart of a text processing method provided for an embodiment of the present application, the method comprising the following steps:

[0036] S102, searching a knowledge base for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item.

[0037] The first question text refers to the question text to be answered. In one embodiment, the question text input by the user through the dialogue interface can be used as the first question text.

[0038] The knowledge base stores multiple knowledge items, which are used to describe the knowledge in the scenario to which the first question text belongs. For example, in the legal consultation scenario, the knowledge items are used to describe legal knowledge, such as contract breach, legal liability, compensation standards, and other knowledge.

[0039] Each knowledge item has a corresponding text, which is also called the original text. The text corresponding to the knowledge item is used to describe the detailed content of the knowledge item. For example, the text corresponding to the knowledge item "anticipatory breach of contract" includes "this is an act in which one party clearly expresses or indicates through its own behavior that it will not perform its contractual obligations before the contract performance period arrives."

[0040] In one embodiment, the knowledge base also stores text corresponding to each knowledge item. In this case, the above S102 includes the following steps: determining the semantic similarity between the text corresponding to each knowledge item and the first question text; determining from the knowledge base a text whose semantic similarity with the first question text is greater than a similarity threshold, and determining the knowledge item corresponding to the text as the first knowledge item similar to the first question text; and using the text corresponding to the first knowledge item as the first text. The semantic similarity between two texts can be represented by cosine similarity, Euclidean distance, etc., which is not limited in the embodiment of the present application.

[0041] For example, the first question text is "What are the legal liabilities and compensation standards for breach of contract?" The knowledge items and the texts corresponding to the knowledge items stored in the knowledge base are as follows:

[0042] Anticipatory breach of contract: This is an act in which one party clearly expresses or indicates through its own behavior that it will not perform its contractual obligations before the contract performance deadline arrives;

[0043] The way to bear the liability for breach of contract: including continuing to perform, taking remedial measures, compensating for losses, etc.;

[0044] Jurisdictional court: refers to the court that has the power to adjudicate cases in accordance with the law during litigation activities.

[0045] According to the semantic similarity between the texts corresponding to the above-mentioned knowledge items and the first question text, it can be obtained that the knowledge items "anticipated breach of contract" and "way of assuming liability for breach of contract" are the first knowledge items similar to the first question text, and then the first text corresponding to the first knowledge item is obtained.

[0046] In the above method, since the text corresponding to the knowledge item describes the detailed content of the knowledge item, the first knowledge item similar to the first question text is determined according to the semantic similarity between the text corresponding to each knowledge item and the first question text. This ensures that the first knowledge item and the first question text are highly similar in semantics, thereby ensuring that the first knowledge item can be used to answer the first question text more accurately.

[0047] In another embodiment, in addition to storing multiple knowledge items, the knowledge base also stores text location information corresponding to each knowledge item. The text location information corresponding to the knowledge item is used to accurately obtain the text corresponding to the knowledge item, for example, including but not limited to the index number of the text corresponding to the knowledge item in the text database or cloud storage platform to which it belongs.

[0048] In this case, the above S102 includes the following steps: determining the semantic similarity between the keywords in the first question text and the knowledge items in the knowledge base; determining from the knowledge base a knowledge item whose semantic similarity with the keywords is greater than a similarity threshold as a first knowledge item similar to the first question text; further, based on the text positioning information corresponding to the first knowledge item, obtaining the first text corresponding to the first knowledge item.

[0049] For example, the first question text is "What are the legal liabilities and compensation standards for contract breach?", which contains the keywords "contract breach", "legal liabilities", and "compensation standards". The knowledge items stored in the knowledge base and the text positioning information corresponding to the knowledge items are as follows:

[0050] Anticipated default: 1001;

[0051] Actual breach: 1002;

[0052] Continue to perform: 2001;

[0053] Remedial measures taken: 2002;

[0054] Compensation for damages: 2003;

[0055] Payment of liquidated damages: 2004;

[0056] Deposit penalty: 2005;

[0057] Agreed compensation: 3001;

[0058] Statutory damages: 3002;

[0059] Jurisdictional court: 4001.

[0060] Based on the semantic similarity between the above-mentioned knowledge items and the keywords, it can be determined that the knowledge items "anticipated breach of contract" and "actual breach of contract" are similar to the keyword "contract breach", the knowledge items "continue to perform", "take remedial measures", "compensate for losses", "pay liquidated damages" and "deposit penalty" are similar to the keyword "legal liability", and the knowledge items "statutory compensation" and "agreed compensation" are similar to the keyword "compensation standard", and then these knowledge items are determined as the first knowledge items similar to the first question text, and then according to the text positioning information corresponding to these first knowledge items, the first text corresponding to the first knowledge items is obtained from other places such as document databases or cloud storage platforms.

[0061] In the above method, considering that in some scenarios, the text corresponding to the knowledge item is long, if such text is stored in the knowledge base, it will increase the storage pressure of the knowledge base, increase the retrieval and reading efficiency of the knowledge base, and thus affect the efficiency of answering the first question text. For this reason, the knowledge items and their corresponding text location information are stored in the knowledge base, and the text corresponding to the knowledge items is stored elsewhere, such as a special document database or cloud storage platform. Such external storage areas have larger capacity and more flexible storage architecture, and can easily cope with the storage needs of long texts. In addition, according to the semantic similarity between each knowledge item and the keywords of the first question text, the first knowledge item similar to the first question text and the first text corresponding to the first knowledge item are determined, which can significantly improve the search efficiency.

[0062] The above describes some implementation methods of the above S102. Of course, it should be understood that the above S102 can also be implemented in other ways, and the present application embodiment does not limit this.

[0063] It is worth noting that, in the case where the first text is long, the first text can also be segmented to obtain multiple segments, each segment having corresponding text positioning information. In this case, the text positioning information corresponding to the first knowledge item includes the text positioning information of each segment obtained by segmenting the first text.

[0064] Secondly, the above semantic similarity can be set according to actual needs, and the embodiment of the present application does not limit this, so as to better control the accuracy and performance of the query results output by the knowledge base.

[0065] In addition, the correspondence between knowledge items in the knowledge base and text location information can be recorded through metadata or mapping tables.

[0066] S104, searching the knowledge graph for target entities related to the first text and the relationships between the target entities.

[0067] The knowledge graph stores multiple entities and the relationships between them. Specifically, each entity corresponds to a node in the knowledge graph, and the edge between two nodes represents the relationship between the entities corresponding to the two nodes.

[0068] The entities and the relationships between entities are different in different scenarios.

[0069] For example, taking the legal consultation scenario as an example, the above-mentioned multiple entities may include, but are not limited to: legal documents, legal clauses, judicial institutions, legal roles, legal concepts, legal events, legal entities, geographical locations, etc. Among them, legal documents may include, but are not limited to: legal provisions, judgments, contracts, legal opinions, case files, etc. Legal clauses may include, but are not limited to: regulations, rules, clauses in laws and regulations, etc. Judicial institutions may include, but are not limited to: courts, procuratorates, arbitration institutions. Legal roles may include, but are not limited to: judges, lawyers, prosecutors, parties, defendants, plaintiffs, etc. Legal concepts may include, but are not limited to: responsibilities, rights, obligations, crimes, civil acts, etc. Legal events may include, but are not limited to: case submission, trial, ruling, appeal, etc. Legal entities may include, but are not limited to: legal persons, enterprises, government agencies, non-profit organizations, etc. Geographical locations may include, but are not limited to: jurisdictions, court locations, etc.

[0070] The relationship between entities may include, but is not limited to, several major categories such as normative relationship, hierarchical relationship, causal relationship, entity relationship, temporal relationship, spatial relationship, and contractual relationship. Among them, normative relationship may include, but is not limited to, subcategories such as applicable (such as a legal provision applies to a case), cited (such as a legal document cites another legal document), and involving (such as a case involves a certain legal role). Hierarchical relationship may include, but is not limited to, subcategories such as subordinate (such as a judicial institution is subordinate to a higher-level judicial system), contain (such as a legal document contains several legal provisions or regulations). Causal relationship may include, but is not limited to, subcategories such as: cause (such as a legal event causes another legal event to occur), result (such as a judgment causes specific legal consequences), etc. Entity relationship may include, but is not limited to, subcategories such as agency (such as a lawyer represents a client or defendant), accusation (such as the plaintiff accuses the defendant of a certain behavior), defense (such as the defendant defends himself from a certain accusation), etc. Temporal relationship may include, but is not limited to, subcategories such as occurring (such as a legal event occurs at a specific time), continuing (such as a legal procedure or case continues from a certain point in time to another point in time), etc. Spatial relations may include, but are not limited to, subcategories such as location (e.g., a judicial institution or event occurs in a specific geographic location), jurisdiction (e.g., a court has jurisdiction over legal matters in a certain place), etc. Contractual relations may include, but are not limited to, subcategories such as the foregoing (e.g., a contract is signed by certain entities), performance (e.g., obligations in a contract are performed by a certain entity), etc.

[0071] Each entity has corresponding attribute information. The attribute information of an entity may include, for example, but is not limited to, at least one of the following information: text corresponding to the entity, text location information corresponding to the entity, etc. The text corresponding to the entity may be the text of the entity source, or may be the text used to describe the detailed content of the entity. The text location information corresponding to the entity is used to indicate the storage location to which the text corresponding to the entity belongs, and is used to accurately obtain the text corresponding to the entity.

[0072] In the embodiment of the present application, the above S104 can be implemented in various appropriate ways.

[0073] In one embodiment, the above S104 includes the following steps:

[0074] S141, matching the text positioning information of the first text with the text positioning information corresponding to each entity.

[0075] Specifically, for each entity, if the text positioning information corresponding to the entity includes the text positioning information of the first text, it is determined that the two match; otherwise, it is determined that the two do not match.

[0076] S142: Based on the matching result, determine a target entity related to the first text from the multiple entities.

[0077] As an example, for each entity, if the text positioning information corresponding to the entity matches the text positioning information of the first text, it indicates that the text corresponding to the entity contains the first text, and thus the entity is determined to be a target entity related to the first text.

[0078] As another example, for each entity, if the text positioning information corresponding to the entity includes the text positioning information of the first text, the entity is determined as the first entity; based on the relationship between multiple entities, a second entity related to the first entity is determined from the multiple entities, wherein the second entity at least includes an entity that has a relationship with the first entity; the first entity and the second entity are determined as target entities related to the first text.

[0079] For example, according to the matching result of the text positioning information, it is determined that the first entity includes entity 1, entity 2, and entity 3. If in the knowledge graph, there is a connection edge between the node corresponding to entity 1 and the node corresponding to entity 4, and there is a connection edge between the node corresponding to entity 2 and the node corresponding to entity 5, then entity 4 and entity 5 are determined as the second entity.

[0080] Optionally, the second entity may also include an entity that has an indirect relationship with the first entity, such as a connection edge between the node corresponding to entity 4 and the node corresponding to entity 6, and a connection edge between the node corresponding to entity 6 and the node corresponding to entity 7, then the second entity may also be determined from entity 6 and entity 7, which may be determined specifically according to a preset expansion depth. For example, if the expansion depth is 3, entity 6 is also determined as the second entity; if the expansion depth is 4, entity 7 is also determined as the second entity.

[0081] S143, determining the relationship between target entities from the relationships between multiple entities.

[0082] In the above method, not only the first entity that matches the first text in the text positioning information is used as the target entity, but the selection scope of the target entity is also expanded to entities connected to the first entity and other entities connected to the entity, etc. The target entities thus determined and the relationships between them can provide richer knowledge for the language model, assisting the language model to answer the first question text more accurately.

[0083] In another embodiment, the above S104 includes the following steps: performing entity recognition on the first text to obtain entities contained in the first text; determining target entities related to the first text from the knowledge graph based on the matching results between the entities contained in the first text and each entity in the knowledge graph; and determining the relationship between target entities from the relationships between multiple entities.

[0084] For example, for each entity in the knowledge graph, if the entity matches an entity contained in the first text, such as the entity belongs to an entity contained in the first text, the entity is determined as a target entity related to the first text. In addition, entities connected to the determined target entity can also be determined from the knowledge graph as target entities.

[0085] The above describes some implementation methods of the above S104. Of course, it should be understood that the above S104 can also be implemented in other ways, which are not limited in the present embodiment.

[0086] S106, generating a response text corresponding to the first question text through a language model based on the first text, the target entity and the relationship between the target entities.

[0087] In one embodiment, the above S106 includes the following steps: directly inputting the first text, the target entity and the relationship between the target entities into the language model for reasoning to obtain the answer text corresponding to the first question text.

[0088] In another embodiment, the above S106 includes the following steps:

[0089] S161, concatenate the first text and the target entity to obtain a second text.

[0090] As an example, the first text and the target entity are directly concatenated to obtain the second text. For example, the first text is:

[0091] As another example, when the target entity includes a first entity and a second entity, the first entity and the first text are concatenated to obtain a first segment; based on the text positioning information corresponding to the second entity, the text corresponding to the second entity is obtained; the text corresponding to the second entity and the second entity are concatenated to obtain a second segment; the first segment and the second segment are concatenated to obtain a second text.

[0092] For example, the first entity includes case A and judgment B, the first text corresponding to case A includes text 1, and the first text corresponding to judgment B is text 2, then the first segment is as follows:

[0093] Case A: Document 1

[0094] Judgment B: Text 2

[0095] The second entity includes legal clause C. The second text corresponding to legal clause C is text 3. Then the following second segment is obtained:

[0096] Legal Articles C: Article 3

[0097] Furthermore, the first segment and the second segment are concatenated to obtain the following second text:

[0098] Case A: Document 1

[0099] Judgment B: Text 2

[0100] Legal Articles C: Article 3

[0101] In the above method, by concatenating the target entity and its corresponding text, the structure, pattern, etc. of the text corresponding to the target entity and the detailed content of the target entity can be more accurately reflected, assisting the language model to learn this information, thereby generating a more accurate and rigorous response text.

[0102] S162, generating a third text for describing the relationship between target entities.

[0103] For example, if the relationship between the judgment C and case A is "applicable", and the relationship between the judgment B and legal clause C is "referenced", the third text "the judgment B is applicable to case A, and the judgment B references legal clause C" is generated.

[0104] S163, generating a first prompt word based on the second text, the third text and the first question text.

[0105] The first prompt word is used to instruct to answer the first question text based on the second text and the third text.

[0106] For example, continuing with the second text, the third text and the first question text in the above examples, the first prompt word is obtained as follows:

[0107] Case A: Document 1

[0108] Judgment B: Text 2

[0109] Legal Articles C: Article 3

[0110] The above judgment B applies to case A, and judgment B cites legal provision C.

[0111] Based on the above, please output "What are the legal responsibilities and compensation standards for breach of contract?"

[0112] S164: Input the first prompt word into the language model to obtain a response text corresponding to the first question text.

[0113] The first prompt word generated in the above manner can more accurately reflect the structure, pattern, etc. of the text corresponding to the target entity, the detailed content of the target entity and the relationship between the target entities. This information can enhance language understanding, reasoning, interpretation and polishing capabilities, and generate more accurate and rigorous response texts.

[0114] The above describes some implementation methods of the above S106. Of course, it should be understood that the above S106 can also be implemented in other ways, which are not limited in the present embodiment of the application.

[0115] To facilitate understanding of the text processing method provided in the above embodiment, Figure 2 Provide explanation.

[0116] In one embodiment, the knowledge base stores multiple knowledge items and text location information corresponding to each knowledge item. The knowledge graph stores multiple entities and relationships between multiple entities. Each entity has corresponding attribute information. The attribute information of the entity includes the text location information of the text corresponding to the entity.

[0117] like Figure 2 As shown, first, a first knowledge item similar to the first question text and text location information corresponding to the first knowledge item are searched from the knowledge base, and based on the text location information corresponding to the first knowledge item, a first text corresponding to the first knowledge item is obtained.

[0118] Then, the target entity related to the first text and the relationship between the target entities are searched from the knowledge graph, wherein the target entity includes: a first entity that matches the first text in the text positioning information, and a second entity related to the first entity.

[0119] Further, based on the text positioning information corresponding to the second entity, a second text corresponding to the second entity is obtained.

[0120] Finally, the first text, the second text, the first question text, the target entity and the relationship between the target entities are provided to the language model for integration and reasoning to obtain the answer text corresponding to the first question text.

[0121] In another embodiment of the present application, in order to enable the language model to learn the basic structure and pattern of the text in the corresponding scene before reasoning about the first question text, so as to enhance the language model's language understanding and text generation capabilities of the question text in the scene, the text processing method provided in the embodiment of the present application also improves the pre-training process of the language model, that is, before the above-mentioned S106, the language model is pre-trained in advance according to the training corpus corresponding to the scene to which the first question text belongs.

[0122] Specifically, if Figure 3 As shown, the pre-training method of the language model may include the following steps:

[0123] S302: Obtain a fourth text from the training corpus corresponding to the scene to which the first question text belongs.

[0124] For example, taking the legal consultation scenario as an example, the training corpus corresponding to this scenario may include but is not limited to: legal provisions, academic works, legal cases, judicial decision cases, judgments, rulings, complaints, and replies, etc.

[0125] S304: segment the fourth text to obtain a plurality of third segments.

[0126] As an example, the fourth text may be segmented into paragraphs according to a length defined by the language model, such as 1024 characters, to ensure that the length of each third segment does not exceed the length defined by the language model.

[0127] As another example, the fourth text may be segmented according to punctuation marks included in the fourth text, using a period as a separator, to ensure that each third segment is a complete sentence.

[0128] As another example, the semantic similarity between sentences in the fourth text may be determined, and sentences with similar semantics may be divided into one segment to obtain multiple third segments, so that sentences in the same third segment have the same semantics.

[0129] The above shows some ways to segment the fourth text. Of course, it should be understood that the fourth text can also be segmented in other ways, which are not limited in the embodiments of the present application.

[0130] S306: predict the next segment of the third segment by using the language model to obtain the next predicted segment of the third segment.

[0131] S308: Adjust parameters of the language model based on the next third segment of the third segment and the next predicted segment.

[0132] For example, the above-mentioned multiple third fragments include fragment 1, fragment 2 and fragment 3 arranged in sequence, and the next fragment of fragment 1 is predicted by the language model to obtain fragment 2'; the next fragment of fragment 2 is predicted by the language model to obtain fragment 3'; then, based on the difference between fragment 2 and fragment 2', and the difference between fragment 3 and fragment 3', the loss of the language model is calculated; finally, according to the loss of the language model, the parameters of the language model are adjusted.

[0133] The above is only one round of pre-training process for the language model. In practical applications, the language model can be pre-trained for multiple rounds until the preset training stop condition is met. Among them, the training stop condition can be set according to actual needs, such as the loss convergence of the language model or the pre-training rounds reaching the maximum rounds, etc., which is not limited in the embodiments of the present application.

[0134] One or more embodiments of the present application provide a text processing method that combines a knowledge graph and a knowledge base, uses the knowledge base to search for a first knowledge item that is similar to a first question text to be answered, and tracks the first text corresponding to the first knowledge item. The first text can describe the content of the first knowledge item in detail, and can reflect the text structure, pattern, etc. in the scenario to which the first question text belongs, which can assist the language model in learning this information; then, the target entities related to the first text and the relationships between the target entities in the knowledge graph are searched, which can assist the language model in learning the relationships between the knowledge related to the first question text; further, the language model combines the first text, the target entity and the relationship between the target entities, and the first question text for professional integrated reasoning, which can enhance language comprehension, reasoning, interpretation and polishing capabilities, and generate a more accurate and rigorous answer text.

[0135] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] Based on the same inventive concept, the present application embodiment also provides a text processing device. Figure 4 , is a structural diagram of a text processing device 400 provided in one embodiment of the present application, the device comprising:

[0137] A first search module 410 is used to search the knowledge base for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item;

[0138] A second search module 420 is used to search for a target entity related to the first text and a relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities;

[0139] The generation module 430 is used to generate a response text corresponding to the first question text based on the first text, the target entity and the relationship between the target entities through a language model.

[0140] In another embodiment, each entity has corresponding text location information, and the text location information corresponding to the entity is used to obtain the text corresponding to the entity;

[0141] The second search module is used for:

[0142] Matching the text location information of the first text with the text location information corresponding to each of the entities;

[0143] Based on the matching result, determining a target entity related to the first text from the multiple entities;

[0144] From the relationships between the multiple entities, the relationships between the target entities are determined.

[0145] In another embodiment, when the second search module determines the target entity related to the first text from the multiple entities based on the matching result, the second search module performs the following steps:

[0146] For each of the entities, if the text positioning information corresponding to the entity includes the text positioning information of the first text, the entity is determined as a first entity;

[0147] Based on the relationship between the multiple entities, determine a second entity related to the first entity from the multiple entities, the second entity at least including an entity having a relationship with the first entity;

[0148] The first entity and the second entity are determined as target entities related to the first text.

[0149] In another embodiment, the production module is used to:

[0150] Concatenating the first text and the target entity to obtain a second text;

[0151] generating a third text for describing the relationship between the target entities;

[0152] Generate a first prompt word based on the second text, the third text and the first question text;

[0153] The first prompt word is input into a language model to obtain an answer text corresponding to the first question text.

[0154] In another embodiment, the target entity includes a first entity corresponding to the first text and a second entity related to the first entity, the second entity at least includes an entity connected to the first entity, and each entity has corresponding text positioning information;

[0155] When the generating module concatenates the first text and the target entity to obtain the second text, the generating module performs the following steps:

[0156] Concatenate the first entity and the first text to obtain a first segment;

[0157] Based on the text location information corresponding to the second entity, obtaining the text corresponding to the second entity;

[0158] concatenating the second entity and the text corresponding to the second entity to obtain a second segment;

[0159] The first segment and the second segment are concatenated to obtain a second text.

[0160] In another embodiment, the text processing device further includes:

[0161] An acquisition module, configured to acquire a fourth text from a training corpus corresponding to the scenario to which the first question text belongs;

[0162] A segmentation module, used for segmenting the fourth text to obtain a plurality of third segments;

[0163] A prediction module, configured to predict a next segment of the third segment by using the language model to obtain a next predicted segment of the third segment;

[0164] An adjustment module is used to adjust the parameters of the language model based on a next third segment of the third segment and a next predicted segment.

[0165] Obviously, the text processing device 400 provided in the embodiment of the present application can be used as the above Figure 1 The execution body of the text processing method shown in the figure can thus realize the text processing device in Figure 1 The functions realized are as follows. Since the principles are the same, they will not be described in detail.

[0166] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0167] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0168] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0169] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a text processing device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0170] Searching for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item in the knowledge base;

[0171] Searching for a target entity related to the first text and a relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities;

[0172] Through the language model, based on the first text, the target entity and the relationship between the target entities, an answer text corresponding to the first question text is generated.

[0173] The above application Figure 1The method performed by the text processing device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0174] The electronic device may also perform Figure 1 Method, and implement the text processing device in Figure 1 , Figure 2 , Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0175] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0176] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations:

[0177] Searching for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item in the knowledge base;

[0178] Searching for a target entity related to the first text and a relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities;

[0179] Through the language model, based on the first text, the target entity and the relationship between the target entities, an answer text corresponding to the first question text is generated.

[0180] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the text processing method provided in the embodiment of the present application.

[0181] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0182] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0183] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0185] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A text processing method, characterized in that: include: Searching for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item in the knowledge base; Searching for target entities related to the first text and the relationships between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationships between the plurality of entities; in the case where the first knowledge item includes a knowledge item for describing legal knowledge, the target entity includes at least one of the following entities: a legal document, a legal clause, a judicial institution, a legal role, a legal concept, a legal event, a legal entity, and a geographical location; Concatenating the first text and the target entity to obtain a second text; generating a third text for describing the relationship between the target entities; generating a first prompt word based on the second text, the third text and the first question text, and inputting the first prompt word into a language model to obtain an answer text corresponding to the first question text; The target entity includes a first entity corresponding to the first text and a second entity related to the first entity, the second entity at least includes an entity connected to the first entity, and each entity has corresponding text positioning information; The step of concatenating the first text and the target entity to obtain a second text includes: Concatenate the first entity and the first text to obtain a first segment; Based on the text location information corresponding to the second entity, obtaining the text corresponding to the second entity; concatenating the second entity and the text corresponding to the second entity to obtain a second segment; The first segment and the second segment are concatenated to obtain a second text.

2. The method according to claim 1, characterized in that: Each entity has corresponding text location information, and the text location information corresponding to the entity is used to obtain the text corresponding to the entity; The step of searching, from the knowledge graph, target entities related to the first text and relations between the target entities includes: Matching the text location information of the first text with the text location information corresponding to each of the entities; Based on the matching result, determining a target entity related to the first text from the multiple entities; From the relationships between the multiple entities, the relationships between the target entities are determined.

3. The method according to claim 2, characterized in that The determining, based on the matching result, a target entity related to the first text from the multiple entities comprises: For each of the entities, if the text positioning information corresponding to the entity includes the text positioning information of the first text, the entity is determined as a first entity; Based on the relationship between the multiple entities, determine a second entity related to the first entity from the multiple entities, the second entity at least including an entity having a relationship with the first entity; The first entity and the second entity are determined as target entities related to the first text.

4. The method according to any one of claims 1 to 3, characterized in that Before inputting the first prompt word into a language model to obtain an answer text corresponding to the first question text, the method further includes: Obtaining a fourth text from a training corpus corresponding to the scenario to which the first question text belongs; Segmenting the fourth text to obtain a plurality of third segments; Predicting the next segment of the third segment by using the language model to obtain the next predicted segment of the third segment; Based on a next third segment of the third segment and a next predicted segment, parameters of the language model are adjusted.

5. A text processing device, characterized in that: include: A first search module, used to search the knowledge base for a first knowledge item similar to the first question text and a first text corresponding to the first knowledge item; A second search module is used to search for target entities related to the first text and the relationship between the target entities from a knowledge graph, wherein the knowledge graph stores a plurality of entities and the relationship between the plurality of entities; in the case where the first knowledge item includes a knowledge item for describing legal knowledge, the target entity includes at least one of the following entities: a legal document, a legal clause, a judicial institution, a legal role, a legal concept, a legal event, a legal entity, and a geographical location; A generation module, used for concatenating the first text and the target entity to obtain a second text; generating a third text for describing the relationship between the target entities; generating a first prompt word based on the second text, the third text and the first question text, and inputting the first prompt word into a language model to obtain an answer text corresponding to the first question text; The target entity includes a first entity corresponding to the first text and a second entity related to the first entity, the second entity at least includes an entity connected to the first entity, and each entity has corresponding text positioning information; The generating module obtains the second text by: concatenating the first entity with the first text to obtain a first segment; and obtaining text corresponding to the second entity based on text location information corresponding to the second entity; The second entity and the text corresponding to the second entity are concatenated to obtain a second segment; the first segment and the second segment are concatenated to obtain a second text.

6. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the text processing method as claimed in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the text processing method as claimed in any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product comprises a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps in the text processing method according to any one of claims 1 to 4.

Images