Data extraction method and device, computer equipment and storage medium

Abstract

The invention discloses a data extraction method and device, computer equipment and a storage medium. The method comprises the steps of determining a first text fragment of a text and a preset description text of an element; determining a plurality of second text fragments associated with a preset description text from the first text fragments according to the preset description text; determining a third text fragment associated with the element from the plurality of second text fragments; and performing element extraction on the third text fragment to obtain an element extraction result of a corresponding element in the text. According to the method, the second text fragment possibly related to the element is firstly extracted from the first text fragment of the text, and then the third text fragment is further determined from the second text fragment, so that the extraction of the element is realized, the final element extraction effect can be effectively ensured only by ensuring the execution accuracy of each simple process, and the extraction efficiency of the element is improved. And the extraction accuracy of the elements is improved.

Description

Technical Field

[0001] The present application relates to the field of text processing, and in particular to a data extraction method, device, computer equipment and storage medium. Background Art

[0002] In normal business exchanges and activities, the parties need to sign a large number of contracts to protect the interests of the parties involved and provide legal protection. In order to ensure the legality of the contract and avoid risks, the parties to the contract need to review the contract before it is officially signed. In traditional contract review, legal professionals need to spend a lot of time and energy to review the contract, and the review efficiency is very low; at the same time, due to negligence or different understandings of the terms, review errors and omissions may occur.

[0003] At present, intelligent contract review based on artificial intelligence technology, such as entity recognition model or large language model, is limited by the training effect of the model when extracting contract elements, and there is still the problem of unsatisfactory extraction effect. Summary of the invention

[0004] In response to the above technical problems, the embodiments of the present application provide a data extraction method, apparatus, computer device and storage medium, aiming to solve the problem of unsatisfactory data extraction effect.

[0005] In a first aspect, an embodiment of the present application provides a data extraction method, comprising:

[0006] Determine a first text segment of the text, and a preset description text of the element;

[0007] Determining, from the first text segment according to the preset description text, a plurality of second text segments associated with the preset description text;

[0008] determining a third text segment associated with the element from the plurality of second text segments;

[0009] Perform element extraction on the third text segment to obtain an element extraction result corresponding to the element in the text.

[0010] In a second aspect, an embodiment of the present application provides a data extraction device, comprising:

[0011] A determination module, used to determine a first text segment of the text and a preset description text of the element to be extracted;

[0012] An extraction module, configured to determine, from the first text segment, a plurality of second text segments associated with the preset description text according to the preset description text;

[0013] a screening module, configured to determine a third text segment associated with the element from the plurality of second text segments;

[0014] A processing module is used to extract elements from the third text segment to obtain element extraction results corresponding to the contract elements in the contract text.

[0015] In a third aspect, an embodiment of the present application further provides a computer device, comprising a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the data extraction steps as described above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for loading by a processor to execute the steps of data extraction as described above.

[0017] In a fifth aspect, an embodiment of the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to perform the steps of data extraction as described above.

[0018] The scheme of the embodiment of the present application will first use the preset description text containing the element to determine multiple second text segments associated with the preset description text from the first text segment of the text, and then determine the third text segment associated with the element from the multiple second text segments, and finally process the third text segment to obtain the element extraction result of the corresponding element in the text. The present application realizes the extraction of elements by first extracting the second text segment that may be related to the contract element, and then further determining the third text segment therefrom. Compared with the scheme of directly extracting elements through a one-time process of the model, it only needs to ensure the execution accuracy of each simple process to effectively ensure the extraction effect of the final element and improve the extraction accuracy of the element. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of the steps of a data extraction method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process flow for screening a second text segment provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a process flow for determining a third text segment provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a process flow of determining a third text segment based on a reflection mechanism provided in an embodiment of the present application;

[0024] Figure 5 A complete implementation flow diagram of a data extraction method provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of a data extraction device provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0028] In order to better understand the data extraction method, device, computer equipment and storage medium provided by this application, the relevant application background of the data extraction method is first explained. Among them, the data extraction method provided by this application mainly refers to extracting specified element information from relevant texts, for example, it is particularly suitable for element extraction of contract texts, that is, extracting and specifying contract elements from the signed contract text through artificial intelligence technology, such as Party A's name, deadline date, delivery amount, invoice type, etc., so as to assist legal professionals in reviewing the contract text.

[0029] At present, the relevant technologies for implementing factor extraction mainly include entity extraction models and language question-answering models. The former converts factor extraction into entity extraction, and uses entity recognition models such as BERT-CRF to extract the names of the factors to be extracted in advance, while the latter mainly relies on prompt words, and inputs the prompt words related to the factors and the complete text into the large language model, so that the large language model outputs the corresponding factor extraction results. However, in actual application, especially when there are multiple interfering information in the text, such as multiple time information and multiple quantity amounts, by inputting the complete text into the model for factor extraction, there will be a technical problem that the extracted factor information is not accurate enough.

[0030] In order to solve the above problems, the present application provides a data extraction method, device, computer equipment and storage medium, which first uses some relevant possible description texts containing elements to filter out multiple second text fragments related to the text from the text, and then further filters out the third text fragment most related to the element from the second text fragment, so as to finally extract the elements from the third text fragment to obtain the final element extraction result. Compared with inputting the complete text into the model once to realize element extraction, by decomposing the element extraction process into multiple simple processes, it is only necessary to ensure the execution accuracy of each process to effectively ensure the effect of the final element extraction. Specifically, the data extraction method is usually set in the data extraction device in the form of a computer program, and the data extraction device is usually set in the computer device in the form of a processor, such as the server-side device used by various manufacturers to store contract documents. The data extraction device in the computer device executes the computer program corresponding to the data extraction method to execute the data extraction method provided in the embodiment of the present application.

[0031] For details, please refer to Figure 1 , Figure 1 A schematic flow chart of a data extraction method provided in an embodiment of the present application includes steps S110 to S140:

[0032] S110, determining a first text segment of the text and a preset description text of the element.

[0033] Combined with the aforementioned description of related scenarios, it can be known that text usually refers to text from which element information needs to be extracted, and the first text segment of the text can usually refer to the text segmented in an appropriate manner by technical means such as text segmentation. For example, it is more common to identify specific characters in the text, such as "period", and complete the segmentation of the text based on the "period". Of course, it is also feasible to segment the text according to semantic information to obtain multiple first text segments by other methods, such as semantic segmentation, and the embodiments of the present application are not limited here.

[0034] On the basis of the above, the elements refer to the contract elements that need to be extracted from the above text. For different types of texts, the element information that needs to be extracted in the general sense is different. Specifically, for ease of understanding, the following will take the contract text as an example. At this time, the elements that need to be extracted mainly include Party A's name, deadline date, delivery amount, invoice type, etc. Among them, it should be noted that the elements to be extracted may usually include multiple elements, but for different elements, the corresponding element information can be extracted by separately executing the data extraction method provided in the embodiment of the present application. Therefore, for the convenience of subsequent description, the extraction of a single element will be used as an example for explanation. When multiple elements need to be extracted, based on the scheme provided in the embodiment of the present application, those skilled in the art can undoubtedly determine the extraction method for other elements.

[0035] Furthermore, unlike the conventional method of directly matching the corresponding element information from the text through the elements, in the technical solution provided in the embodiment of the present application, different description texts are further pre-set for different elements. Among them, the preset description text here usually refers to the common description information containing the corresponding extracted elements collected in advance from some sample texts. For example, as a feasible implementation scheme, the preset description text refers to the common description of the context (pre and post) fragment corresponding to a certain element collected from the sample text. Specifically, that is, before determining the preset description text of the element, it is usually also included to collect the corresponding preset description text for each element, which specifically includes:

[0036] Extracting a description text segment containing the elements from the sample text;

[0037] The context segment of the description text segment is used as the preset description text of the element.

[0038] The sample text refers to some complete texts that have been made public based on network technology, such as through web crawlers or reading from relevant text databases. For example, the contract text can be a published contract text read from a website related to the contract document, and then a description text segment containing a certain element is extracted from the text through manual annotation or other technical means. For example, taking the amount element in the contract text as an example, the description text segment can be "the delivery amount is XX yuan", "within a given time limit, XX pays XXX amount XXXX yuan", etc. Of course, on this basis, in order to improve the matching and recognition results of the elements, the context segment of this description text segment, that is, some implicit information associated with the element, can also be used as the preset description text of the element.

[0039] In the actual application process, the embodiment of the present application does not limit the number of preset description texts. For example, the preset description text may only include a single one. Of course, in order to ensure the matching recognition result of the element, the number of preset description texts may also generally include multiple ones. Further, on the basis that the number of preset description texts includes multiple ones, different numbers of preset description texts may be further set for different elements. For example, by statistically analyzing the contract extraction results of the historical text, for some elements that are difficult to extract or have low extraction accuracy, the corresponding preset description texts may be set more, for example, 10 may be set, while for some elements that are relatively easy to extract or have relatively high extraction accuracy, the corresponding preset description texts may be appropriately reduced. For example, as a feasible implementation scheme, the number of preset description texts may be set to 5. Of course, the above-mentioned embodiment is only an explanation of a feasible scheme for collecting different numbers of preset description texts for different elements, and is not a limitation to the embodiment of the present application.

[0040] S120: Determine, from the first text segment according to the preset description text, a plurality of second text segments associated with the preset description text.

[0041] On the basis of the preset description text of the element to be extracted as described above, further, in order to realize the extraction of element information related to the element, in the embodiment of the present application, multiple second text segments associated with the preset description text will be further determined from the first text segment based on these preset description texts. It can be understood that, in combination with the above-mentioned related instructions, since the preset description text usually refers to some common descriptions of context segments corresponding to the elements collected in advance, therefore, based on these preset description texts, the first text segment can be simply screened to determine the multiple second text segments associated with the preset description text, that is, potentially associated with the element.

[0042] Specifically, determining multiple second text segments associated with a preset description text from a first text segment includes multiple implementation schemes, for example, it can be implemented by processing a network model trained based on machine learning. Of course, it is also feasible to use some pre-trained text encoding models, such as General Text Embeddings (GTE, general text embedding, a sentence vector model for generating vector representations of text sentences). Specifically, the text encoding model can be used to convert different texts into different vector representations, so that the vector similarity between these vectors can be used to quickly filter out multiple related texts. For details, please refer to Figure 2 , Figure 2A schematic flow chart of a step of screening a second text segment provided in an embodiment of the present application, specifically, comprising steps S210 to S230:

[0043] S210: Process the first text segment through a text encoding model to obtain a first encoding feature of the first text segment.

[0044] S220, determining a second encoding feature obtained by processing the preset description text by the text encoding model.

[0045] In the embodiment of the present application, combined with the above-mentioned related description, it can be known that by using some trained text encoding models to process the preset description text and the first text segment respectively, the encoding features corresponding to these texts can be obtained, for example, including the first encoding feature of the first text segment and the second encoding feature of the preset description text. Among them, since the preset description text is collected in advance, the preset description text can also be processed in advance by the text encoding model to obtain the second encoding feature, and stored in the vector database in advance, so that when the contract element extraction is needed, it can be directly read from the vector database.

[0046] S230: Determine, from the first text segment, a plurality of second text segments associated with the preset description text according to the similarity between the first coding feature and the second coding feature.

[0047] In the embodiment of the present application, after obtaining the encoding features of the preset description text and the first text segment through the text encoding model, by further calculating the similarity between the first encoding feature and the second encoding feature, text segments with higher similarity to the preset description text can be selected from the first text segment as multiple second text segments. Among them, there are many ways to calculate the similarity between the first encoding feature and the second encoding feature, for example, it can be determined based on the Euclidean distance or cosine similarity between the encoding feature vectors. The embodiment of the present application does not limit the specific implementation scheme for calculating the similarity between the first encoding feature and the second encoding feature.

[0048] Furthermore, when there are multiple preset description texts, it is usually necessary to determine a text segment associated with each preset description text from the first text segment as the final second text segment.

[0049] In addition, as another feasible embodiment of the present application, it can be known from the above-mentioned related description that different numbers of preset description texts can be set for different elements. Then, in the process of determining multiple second text segments associated with the preset description text from the first text segment according to the similarity between the first coding feature and the second coding feature, it can also be the process of selecting different numbers of second text segments depending on the element types of different contract elements, that is, at this time, a certain number of text segments will be determined from the first text segment as the second text segments associated with the preset description text according to the similarity between the first coding feature and the second coding feature, that is, according to the similarity between the first coding feature and the second coding feature, multiple second text segments associated with the preset description text are determined from the first text segment, specifically including:

[0050] Determining, according to the element type of the contract element, the extraction quantity corresponding to the element type;

[0051] According to the similarity between the first coding feature and the second coding feature and the extraction quantity, text segments corresponding to the extraction quantity are determined from the first text segments as multiple second text segments associated with the preset description text.

[0052] Among them, the correlation between the element type and the number of extractions of the element can be implemented by referring to the aforementioned setting of different numbers of preset description texts for different elements, that is, for some elements that are difficult to extract or have low extraction accuracy, the second text fragments screened out can be more, and conversely, for some elements that are relatively easy to extract or have relatively high extraction accuracy, the number of the second text fragments screened out can be appropriately reduced to improve data extraction efficiency. The embodiments of the present application are not described in detail here.

[0053] Furthermore, by sequentially sorting the similarities between the first coding features and the second coding features, the first corresponding number of first text segments that are most similar to the preset description text can be used as multiple second text segments associated with the preset description text for subsequent feature extraction.

[0054] S130: Determine a third text segment associated with the element from the plurality of second text segments.

[0055] After the aforementioned preset description text based on the element extracts several second text fragments that are potentially related to the element from the first text fragment, in order to more accurately realize the element extraction of the element, on the basis of the extraction scheme of the present application, a third text fragment associated with the element will be further determined from the second text fragment.

[0056] It can be understood that the accuracy of the third text segment determined from the second text segment will directly affect the effect of subsequent element extraction. Therefore, in order to ensure the element extraction effect of the data extraction method provided by this application, as a feasible embodiment of this application, a multi-layer model question-answering method that facilitates the introduction of a reflection mechanism is also provided to implement the determination of the third text segment. For details, please refer to Figure 3 , Figure 3 A schematic flow chart of a step of determining a third text segment provided in an embodiment of the present application, specifically, comprising steps S310 to S330:

[0057] S310: Input the plurality of second text segments and the elements into a first model, and output a fourth text segment among the candidate text segments.

[0058] In an embodiment of the present application, by inputting the multiple second text fragments and elements obtained above into the first model, the first model can process the multiple second text fragments and elements, and output a text fragment among the second text fragments, such as the fourth text fragment, as the text fragment that it considers to be most similar to the element.

[0059] Of course, it should be noted that, unlike the above-mentioned method of selecting the second text fragment based on similarity sorting, if only considering using each second text fragment to calculate the similarity between the elements, and finally selecting the text fragment with the highest similarity to the elements in the second text fragment as the fourth text fragment, the output of the fourth text fragment is often not accurate enough, and when the output of the fourth text fragment is wrong, it is also impossible to effectively teach the model and make the model output more accurate results. Therefore, in the embodiment of the present application, the first model used does not analyze different second text fragments independently with the elements, but cross-analyzes and considers the association between multiple second text fragments and elements, that is, the first model not only considers which second text fragment is most accurately corresponding to the element when processing multiple second text fragments and elements, but also considers the differences between the remaining second text fragments and the elements. Therefore, when the selected text fragment is wrong, by teaching the model the experience that the text fragment is not the most accurate related text corresponding to the element, the model can reflect and analyze based on the experience, consider the differences between the text fragment and the element, and further determine the third text fragment that it believes to be most accurately corresponding to the element from the remaining second text fragments until the requirements are met.

[0060] It can be seen that in the above process, the first model is an intelligent entity that can perceive the environment, make decisions and execute actions, that is, it has the ability to gradually complete a given goal through independent thinking and calling tools, that is, it can be understood as an intelligent agent. Therefore, the first model provided in the embodiment of the present application can use a large language model (LLM), that is, it has the ability to output more accurate results by teaching relevant experience inspiration as provided above.

[0061] S320: Input the fourth text segment and the element into the second model to obtain a matching result of the fourth text segment and the element.

[0062] In combination with the above-mentioned related description, it can be known that the fourth text fragment refers to the text fragment that is considered to correspond most accurately to the element, which is selected from the second text fragment by the trained parameters when the first model does not receive external teaching enlightenment, that is, the selection of the fourth text fragment may be correct, and of course there may be errors. Therefore, as a further optional embodiment of the present application, the fourth text fragment and the element will be further input into the second model to determine whether the fourth text fragment is the text fragment that best corresponds to the element in the entire text. In other words, the second model has similar characteristics to the aforementioned first model. Therefore, the second model can also be understood as an intelligent agent, that is, a large language model is selected to realize the judgment of the matching results of the fourth text fragment and the element. Of course, in the actual application process, the first model and the second model are integrated into the same large language model as different sub-model structures for use, and the embodiments of the present application will not be repeated here.

[0063] In the embodiment of the present application, the matching results between the fourth text segment and the contract element generally include two types, namely, a correct match or a match failure, and corresponding to different matching results, different processes need to be executed subsequently.

[0064] S330: In response to the matching result being the first matching result, determining the fourth text segment as the third text segment associated with the element.

[0065] In the embodiment of the present application, the first matching result generally refers to a correct match, that is, when the first model determines that the fourth text segment is the text segment in the second text segment that most accurately corresponds to the element, and the second model confirms that the fourth text segment is the text segment in the text that corresponds to the element, then the fourth text segment can be directly selected as the third text segment associated with the element, so as to be used for subsequent processing of the text segment to extract the element information. Correspondingly, when the first model determines that the fourth text segment is the text segment in the second text segment that most accurately corresponds to the element, but the second model determines that the fourth text segment is not the text segment in the text that most corresponds to the element, then the first model will be taught and enlightened based on the reflection mechanism to guide the first model to make a more accurate judgment. For details, please refer to Figure 4 , Figure 4 A schematic flow chart of the steps for determining a third text segment based on a reflection mechanism provided in an embodiment of the present application is described in detail as follows.

[0066] In the embodiment of the present application, after the fourth text segment and the element are input into the trained second model and the matching result of the fourth text segment and the contract element is obtained, when the matching result does not meet the specific situation, the implementation scheme of the third text segment is re-determined based on the reflection mechanism, wherein, that is, the second text segment, in addition to the aforementioned fourth text segment, generally includes other text segments, that is, the fifth text segment, specifically, including steps S410 to S420:

[0067] S410: In response to the matching result being the second matching result, inputting the fifth text segment and the element into the first model, and outputting the sixth text segment in the second text segment.

[0068] In an embodiment of the present application, corresponding to the aforementioned first matching result, the second matching result usually refers to a matching failure, that is, when the second model determines that the fourth text fragment is not the text fragment that best corresponds to the element in the text, it will choose to input other text fragments except the fourth text fragment, that is, the fifth text fragment and the element into the first model, so that the first model can re-analyze the association between the second text fragment and the element, and combine the teaching inspiration given by the aforementioned second model that the fourth text fragment is not the text fragment that most accurately corresponds to the element, and select a sixth text fragment different from the fourth text fragment from the remaining second text fragments as the text fragment that it believes best corresponds to the contract element.

[0069] S420: In response to the matching result between the sixth text segment and the element being a first matching result, determine the sixth text segment as a third text segment associated with the element.

[0070] In an embodiment of the present application, after the aforementioned first model reselects the sixth text segment that best matches the element, similar to the sixth text segment, the sixth text segment and the element need to be input into the second model to obtain the matching result of the sixth text segment and the element, and when the matching result is the first matching result, that is, when the match is correct, the sixth text segment is selected to be determined as the third text segment associated with the element. Otherwise, in the event of a matching failure, it will be similar to the fourth text segment and continue to be used to teach and inspire the first model, so as to guide the first model to continue to screen out the third text segment that it believes is most likely to correspond to the element from the remaining text segments.

[0071] Specifically, the introduction of teaching enlightenment can usually be to write the matching result as an error, for example, the aforementioned fourth text fragment is written into the error context list. At this time, the error context list will be used as the first model, that is, the prior knowledge base of the large language model, to teach the large language model that the text fragment written in the error context list is not the text fragment that best corresponds to a certain element, so that when the large language model subsequently screens the second text fragment, it can combine the text fragment written in the error context list to improve the accuracy of the determined third text fragment. That is, through continuous reflection, that is, constantly writing the wrong text fragments into the error context list, it is possible to accumulate the prior knowledge base of the large language model, thereby continuously improving the accuracy of the large language model screening the third text fragment that most accurately corresponds to the element.

[0072] Of course, it should be noted that in the aforementioned process of continuously reflecting on whether the selected text fragment is the most capable of corresponding to the element, a preset iteration threshold can also be set based on actual needs, that is, the number of times the model repeatedly selects the text fragment. For example, the preset iteration threshold can be selected as 5, that is, after the large language model repeatedly selects the text fragment 5 times, the last selected text fragment is used as the third text fragment, and subsequent processing is directly performed.

[0073] It can be understood that through the aforementioned multi-stage processing, that is, first screening out the text segment that most accurately corresponds to the element, and then confirming whether the text segment is the most corresponding to the element in the text, and if the confirmation result is wrong, by writing the erroneous text segment into the error context list of the large language model, teaching the large language model as the prior knowledge base of the large language model, it is possible to continuously accumulate the knowledge base of the large language model based on the reflection mechanism, thereby ultimately improving the large language model to accurately identify the text segment that most accurately corresponds to the element.

[0074] S140, extracting elements from the third text segment to obtain element extraction results corresponding to the contract elements in the contract text.

[0075] In an embodiment of the present application, after the aforementioned screening of the third text segment that most accurately corresponds to the element is implemented, for example, the screening of the third text segment is implemented by introducing a large language model with a reflection mechanism, and then the third text segment containing the element is further processed, that is, element extraction, and the element extraction result of the element can be extracted from the third text segment.

[0076] For example, as a common feasible implementation scheme, the element, the third text segment and the Few-shot template (few samples) can be combined and input into the large language model to obtain the element extraction result in the contract text. Of course, the above is only a feasible implementation scheme and is not a limitation of the present application. It is also feasible to use other methods, use elements as named entities, and retrieve in the third text segment based on the entity recognition model to obtain the element extraction result of the element. The embodiments of the present application will not be repeated here.

[0077] Of course, it should be noted that, similar to the aforementioned determination of the third text segment from the second text segment, when a large language model is used to implement the extraction of elements from elements, a reflection mechanism can also be introduced to further improve the effect of the element extraction result. That is to say, as a further optional implementation scheme of the present application, the third text segment is subjected to element extraction to obtain the element extraction result corresponding to the element in the text, which can generally also include:

[0078] Inputting the third text segment and the element into a third model to obtain a first element extraction result;

[0079] Inputting the first element extraction result and the element into a fourth model to obtain a matching result between the element and the first element extraction result;

[0080] In response to the matching result being a first matching result, using the first element extraction result as an element extraction result corresponding to the element in the text;

[0081] In response to the matching result being a second matching result, inputting a third text segment and the element except the first element extraction result into the third model, and outputting a second element extraction result;

[0082] In response to the matching result between the second element extraction result and the element being a first matching result, the second element extraction result is determined as the element extraction result corresponding to the element in the text.

[0083] The explanation of the reflection mechanism has been provided above, and this embodiment will not be repeated here. For details, please refer to the above Figure 3 as well as Figure 4In this embodiment, the third model and the fourth model are similar to the first model and the second model, and usually a large language model can be selected or integrated into the large language model. When it is determined that the matching result between the element and the first element extraction result is incorrect, the first element extraction result will be written into the error element extraction result list as a priori knowledge base of the large language model to teach the large language model that the element extraction results in the error element extraction result list are accurate, so that the large language model can combine the element extraction results that are constantly superimposed in the error element extraction result list to more accurately extract subsequent element information.

[0084] Of course, in order to more accurately extract the elements in the third text segment, in the process of processing the third text segment to obtain the element extraction result in the text, the third text segment can also be used as a basis to continue to expand a segment forward and backward as a reference text segment corresponding to the element, so as to assist in the extraction of element information and avoid element extraction errors caused by incomplete description of the context segment. That is to say, the third text segment is processed to obtain the element extraction result corresponding to the element in the text, which specifically also includes:

[0085] A context segment of the third text segment in the contract text is extracted to obtain a reference text segment.

[0086] The third text segment and the reference text segment are processed to obtain an element extraction result corresponding to the element in the text.

[0087] Using the scheme of the embodiment of the present application, the preset description text containing the element is first used to determine multiple second text segments associated with the preset description text from the first text segment, and then the third text segment associated with the element is determined from the multiple second text segments, and finally the third text segment is processed to obtain the element extraction result corresponding to the element in the text. The present application realizes the extraction of elements by first extracting the second text segment that may be related to the element, and then further determining the third text segment therefrom. Compared with the scheme of directly extracting elements through a one-time process of the model, it only needs to ensure the execution accuracy of each simple process to effectively ensure the extraction effect of the final element and improve the extraction accuracy of the element.

[0088] In order to more clearly understand the complete implementation process of the data extraction method provided in the embodiment of the present application, the following will be combined with the aforementioned Figure 1 to Figure 4 The content provided, and taking the contract text as an example, provides a complete implementation process of a method for extracting data from the contract text. For details, please refer to Figure 5 In the complete implementation flow diagram of the data extraction method provided in this application, the following steps are specifically included:

[0089] 1. The contract text to be extracted is divided into pieces according to preset characters, such as "periods", to obtain multiple contract text fragments, where the number of contract text fragments is usually more than a hundred. Then, each contract text fragment is encoded using a pre-trained vector model (such as GTE) to obtain text vectors corresponding to different contract text fragments and store them in a vector database.

[0090] 2. Collect common descriptions of the context fragments corresponding to each element to be extracted from the sample contract text, and collect a total of K preset description texts. For contract elements that are difficult to extract or have low extraction accuracy, more preset description texts can be collected, such as K=10. For contract elements that are easier to extract or have high extraction accuracy, fewer preset description texts can be collected, such as K=5.

[0091] 3. The collected preset description text is processed by the same pre-trained vector model as in process 1 to obtain the corresponding K vectors. Then, the candidate text segments that are most similar to the K vectors are retrieved from the vector database.

[0092] 4. Input K candidate text segments and contract elements, such as the names of contract elements, into the large language model, and let it select the most accurate text segment corresponding to the contract element from the K candidate text segments.

[0093] 5. Input the selected text fragment, contract text and element name into the large language model to let it judge whether the text fragment is the text fragment that best corresponds to the contract element in the contract text. If the judgment result is correct, the text fragment is used as the target text fragment for the next step. If the judgment is wrong, the selected text fragment is added to the error context list to tell the large language model that the text fragment selected in this list is wrong, and let it reselect a candidate text fragment. If the number of errors is greater than the iteration threshold M (for example, M can be 5), the candidate text fragment obtained last time is selected as the target text fragment, and the execution continues backward.

[0094] 6. In order to prevent the target text segment from causing incorrect element extraction due to incomplete description, the target text segment can be used as the basis to extend one segment forward and one segment backward respectively as the final text segment corresponding to the contract element.

[0095] 7. The name of the contract element, the final text fragment corresponding to the contract element and the few-shot template are combined and input into the large language model to obtain the element extraction result.

[0096] 8. Input the name of the contract element, the element extraction result, and the final text fragment corresponding to the contract element into the large language model, and let it judge again whether the element extraction result is correct.

[0097] 9. If the judgment result is correct, the task ends and the corresponding text segment and feature extraction result are returned; if the judgment result is wrong, this extraction result is added to the list of wrong feature extraction results, telling the large language model that the feature results in this list are extracted incorrectly and asking it to extract again. If the number of judgment errors is greater than the iteration threshold M (M can be 5), the last feature extraction result is selected as the final result and the task ends.

[0098] In order to better implement the data extraction method provided in the embodiment of the present application, the present embodiment also provides a data extraction device, which can be integrated in the server. Figure 6 As shown, the data extraction device may include:

[0099] A determination module 610, configured to determine a first text segment of a text and a preset description text of an element;

[0100] An extraction module 620, configured to determine, from the first text segment, a plurality of second text segments associated with the preset description text according to the preset description text;

[0101] A screening module 630, configured to determine a third text segment associated with the element from the plurality of second text segments;

[0102] The processing module 640 is used to extract elements from the third text segment to obtain element extraction results corresponding to the elements in the text.

[0103] In some embodiments of the present application, the screening module 630 is further used to input the plurality of second text segments and the elements into the first model, and output a fourth text segment in the second text segments;

[0104] Inputting the second text segment and the element into a second model to obtain a matching result between the fourth text segment and the element;

[0105] In response to the matching result being the first matching result, the fourth text segment is determined as the third text segment associated with the element.

[0106] In some embodiments of the present application, the screening module 630 is further configured to, in response to the matching result being the second matching result, input the fifth text segment in the second text segment and the element into the first model, and output the sixth text segment in the second text segment;

[0107] In response to the matching result between the sixth text segment and the element being a first matching result, the sixth text segment is determined as a third text segment associated with the element.

[0108] In some embodiments of the present application, the extraction module 620 is further configured to process the first text segment through a text encoding model to obtain a first encoding feature of the first text segment.

[0109] Determine a second encoding feature obtained by processing the preset description text by the text encoding model;

[0110] According to the similarity between the first coding feature and the second coding feature, a plurality of second text segments associated with the preset description text are determined from the first text segment.

[0111] In some embodiments of the present application, the extraction module 620 is further used to determine the extraction quantity corresponding to the element type according to the element type of the element;

[0112] According to the similarity between the first coding feature and the second coding feature and the extraction quantity, text segments corresponding to the extraction quantity are determined from the first text segments as multiple second text segments associated with the preset description text.

[0113] In some embodiments of the present application, the processing module 640 is further used to extract a context segment of the third text segment in the text to obtain a reference text segment;

[0114] The third text segment and the reference text segment are processed to obtain an element extraction result corresponding to the element in the text.

[0115] In some embodiments of the present application, the determination module 610 is further used to extract a description text segment containing the element from the sample text;

[0116] The context segment of the description text segment is used as the preset description text of the element.

[0117] Using the scheme of the embodiment of the present application, the preset description text containing the element is first used to determine multiple second text segments associated with the preset description text from the first text segment of the text, and then the third text segment associated with the element is determined from the multiple second text segments, and finally the third text segment is processed to obtain the element extraction result corresponding to the element in the text. The present application realizes the extraction of elements by first extracting the second text segment that may be related to the element, and then further determining the third text segment therefrom. Compared with the scheme of directly extracting elements through a one-time process of the model, it only needs to ensure the execution accuracy of each simple process to effectively ensure the extraction effect of the final element and improve the extraction accuracy of the element.

[0118] Based on the same inventive concept, the embodiment of the present application also provides a computer device, which may be a server or a terminal device, and includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above data extraction method when executing the computer program. Thereby, various functions are realized, such as executing the following steps:

[0119] Determine a first text segment of the text, and a preset description text of the element;

[0120] Determining, from the first text segment according to the preset description text, a plurality of second text segments associated with the preset description text;

[0121] determining a third text segment associated with the element from the plurality of second text segments;

[0122] Perform element extraction on the third text segment to obtain an element extraction result corresponding to the contract in the text.

[0123] Using the scheme of the embodiment of the present application, the preset description text containing the element is first used to determine multiple second text segments associated with the preset description text from the first text segment of the text, and then the third text segment associated with the element is determined from the multiple second text segments, and finally the third text segment is processed to obtain the element extraction result corresponding to the element in the text. The present application realizes the extraction of elements by first extracting the second text segment that may be related to the element, and then further determining the third text segment therefrom. Compared with the scheme of directly extracting elements through a one-time process of the model, it only needs to ensure the execution accuracy of each simple process to effectively ensure the extraction effect of the final element and improve the extraction accuracy of the element.

[0124] In one embodiment, the computer device is a terminal device, for example, its internal structure diagram can be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data extraction method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0125] Those skilled in the art will understand that Figure 7 The structure shown is only a block diagram of a portion of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0126] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0127] Since the computer program stored in the computer-readable storage medium can execute any one of the data extraction methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the data extraction methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0128] Using the scheme of the embodiment of the present application, the preset description text containing the element is first used to determine multiple second text segments associated with the preset description text from the first text segment of the text, and then the third text segment associated with the element is determined from the multiple second text segments, and finally the third text segment is processed to obtain the element extraction result corresponding to the element in the text. The present application realizes the extraction of elements by first extracting the second text segment that may be related to the element, and then further determining the third text segment therefrom. Compared with the scheme of directly extracting elements through a one-time process of the model, it only needs to ensure the execution accuracy of each simple process to effectively ensure the extraction effect of the final element and improve the extraction accuracy of the element.

[0129] Based on the same inventive concept, the embodiment of the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations in the above embodiments.

[0130] It should be noted that the object data (including but not limited to user device information, user personal information, etc.) and conversation data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0131] Any reference to the memory, database or other media used in the embodiments provided in the present application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (Re RAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0132] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0133] In the above data extraction device, computer equipment, computer readable storage medium, and computer program product embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described data extraction device, computer readable storage medium, computer program product, computer equipment and its corresponding units can refer to the description of the data extraction method in the above embodiment, and will not be repeated here.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above is a detailed introduction to a data extraction method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data extraction method, characterized in that: include: Determine a first text segment of the text, and a preset description text of the element; Determining, from the first text segment according to the preset description text, a plurality of second text segments associated with the preset description text; determining a third text segment associated with the element from the plurality of second text segments; Perform element extraction on the third text segment to obtain an element extraction result corresponding to the element in the text.

2. The method according to claim 1, characterized in that The determining a third text segment associated with the element from the plurality of second text segments comprises: Inputting the plurality of second text segments and the elements into the first model, and outputting a fourth text segment among the second text segments; Inputting the fourth text segment and the element into the second model to obtain a matching result between the fourth text segment and the element; In response to the matching result being the first matching result, the fourth text segment is determined as the third text segment associated with the element.

3. The method according to claim 2, characterized in that The second text segment includes a fourth text segment and a fifth text segment. After the fourth text segment and the element are input into the second model to obtain a matching result between the fourth text segment and the element, the method further includes: In response to the matching result being a second matching result, inputting the fifth text segment and the element into the first model, and outputting a sixth text segment in the second text segment; In response to the matching result between the sixth text segment and the element being a first matching result, the sixth text segment is determined as a third text segment associated with the element.

4. The method according to claim 1, characterized in that: The step of determining, from the first text segment, a plurality of second text segments associated with the preset description text according to the preset description text comprises: Processing the first text segment through a text encoding model to obtain a first encoding feature of the first text segment; Determine a second encoding feature obtained by processing the preset description text by the text encoding model; According to the similarity between the first coding feature and the second coding feature, a plurality of second text segments associated with the preset description text are determined from the first text segment.

5. The method according to claim 4, characterized in that Before determining the plurality of second text segments associated with the preset description text from the first text segment based on the similarity between the first coding feature and the second coding feature, the method includes: determining the extraction quantity corresponding to the element type based on the element type of the element; According to the similarity between the first coding feature and the second coding feature and the extraction quantity, text segments corresponding to the extraction quantity are determined from the first text segments as multiple second text segments associated with the preset description text.

6. The method according to claim 1, characterized in that The extracting elements from the third text segment to obtain an element extraction result corresponding to the element in the first text includes: Extracting a context segment of the third text segment in the text to obtain a reference text segment; Perform element extraction on the third text segment and the reference text segment to obtain element extraction results corresponding to the elements in the text.

7. The method according to any one of claims 1 to 6, characterized in that: Before determining the preset description text of the element, the method further includes: Extracting a description text segment containing the elements from the sample text; The context segment of the description text segment is used as the preset description text of the element.

8. A data extraction device, characterized in that: include: A determination module, used to determine a first text segment of the text, and a preset description text of the element; An extraction module, configured to determine, from the first text segment, a plurality of second text segments associated with the preset description text according to the preset description text; a screening module, configured to determine a third text segment associated with the element from the plurality of second text segments; The processing module is used to extract elements from the third text segment to obtain element extraction results corresponding to the elements in the text.

9. A computer device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the data extraction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps of the data extraction method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used by a processor to execute the steps of the data extraction method according to any one of claims 1 to 7.

Images