Information extraction method and field selection method

Abstract

The embodiments of this specification provide an information extraction method and a field selection method, wherein the information extraction method includes: displaying a field selection page containing a business sample document according to business query information, and determining the query field corresponding to the business query information; in response to a selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document; recalling a candidate document paragraph set in a business document library according to the query field, and inputting the query field and the selection field into a large language model; recalling a target document paragraph in the candidate document paragraph set through the large language model, and extracting the target information associated with the business query information in the target document paragraph by referring to the selection field using the large language model. The information extraction operation is driven by user-defined selection fields, thereby effectively improving the accuracy of information extraction.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of information processing technology, and in particular to an information extraction method and a field selection method. Background Art

[0002] With the development of the Internet and computer technology, information extraction tasks are applied in most business scenarios. The execution of information extraction tasks can extract subsets with representative and specific features from a large amount of data, so that the subsets can be used for subsequent analysis and processing. In the prior art, the execution of information extraction tasks mostly relies on random sampling or algorithm implementation. In the random sampling strategy, the problem of insufficient representativeness of the extraction results is often accompanied, which will seriously affect the accuracy of the analysis. The extraction algorithm can ensure the representativeness of the extraction results by configuring operators, but the extraction algorithm is very dependent on the hardware performance of the computing device when processing large-scale data. In the case of high computational complexity, it is easy to cause the extraction time to become longer and the efficiency to be lower. In addition, whether it is random sampling or extraction algorithm, when facing the situation where the same word has different meanings in different scenarios, for example, "fundamentals" in the stock investment scenario means financial status, profitability, market competitiveness, etc., and in the macroeconomic scenario means GDP growth rate, inflation rate, unemployment rate, etc., at this time, the algorithm cannot achieve the balance of the multiple meanings of a word, which will lead to deviations in the extraction results in a certain dimension. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, the embodiments of this specification provide an information extraction method. One or more embodiments of this specification also relate to an information extraction device, a field selection method, a field selection device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, there is provided an information extraction method, comprising:

[0005] Displaying a field selection page containing a business sample document according to the business query information, and determining a query field corresponding to the business query information;

[0006] In response to a selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document;

[0007] Recalling a candidate document paragraph set in a business document library according to the query field, and inputting the query field and the selected field into a large language model;

[0008] The target document paragraph is recalled from the candidate document paragraph set by using the large language model, and the target information associated with the business query information is extracted from the target document paragraph by using the large language model with reference to the selected field.

[0009] According to a second aspect of an embodiment of this specification, a field selection method is provided, which is applied to a client and includes:

[0010] Receiving business query information submitted for a target business, and determining a query field corresponding to the business query information;

[0011] Determine a business sample document according to the business query information, and display the business sample document through a field selection page;

[0012] In response to a selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document;

[0013] The information extraction task associated with the target business is performed based on the selected field and the query field.

[0014] According to a third aspect of an embodiment of this specification, another information extraction method is provided, including:

[0015] Determine a selected field associated with a query field in the business query information;

[0016] Recalling a candidate document paragraph set in a business document library according to the query field;

[0017] Inputting the query field and the selected field into a large language model, and recalling a target document paragraph from the candidate document paragraph set through the large language model;

[0018] The target information associated with the business query information is extracted from the target document paragraph by using the large language model and referring to the selected field.

[0019] According to a fourth aspect of the embodiments of this specification, there is provided an information extraction device, including:

[0020] A selection module is configured to display a field selection page containing a business sample document according to the business query information, and determine a query field corresponding to the business query information;

[0021] A determination module is configured to display a field selection page containing a business sample document according to the business query information, and determine a query field corresponding to the business query information;

[0022] An input module, configured to recall a candidate document paragraph set in a business document library according to the query field, and input the query field and the selected field into a large language model;

[0023] The extraction module is configured to recall the target document paragraph from the candidate document paragraph set by using the large language model, and extract the target information associated with the business query information from the target document paragraph by using the large language model and referring to the selected field.

[0024] According to a fifth aspect of an embodiment of this specification, a field selection device is provided, which is applied to a client and includes:

[0025] An information receiving module is configured to receive business query information submitted for a target business and determine a query field corresponding to the business query information;

[0026] A document determination module is configured to determine a business sample document according to the business query information, and display the business sample document through a field selection page;

[0027] A field selection module is configured to determine, in the business sample document, a selection field associated with the query field in response to a selection instruction submitted through the field selection page;

[0028] The task execution module is configured to execute the information extraction task associated with the target business based on the selected field and the query field.

[0029] According to a sixth aspect of the embodiments of this specification, another information extraction device is provided, including:

[0030] A field determination module is configured to determine a selected field associated with a query field in the business query information;

[0031] A document recall module, configured to recall a candidate document paragraph set in a business document library according to the query field;

[0032] An input model module is configured to input the query field and the selected field into a large language model, and to recall a target document paragraph from the candidate document paragraph set through the large language model;

[0033] The information extraction module is configured to use the large language model to extract the target information associated with the business query information in the target document paragraph with reference to the selected field.

[0034] According to a seventh aspect of an embodiment of this specification, there is provided a computing device, including:

[0035] Memory and processor;

[0036] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned information extraction method or field selection method are implemented.

[0037] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned information extraction method or field selection method are implemented.

[0038] According to the ninth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which implement the steps of the above-mentioned information extraction method or field selection method when executed by a processor.

[0039] The information extraction method provided in this embodiment can display a field selection page containing a business sample document to the user in order to ensure the representativeness of the information extraction results, improve the efficiency of information extraction and balance the weights of various key indicators. At the same time, the query field corresponding to the business query information submitted by the user can be determined; on this basis, the selected field with an associated relationship with the query field can be determined in the business sample document in response to the selection instruction submitted through the field selection page; the user can customize the associated relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. Therefore, the candidate document paragraph set can be recalled in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph (multiple) associated with the business query information can be obtained in depth. Finally, the target information associated with the business query information can be extracted from the target document paragraph by referring to the selected field using the large language model. When extracting information, it can not only be completed based on business query information, but also the target document paragraphs with different field representations but the same meaning can be obtained by selecting the association between the field and the query field, so that the target information can be extracted quickly and accurately for downstream business analysis and processing, effectively improving the efficiency and accuracy of information extraction, and avoiding the impact of unbalanced information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of an information extraction method provided by an embodiment of this specification;

[0041] Figure 2 is a flow chart of an information extraction method provided by an embodiment of this specification;

[0042] Figure 3It is a schematic diagram of a field selection page in an information extraction method provided by an embodiment of this specification;

[0043] Figure 4 is a flow chart of another information extraction method provided by an embodiment of this specification;

[0044] Figure 5 is a processing flow chart of an information extraction method provided by an embodiment of this specification;

[0045] Figure 6 is a flow chart of a field selection method provided by an embodiment of this specification;

[0046] Figure 7 is a structural schematic diagram of an information extraction device provided by an embodiment of this specification;

[0047] Figure 8 is a schematic diagram of the structure of another information extraction device provided by an embodiment of this specification;

[0048] Fig. 9 It is a structural schematic diagram of a field selection device provided by an embodiment of this specification;

[0049] Fig.10 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0050] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0051] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0053] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0054] In this specification, an information extraction method and a field selection method are provided. This specification also relates to an information extraction device, a field selection device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0055] See also Figure 1As shown in the schematic diagram, the information extraction method provided in this embodiment can display a field selection page containing a business sample document to the user in order to ensure the representativeness of the information extraction result, improve the efficiency of information extraction and balance the weight of each key indicator. At the same time, the query field corresponding to the business query information submitted by the user can be determined; on this basis, the selected field with an associated relationship with the query field can be determined in the business sample document in response to the selection instruction submitted through the field selection page; the user can customize the association relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. It is also possible to recall a set of candidate document paragraphs in the business document library according to the query field to preliminarily obtain document paragraphs associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph (multiple) associated with the business query information can be obtained in depth. Finally, the large language model can be used to refer to the selected field to extract the target information associated with the business query information in the target document paragraph. When extracting information, it can not only be completed based on business query information, but also the target document paragraphs with different field representations but the same meaning can be obtained by selecting the association between the field and the query field, so that the target information can be extracted quickly and accurately for downstream business analysis and processing, effectively improving the efficiency and accuracy of information extraction, and avoiding the impact of unbalanced information extraction.

[0056] See also Figure 2 , Figure 2 A flowchart of an information extraction method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0057] Step S202: displaying a field selection page containing business sample documents according to the business query information, and determining a query field corresponding to the business query information.

[0058] The information extraction method provided in this embodiment can be applied to any scenario, such as intelligent document scenarios (information extraction can extract key information from text to quickly and accurately answer user questions), data mining and analysis scenarios (information extraction can be used to extract valuable information from text data such as social media, news reports, customer feedback, etc., to conduct market trend analysis, brand reputation monitoring, customer behavior prediction, etc.), public opinion monitoring scenarios (information extraction can be used to monitor public opinions and emotional changes on channels such as social media and news reports in real time to provide timely and accurate information support for decision-making), financial field scenarios (information extraction can be used to extract key financial information from texts such as news reports, announcements, and financial statements, such as stock price changes, company performance, policy changes, etc., to provide decision support for investors), medical and health scenarios (information extraction can extract disease information, drug information, treatment methods, etc. from texts such as medical literature and medical records to provide doctors with auxiliary diagnosis, treatment plan formulation, etc.), cross-language information retrieval scenarios (helping users find relevant information in texts in different languages), etc.

[0059] In addition to the above-mentioned scenarios directly used for information extraction, the information extraction method provided in this embodiment can also be applied to information entry scenarios. For example, in the information entry scenario of financial documents, the information extraction method provided in this embodiment can extract information such as "issuer name" and "main provider" from the "prospectus" on demand, and enter it into the financial system to facilitate business use. On the other hand, it can also be applied to compliance review scenarios, such as compliance review of financial documents. The information extraction method provided in this embodiment can extract information on demand from informal documents (key words in the document are not standardized enough), and then conduct compliance review on the extracted information, thereby avoiding the impact of document errors and improving the efficiency of compliance review.

[0060] This embodiment illustrates the information extraction method by taking its application in a financial scenario as an example. Descriptions in other scenarios may refer to the same or corresponding descriptions in this embodiment, and this embodiment will not be elaborated in detail here.

[0061] Specifically, the business query information specifically refers to the information query text submitted by the user when the user needs to query the information related to the target business. For example, if the user needs to query the breach of contract compensation information in the contract document, the user can submit the business query information "how much liquidated damages does Party A need to pay for breach of contract", and then the relevant information can be extracted from the contract document signed by Party A and Party B according to the business query information, so as to facilitate the user's use. In specific implementation, the content involved in the business query information can be edited and input according to actual needs, and this embodiment does not make any limitation here.

[0062] Correspondingly, the business sample document specifically refers to a sample document of the type of associated business query information, which records the text description information of the field related to the business query information. When the user enters the business query information, the associated sample document can be displayed to the user, so that the user can select the field in the sample document and establish an association relationship with the query field, so as to recall the target document based on this. It can be understood that the business sample document displays the text content that matches the text description of the current field to the user, so that the user can determine the relevance of the content of his own expression with the text description in the field by reading and selecting the business sample document, and then establish an association relationship between multiple fields with the same meaning but different expressions, so as to facilitate the subsequent text recall. For example, the business sample document is a standard lease contract, which records information such as Party A, Party B, signing matters, breach of contract clauses, precautions, etc., and the document records text fields with different meanings in the lease contract scenario. For example, the subject matter of the lease refers to the leased items (such as cars, houses, computers, mobile phones, etc.). Users can select the fields involved in the business sample document to establish an association between the selected fields and the query fields. Based on this association, it can support the subsequent recall of documents to ensure that all related documents will be recalled, thereby ensuring the accuracy of information extraction. It should be noted that the business sample document is a pre-set sample, and different business types correspond to different documents, which is convenient for use in the application stage.

[0063] Correspondingly, the field selection page specifically refers to a page that displays business sample documents and supports users to select fields. The sample documents are displayed on this page, and an editing area for selecting fields is provided. Fields with inconsistent expressions but the same meaning are associated with each other. For example, if T+1 text appears in a document, it can be associated with the redemption period by selecting it, so that the document can be recalled in the subsequent use stage.

[0064] Correspondingly, the query field specifically refers to the key field extracted from the business query information, or the standard field associated with the key field, which is used to normalize the flexible expression key field in the business query information into a standard field, thereby ensuring that the roughly recalled document paragraphs are more accurate. For example, if the business query information is "What is the net profit of Company H", the key field "net profit" can be extracted from the business query information and used as the query field. The key field "net profit" can also be edited as "net profit" by the user and used as the query field. In specific implementation, after obtaining the business query information, the query field determined according to the business query information can be one or more, so that the relevant document paragraph recall operation can be performed based on the determined one or more query fields.

[0065] Based on this, in order to ensure the representativeness of the information extraction results, improve the efficiency of information extraction and balance the weights of various key indicators, the field selection page containing the business sample document can be displayed to the user, and the query field corresponding to the business query information submitted by the user can be determined; on this basis, the selected field with an associated relationship with the query field can be determined in the business sample document in response to the selection instruction submitted through the field selection page; the user can customize the associated relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. Therefore, the candidate document paragraph set can be recalled in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph associated with the business query information can be obtained in depth. Finally, the large language model can be used to extract the target information associated with the business query information in the target document paragraph according to the selected field.

[0066] Furthermore, considering that different information extraction requirements require different information contents to be extracted, in order to support the extraction operation that matches the user's expectations, a business sample document can be provided before information extraction, so that the user can provide more field selections by selecting fields, thereby covering richer documents. In order to support this implementation, a sample document matching the current information extraction requirement type can be provided to the user. In this embodiment, the specific implementation method is as follows:

[0067] Obtain business query information and determine the information query type corresponding to the business query information; extract business sample documents from the sample document library according to the information query type, and load the business sample documents to the field selection page for display; wherein the field selection page includes a document display area and a selection custom area, the document display area is used to display the business sample documents, and the selection custom area is used to establish an association relationship between the query field and the selection field.

[0068] Specifically, the information query type refers to the field of information association that determines the business query information. It is used to select sample documents corresponding to the field to establish associations between relevant fields before information extraction, thereby ensuring that the information extraction operation is completed under the premise of clear business meaning; for example, if the information query type corresponding to the business query information is a contract type, then a standard contract document can be selected as a business sample document, or if the information query type is a financial transaction type, then a financial-related document can be selected as a business sample document.

[0069] Correspondingly, the sample document library specifically refers to a database storing sample documents of different types. Correspondingly, the document display area specifically refers to an area displaying document content, and the business sample documents determined based on the query field will be displayed in the document display area; the selected custom area specifically refers to an area used to edit fields, and is used to establish an association relationship between the selected field and the query field.

[0070] Based on this, considering that the fields in different fields in actual application scenarios have many business attributes, each field may contain deep business meanings in addition to the same literal expression; under this premise, in order to be able to recall documents with stronger relevance for information extraction based on business query information when the field representation is different, the information query type corresponding to the business query information can be determined; at this time, business sample documents can be extracted from the sample document library according to the information query type, so that users can subsequently select fields for business sample documents, thereby improving the richness of the fields used in the recalled documents. By loading business sample documents to the field selection page for display, users can flexibly select fields on demand; by selecting in the business sample documents displayed in the document display area, the corresponding selected fields can be displayed in the selected custom area according to the selection behavior, and the association relationship between the query field and the selected field can be established, so that the document paragraph recall and information extraction can be completed based on the association relationship in the future.

[0071] In actual applications, the field selection function provided on the field selection page can be understood as supporting users to unify the meaning of fields. For example, "professional vocabulary A" is a vocabulary defined internally by the customer, and may be expressed as 100,000 yuan, 100,000 yuan, 100,000 yuan, or 100,000 yuan in financial statement documents. If the "professional vocabulary A" field exists in the business query information submitted by the user, without field selection, the conventional model has no knowledge related to vocabulary A, resulting in ignoring document paragraphs recording 100,000 yuan, 100,000 yuan, 100,000 yuan, or 100,000 yuan, thereby affecting the accuracy of information extraction. Therefore, the field selection function can support users to establish associations between fields with deep business meanings, help the large language model understand, thereby improving the coverage of recalled document paragraphs and improving recall accuracy.

[0072] In addition, the field selection function can help the system accurately output the result range. For example, if the output range of "net profit" is not standardized, the result may be "Company B's net profit is 100,000 yuan", or "net profit is 100,000 yuan", or "100,000", resulting in the inability to normalize the output. With the field selection function, the system can refer to the selected fields to output the expected results, thereby meeting the needs of downstream business.

[0073] For example, a user needs to use the annual net profit information of H Company, but it takes a long time to collect statistics for this information. The reports from January to December record the net profit information of each month. Therefore, you only need to filter out the monthly net profit information of H Company from a large number of report documents to calculate the annual net profit information of H Company. At this time, the business query information "Monthly net profit of H Company" submitted by the user is received. In order to filter out the report document that records the net profit of H Company from a large number of report documents, you can first determine the sample document associated with the current scenario. The sample document is a sample document that records financial information. By loading the sample document into the page, you can display it as follows: Figure 3 The field selection page shown in (a) in the figure. The business sample document is displayed in the left area of ​​the page, and the field selection function is displayed in the right area of ​​the page. The user can enter the specified field for field 1, field 2...field n, and then select the field in the document to add the selected field to the value column, and then establish an association between the specified field and the selected field, so as to facilitate subsequent document recall and information extraction. At the same time, the key field "monthly net profit" in the business query information "monthly net profit of H enterprise" can also be determined. After standardization, the query field "monthly income" will be obtained for subsequent use.

[0074] In summary, by displaying the document display area and selecting the custom area on the page, users can easily establish associations between fields with different expressions but the same meaning before information extraction, thereby avoiding missing key documents during the document recall stage and ensuring the accuracy of information extraction.

[0075] Step S204: In response to the selection instruction submitted through the field selection page, a selection field associated with the query field is determined in the business sample document.

[0076] Specifically, after the field selection page containing the business sample document is displayed as above, the user can be supported to select the fields contained in the business sample document in the field selection page. The selection instruction can be collected based on the user's selection behavior for the fields in the business sample document. The logic for determining the selected field can be driven based on the selection instruction. The logic is executed to determine the selected field that has an association with the query field in the business sample document, and then the association relationship between the selected field and the query field is established, so that when the document paragraph is recalled in the large language model, the document paragraph recall can be completed by understanding the semantic relationship between the fields.

[0077] The selection instruction specifically refers to the instruction triggered when the user selects the field to be selected in the business sample document. The triggering method of the instruction can be a field box selection method, such as Figure 3In the diagram (b), the user can trigger the selection command by sliding the cursor in the business sample document, and then select the field "T+5 days". At this time, in order to facilitate the user to determine the content of the selected field, the selected field can be highlighted with a set color. Correspondingly, the selected field specifically refers to the field value selected by the user in the sample document, such as Figure 3 In the diagram shown in (b), the field selected by the selection instruction triggered at this time is "T+5 days"; it can be understood that the selected field is the value selected by the user for the query field in the business sample document, so that when the document is recalled, the document paragraphs containing the same meaning as the query field but not described according to the query field can be recalled. For example, the information that the user needs to query is related to the "redemption period", and the document contains a field expressed as "T+1". The user can select the field expressed as "T+1" and associate it with the "redemption period"; or the information that the user needs to query is related to the "lease period", and the document contains "lease time from January to December", the user can select "lease time from January to December" and associate it with the "lease period", and then use it when recalling document paragraphs later.

[0078] Furthermore, considering that when a user selects a field in a sample document, the selection result may be affected by an error operation, a field library may be provided to complete the incompletely selected fields. In this embodiment, the specific implementation is as follows:

[0079] Receive a selection instruction submitted through the field selection page; determine an initial selection field in the business sample document according to the selection instruction, and perform an integrity check on the initial selection field; if the check fails, extract business fields from a preset field library according to the initial selection field; and use the business field as a selection field that has an associated relationship with the query field.

[0080] Specifically, the initially selected field specifically refers to the field determined in the business sample document according to the user's selection instruction, wherein the selection method of the initially selected field can refer to the description of the selected field in the above embodiment, and this embodiment will not be repeated here. Correspondingly, the integrity check specifically refers to detecting whether the initially selected field is a full name, or whether a field is missing. Correspondingly, the business field specifically refers to the field of the complete expression stored in the field library.

[0081] Based on this, considering that when users select fields, they may only select part of the field content of the target field due to misoperation or unfamiliarity with the meaning of the field, in order to avoid this selection behavior affecting the recall accuracy, an integrity check can be performed. That is to say, after receiving the selection instruction submitted through the field selection page, the initial selected field can be determined in the business sample document according to the selection instruction. At this time, an integrity check can be performed on the initial selected field; if the test fails, it means that the initial selected field is incomplete and cannot be used directly. Therefore, the business field can be extracted from the preset field library according to the initial selected field; thereby, the business field can be used as a selected field with an associated relationship with the query field for subsequent use.

[0082] In actual applications, when extracting business fields from the field library, you can use the method of calculating field similarity to complete it, and select the business field with the highest field similarity as the selected field for subsequent use.

[0083] For example, the user needs to select the "H Co., Ltd." field in the sample document, but due to an error, selects "H Shares", or "H Shares Limited", or "H Shares Have". At this time, based on the field selected by the user, the field library can search for a business field that matches the currently selected field to obtain "H Co., Ltd.". Therefore, the field content can be automatically completed in the selected custom area for subsequent use.

[0084] In summary, by performing integrity check on the selected fields, it can be ensured that the final selected fields are more consistent with the actual expression, which can effectively improve the recall accuracy of document paragraphs and thus ensure the accuracy of information extraction.

[0085] Furthermore, in order to ensure that the target information extracted subsequently is more accurate, in addition to selecting the fields that match the business query information, auxiliary selection fields can also be selected, thereby supporting the large language model to mine the field relationship in the document through semantic analysis to improve the accuracy of information extraction. In this embodiment, the specific implementation method is as follows:

[0086] In response to a selection instruction submitted through the field selection page, an auxiliary selection subfield having a collaborative relationship with the query field and a base selection subfield having a field association relationship with the query field are determined in the business sample document; the auxiliary selection subfield and the base selection subfield are used as selection fields having an association relationship with the query field.

[0087] Specifically, the auxiliary selected subfield refers to the selected subfield that has a collaborative relationship with the query field. The selected subfield does not belong to the business query information, but can be used as an associated field to improve the recall accuracy of subsequent document paragraphs. It can be understood that the auxiliary selected subfield is indirectly associated with the query field. For example, if the business query information is "the monthly net profit of H enterprise", in addition to selecting the field associated with the business query information, you can also select "gross profit 100,000 yuan", "total sales volume 1,000 pieces", "cost 50,000 yuan" and other fields as auxiliary selected fields. Correspondingly, the base selected subfield refers to the selected subfield that has a field association relationship with the query field, which can be understood as the base selected subfield having a direct relationship with the query field.

[0088] Based on this, in order to improve the accuracy of the subsequent recall of document paragraphs, so as to accurately extract the target information of the associated business query information, in response to the selection instruction submitted through the field selection page, the auxiliary selection subfield having a collaborative relationship with the query field can be determined in the business sample document. Through the auxiliary selection subfield, the document paragraph having an indirect association with the business query information can be recalled in the document paragraph recall stage to ensure the accuracy of subsequent information extraction. At the same time, the base selection subfield having a field association relationship with the query field can also be determined. The base selection subfield has a direct association relationship with the query field, so it can be ensured that the directly associated document paragraph will not be omitted. Therefore, the auxiliary selection subfield and the base selection subfield can be used as the selection field having an association relationship with the query field, so as to facilitate the subsequent document paragraph recall operation.

[0089] Using the above example, when showing the user Figure 3 After the field selection page shown in (a) is displayed, users can select fields. For example, in the sample document, it is recorded that {the company's net income last year was 10 million yuan, and this year's income increased by 50% compared to last year, reaching 15 million yuan, and the sales volume of product A also exceeded expectations, reaching 20,000 units...}, users can associate fields such as "10 million yuan" and "15 million yuan" with the field "net profit" by selecting. In addition, users can select the "20,000 units" field in the document to establish a relationship with the "sales volume" field and use it as an auxiliary selection field; it should be noted that during the field selection stage, users can select fields such as Figure 3 As shown in (a), field 1, field 2, ... field n are respectively set with field names, i.e., fields associated with business query information. Then, any field can be selected. After the selection is completed, the corresponding selected field can be added to the value column corresponding to the field, such as Figure 3In the diagram shown in (b), the field name of field 1 is "net profit", and the corresponding value is "10 million", the field name of field 2 is "sales volume", and the corresponding value is "20,000 pieces", the field name of field 3 is "net profit", and the corresponding value is "15 million yuan"... The field name of field n is "T+5 days", and the corresponding value is "the number of days for redemption funds to arrive". The above fields can be used to perform document recall operations later to obtain documents that record monthly profit information.

[0090] In summary, by constructing the selection field by combining the base selection field and the auxiliary selection field, it can be ensured that the recall accuracy can be improved during the subsequent document paragraph recall, and redundant documents can be screened out, thereby improving the efficiency and accuracy of information extraction.

[0091] Step S206: Recall a candidate document paragraph set in the business document library according to the query field, and input the query field and the selected field into a large language model.

[0092] Specifically, after obtaining the query field and the selected field as described above, the document paragraph recall operation can be performed based on the query field and the selected field. Considering that the number of document paragraphs that need to be recalled is large, direct recall will occupy more computing resources and take more time. Therefore, the candidate document paragraph set can be recalled in the business document library based on the query field to complete the preliminary document paragraph recall operation. In order to improve the accuracy and efficiency of document paragraph recall and improve the accuracy of information extraction, a large language model can be selected for precise document paragraph recall and information extraction. By inputting the query field and the selected field into the large language model, document paragraph recall and information extraction can be completed quickly and efficiently.

[0093] Among them, the business document library specifically refers to a database that records the documents needed to be used in the current business scenario, and the documents contained in the document library are all provided by the business party to which the business belongs. Correspondingly, the candidate recall document paragraph set specifically refers to a set consisting of multiple candidate recall document paragraphs, and each candidate recall document paragraph is associated with a query field. When recalling candidate document paragraphs, each business document in the business document library can be segmented to obtain at least one document paragraph corresponding to each business document. After that, the query field is used as the basis for each business document. The document paragraphs that have a relationship with the query document can be screened out, and then used as candidate document paragraphs and constitute a candidate document paragraph set for subsequent fine recall processing. In addition, it should be noted that since the large language model can have the ability to recall document paragraphs by inputting the prompt words of the associated document recall operation, in order to support the document paragraph recall operation of the large language model, the recall interface set by the large language model can be used to connect to the candidate document paragraph set, and support the fine recall stage after the rough recall. The candidate document paragraph set can be accessed through this interface, thereby supporting the large model to complete the fine recall operation according to the input information.

[0094] Furthermore, when recalling candidate document paragraphs according to the query field, in order to ensure that the initially recalled document paragraphs are more accurate, the association between the document paragraphs and the fields can be determined by calculating the recall score, thereby ensuring the accuracy of the document recall. In this embodiment, the specific implementation method is as follows:

[0095] Determine the initial candidate document paragraphs contained in the candidate documents in the business document library; use a preset recall algorithm to calculate the document recall score between the query field and the initial candidate document paragraphs contained in each candidate document; recall a set number of candidate document paragraphs in the business document library according to the document recall score, and create a candidate document paragraph set based on the set number of candidate document paragraphs.

[0096] Specifically, the initial candidate document paragraph specifically refers to the document paragraph determined after paragraph division of any candidate document in the business document library. Correspondingly, the preset recall algorithm specifically refers to the algorithm for sparse recall calculation, such as the BM25 algorithm can be used to achieve sparse recall. Correspondingly, the document recall score specifically refers to the score of the word frequency correlation between the query field and each initial candidate document paragraph, and the higher the score, the higher the correlation. The set number refers to the number of initial candidate document paragraphs with the TOP K scores selected, which can be set according to actual needs, and this embodiment does not make any restrictions here.

[0097] Based on this, in order to be able to quickly recall complete document paragraphs in the rough recall stage, the initial candidate document paragraphs contained in the candidate documents in the business document library can be determined first; then, the initial candidate document paragraphs are used as the minimum recall unit to calculate the document recall score between the query field and the initial candidate document paragraphs contained in each candidate document using a preset recall algorithm; the document recall score reflects the correlation between each initial candidate document paragraph and the query field, so as to determine the document paragraph with stronger correlation by sorting the scores. At this point, a set number of candidate document paragraphs can be recalled in the business document library according to the document recall score, and a candidate document paragraph set can be created based on the set number of candidate document paragraphs, so that the semantic analysis capability of the large language model can be used to complete the fine recall processing later.

[0098] In practical applications, the BM25 algorithm is used for coarse recall, which can better capture the word frequency correlation between document paragraphs and query fields, thereby effectively improving the recall efficiency. In addition, other algorithms can also be used to complete coarse recall, and this embodiment does not make any limitation here.

[0099] Continuing with the above example, after obtaining the query field "monthly income" and selecting the fields "10 million" and "15 million yuan", a coarse recall can be performed in the database storing reports based on the query field "monthly income" according to the BM25 algorithm. According to the recall results, 20 document paragraphs recording net profit information will be obtained. After that, fine recall can be completed based on the 20 document paragraphs in combination with the large language model, thereby ensuring the accuracy of information extraction.

[0100] In summary, by adopting the coarse recall method to recall document paragraphs, the number of document paragraphs that need to be processed by the fine recall operation can be effectively reduced, thereby effectively improving the efficiency of information extraction.

[0101] In addition, considering that when users submit business query information, the expression of the business query information may not be standardized due to different expressions, if information extraction is performed based on this, it may affect the accuracy of information extraction. Therefore, the business query information can be updated by entity extraction detection. In this embodiment, the specific implementation method is as follows:

[0102] Extract query entities from the business query information, and detect whether the query entity meets the information extraction condition; if not, extract business entities from the business sample document according to the query entity; use the business entity to update the business query information to obtain target business query information; use the target business query information as the business query information, and execute the step of recalling a set of candidate document paragraphs in a business document library according to the query field.

[0103] Specifically, the query entity refers to the entity in the business query information. For example, the business query information is "the net profit of H Company". By performing entity extraction on the business query information, the entity "H Company" can be obtained. Correspondingly, the information extraction condition refers to the condition for detecting whether the query entity in the business query information is a standard entity name. For example, H Company can also be expressed as H Co., Ltd., H Enterprise, etc. By detecting the information extraction condition of the entity, it can be ensured that the document paragraph recall operation is performed using the standard entity. Correspondingly, the business entity refers to the standard entity associated with the query entity in the business sample document.

[0104] Based on this, considering that different users may express the same entity in different ways, which may affect the accuracy of information extraction, the query entity can be extracted from the business query information, and the query entity can be detected to see if it meets the information extraction conditions; if so, subsequent operations can be performed. If not, it means that the query entity naming is not standardized, so the business entity can be extracted from the business sample document according to the query entity; then the business query information can be updated using the business entity to obtain the target business query information according to the update result; thereafter, the target business query information is used as the business query information, and the step of recalling the candidate document paragraph set in the business document library according to the query field is performed.

[0105] Continuing with the above example, after the user enters the business query information of "Monthly Net Profit of H Enterprise", the query entity "H Enterprise" can be extracted. Through detection, it is determined that the entity is a non-standard entity, so it can be changed. Through detection, it is determined that the standard entity corresponding to "H Enterprise" is named "H Co., Ltd.", and then the entity can be used to update the business problem information, and then the query information of "Monthly Net Profit of H Co., Ltd." can be obtained, and then information extraction operations can be performed on this basis.

[0106] In summary, by updating business query information in an entity normalization manner, it is possible to avoid affecting the accuracy of field extraction due to incomplete entities in the business query information when extracting query fields, and the accuracy of information extraction can be effectively improved by updating entities.

[0107] Step S208 , recalling a target document paragraph from the candidate document paragraph set by using the large language model, and extracting target information associated with the business query information from the target document paragraph by using the large language model and referring to the selected field.

[0108] Specifically, after the query field and the selected field are input into the large language model, in order to accurately extract information that meets the business query information usage requirements, the target document paragraph can be first recalled from the candidate document paragraph set through the large language model, and then the target information associated with the business query information can be extracted from the target document paragraph by referring to the selected field using the large language model, so as to facilitate the subsequent use of the target information for corresponding business processing and analysis. Among them, the target document paragraph specifically refers to the document paragraph obtained from the candidate document paragraph set through the large language model for precise recall processing, and correspondingly, the target information specifically refers to the field value extracted from the target document paragraph that matches the business query information.

[0109] It should be noted that since the large language model is a deep learning model trained using a large amount of text data, it can generate natural language text or understand the meaning of language text. Therefore, in order to enable the large language model to be used for document paragraph recall and information extraction, the recall prompt words associated with the document paragraph recall task can be input when the large language model performs document paragraph recall, so that the large language model determines that the document paragraph recall task needs to be performed at present, and then the target document paragraph is recalled from the candidate document paragraph set. When the large language model performs information extraction, the extraction prompt words associated with the information extraction task can be input, so that the large language model determines that the information extraction task needs to be performed at present, and then the target information of the associated business query information is extracted from the target document paragraph for use by downstream businesses. Among them, the recall prompt words and extraction prompt words can be set according to actual needs.

[0110] Furthermore, when recalling the target document paragraph from the candidate document paragraph set, in order to realize that the query field and the selected field can simultaneously assist in recalling the document paragraph, a large language model can be used to mine the relationship between the fields based on semantic analysis, thereby ensuring the recall accuracy of the target document paragraph. In this embodiment, the specific implementation method is as follows:

[0111] The semantic relationship between the query field and the selected field and each candidate document paragraph in the candidate document paragraph set is constructed through the large language model; the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set is calculated according to the semantic relationship; and the target document paragraph is recalled in the candidate document paragraph set according to the semantic similarity.

[0112] Specifically, the semantic relationship refers to the semantic association between the query field and the selected field and each candidate document paragraph. This semantic relationship can reflect the intrinsic connection between the field and the document paragraph, thereby facilitating the large language model to complete the precise recall operation according to the intrinsic connection. Correspondingly, the semantic similarity is the score that represents the semantic relevance between each candidate document paragraph and the query field and the selected field. The higher the similarity, the higher the relevance.

[0113] Based on this, after the query field and the selected field are input into the large language model, the selected field can be used as auxiliary information to improve the large language model when recalling document paragraphs according to the query field, so that when the large language model performs precise document recall, it can mine the semantic relationship between the query field and the selected field and each candidate document paragraph in the candidate document paragraph set; then the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set is calculated according to the semantic relationship; the semantic similarity is used to reflect the strength of the intrinsic correlation between each candidate document paragraph and the field, so that the target document paragraph can be recalled in the candidate document paragraph set according to the semantic similarity, so as to perform subsequent information extraction operations.

[0114] In practical applications, the powerful semantic understanding ability of the large language model can be used to determine the correlation between each document paragraph and the query field and the selected field. In order to ensure the recall accuracy, the selected field can be used as a prompt word for the large language model to recall the document paragraph, helping the large language model to understand the meaning of the field, thereby ensuring the recall accuracy of the target document paragraph.

[0115] In summary, by recalling the target document paragraphs through semantic analysis, a high match between the target document paragraphs and the business query information can be ensured, thereby ensuring the accuracy of subsequent information extraction.

[0116] Furthermore, when extracting target information, in order to make the expression of the target information more uniform, it can be updated according to the information structure of the selected field. In this embodiment, the specific implementation is as follows:

[0117] The information structure corresponding to the selected field is determined by the large language model, and the initial information associated with the business query information is extracted from the target document paragraph; the initial information is updated based on the information structure to obtain the target information associated with the business query information.

[0118] Specifically, the information structure specifically refers to the field composition structure corresponding to the selected field, which is used to simplify the initial information. Correspondingly, the initial information specifically refers to the information extracted from the target document paragraph that has not been simplified. By updating its structure, the structure of the target information can be guaranteed to be more unified and standardized.

[0119] Based on this, when extracting target information, in order to ensure the comprehensiveness and accuracy of the extracted information, the information structure corresponding to the selected field can be determined through a large language model, as well as the initial information associated with the business query information in the target document paragraph; then the initial information can be updated based on the information structure, so that the target information associated with the business query information can be obtained for subsequent use.

[0120] Following the above example, after obtaining 20 document paragraphs, the query field "monthly income" and the selected fields "10 million", "15 million yuan" and "20,000 pieces" can be input into the large language model, and 12 document paragraphs can be screened out from the 20 document paragraphs through the semantic analysis capability of the large language model. At this time, the 12 screened document paragraphs can be used as target document paragraphs. Further, information can be extracted from the 12 document paragraphs according to "monthly net profit of H Co., Ltd." to obtain 12 initial information, namely: net profit *** in January, company this month..., net income *** in February, net profit *** in March, first quarter..., April... On this basis, considering that the extracted information is not uniformly expressed and is not convenient for subsequent use, each information can be standardized with reference to the structure of the selected field, and then the following target information can be obtained, namely, net profit 50,000 in January; net profit 30,000 in February; net profit 40,000 in March...; so that users can use the annual profit by accumulating the net profit from January to December.

[0121] The information extraction method provided in this embodiment can display a field selection page containing a business sample document to the user in order to ensure the representativeness of the information extraction results, improve the efficiency of information extraction and balance the weights of various key indicators. At the same time, the query field corresponding to the business query information submitted by the user can be determined; on this basis, the selected field with an associated relationship with the query field can be determined in the business sample document in response to the selection instruction submitted through the field selection page; the user can customize the associated relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. Therefore, the candidate document paragraph set can be recalled in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph (multiple) associated with the business query information can be obtained in depth. Finally, the target information associated with the business query information can be extracted from the target document paragraph by referring to the selected field using the large language model. When extracting information, it can not only be completed based on business query information, but also the target document paragraphs with different field representations but the same meaning can be obtained by selecting the association between the field and the query field, so that the target information can be extracted quickly and accurately for downstream business analysis and processing, effectively improving the efficiency and accuracy of information extraction, and avoiding the impact of unbalanced information extraction.

[0122] See also Figure 4 , Figure 4 A flowchart of another information extraction method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0123] Step S402: determine a selected field associated with a query field in the business query information.

[0124] Step S404: Retrieve a candidate document paragraph set from the business document library according to the query field.

[0125] Step S406: input the query field and the selected field into a large language model, and use the large language model to recall target document paragraphs from the candidate document paragraph set.

[0126] Step S408: using the large language model and referring to the selected field, extracting target information associated with the business query information from the target document paragraph.

[0127] The information extraction method provided in this embodiment, in order to ensure the representativeness of the information extraction results, improve the information extraction efficiency and balance the weights of each key indicator, can determine the selected field associated with the query field in the business sample document, and then recall the candidate document paragraph set in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model, so as to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then deeply obtain the document paragraph (multiple) associated with the business query information. Finally, the target information associated with the business query information can be extracted from the target document paragraph by referring to the selected field using the large language model. When performing information extraction, it can not only be completed based on the business query information, but also the target document paragraph with different field representation but the same meaning can be obtained through the association between the selected field and the query field, so that the target information can be accurately extracted for downstream business analysis and processing, which effectively improves the efficiency and accuracy of information extraction, and can avoid the impact of unbalanced information extraction.

[0128] The following combination Figure 5 , taking the application of the information extraction method provided in this specification in the information extraction and entry scenario as an example, the information extraction method is further described. Among them, Figure 5 A processing flow chart of an information extraction method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0129] Step S502: Acquire business query information and determine the information query type corresponding to the business query information.

[0130] Step S504: extract business sample documents from the sample document library according to the information query type, and load the business sample documents to the field selection page for display.

[0131] The field selection page includes a document display area and a selection custom area. The document display area is used to display business sample documents, and the selection custom area is used to establish an association relationship between the query field and the selected field.

[0132] For example, a user inputs the business query information "How much money did Company H make this year?" In order to accurately extract the company's net profit value from the company's 10 documents for entry into the system, you can first determine the business sample document associated with the business query information. At this time, you can select "Prospectus" as the business sample document and display it in the document display area on the field selection page. At the same time, the selected custom area for establishing the association between the selected field and the query field is displayed on the field selection page.

[0133] Step S506: determine the query field corresponding to the business query information.

[0134] Step S508: receiving a selection instruction submitted through the field selection page.

[0135] Step S510: determining an initial selected field in the business sample document according to the selection instruction, and performing integrity check on the initial selected field.

[0136] Step S512: If the detection fails, extract the business field from the preset field library according to the initially selected field.

[0137] Step S514: The business field is used as a selected field associated with the query field.

[0138] Following the above example, after obtaining the business query information "How much money did H Company earn this year", considering that the business query information is not accurate enough in expressing its meaning, it may lead to omissions in the subsequent recall documents. Therefore, the key field "How much money did H Company earn this year" can be extracted from the business query information. At this time, it can be standardized to obtain the query field "Company Net Profit" for subsequent use. Further, after the field selection page is displayed to the user, the user can select the field to be selected in the business sample document. According to the user's selection instruction, the field selected by the user will be displayed in the selection custom area. For example, if the user selects "H Limited" and "10 million yuan", the integrity of the field selected by the user can be checked. If it is detected that "H Limited" is incomplete, similarity calculation can be performed based on the selected field "H Limited" and the fields stored in the field library. According to the similarity calculation result, the field with the highest similarity is selected as the business field corresponding to "H Limited". At this time, the business field obtained is "H Co., Ltd.", which can be used as the selected field for subsequent document recall.

[0139] Step S516: Select a set number of candidate document paragraphs in the business document library according to the query field using the BM25 algorithm, and create a candidate document paragraph set based on the set number of candidate document paragraphs.

[0140] Using the above example, after obtaining the selected field and the query field, you can first perform a rough recall in 10 documents based on the query field according to the BM25 algorithm, so as to find the document paragraph containing the query field "company net profit" in the 10 documents and then perform subsequent information extraction operations. In the rough recall stage, you need to first split the document into one or more document paragraphs, and then use the BM25 algorithm to calculate the literal similarity between the query field and each document paragraph. After that, by sorting the literal similarity, you can select the top 3 document paragraphs with the highest similarity to form a candidate document paragraph set, which will then be used for subsequent fine recall and information extraction.

[0141] Step S518: input the query field and the selected field into the large language model.

[0142] Step S520: constructing a semantic relationship between the query field and the selected field and each candidate document paragraph in the candidate document paragraph set through a large language model.

[0143] Step S522 , calculating the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set according to the semantic relationship, and recalling the target document paragraph in the candidate document paragraph set according to the semantic similarity.

[0144] Continuing with the above example, after determining the three document paragraphs, the query field and the selected field can be input into the large language model, and then the large language model can be used to perform precise recall on the three document paragraphs. According to the recall results, it is determined that document paragraph 1 contains the company's net profit information. Therefore, document paragraph 1 can be used as the target document paragraph for subsequent extraction of relevant information in the document paragraph for entry into the system.

[0145] Step S524, determining the information structure corresponding to the selected field through the large language model, and extracting the initial information associated with the business query information in the target document paragraph.

[0146] Step S526: Update the initial information based on the information structure to obtain target information associated with the business query information.

[0147] Continuing with the above example, after determining document paragraph 1, the target information of the related business query information "How much money did Company H make this year" can be extracted in document paragraph 1, and the result is "Company H's net profit this year is 15 million yuan, a year-on-year increase..." At this time, in order to facilitate system entry, the above target information can be updated with reference to the information structure of the selected field "10 million yuan", and the final output result "15 million yuan" can be obtained for use in system entry.

[0148] In summary, in order to ensure the representativeness of the information extraction results, improve the efficiency of information extraction and balance the weights of various key indicators, the field selection page containing the business sample document can be displayed to the user, and the query field corresponding to the business query information submitted by the user can be determined; on this basis, the selected field with an associated relationship with the query field can be determined in the business sample document in response to the selection instruction submitted through the field selection page; the user can customize the associated relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. Therefore, the candidate document paragraph set can be recalled in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph (multiple) associated with the business query information can be obtained in depth. Finally, the large language model can be used to refer to the selected field to extract the target information associated with the business query information in the target document paragraph. When extracting information, it can not only be completed based on business query information, but also the target document paragraphs with different field representations but the same meaning can be obtained by selecting the association between the field and the query field, so that the target information can be extracted quickly and accurately for downstream business analysis and processing, effectively improving the efficiency and accuracy of information extraction, and avoiding the impact of unbalanced information extraction.

[0149] See also Figure 6 , Figure 6 A flow chart of a field selection method provided according to an embodiment of the present specification is shown. The method is applied to a client and specifically includes the following steps.

[0150] Step S602, receiving business query information submitted for a target business, and determining a query field corresponding to the business query information;

[0151] Step S604, determining a business sample document according to the business query information, and displaying the business sample document through a field selection page;

[0152] Step S606, in response to the selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document;

[0153] Step S608: performing an information extraction task associated with the target business based on the selected field and the query field.

[0154] A field selection method provided in this embodiment is applied to the client to provide the user with a field selection function, so that the information extraction task can be subsequently performed based on the field selected by the user. The specific description of the field selection method can be found in the description in the above-mentioned information extraction method embodiment, and this embodiment will not be elaborated in detail here.

[0155] Corresponding to the above method embodiment, this specification also provides an information extraction device embodiment, Figure 7 FIG. 1 shows a schematic diagram of the structure of an information extraction device provided by an embodiment of the present specification. Figure 7 As shown, the device comprises:

[0156] The selection module 702 is configured to display a field selection page containing a business sample document according to the business query information, and determine a query field corresponding to the business query information;

[0157] The determination module 704 is configured to display a field selection page containing a business sample document according to the business query information, and determine a query field corresponding to the business query information;

[0158] An input module 706 is configured to recall a candidate document paragraph set in a business document library according to the query field, and input the query field and the selected field into a large language model;

[0159] The extraction module 708 is configured to recall the target document paragraph from the candidate document paragraph set by using the large language model, and extract the target information associated with the business query information from the target document paragraph by using the large language model and referring to the selected field.

[0160] In an optional embodiment, the selection module 702 is further configured to:

[0161] Obtain business query information and determine the information query type corresponding to the business query information; extract business sample documents from the sample document library according to the information query type, and load the business sample documents to the field selection page for display; wherein the field selection page includes a document display area and a selection custom area, the document display area is used to display the business sample documents, and the selection custom area is used to establish an association relationship between the query field and the selection field.

[0162] In an optional embodiment, the determining module 704 is further configured to:

[0163] Receive a selection instruction submitted through the field selection page; determine an initial selection field in the business sample document according to the selection instruction, and perform an integrity check on the initial selection field; if the check fails, extract business fields from a preset field library according to the initial selection field; and use the business field as a selection field that has an associated relationship with the query field.

[0164] In an optional embodiment, the determining module 704 is further configured to:

[0165] In response to a selection instruction submitted through the field selection page, an auxiliary selection subfield having a collaborative relationship with the query field and a base selection subfield having a field association relationship with the query field are determined in the business sample document; the auxiliary selection subfield and the base selection subfield are used as selection fields having an association relationship with the query field.

[0166] In an optional embodiment, the input module 706 is further configured to:

[0167] Determine the initial candidate document paragraphs contained in the candidate documents in the business document library; use a preset recall algorithm to calculate the document recall score between the query field and the initial candidate document paragraphs contained in each candidate document; recall a set number of candidate document paragraphs in the business document library according to the document recall score, and create a candidate document paragraph set based on the set number of candidate document paragraphs.

[0168] In an optional embodiment, the extraction module 708 is further configured to:

[0169] The semantic relationship between the query field and the selected field and each candidate document paragraph in the candidate document paragraph set is constructed through the large language model; the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set is calculated according to the semantic relationship; and the target document paragraph is recalled in the candidate document paragraph set according to the semantic similarity.

[0170] In an optional embodiment, the extraction module 708 is further configured to:

[0171] The information structure corresponding to the selected field is determined by the large language model, and the initial information associated with the business query information is extracted from the target document paragraph; the initial information is updated based on the information structure to obtain the target information associated with the business query information.

[0172] In an optional embodiment, the device further comprises:

[0173] Extract query entities from the business query information, and detect whether the query entity meets the information extraction condition; if not, extract business entities from the business sample document according to the query entity; use the business entity to update the business query information to obtain target business query information; use the target business query information as the business query information, and execute the step of recalling a set of candidate document paragraphs in a business document library according to the query field.

[0174] The information extraction device provided in this embodiment can display a field selection page containing a business sample document to the user in order to ensure the representativeness of the information extraction results, improve the efficiency of information extraction and balance the weights of various key indicators. At the same time, it can determine the query field corresponding to the business query information submitted by the user; on this basis, it can respond to the selection instruction submitted through the field selection page, and determine the selected field with an association relationship with the query field in the business sample document; it enables the user to customize the association relationship of the extracted field through the field selection page, thereby achieving a wider range of extraction coverage. Therefore, the candidate document paragraph set can be recalled in the business document library according to the query field to preliminarily obtain the document paragraph associated with the business query information. Then the query field and the selected field can be input into the large language model to realize the recall of the target document paragraph in the candidate document paragraph set through the large language model, and then the document paragraph (multiple) associated with the business query information can be obtained in depth. Finally, the target information associated with the business query information can be extracted from the target document paragraph by referring to the selected field using the large language model. When extracting information, it can not only be completed based on business query information, but also the target document paragraphs with different field representations but the same meaning can be obtained by selecting the association between the field and the query field, so that the target information can be extracted quickly and accurately for downstream business analysis and processing, effectively improving the efficiency and accuracy of information extraction, and avoiding the impact of unbalanced information extraction.

[0175] The above is a schematic scheme of an information extraction device of this embodiment. It should be noted that the technical scheme of the information extraction device and the technical scheme of the above information extraction method belong to the same concept, and the details not described in detail in the technical scheme of the information extraction device can be referred to the description of the technical scheme of the above information extraction method.

[0176] Corresponding to the above method embodiment, this specification also provides another information extraction device embodiment, Figure 8 FIG. 2 shows a schematic diagram of the structure of another information extraction device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises:

[0177] The field determination module 802 is configured to determine a selected field associated with a query field in the business query information;

[0178] A document recall module 804 is configured to recall a candidate document paragraph set in a business document library according to the query field;

[0179] An input model module 806 is configured to input the query field and the selected field into a large language model, and to recall a target document paragraph from the candidate document paragraph set through the large language model;

[0180] The information extraction module 808 is configured to extract the target information associated with the business query information from the target document paragraph by using the large language model and referring to the selected field.

[0181] The above is a schematic scheme of another information extraction device of this embodiment. It should be noted that the technical scheme of the information extraction device and the technical scheme of the above information extraction method belong to the same concept, and the details not described in detail in the technical scheme of the information extraction device can be referred to the description of the technical scheme of the above information extraction method.

[0182] Corresponding to the above method embodiment, this specification also provides a field selection device embodiment, Fig. 9 FIG. 1 shows a schematic diagram of the structure of a field selection device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:

[0183] The information receiving module 902 is configured to receive business query information submitted for a target business and determine a query field corresponding to the business query information;

[0184] The document determination module 904 is configured to determine a business sample document according to the business query information, and display the business sample document through a field selection page;

[0185] A field selection module 906 is configured to determine, in the business sample document, a selection field associated with the query field in response to a selection instruction submitted through the field selection page;

[0186] The task execution module 908 is configured to execute the information extraction task associated with the target business based on the selected field and the query field.

[0187] The above is a schematic scheme of a field selection device of this embodiment. It should be noted that the technical scheme of the field selection device and the technical scheme of the field selection method described above are of the same concept, and the details not described in detail in the technical scheme of the field selection device can be found in the description of the technical scheme of the field selection method described above.

[0188] Fig.10The block diagram of a computing device 1000 according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and the database 1050 is used to store data.

[0189] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0190] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0191] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1000 may also be a mobile or stationary server.

[0192] The processor 1020 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned information extraction method or field selection method.

[0193] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned information extraction method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned information extraction method or field selection method.

[0194] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned information extraction method or field selection method.

[0195] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned information extraction method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned information extraction method or field selection method.

[0196] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which implement the steps of the above-mentioned information extraction method or field selection method when executed by a processor.

[0197] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned information extraction method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above-mentioned information extraction method or field selection method.

[0198] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0199] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0200] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0201] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0202] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. An information extraction method, characterized in that: include: Displaying a field selection page containing a business sample document according to the business query information, and determining a query field corresponding to the business query information; In response to a selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document; Recalling a candidate document paragraph set in a business document library according to the query field, and inputting the query field and the selected field into a large language model; The semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set is calculated by the large language model, the target document paragraph is recalled from the candidate document paragraph set according to the semantic similarity, and the target information associated with the business query information is extracted from the target document paragraph by referring to the selected field using the large language model.

2. The information extraction method according to claim 1, characterized in that: The display of the field selection page containing the business sample document according to the business query information includes: Acquire business query information, and determine the information query type corresponding to the business query information; Extracting business sample documents from the sample document library according to the information query type, and loading the business sample documents to the field selection page for display; The field selection page includes a document display area and a selection custom area. The document display area is used to display the business sample document, and the selection custom area is used to establish an association relationship between the query field and the selected field.

3. The information extraction method according to claim 1, characterized in that: The step of determining, in response to the selection instruction submitted through the field selection page, a selected field associated with the query field in the business sample document includes: Receiving a selection instruction submitted through the field selection page; Determining an initial selection field in the business sample document according to the selection instruction, and performing integrity detection on the initial selection field; If the detection fails, extracting business fields from a preset field library according to the initially selected fields; The business field is used as a selection field associated with the query field.

4. The information extraction method according to claim 1, characterized in that: The step of determining, in response to the selection instruction submitted through the field selection page, a selected field associated with the query field in the business sample document includes: In response to a selection instruction submitted through the field selection page, determining in the business sample document an auxiliary selection subfield having a collaborative relationship with the query field and a reference selection subfield having a field association relationship with the query field; The auxiliary selection subfield and the reference selection subfield are used as selection fields associated with the query field.

5. The information extraction method according to claim 1, characterized in that: The step of recalling a candidate document paragraph set in a business document library according to the query field includes: Determine initial candidate document paragraphs included in the candidate documents in the business document library; Calculating a document recall score between the query field and the initial candidate document paragraphs contained in each candidate document using a preset recall algorithm; A set number of candidate document paragraphs are recalled from the business document library according to the document recall score, and a candidate document paragraph set is created based on the set number of candidate document paragraphs.

6. The information extraction method according to claim 1, characterized in that: Calculating the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set by using the large language model, and recalling the target document paragraph in the candidate document paragraph set according to the semantic similarity, comprises: Constructing a semantic relationship between the query field and the selected field and each candidate document paragraph in the candidate document paragraph set by using the large language model; Calculate the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set according to the semantic relationship; Recall the target document paragraph from the candidate document paragraph set according to the semantic similarity.

7. The information extraction method according to claim 1, characterized in that: The extracting target information associated with the business query information in the target document paragraph by using the large language model and referring to the selected field includes: Determining the information structure corresponding to the selected field through the large language model, and extracting the initial information associated with the business query information in the target document paragraph; The initial information is updated based on the information structure to obtain target information associated with the business query information.

8. The information extraction method according to any one of claims 1 to 7, characterized in that: Before the step of recalling a candidate document paragraph set in the business document library according to the query field is performed, the method further includes: Extracting a query entity from the business query information, and detecting whether the query entity meets an information extraction condition; If not, extracting business entities from the business sample document according to the query entity; Using the business entity to update the business query information to obtain target business query information; The target business query information is used as the business query information, and a step of recalling a candidate document paragraph set in a business document library according to the query field is performed.

9. A field selection method, characterized in that: Applied to the client, including: Receiving business query information submitted for a target business, and determining a query field corresponding to the business query information; Determine a business sample document according to the business query information, and display the business sample document through a field selection page; In response to a selection instruction submitted through the field selection page, determining a selection field associated with the query field in the business sample document; The target business-related information extraction task is performed based on the selected field and the query field, wherein the information extraction task is to use a large language model to recall the target document paragraphs in a candidate document paragraph set by calculating semantic similarity according to the selected field and the query field, and extract the target information associated with the business query information in the target document paragraphs.

10. An information extraction method, characterized in that: include: Determine a selected field associated with a query field in the business query information; Recalling a candidate document paragraph set in a business document library according to the query field; Inputting the query field and the selected field into a large language model, calculating the semantic similarity corresponding to each candidate document paragraph in the candidate document paragraph set by the large language model, and recalling the target document paragraph in the candidate document paragraph set according to the semantic similarity; The target information associated with the business query information is extracted from the target document paragraph by using the large language model and referring to the selected field.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 10 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1 to 10.

13. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images