Information extraction method, electronic equipment, storage medium and program product

Abstract

The invention discloses an information extraction method, an electronic device, a storage medium and a program product, and relates to the technical field of computers.The method comprises the steps that a to-be-extracted document is obtained, and the to-be-extracted document comprises at least one to-be-processed text and at least one preset extraction field; inputting all the preset extraction fields into a pre-constructed prediction model to obtain a prediction industry category corresponding to the to-be-extracted document; according to the predicted industry category, obtaining industry terms corresponding to the predicted industry category from a preset database; generating prompt words according to all preset extraction fields and industry terms; and according to the prompt word and each preset extraction field, performing information extraction on each to-be-processed text included in the to-be-extracted document to obtain an extraction field content corresponding to each to-be-processed text. According to the method and the device, the cue word is generated in combination with the industry term corresponding to the preset extraction field, the document industry expression habit can be fit, and the accuracy and the efficiency of information extraction are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an information extraction method, electronic device, storage medium, and program product. Background Technology

[0002] In data management, documents, as the core carriers of structured and unstructured data, are widely used in fields such as healthcare and manufacturing. Taking a spreadsheet document in manufacturing as an example, a spreadsheet document stores records of "equipment defect descriptions," requiring further extraction of key fields such as "defect location" and "defect type" from these descriptions. With the rapid development of pre-trained language models, information extraction technologies based on natural language understanding have shown great potential. Cue words, as the bridge connecting pre-trained language models and actual extraction needs, directly determine the model's extraction performance. However, in related technologies, manually generating clue words not only affects information extraction efficiency but also limits the accuracy of model extraction results due to the expertise of personnel. Summary of the Invention

[0003] This application provides an information extraction method, electronic device, storage medium, and program product to at least solve the problems of low efficiency and inaccuracy in information extraction.

[0004] This application provides an information extraction method applied to an information extraction device, the method comprising:

[0005] Obtain the document to be extracted, wherein the document to be extracted includes at least one text to be processed and at least one preset extraction field;

[0006] Input all preset extraction fields into the pre-built prediction model to obtain the predicted industry category corresponding to the document to be extracted;

[0007] Based on the predicted industry category, retrieve the industry terms corresponding to the predicted industry category from the preset database;

[0008] Based on all preset extraction fields and industry terms, generate prompt words corresponding to the document to be extracted;

[0009] Based on the prompt words and each preset extraction field, information is extracted from each text to be processed in the document to be extracted, and the content of the extraction field corresponding to each text to be processed is obtained.

[0010] This application also provides an information extraction device, comprising:

[0011] The first acquisition module is used to acquire the document to be extracted, wherein the document to be extracted includes at least one text to be processed and at least one preset extraction field;

[0012] The determination module is used to input all preset extraction fields into the pre-built prediction model to obtain the predicted industry category corresponding to the document to be extracted;

[0013] The second acquisition module is used to retrieve industry terms corresponding to the predicted industry category from a preset database based on the predicted industry category.

[0014] The generation module is used to generate prompt words corresponding to the document to be extracted based on all preset extraction fields and industry terms;

[0015] The extraction module is used to extract information from each text to be processed in the document based on prompt words and preset extraction fields, and obtain the content of the extraction field corresponding to each text to be processed.

[0016] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described information extraction methods.

[0017] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described information extraction methods.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described information extraction methods.

[0019] This application employs a predictive model to predict the industry category of the document to be extracted, and uses the predicted industry category to retrieve the corresponding industry terminology from a pre-set database. Furthermore, based on the industry terminology and pre-set extraction fields, prompt words are generated. Since the pre-set database stores terminology from different industries, it can automatically match and generate prompt words for documents from different industries, enabling information extraction from multiple industry documents. This allows the prompt words generated by combining industry terminology to better fit the context and professional expression habits of the document's industry, effectively expanding the applicability of information extraction and improving the efficiency and accuracy of prompt word generation. Furthermore, extracting information from the text to be processed based on the prompt words and pre-set extraction fields allows for more precise location of content related to the pre-set extraction fields within the text, reducing misunderstandings caused by industry differences and improving the accuracy of the extracted information. Attached Figure Description

[0020] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an information extraction method provided in this application embodiment;

[0022] Figure 2 This is an application diagram of an information extraction system provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of an information extraction device provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] First, the application scenarios of the embodiments of this application will be introduced by way of example.

[0029] In data management, documents (such as tables and lists) serve as the core carriers of structured and unstructured data, and are widely used in key scenarios such as manufacturing and finance. Taking quality inspection instructions in the production field as an example, these documents contain text describing defects, such as "a 3cm scratch on the left side of the casing, and slight deformation at the corners." In the process of managing quality inspection instructions, it is often necessary to extract key fields from this text to automate business processes and perform statistical data analysis.

[0030] In related technologies, there are three main methods for extracting data from documents: Method 1 relies on manual reading of the document line by line, manually identifying and filling in the content of preset fields. This method is time-consuming and labor-intensive, and prone to problems such as information omission due to human error and statistical bias due to subjectivity. Method 2 involves designers writing rule scripts to extract the content of preset fields in the document. For example, extracting text content based on rules such as regular expressions. However, this method is only suitable for document processing scenarios with highly standardized formats, and its adaptability is poor for unstructured data with complex semantics and flexible expression. Method 3 uses pre-trained language models to process the text in the document, and manually designs prompt words and manually integrates them to return the content of preset fields. In practical applications, this method has encountered problems such as insufficient adaptability of prompt words, waste of computing resources, and poor accuracy of results.

[0031] In view of this, embodiments of this application provide an information extraction method to improve the efficiency and accuracy of information extraction.

[0032] Figure 1 This is a flowchart of an information extraction method provided according to an embodiment of the present invention. The execution subject of this method is an information extraction device, such as... Figure 1 As shown, the process includes:

[0033] S101, Obtain the document to be extracted.

[0034] The document to be extracted includes at least one text to be processed and at least one preset extraction field.

[0035] Specifically, the document to be extracted refers to the document for which information extraction operations are required. This application embodiment does not specifically limit the format of the document to be extracted; it can be a table document (i.e., a document that carries data in a row-column structure), a list document (i.e., a document that carries data in a list structure), etc. In addition to containing the text to be processed for information extraction, such as "There is a 3cm scratch on the left side of the shell, and the corner is slightly deformed," the document to be extracted also contains preset extraction fields, which are pre-defined categories of information to be extracted from the text. Continuing with the example of "There is a 3cm scratch on the left side of the shell, and the corner is slightly deformed," the corresponding preset extraction fields could be defect location, defect length, defect type, severity level, etc.

[0036] In one possible implementation, the document to be extracted is obtained through its storage address. Once the document is obtained, a header analysis algorithm identifies two types of data columns: one type consists of individual text characters that are the text to be processed, and the other type consists of field content that needs to be filled based on preset extraction fields, i.e., the field content to be extracted.

[0037] S102, input all preset extraction fields into the pre-built prediction model to obtain the predicted industry category corresponding to the document to be extracted.

[0038] Specifically, the pre-built prediction model refers to a pre-trained model used to predict the industry category of the document to be extracted. This model is trained on data with industry labels and can obtain the industry category of the document to be extracted based on the preset extraction fields. For example, by inputting defect location, defect type, etc., into the prediction model, the model can predict that the document to be extracted belongs to the manufacturing industry.

[0039] For example, the pre-built prediction model can be a deep learning model, a machine learning model, etc., and this application does not specifically limit it.

[0040] S103, Based on the predicted industry category, retrieve the industry terms corresponding to the predicted industry category from the preset database.

[0041] Specifically, the default database stores industry-specific terms for different sectors. Taking manufacturing as an example, the default database may contain industry terms such as physical damage and stress deformation.

[0042] S104: Generate prompt words corresponding to the document to be extracted based on all preset extraction fields and industry terms.

[0043] Specifically, prompt words are used to guide statements during the information extraction process. In this embodiment, prompt words obtained by combining preset extraction fields and industry terminology can make the extraction process more accurate. For example, for the text to be processed describing defects in manufacturing, the prompt words could be "Extract the defect location and defect type from the text to be processed, where the defect location can be the front or back, and the defect type can be scratches or deformations".

[0044] In one possible implementation, all preset extraction fields and industry terms are input into a pre-built prompt word generation model to generate prompt words.

[0045] For example, the pre-built prompt word generation model can be a pre-trained language model (such as a Bidirectional Encoder Representations from Transformers (BERT) model, a Large Language Model (LLM) model, a Generative Pre-trained Transformer (GPT) model, etc.), which uses natural language processing technology to obtain prompt words by combining preset extraction fields and industry terms.

[0046] S105, based on the prompt words and each preset extraction field, extract information from each text to be processed in the document to be extracted, and obtain the content of the extraction field corresponding to each text to be processed.

[0047] Specifically, extracting field content refers to extracting specific information from the text to be processed that corresponds to the preset extraction field. For example, the defect type extracted from "there is a 3cm scratch on the left side of the shell" is "scratches", and the defect location is "left side".

[0048] In one possible implementation, prompt words and preset extraction fields are input into a pre-built information extraction model to extract information from each text to be processed in the document, thereby obtaining the content of the extraction fields corresponding to each text to be processed.

[0049] For example, the information extraction model can also be the pre-trained language model mentioned above.

[0050] In this embodiment, a predictive model is used to predict the industry category of the document to be extracted. The predicted industry category is then used to retrieve corresponding industry terms from a pre-set database. Furthermore, based on the industry terms and pre-set extraction fields, prompt words are generated. Since the pre-set database stores terms from different industries, prompt words can be automatically matched and generated for documents from different industries, enabling information extraction from multiple industry documents. This allows prompt words generated by combining industry terms to better fit the context and professional expression habits of the document's industry, effectively expanding the applicability of information extraction and improving the efficiency and accuracy of prompt word generation. Furthermore, extracting information from the text to be processed based on prompt words and pre-set extraction fields allows for more precise location of content related to the pre-set extraction fields within the text, reducing misunderstandings caused by industry differences and improving the accuracy of the extracted information.

[0051] In some embodiments, based on the foregoing embodiments, prompt words corresponding to the document to be extracted are generated according to preset extraction fields and industry terms. There are multiple implementation methods. The following uses Method 1 and Method 2 to illustrate how to generate prompt words.

[0052] Method 1: Generate prompt words corresponding to the document to be extracted through the following steps:

[0053] a1 retrieves multiple historical documents.

[0054] Specifically, a historical document contains at least one text and an extraction field. A historical document refers to a document that has already undergone information extraction; it can be used as a reference document for the document to be processed. Historical prompts refer to prompts corresponding to historical documents, used to record the extraction logic for those documents.

[0055] a2, selects the target historical document that corresponds to the document to be extracted from multiple historical documents.

[0056] Specifically, the target historical document refers to a document that is highly similar to the document to be extracted in terms of industry category, extraction fields, or text features.

[0057] For example, the target historical document corresponding to the document to be extracted can be selected from multiple historical documents in the following way:

[0058] First, obtain the industry category corresponding to each historical document, as well as the extraction fields and text included in each historical document.

[0059] Here, the industry category corresponding to the historical documents can be determined in a similar way to the industry category corresponding to the documents to be extracted, which will not be elaborated here.

[0060] Next, based on the industry category of the document to be extracted, the preset extraction fields, and at least one text to be processed, a feature vector corresponding to the document to be extracted is generated.

[0061] For example, the industry category, the preset extraction field, and at least one text to be processed are respectively encoded into their corresponding text vectors. The text vectors corresponding to the industry category, the preset extraction field, and the text to be processed are then concatenated to generate a feature vector corresponding to the document to be extracted.

[0062] Then, based on the industry category, extracted fields, and text corresponding to the first historical document, feature vectors corresponding to the first historical document are generated.

[0063] The method for generating feature vectors for historical documents is similar to that for generating feature vectors for documents to be extracted, and will not be repeated here.

[0064] Finally, after generating feature vectors corresponding to all historical documents, the target historical document is determined based on the feature vectors corresponding to the document to be extracted and the feature vectors corresponding to each historical document.

[0065] Optionally, the similarity between the feature vector of the document to be extracted and the feature vector of each historical document can be calculated, such as cosine similarity, and the historical document with the highest similarity can be used as the target historical document.

[0066] In this way, by converting industry categories, extraction fields, and text content into structured feature vectors, quantitative comparison of multi-dimensional information can be achieved, avoiding the subjectivity of manual screening. This makes the matching between target historical documents and documents to be extracted more accurate. Furthermore, the historical prompts corresponding to the selected target historical documents are more valuable and can help generate prompts that are more relevant to the current scenario, ultimately improving the accuracy and efficiency of information extraction.

[0067] a3 retrieves historical suggestion words corresponding to the target historical document.

[0068] a4 generates prompts corresponding to the document to be extracted based on preset extraction fields, industry terms, and historical prompts.

[0069] Optionally, all preset extraction fields, industry terms, and historical prompt words can be input into a pre-built prompt word generation model to generate prompt words.

[0070] In this implementation, since the historical suggestion words of the target historical documents are effective templates that have been verified in practice, the selected "target historical documents" will be more in line with the new scenario. Furthermore, by combining the current preset extraction fields and industry terms, the accuracy and adaptability of the suggestion words can be improved.

[0071] Method 2: By using a pre-defined prompt word template, preset extraction fields and industry terms are dynamically filled into the prompt word template to generate prompt words.

[0072] For example, a pre-defined prompt template could be: "Please extract the following fields from the text to be processed: {field list}. Where {field 1} specifies the exact location; {field 2} refers to the terminology: {terminology list}; {field 3} is based on the terminology {terminology classification}." When the preset extraction fields are defect location, defect type, and severity level, and the industry terms are scratch, deformation, and crack, with terminology classifications of minor, moderate, and severe, the generated prompt could be: "Please extract the following fields from the text to be processed: defect location, defect type, and severity level. Wherein, defect location must specify the exact location; defect type refers to the terminology: scratch, deformation, crack; severity level must be determined based on the terminology classification of minor, moderate, and severe."

[0073] In some embodiments, based on any of the foregoing embodiments, the document to be extracted further includes at least one reference text, and the preset field content corresponding to each preset extraction field for all reference texts.

[0074] Specifically, the preset field content in the reference text refers to the correct field content corresponding to each preset extraction field that is known in advance, used to verify the validity of the prompt words.

[0075] Based on the prompt words and each preset extraction field, before extracting information from each text to be processed in the document and obtaining the content of the extraction field corresponding to each text, the method also includes the following:

[0076] b1. Input the prompt word, each preset extraction field and the first reference text into the pre-built information extraction model to obtain the extraction field content corresponding to each preset extraction field of the first reference text.

[0077] The first reference text can be any one of at least one reference text.

[0078] Information extraction models can be pre-trained models used to extract information from specific fields in text, such as pre-trained language models.

[0079] b2. After obtaining the content of the extracted fields corresponding to each preset extraction field for all reference texts, the prompt words are verified based on the content of the extracted fields corresponding to each preset extraction field for all reference texts and the preset field content, and the first verification result is obtained.

[0080] Specifically, the first verification result is the accuracy of the prompt words verified based on the reference text.

[0081] b3, randomly select a first number of target texts from at least one text to be processed.

[0082] b4. According to the preset number of loops, input the prompt words, each preset extraction field and each target text into the information extraction model in each loop to obtain the extraction field content corresponding to each target text in each preset extraction field in each loop.

[0083] Specifically, the preset number of loops refers to the number of times the target text is repeatedly extracted from the fields. For example, the predicted number of loops is 3. Within each loop, the prompt words, preset extraction fields, and each target text are input into the information extraction model to obtain the extracted field content corresponding to each target text in each preset extraction field.

[0084] It should be noted that the initial quantity and preset number of cycles can be set according to the actual situation, and there is no limitation here.

[0085] b5 verifies the prompt words based on the content of each preset extraction field obtained in each loop for all target texts, and obtains the second verification result.

[0086] Specifically, the second verification result is to extract the target text multiple times to verify whether the prompt words can consistently output similar results, that is, to verify the consistency of the prompt words.

[0087] b6. Based on the first and second verification results, verify the validity of the prompt words. After confirming the validity of the prompt words, extract information from each text to be processed in the document to be extracted according to the prompt words and each preset extraction field, and obtain the extraction field content corresponding to each text to be processed.

[0088] In one possible implementation, in b2 above, the first verification result is determined as follows:

[0089] c1, Based on the content of the preset field and the content of the extracted field corresponding to the first preset extraction field in the first reference text, determine the semantic similarity corresponding to the first preset extraction field.

[0090] The first preset extraction field is any one of the preset extraction fields.

[0091] Specifically, semantic similarity refers to the degree of semantic matching between the content of the preset field and the content of the extracted field. The greater the semantic similarity, the closer the two are.

[0092] For example, a pre-trained language model is used to convert the preset field content and the extracted field content into their respective vectors. Cosine similarity is then used to calculate the vector similarity between the vector corresponding to the preset field content and the vector corresponding to the extracted field content, which is then used as the semantic similarity.

[0093] c2, after determining the semantic similarity of all reference texts in each preset extraction field, determine the first verification result based on the semantic similarity of all reference texts in each preset extraction field.

[0094] The first verification result is based on the semantic similarity of all reference texts on each preset extraction field, comprehensively judging the extraction effect of prompt words on reference texts.

[0095] Optionally, in c2 above, the first verification result is determined as follows:

[0096] First, count the number of first-class fields corresponding to the first reference text.

[0097] The first category of fields consists of pre-defined extraction fields whose semantic similarity is greater than a preset similarity threshold. Specifically, the first category of fields refers to those fields in the first reference text whose semantic similarity to the content of a pre-defined field in the first extracted text is greater than the preset similarity threshold. The preset similarity threshold can be limited according to actual circumstances; for example, it can be set to 0.85. For instance, if the preset extraction fields are defect location and defect type, and the preset similarity threshold is 0.85, and the semantic similarity of defect location and defect type are 0.9 and 0.8 respectively, then the defect location pre-defined extraction field is considered a first category field.

[0098] Then, after counting the number of the first type of fields corresponding to all reference texts, count the number of target reference texts.

[0099] The target reference text is the reference text in which the number of the first type of fields is greater than the number of the second type of fields.

[0100] For example, if the second quantity is 7, and a reference text contains 8 fields of the first type, then that reference text is the target text. Here, the second quantity can be determined based on the total quantity of the reference text, such as using a preset proportion (e.g., 70%) of the total quantity as the second quantity.

[0101] Finally, the effectiveness of the prompt is determined based on the number of target reference texts and the total number of all reference texts.

[0102] For example, a prompt is determined to be valid when the ratio between the number of target reference texts and the total number is greater than a first ratio threshold. Conversely, a prompt is determined to be valid when the ratio between the number of target reference texts and the total number is less than or equal to the first ratio threshold.

[0103] The first proportional threshold can be set according to the actual situation, and this application embodiment does not limit it.

[0104] In this way, we not only focus on the extraction accuracy of prompt words in a single field through the first type of field, but also on the extraction success rate of multiple reference texts as a whole through the target reference text, ensuring the stability of prompt words in multi-field and multi-text scenarios.

[0105] Optionally, in c2 above, based on the semantic similarity of all reference texts on each preset extraction field, the average semantic similarity corresponding to all reference texts is calculated. When the average semantic similarity is greater than the preset semantic similarity threshold, the first verification result is determined to be a valid prompt word; when the average semantic similarity is less than or equal to the preset semantic similarity threshold, the first verification result is determined to be an invalid prompt word.

[0106] In this way, by using semantic similarity rather than exact string matching, and taking into account the differences in textual expression, it is more in line with the linguistic diversity in actual data management scenarios.

[0107] In one possible implementation, in b5 above, the second verification result is determined as follows:

[0108] d1, based on the content of the extracted fields corresponding to each preset extracted field obtained in each loop of the first target text, count the number of second type fields in the first target text.

[0109] The second type of field is a preset extraction field where the content of the extraction field corresponding to the first target text is inconsistent in different loops.

[0110] For example, whether the content of the extracted fields is consistent can be determined by the semantic similarity between the vectors corresponding to the extracted field content. For example, two extracted field contents with a semantic similarity greater than a preset similarity threshold are considered consistent. Two extracted field contents with a semantic similarity less than or equal to the preset similarity threshold are considered inconsistent.

[0111] d2, After determining the number of second-class fields in all target texts, determine the second verification result based on the number of second-class fields in all target texts.

[0112] Optionally, a second verification result is determined based on the number of second-type fields in all target texts, specifically including the following steps:

[0113] First, filter out target texts from all target texts where the number of second-category fields is greater than the number of third-category fields.

[0114] Then, based on the fact that the number of target texts in the second category is greater than the number in the third category, the second verification result is determined.

[0115] For example, target texts with a greater number of second-category fields than third-category fields are considered non-compliant texts. The proportion of non-compliant texts in all target texts (i.e., the proportion of target texts with a greater number of second-category fields than third-category fields) is determined to establish a second verification result. For instance, if the proportion of non-compliant texts is less than a second percentage threshold, the second verification result is that the prompt is valid; otherwise, the prompt is invalid.

[0116] Here, a third quantity is set as the maximum number of unstable fields allowed in a single text. Target texts with poor stability are filtered out, that is, target texts with more than three types of fields. The second verification result is then determined through the target texts with poor stability to determine whether the prompt words can remain consistent in multiple extractions and to verify the reliability of the prompt words.

[0117] Optionally, the total number of the second type of fields in all target texts is summed to obtain the total number of the second type of fields corresponding to all target texts. If the total number of the second type of fields is greater than a preset threshold, the second verification result is determined to be an invalid prompt word. If the total number of the second type of fields is less than or equal to the preset threshold, the second verification result is determined to be a valid prompt word.

[0118] Alternatively, the average number of second-category fields in all target texts can be calculated. Based on this average number of second-category fields and a preset average threshold, a second verification result is determined. When the average number of second-category fields is greater than the preset average threshold, the second verification result is determined to be an invalid prompt. When the average number of second-category fields is less than or equal to the preset average threshold, the second verification result is determined to be a valid prompt.

[0119] In addition, after obtaining the prompt words, manual adjustments can be made to further ensure the accuracy of the prompt words.

[0120] In this embodiment, the first verification result obtained through the reference text ensures that the prompt words can correctly extract the field content, reducing errors in extraction caused by ambiguous prompt words. Simultaneously, a second verification result is obtained by repeatedly verifying the target text to avoid randomness in the model output and ensure stable extraction results. By performing two verifications on the prompt words to confirm their effectiveness, the accuracy of extracted field content is improved, reducing the need for manual correction of the prompt words and subsequent extracted field content. This ensures the reliability of field extraction from the very beginning, i.e., from the generation of the prompt words.

[0121] In some embodiments, the information extraction device includes multiple processing units, each of which is equipped with an information extraction model. Each processing unit, as a module with independent computing capabilities within the information extraction device, such as a server node or virtual machine, is used to individually process information extraction tasks. Taking a server cluster as an example, the processing units are the server nodes within the cluster.

[0122] Based on the prompt words and preset extraction fields, information is extracted from each text to be processed in the document, resulting in the extraction field content corresponding to each text. The specific steps include the following:

[0123] First, obtain the status identifier corresponding to each processing unit.

[0124] Specifically, a status identifier is a marker that indicates the current working state of the processing unit. For example, a status identifier can be an idle identifier or a busy identifier.

[0125] For example, the status identifier of each processing unit can be queried in real time through the internal communication records of the information extraction device (such as heartbeat detection and status synchronization interface). For instance, a status query request is sent to each processing unit according to a preset period, the status identifier returned by each processing unit is received, and recorded in the task scheduling table of the information extraction device.

[0126] Then, the target processing unit with the status identifier of "idle" is selected from each processing unit.

[0127] For example, the information processing unit also includes a scheduling unit, which, after obtaining the status identifiers of each processing unit, filters out all processing units marked as idle.

[0128] Next, at least one text to be processed is grouped to obtain texts to be processed corresponding to each target processing unit.

[0129] For example, the texts to be processed can be grouped based on the performance data of the target processing unit (such as CPU utilization, memory size, etc.). Alternatively, the texts can be grouped according to the principle of average distribution to obtain the texts to be processed corresponding to each target processing unit.

[0130] Finally, the prompt words, preset extraction fields, and the text to be processed corresponding to the first target processing unit are input into the information extraction model of the first target processing unit to obtain the extraction field content of the text to be processed corresponding to the first target processing unit.

[0131] Among them, the first target processing unit is one of the target processing units whose status is marked as idle.

[0132] In one possible implementation, the method provided in this application embodiment further includes the following:

[0133] If the information extraction model of the first target processing unit fails to output the content of the extracted fields of the first text to be processed within the preset time period, the prompt word, the preset extraction fields, and the first text to be processed are input into the information extraction model of the first target processing unit again until the preset conditions are met and the input operation stops.

[0134] The first text to be processed is one of the texts to be processed corresponding to the first target processing unit. The preset condition is to reach a preset number of inputs, or to obtain the extracted field content of the first text to be processed.

[0135] Optionally, the relationship between the time interval between two consecutive input operations and the number of inputs corresponding to the input operation is expressed as follows:

[0136] Δt i-1,i =a×b i

[0137] Where, Δt i-1,i is the time interval between the (i-1)th input operation and the i-th input operation; a is the base time interval, and b is the exponential parameter. a and b can be set according to the actual situation. For example, a can be set to 1 second and b can be set to 2 seconds.

[0138] Thus, considering that the processing unit may time out when processing text due to network jitter or excessive instantaneous resource consumption, retries can avoid such problems. However, frequent retries immediately after a failure may cause failed processing tasks to continuously occupy the processing unit's resources, increasing the pressure on the processing unit. Therefore, an exponentially increasing time interval between two adjacent input operations is adopted. That is, as the number of retries increases, the time interval between two adjacent inputs lengthens. This effectively reduces the impact of high-frequency retries in a short period of time on the model or processing unit, alleviates the pressure on the processing unit, and avoids overload. At the same time, setting a preset number of inputs prevents invalid tasks from being infinitely retried and wasting the processing unit's resources.

[0139] In this embodiment, information extraction tasks are assigned only to idle target processing units, avoiding the assignment of text to busy processing units. This further prevents task timeouts due to processing unit overload, maximizing the utilization of internal resources in the information extraction device. Furthermore, parallel text processing by multiple target processing units further improves the efficiency of information extraction. In addition, distributing information extraction tasks across multiple processing units allows other processing units to continue the task even if one unit fails, reducing the risk of overall task failure.

[0140] In some embodiments, based on any of the foregoing embodiments, the method provided in this application further includes the following:

[0141] First, obtain the number of historical texts processed and the number of timeouts corresponding to the first target processing unit within the preset historical time period based on the current time.

[0142] Specifically, the timeout count refers to the number of times within a preset historical time period, the first target processing unit fails to return a result after processing the text for a preset duration. For example, if one out of 10 texts takes longer than the preset duration to process, the timeout count is 1.

[0143] For example, the information extraction device acquires the number of processed historical texts and the number of timeouts corresponding to the first target processing unit according to a preset period. The preset historical time period can be 1 hour, 30 minutes, etc.

[0144] Then, based on the number of processed historical texts and the number of timeouts, the timeout rate corresponding to the first target processing unit is determined.

[0145] For example, the timeout rate is the ratio between the number of timeouts and the amount of historical text that has been processed, which can reflect the efficiency stability of the processing unit.

[0146] Finally, when the timeout rate exceeds the preset timeout rate, the number of texts to be processed corresponding to the first target processing unit is reduced according to the preset quantity adjustment rules.

[0147] For example, when the timeout rate exceeds a preset timeout rate, the number of texts to be processed in the current first target processing unit is reduced by a preset ratio or a preset reduction amount. For example, the number of texts to be processed is reduced by 80%, or by 20 texts. At this time, a minimum number of texts that the target processing unit needs to process can also be set, for example, the target processing unit must retain at least 2 texts to be processed.

[0148] In this embodiment, the load capacity of the processing unit is determined by historical text, reducing the amount of text to be processed corresponding to processing units that frequently time out, avoiding risks such as low information extraction efficiency and timeouts due to overload, realizing dynamic resource allocation in the information extraction device to cope with performance fluctuations of the processing unit, and ensuring stable operation of the information extraction device in a dynamic environment.

[0149] In some embodiments, based on the foregoing embodiments, the method provided in this application further includes the following:

[0150] e1 retrieves the processing time for each historical text.

[0151] Specifically, historical text refers to text that the processing unit has already processed within a preset historical time period, i.e., text for which information extraction has been performed. Processing time refers to the time from when the processing unit receives historical text to when it returns the extracted field content of that text.

[0152] e2. Based on the number of historical texts and the processing time corresponding to each historical text, determine the average response time corresponding to the first target processing unit.

[0153] Specifically, the average response time is the average time taken for the first target processing unit to process all its corresponding historical texts within a preset historical period, reflecting the processing efficiency during that period. The average response time is the sum of the processing times for all historical texts, divided by the number of historical texts.

[0154] e3, obtain the historical average response time corresponding to the first target processing unit.

[0155] Specifically, the historical average response time refers to the long-term average processing time of the first target processing unit before a preset historical time period. For example, it is the average time taken for the first target processing unit to process all processed texts within the preset historical time period, based on the initial moment of the preset historical time period. For instance, if the first target processing unit processed 50 historical texts in the past 7 days, with a total processing time of 500 seconds, then the historical average response time is 500 / 50 seconds.

[0156] e4. When the average response time is less than the historical average response time, determine the response time difference corresponding to the first target processing unit based on the average response time and the historical average response time.

[0157] Specifically, the response time difference refers to the difference between the average response time and the historical average response time, which reflects the magnitude of efficiency changes in the processing unit. Thus, when the "average response time" (processing efficiency in the current time period) of the first target processing unit is less than the "historical average response time" (long-term benchmark efficiency), and the difference exceeds the "preset proportion of historical average response time" (e.g., exceeding 20%), it indicates that the current efficiency of the processing unit has significantly improved (e.g., processing speed has increased), and there is underutilized redundant computing power.

[0158] e5. When the response time difference is greater than the historical average response time of the preset ratio, and the timeout rate is less than or equal to the preset timeout rate, the number of texts to be processed corresponding to the first target processing unit is increased according to the preset quantity adjustment rules.

[0159] The preset ratio and preset timeout rate can be limited according to the actual situation. For example, the preset ratio and preset timeout rate are set to 15% and 20%, respectively.

[0160] For example, the preset quantity adjustment rule can be to increase the number of texts to be processed by 120% when the response time difference is greater than a preset proportion of the historical average response time, and the timeout rate is less than or equal to the preset timeout rate. Alternatively, it can be to increase the number of texts to be processed by a preset quantity (e.g., 20). Of course, the maximum number of texts that each target processing unit can process can also be set, for example, a maximum of 20 texts per target processing unit, to avoid overloading the target processing unit.

[0161] In this embodiment of the application, the number of texts to be processed by the first target processing unit can only be increased when the response time difference is greater than the historical average response time of a preset proportion and the timeout rate is less than or equal to the preset timeout rate. This ensures the stability of the first target processing unit and makes full use of the redundant computing power of the first target processing unit. Based on the performance changes of the processing unit, the number of texts to be processed is dynamically adjusted, thereby improving the flexibility of the information extraction device and enhancing the load balancing capability.

[0162] The above mainly describes the solution provided by the embodiments of this application from a methodological perspective.

[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0164] This application also provides an information extraction system, which includes a task configuration module, a dynamic prompt word generator, a confidence verification processing unit, an adaptive scheduler, a large model working pool, and a state manager.

[0165] The task configuration module serves as the entry point for interaction with the system, enabling task configuration. Users can configure tasks, submit documents to be extracted, and configure parameters through the interface. Upon receiving the document to be extracted, the task configuration module automatically parses basic information such as the number of rows and columns, and generates a visual configuration interface. Parameter configuration allows users to select whether to enable manual editing of prompt words (disabled by default), set the first and second percentage thresholds, and the first quantity (adjustable from 5 to 10 lines). Responding to document submission and parameter configuration, the task configuration module encapsulates the document to be extracted and the configured parameters into a standardized task object, passes it to the dynamic prompt word generator through an internal interface, and simultaneously registers basic task information with the state manager, enabling full lifecycle tracking of the task.

[0166] The dynamic prompt generator, as the core module for automated prompt generation, interacts with the task configuration module and the confidence verification processing unit. Based on the task records, the dynamic prompt generator retrieves the relevant documents to be extracted and uses a header analysis algorithm to identify the column structure of the documents, namely three types of key columns: the text to be processed prefixed with "DataSource-", the column containing the preset extraction fields, and a preset column used to mark the text category (text to be processed, reference text). Based on the column identification results, the dynamic prompt generator calls a prediction model to determine the industry category corresponding to the document to be extracted, matches it with the corresponding industry knowledge base (including terminology system and extraction rule templates), and generates initial prompts.

[0167] Table 1 shows a portion of the document to be extracted. In Table 1, the reference text in the document to be extracted is determined based on the field content of the preset columns. When the field content of the preset column is "Yes", the text corresponding to the row containing "Yes" is the reference text. When the field content of the preset column is "No", the text corresponding to the row containing "No" is the text to be processed. In Table 1, the reference text is "A 2.5mm slight scratch exists on the front of the casing, no missing corner" and "A 3mm long scratch was found on the bottom, of moderate depth, with a missing corner on the right side". The preset extraction fields in the table are "Scratches present", "Scratches location", "Scratches length", "Scratches depth", "Cut corners present", and "Cut corner location".

[0168] Table 1

[0169]

[0170] Taking Table 1 as an example, the generated initial prompts are as follows:

[0171] "You are a quality inspector. Please interpret the quality inspection report. Extract features based on the report content and generate a list containing 7 features in sequence (features not described are marked as N / A). The output is in strict JSON format: {"results":[value1,value2,...]}, where each value adopts a uniform format: "Description: Detailed explanation, Encoding: Encoded value / numerical value / N / A".

[0172] Feature items and coding rules (in fixed order):

[0173] 1. Are there scratches? → [Description] [Encoding rule: Yes = 1, No = 2];

[0174] 2. Scratch location → [Description][Coding rules: Front = 1, Back = 2, Bottom = 3];

[0175] 3. Scratch length → [Description] [Coding rule: Report value (mm) / N / A];

[0176] 4. Scratch Depth → [Description] [Coding Rules: Minor = 1, Moderate = 2, Severe = 3];

[0177] 6. Is there a missing corner? → [Description] [Encoding rule: Yes = 1, No = 2];

[0178] 7. Missing corner location → [Description][Encoding rule: Left side = 1, Right side = 2].

[0179] Output requirements:

[0180] 1. The result array always contains 7 items (strictly corresponding to the feature order);

[0181] 2. Format for each item: "Description: [Human-readable description of feature status], Encoding: [Encoded value / Measurement value / N / A]";

[0182] 3. All measurements are in millimeters (mm);

[0183] 4. For features not mentioned in the description, mark them as "not described" and use "N / A" for the encoded parts;

[0184] 5. The JSON format contains only the "results" key, whose values ​​are arrays of strings.

[0185] If the manual editing option is enabled in the task configuration, the dynamic prompt generator will convert the prompts into rich text format and push them to the object editing interface through the interactive interface. After the object is confirmed or modified, the optimized prompts will be received and synchronously transmitted to the confidence verification processing unit for validity verification, forming a closed-loop process of "automatic generation - manual optimization - pre-verification".

[0186] The confidence verification processing unit is used to verify the validity of the prompt words. The confidence verification processing unit receives the prompt words, the document to be extracted, and the column structure information (i.e., the three types of key column information mentioned above) sent by the dynamic prompt word generator, and performs dual verification on the prompt words in two stages.

[0187] The first stage involves using the reference text (i.e., the text corresponding to the preset column "Yes") and the information extraction model to extract the content of the extracted fields from the reference text based on the prompt words. The semantic consistency is used to determine the semantic matching degree between the preset field content and the extracted field content, and the prompt words are verified based on the semantic matching degree.

[0188] The second stage involves randomly selecting a first number of target texts from the text to be processed. For each target text, following a preset number of loops, the prompt word, each preset extraction field, and the target text are input into the information extraction model within each loop to obtain the extracted field content corresponding to the target text in each preset extraction field. The prompt word is validated based on the differences between the extracted field content corresponding to different loops.

[0189] During the verification process, the confidence verification processing unit records the verification log in real time (including the specific fields and differences of the failed items), and generates a standardized verification report upon completion. If both the first and second stages indicate that the prompt words are valid, the prompt words are synchronized to the adaptive scheduler to start the formal processing flow, that is, to extract information from each text to be processed. If the verification prompt words are invalid in any stage, the invalid prompt word information is pushed to the status manager, triggering the task termination process and displaying the detailed reasons.

[0190] The adaptive scheduler, as the central module for resource scheduling and task execution in this system, interacts with other modules in the system through data exchange or control. During initialization, the adaptive scheduler creates a pool of basic processing units (the number of texts to be processed in the pool is configurable). Each processing unit is an independent thread entity with complete capabilities for task reception, data processing, and result writing.

[0191] After receiving the "prompt word valid" signal from the confidence verification processing unit, the adaptive scheduler splits the tasks corresponding to the documents to be extracted into line ranges according to the complete task information (by default, each group consists of 100 lines of text to be processed), forming a task queue. Then, through the thread pool management algorithm, it traverses the currently idle processing units and allocates the line groups in the task queue (task queue: taskId / groupId / filePath / prompt / rowStart / rowEnd / currentRow / failedNum / status).

[0192] The adaptive scheduler periodically summarizes the performance data of the processing units (response time and number of timeouts of the information extraction model), and reads the performance indicators within the time window from the database based on the preset dynamic adjustment rules (evaluated every 30 minutes): when the timeout rate exceeds 20%, the amount of text to be processed is automatically reduced by 20%; when the average response time decreases by more than 15% and the timeout rate is less than 5%, the amount of text to be processed is increased by 20%, ensuring that the allocation of text to be processed is dynamically matched with the processing capacity of the processing units, and avoiding overload or idleness of the processing units.

[0193] The large model working pool, serving as the core of the actual execution information extraction, consists of multiple processing units managed by the adaptive scheduler and interacts with the adaptive scheduler and state manager in real time.

[0194] Each processing unit independently receives row group tasks (including file path, row range, prompt words, and column mapping relationships) assigned by the adaptive scheduler, and executes a standardized processing flow: reads the original data column content of the specified row and merges it into input text, constructs a model request according to the prompt word format requirements, calls the application programming interface (API) of the pre-trained language model (supporting multi-model switching interface), parses the returned extracted field content and writes it into the target column, and finally records the processing result in the database, updates the current processing row number (currentRow) and the number of processing failures (failedNum) of the task according to the task number (taskId).

[0195] During processing, the processing unit records key performance data in real time: the response time of each model call (accurate to milliseconds), whether it timed out (based on a 20-second threshold), and the completeness of extracted field content (coverage of required fields). This data is written to the database performance table via an asynchronous interface, providing data support for the adaptive scheduler to dynamically adjust. If a model call fails (e.g., due to network errors or formatting issues), the processing unit initiates a built-in retry mechanism (up to 3 times, with exponential backoff intervals). If the failure rate exceeds the threshold, the row status is marked as "processing failed," awaiting subsequent retry scheduling.

[0196] If a processing unit fails to extract document information using the model, a retry mechanism is triggered:

[0197] First attempt at the task: failed (e.g., network timeout).

[0198] First retry: Execute again after a certain period of time. Assuming the initial interval is 1 second, then the first retry interval is 1 second (2...). 0 Second).

[0199] If the first retry fails, the second retry interval becomes 2 seconds. 0 Second).

[0200] If the second retry also fails, the third retry will proceed with an interval of 4 seconds (2). 2 Second).

[0201] If the third retry still fails: the maximum number of attempts (3) has been reached, stop retrying and return "Task failed".

[0202] The interval here increases exponentially from 1 second to 2 seconds to 4 seconds, which is called "exponential backoff"; there are a maximum of 3 retries, that is, the total number of attempts is the initial 1 attempt + 3 retries = 4 attempts.

[0203] As the monitoring and management center of the system, the Status Manager provides visualized monitoring by summarizing various metrics based on database data: task-level progress (completed rows / total rows), resource-level status (total processing units / idle units / load rate), first verification results, and second verification results. These metrics can be displayed in real time through a web console. Furthermore, the Status Manager offers functions such as viewing task logs, pausing / resuming tasks, deleting tasks, and exporting results, providing a convenient and controllable operational view.

[0204] Figure 2 This is a schematic diagram illustrating the application of an information extraction system. Figure 2 In the process, the task configuration module receives the task configuration, encapsulates the document to be extracted and the configured parameters into a standardized task object, and sends it to the dynamic prompt generator. The dynamic prompt generator determines the industry category corresponding to the document to be extracted and matches it with the industry knowledge base from the database based on the industry category. Based on the matched industry knowledge base and preset extraction fields, it generates prompt words corresponding to the document to be extracted. When the prompt word manual editing function is enabled, a prompt word editing interface is provided, and the edited prompt words are input into the confidence verification processing unit. When the prompt word is valid, it is input into the adaptive scheduler. The adaptive scheduler groups at least one text to be processed, obtaining the text to be processed corresponding to each processing unit. The extraction tasks corresponding to the text to be processed are assigned to the processing units in the large model working pool, thereby realizing the information extraction of each text to be processed. In addition, the large model working pool also records the processing results of each text to be processed in the database. Furthermore, the database also contains performance data of each processing unit (such as timeout count, timeout rate, response time, etc.) so that the adaptive scheduler can adjust the text to be processed corresponding to each processing unit based on the performance data. The status manager aggregates various metrics based on different types of data in the database to monitor each extraction task.

[0205] The dynamic prompt generator in the information extraction system of this application embodiment can automatically generate structured information prompts based on the text in the table list and industry knowledge base rules, lowering the system's usage threshold, enabling flexible adaptation to different documents, and retaining a manual intervention entry point to address the limitations of extreme modes of complete automation and complete manual operation. The confidence verification processing unit achieves pre-quality control through a dual mechanism: pre-set item verification uses semantic consistency judgment to solve the problem that traditional string comparison cannot handle synonymous expressions (e.g., "3 days" and "72 hours" can match correctly); sampling and repeated verification detects the stability of the results through three independent calls, intercepting large-scale errors caused by prompt ambiguity in advance. Pre-verification of prompts ensures the quality and stability of model extraction beforehand, avoiding resource and time waste caused by discovering anomalies after large-scale model calls. The adaptive scheduler, based on a dynamic adjustment mechanism using 30-minute window performance data, dynamically expands and shrinks the number of texts to be processed according to the text processing latency, fully utilizing model capabilities and improving processing efficiency while ensuring processing effectiveness. In addition, by matching the number of texts to be processed with the model's processing capabilities in real time, the system can cope with performance fluctuations caused by different time periods and document types, and maintain continuous and efficient operation.

[0206] This application also provides an information extraction device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0207] This embodiment provides an information extraction device, such as... Figure 3 As shown, it includes:

[0208] The first acquisition module 301 is used to acquire a document to be extracted, wherein the document to be extracted includes at least one text to be processed and at least one preset extraction field;

[0209] The determination module 302 is used to input all preset extraction fields into the pre-built prediction model to obtain the predicted industry category corresponding to the document to be extracted;

[0210] The second acquisition module 303 is used to acquire industry terms corresponding to the predicted industry category from a preset database based on the predicted industry category.

[0211] The generation module 304 is used to generate prompt words corresponding to the document to be extracted based on all preset extraction fields and industry terms;

[0212] The extraction module 305 is used to extract information from each text to be processed in the document to be extracted based on the prompt words and each preset extraction field, and to obtain the content of the extraction field corresponding to each text to be processed.

[0213] In one possible implementation, the document to be extracted also includes at least one reference text, and the preset field content corresponding to each preset extraction field for all reference texts; the device also includes a verification module.

[0214] The verification module is used to input the prompt words, each preset extraction field and the first reference text into the pre-built information extraction model to obtain the extraction field content corresponding to the first reference text in each preset extraction field, wherein the first reference text is any one of at least one reference text;

[0215] After obtaining the content of the extraction field corresponding to each preset extraction field of all reference texts, the prompt words are verified based on the content of the extraction field corresponding to each preset extraction field of all reference texts and the preset content, and the first verification result is obtained.

[0216] Randomly select a first number of target texts from at least one text to be processed;

[0217] According to the preset number of loops, the prompt words, each preset extraction field and each target text are input into the information extraction model in each loop to obtain the content of the extraction field corresponding to each target text in each preset extraction field in each loop.

[0218] The prompt words are verified based on the content of each preset extraction field obtained in each loop for all target texts, and a second verification result is obtained.

[0219] Based on the first and second verification results, the validity of the prompt words is verified. After confirming that the prompt words are valid, information is extracted from each text to be processed in the document to be extracted according to the prompt words and each preset extraction field, so as to obtain the extraction field content corresponding to each text to be processed.

[0220] In one possible implementation, the verification module is specifically used to determine the semantic similarity corresponding to the first preset extraction field based on the preset field content and extraction field content corresponding to the first preset extraction field of the first reference text, wherein the first preset extraction field is any one of the preset extraction fields;

[0221] Once the semantic similarity of all reference texts in each preset extraction field is determined, the first verification result is determined based on the semantic similarity of all reference texts in each preset extraction field.

[0222] In one possible implementation, the verification module is specifically used to count the number of first-class fields corresponding to the first reference text, wherein the first-class fields are preset extraction fields whose semantic similarity is greater than a preset similarity threshold;

[0223] After counting the number of first-class fields for all reference texts, count the number of target reference texts. Target reference texts are those in which the number of first-class fields is greater than the number of second-class fields.

[0224] The effectiveness of a prompt is determined based on the number of target reference texts and the total number of all reference texts.

[0225] In one possible implementation, the verification module is specifically used to determine that the prompt word is valid when the ratio between the number of target reference texts and the total number is greater than a first ratio threshold.

[0226] In one possible implementation, the verification module is specifically used to count the number of second-type fields in the first target text based on the content of the extracted fields corresponding to each preset extracted field obtained in each loop of the first target text. The second-type fields are preset extracted fields whose content of the extracted fields corresponding to the first target text is inconsistent in different loops.

[0227] After determining the number of second-type fields in all target texts, the second verification result is determined based on the number of second-type fields in all target texts.

[0228] In one possible implementation, the validation module is specifically used to filter target texts from all target texts where the number of second-category fields is greater than the number of third-category fields;

[0229] The second verification result is determined based on the number of target texts in the second category being greater than the number in the third category.

[0230] In one possible implementation, the information extraction device includes multiple processing units, each of which is equipped with an information extraction model.

[0231] In one possible implementation, the extraction module 305 is specifically used to obtain the status identifiers corresponding to each processing unit.

[0232] Select the target processing unit whose status is marked as idle from all processing units;

[0233] At least one text to be processed is grouped to obtain texts to be processed corresponding to each target processing unit;

[0234] The prompt word, the preset extraction field, and the text to be processed corresponding to the first target processing unit are input into the information extraction model of the first target processing unit to obtain the extraction field content of the text to be processed corresponding to the first target processing unit. The first target processing unit is one of the target processing units whose status is marked as idle.

[0235] In one possible implementation, the device further includes: an adjustment module, used to obtain the number of historical texts processed and the number of timeouts corresponding to the first target processing unit within a preset historical time period based on the current time.

[0236] The timeout rate corresponding to the first target processing unit is determined based on the number of historical texts processed and the number of timeouts.

[0237] When the timeout rate is greater than the preset timeout rate, the number of texts to be processed corresponding to the first target processing unit is reduced according to the preset quantity adjustment rules.

[0238] In one possible implementation, the adjustment module is also used to obtain the processing time corresponding to each historical text.

[0239] Based on the number of historical texts and the processing time corresponding to each historical text, the average response time corresponding to the first target processing unit is determined.

[0240] Obtain the historical average response time corresponding to the first target processing unit;

[0241] When the average response time is less than the historical average response time, the response time difference corresponding to the first target processing unit is determined based on the average response time and the historical average response time.

[0242] When the response time difference is greater than the historical average response time of the preset ratio, and the timeout rate is less than or equal to the preset timeout rate, the number of texts to be processed corresponding to the first target processing unit is increased according to the preset quantity adjustment rules.

[0243] In one possible implementation, the generation module 304 is also used to obtain multiple historical documents;

[0244] Select the target historical document that corresponds to the document to be extracted from multiple historical documents;

[0245] Retrieve historical suggestion words corresponding to the target historical document;

[0246] Based on preset extraction fields, industry terms, and historical suggestion words, generate suggestion words corresponding to the document to be extracted.

[0247] In one possible implementation, the generation module 304 is also used to input all preset extraction fields, industry terms and historical prompt words into a pre-built prompt word generation model to generate prompt words.

[0248] For a description of the features in the embodiment corresponding to the information extraction device, please refer to the relevant description of the embodiment corresponding to the information extraction method, which will not be repeated here.

[0249] Embodiments of this application also provide an electronic device, such as... Figure 4 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described information extraction method embodiments.

[0250] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described information extraction method embodiments when it is run.

[0251] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0252] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described information extraction method embodiments.

[0253] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described information extraction method embodiments.

[0254] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0255] The foregoing has provided a detailed description of the information extraction method, electronic device, storage medium, and program product provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. An information extraction method characterized by, The method is applied to an information extraction device and comprises the following steps: acquiring a document to be extracted, wherein the document to be extracted comprises at least one text to be processed and at least one preset extraction field; inputting all the preset extraction fields into a pre-built prediction model to obtain a predicted industry category corresponding to the document to be extracted; acquiring, according to the predicted industry category, an industry term corresponding to the predicted industry category from a preset database; generating a prompt word corresponding to the document to be extracted according to all the preset extraction fields and the industry term; respectively extracting information from each text to be processed included in the document to be extracted according to the prompt word and each preset extraction field to obtain extraction field content corresponding to each text to be processed.

2. The method of claim 1, wherein, The document to be extracted further comprises at least one reference text and preset field content corresponding to each reference text in each preset extraction field; before the step of respectively extracting information from each text to be processed included in the document to be extracted according to the prompt word and each preset extraction field to obtain extraction field content corresponding to each text to be processed, the method further comprises the following steps: inputting the prompt word, each preset extraction field and a first reference text into a pre-built information extraction model to acquire extraction field content corresponding to each preset extraction field of the first reference text, wherein the first reference text is any one of the at least one reference text; after acquiring extraction field content corresponding to each preset extraction field of all the reference texts, verifying the prompt word according to the extraction field content and preset field content corresponding to each preset extraction field of all the reference texts to obtain a first verification result; randomly selecting a first number of target texts from the at least one text to be processed; inputting the prompt word, each preset extraction field and each target text into the information extraction model in each cycle according to a preset number of cycles to acquire extraction field content corresponding to each preset extraction field of each target text in each cycle; verifying the prompt word according to the extraction field content corresponding to each preset extraction field of all the target texts acquired in each cycle to obtain a second verification result; verifying the validity of the prompt word according to the first verification result and the second verification result, so as to extract information from each text to be processed included in the document to be extracted according to the prompt word and each preset extraction field to obtain extraction field content corresponding to each text to be processed after determining that the prompt word is valid.

3. The method of claim 2, wherein, The when all the reference texts in each of the preset extraction fields respectively corresponding extraction field content is obtained, according to all the reference texts in each of the preset extraction fields respectively corresponding extraction field content and the preset field content, the prompt word is verified, and a first verification result is obtained, including: According to the first reference text in the first preset extraction field corresponding to the preset field content and the extraction field content, the semantic similarity corresponding to the first preset extraction field is determined, wherein the first preset extraction field is any one of the preset extraction fields; When determining the semantic similarity corresponding to each of the preset extraction fields of all the reference texts, the first verification result is determined according to the semantic similarity corresponding to each of the preset extraction fields of all the reference texts.

4. The method of claim 3, wherein, The when determining the semantic similarity corresponding to each of the preset extraction fields of all the reference texts, the first verification result is determined according to the semantic similarity corresponding to each of the preset extraction fields of all the reference texts, including: The number of the first type field corresponding to the first reference text is counted, wherein the first type field is a preset extraction field with a semantic similarity greater than a preset similarity threshold; When the number of the first type field corresponding to all the reference texts is counted respectively, the number of target reference texts is counted, wherein the target reference text is a reference text in which the number of the first type field is greater than a second number among all the reference texts; According to the number of the target reference texts and the total number of all the reference texts, it is determined whether the prompt word is effective.

5. The method of claim 4, wherein, The according to the number of the target reference texts and the total number of all the reference texts, it is determined whether the prompt word is effective, including: When the ratio between the number of the target reference texts and the total number is greater than a first proportion threshold, it is determined that the prompt word is effective.

6. The method according to any one of claims 2-5, characterized in that, The according to the extraction field content corresponding to each of the preset extraction fields respectively obtained by all the target texts in each of the cycles, the prompt word is verified, and a second verification result is obtained, including: According to the extraction field content corresponding to each of the preset extraction fields respectively obtained by the first target text in each of the cycles, the number of the second type field in the first target text is counted, wherein the second type field is a preset extraction field with inconsistent extraction field content corresponding to the first target text in different cycles; When the number of the second type field in all the target texts is determined respectively, the second verification result is determined according to the number of the second type field in all the target texts.

7. The method of claim 6, wherein, The when the number of the second type field in all the target texts is determined respectively, the second verification result is determined according to the number of the second type field in all the target texts, including: From all the target texts, the target text with the number of the second type field greater than a third number is screened; According to the target text with the number of the second type field greater than the third number, the second verification result is determined. The according to the extraction field content corresponding to each of the preset extraction fields respectively obtained by all the target texts in each of the cycles, the prompt word is verified, and a second verification result is obtained, including: According to the extraction field content corresponding to each of the preset extraction fields respectively obtained by the first target text in each of the cycles, the number of the second type field in the first target text is counted, wherein the second type field is a preset extraction field with inconsistent extraction field content corresponding to the first target text in different cycles; When the number of the second type field in all the target texts is determined respectively, the second verification result is determined according to the number of the second type field in all the target texts. The when the number of the second type field in all the target texts is determined respectively, the second verification result is determined according to the number of the second type field in all the target texts, including: From all the target texts, the target text with the number of the second type field greater than a third number is screened; According to the target text with the number of the second type field greater than the third number, the second verification result is determined.

8. The method of claim 1, wherein, The information extraction device comprises a plurality of processing units, each of which is provided with an information extraction model; and each of the to-be-extracted documents is subjected to information extraction according to the prompt word and each of the preset extraction fields, so as to obtain the extraction field content corresponding to each of the to-be-processed texts. Obtain the state identifier corresponding to each of the processing units; Select a target processing unit with an idle identifier from the processing units; Group at least one of the to-be-processed texts to obtain the to-be-processed texts corresponding to each of the target processing units; Input the prompt word, the preset extraction field, and the to-be-processed texts corresponding to the first target processing unit into the information extraction model of the first target processing unit to obtain the extraction field content of the to-be-processed texts corresponding to the first target processing unit, wherein the first target processing unit is one of the target processing units with an idle identifier.

9. The method of claim 8, wherein, The method further comprises: Obtain the number of processed historical texts and the number of timeout times of the first target processing unit within a preset historical time period based on the current time; Determine the timeout rate of the first target processing unit according to the number of processed historical texts and the number of timeout times; When the timeout rate is greater than a preset timeout rate, adjust the number of to-be-processed texts corresponding to the first target processing unit according to a preset number adjustment rule.

10. The method of claim 9, wherein, The method further comprises: Obtain the processing time corresponding to each of the historical texts; Determine the average response time of the first target processing unit according to the number of historical texts and the processing time corresponding to each of the historical texts; Obtain the historical average response time corresponding to the first target processing unit; When the average response time is less than the historical average response time, determine the response time difference value of the first target processing unit according to the average response time and the historical average response time; When the response time difference value is greater than a preset proportion of the historical average response time, and the timeout rate is less than or equal to the preset timeout rate, increase the number of to-be-processed texts corresponding to the first target processing unit according to the preset number adjustment rule.

11. The method of claim 1, wherein, The method further comprises: Obtain a plurality of historical documents; Select a target historical document corresponding to the to-be-extracted document from the plurality of historical documents; Obtain a historical prompt word corresponding to the target historical document; Generate a prompt word corresponding to the to-be-extracted document according to the preset extraction field, the industry term, and the historical prompt word.

12. The method of claim 11, wherein, The method further comprises: Input all the preset extraction fields, the industry term, and the historical prompt word into a pre-constructed prompt word generation model to generate the prompt word.

13. An electronic device, comprising: The method further comprises: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the information extraction method according to any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the information extraction method according to any one of claims 1-12.

15. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the information extraction method according to any one of claims 1-12.

Images