A server-unaware information extraction method

Through the server-free information extraction method, the word embedding model matches the information extraction function, the problem of poor universality of the information extraction method in the existing technology is solved, the standardization of the information extraction results and algorithm reuse are realized, and multimodal information is improved Generality of extraction tasks.

CN118467734BActive Publication Date: 2024-11-12BEIJING BIG DATA ADVANCED TECH RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410632156.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-21
Publication Date
2024-11-12
Estimated Expiration
2044-05-21

AI Technical Summary

Technical Problem

The existing information extraction methods are mainly designed for specific fields, resulting in poor universality and the inability to achieve effective universality of multimodal information extraction tasks.

Method used

The server-free information extraction method is adopted, and the original data files and information extraction templates are obtained, and the word embedding model matches the information extraction function is used to generate and execute information extraction tasks to achieve standardization of the information extraction results and Reuse.

Benefits of technology

It improves the universality of information extraction, eliminates the problem that information algorithms are difficult to reuse in different modalities, realizes the standardized execution of information extraction algorithms and the reuse of functions, and blocks the heterogeneity of the algorithm implementation method and operating environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118467734B_ABST
    Figure CN118467734B_ABST
Patent Text Reader

Abstract

The present application provides a server-unaware information extraction method, which relates to the technical field of information extraction. The method comprises: obtaining an original data file and an information extraction template name; performing standardized preprocessing on the original data file and converting it into a standard format file; retrieving a target template from a database according to the information extraction template name, wherein the target template comprises at least: a field, an explanatory text for each field, and remark information for each field; obtaining one or more matching candidate information extraction functions for each field in the target template; determining a target information extraction function from the candidate information extraction functions; generating an information extraction task according to the target information extraction function for each field; and executing the information extraction task using the standard format file to obtain an information extraction result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of information extraction, and in particular to a server-unaware information extraction method. Background Art

[0002] The goal of Universal Information Extraction is to automatically extract structured information from unstructured text. That is, the input of Universal Information Extraction is specific extraction requirements and the text to be extracted, and the output is a structured knowledge structure, including but not limited to text entity structure, relationship structure between entities, and multi-emotion structure. Universal Information Extraction is a key task in knowledge graph construction and natural language understanding.

[0003] However, existing research work on information extraction mainly proposes a set of unique information extraction algorithms for each specialized field, and implements information extraction in a case-by-case manner, which leads to poor versatility and fails to achieve the purpose of better information extraction.

[0004] Therefore, there is an urgent need to propose a server-unaware information extraction method to achieve universality for multimodal information extraction tasks. Summary of the invention

[0005] In view of the above problems, an embodiment of the present application provides a server-unaware information extraction method to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect of an embodiment of the present application, a server-unaware information extraction method is provided, which is applied to a server-unaware information extraction device, and the method includes:

[0007] Get the original data file and information extraction template name;

[0008] Performing standardization preprocessing on the raw data file and converting it into a standard format file;

[0009] Extracting a template name according to the information, retrieving a target template in a database, wherein the target template includes at least: fields, explanatory text of each field, and remark information of each field;

[0010] For each field in the target template, obtaining one or more matching candidate information extraction functions;

[0011] Determining a target information extraction function from the candidate information extraction functions;

[0012] Generate an information extraction task according to the target information extraction function of each field;

[0013] The standard format file is used to execute the information extraction task and obtain the information extraction result.

[0014] In a possible implementation, acquiring one or more matching candidate information extraction functions for each field in the target template includes:

[0015] Using a word embedding model, converting the explanatory text of the field in the target template into a first word vector;

[0016] Using the word embedding model, convert the function description text of each information extraction function in the algorithm function library into a second word vector;

[0017] Using a cosine similarity algorithm, respectively calculate the matching value between the first word vector and each of the second word vectors;

[0018] One or more information extraction functions with the highest matching values ​​are determined as the candidate information extraction functions.

[0019] In a possible implementation, the information extraction device includes a display module, and the determining of a target information extraction function from the candidate information extraction functions includes:

[0020] The name of the candidate information extraction function and the function function description text are displayed through the display module;

[0021] Instruction information sent by a user is received, wherein the instruction information includes a target information extraction function selected by the user from the candidate information extraction functions.

[0022] In a possible implementation, determining a target information extraction function from the candidate information extraction functions includes:

[0023] The candidate information extraction function with the highest matching value is determined as the target information extraction function.

[0024] In a possible implementation, the using the standard format file to perform the information extraction task and obtain the information extraction result includes:

[0025] Using the standard format file as a parameter, accessing the target information extraction functions in the information extraction task one by one, and obtaining the return result of each field;

[0026] The returned results of all fields are integrated into the information extraction result.

[0027] In a possible implementation, the target information extraction function is stored in a Server less system, and accessing the target information extraction functions in the information extraction task one by one to obtain the return result of each field includes:

[0028] Sending an HTTP request to the Serverless system to access the target information extraction function in the Serverless system, wherein the HTTP request at least includes the standard format file;

[0029] An HTTP response returned by the Serverless system is received, wherein the HTTP response includes: a return result of a field corresponding to the target information extraction function.

[0030] In a possible implementation, the word embedding model is trained by the following steps:

[0031] Acquire a training sample set, the training sample set comprising: a plurality of sample pairs consisting of an explanatory text of a sample field and a function description text of a sample information extraction function, and a label of each sample pair, the label being used to indicate whether the sample field matches the sample information extraction function;

[0032] Selecting a target sample pair from the training sample set;

[0033] Inputting the explanatory text of the sample field in the target sample pair into the initialization model to obtain a first sample word vector;

[0034] Inputting the function description text in the target sample pair into the initialization model to obtain a second sample word vector;

[0035] Calculating a matching value between the first sample word vector and the second sample word vector by using a cosine similarity algorithm;

[0036] Calculating a loss function value according to the matching value and the label of the sample pair;

[0037] According to the loss function value, updating the parameters of the initialization model;

[0038] Reselect the target sample pair from the training sample set to train the initialization model until the loss function value converges, thereby obtaining a trained word embedding model.

[0039] In a possible implementation, the remark information of each field indicates whether the field is a mandatory field or a non-mandatory field, and the step of obtaining one or more matching candidate information extraction functions for each field in the target template further includes:

[0040] In the case where the remark information indicates that the field is a required field, when there is no candidate information extraction function matching the field in the algorithm function library, a prompt message is sent to the user to terminate the current information extraction;

[0041] In the case where the remark information indicates that the field is a non-mandatory field, when there is no candidate information extraction function matching the field in the algorithm function library, the field is ignored and the search for the candidate information extraction function matching the next field continues.

[0042] The second aspect of the embodiment of the present application further provides a server-unaware information extraction device, the device comprising:

[0043] File acquisition module, used to obtain the original data file and information extraction template name;

[0044] A standardization preprocessing module, used for performing standardization preprocessing on the raw data file and converting it into a standard format file;

[0045] A template retrieval module is used to extract a template name according to the information, and retrieve a target template in a database, wherein the target template includes at least: fields, explanatory text of each field, and remark information of each field;

[0046] A candidate information extraction function acquisition module, used to acquire one or more matching candidate information extraction functions for each field in the target template;

[0047] A target information extraction function configuration module, used to determine a target information extraction function from the candidate information extraction functions;

[0048] A task generation module, used to generate an information extraction task according to the target information extraction function of each field;

[0049] The execution module is used to use the standard format file to execute the information extraction task and obtain the information extraction result.

[0050] In a possible implementation, the candidate information extraction function acquisition module includes:

[0051] A first word vector conversion submodule, configured to convert the explanatory text of the field in the target template into a first word vector by using a word embedding model;

[0052] A second word vector conversion submodule, used to convert the function description text of each information extraction function in the algorithm function library into a second word vector by using the word embedding model;

[0053] A matching value calculation submodule, used to calculate the matching value between the first word vector and each of the second word vectors respectively by using a cosine similarity algorithm;

[0054] The candidate information extraction function determination module is used to determine one or more information extraction functions with the highest matching values ​​as the candidate information extraction functions.

[0055] In a possible implementation, the target information extraction function configuration module includes:

[0056] A display submodule, used for displaying the name of the candidate information extraction function and the function function description text through the display module;

[0057] The first determination submodule is used to receive indication information sent by a user, wherein the indication information includes a target information extraction function selected by the user from the candidate information extraction functions.

[0058] In a possible implementation, the target information extraction function configuration module includes:

[0059] The second determination submodule is used to determine the candidate information extraction function with the highest matching value as the target information extraction function.

[0060] In a possible implementation, the execution module includes:

[0061] A first execution submodule is used to use the standard format file as a parameter, access the target information extraction functions in the information extraction task one by one, and obtain the return result of each field;

[0062] The second execution submodule is used to integrate the returned results of all fields into the information extraction result.

[0063] In a possible implementation manner, the target information extraction function is stored in a Serverless system, and the first execution submodule includes:

[0064] A first execution unit is configured to send an HTTP request to the Serverless system to access the target information extraction function in the Serverless system, wherein the HTTP request includes at least the standard format file;

[0065] The second execution unit is used to receive the HTTP response returned by the Serverless system, wherein the HTTP response includes: the return result of the field corresponding to the target information extraction function.

[0066] In a possible implementation, the device further includes a model training module, configured to obtain the word embedding model through training by the following steps:

[0067] Acquire a training sample set, the training sample set comprising: a plurality of sample pairs consisting of an explanatory text of a sample field and a function description text of a sample information extraction function, and a label of each sample pair, the label being used to indicate whether the sample field matches the sample information extraction function;

[0068] Selecting a target sample pair from the training sample set;

[0069] Inputting the explanatory text of the sample field in the target sample pair into the initialization model to obtain a first sample word vector;

[0070] Inputting the function description text in the target sample pair into the initialization model to obtain a second sample word vector;

[0071] Calculating a matching value between the first sample word vector and the second sample word vector by using a cosine similarity algorithm;

[0072] Calculating a loss function value according to the matching value and the label of the sample pair;

[0073] According to the loss function value, updating the parameters of the initialization model;

[0074] Reselect the target sample pair from the training sample set to train the initialization model until the loss function value converges, thereby obtaining a trained word embedding model.

[0075] In a possible implementation manner, the remark information of each field indicates whether the field is a mandatory field or a non-mandatory field, and the candidate information extraction function acquisition module further includes:

[0076] A prompt information sending submodule is used to send a prompt information to the user to terminate the current information extraction when there is no candidate information extraction function matching the field in the algorithm function library when the field indicated by the remark information is a required field;

[0077] The search submodule is used to ignore the field when the remark information indicates that the field is a non-mandatory field and there is no candidate information extraction function matching the field in the algorithm function library, and continue to search for a candidate information extraction function matching the next field.

[0078] The third aspect of the embodiments of the present application also provides an electronic device, including a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps in the server-unaware information extraction method described in the first aspect of the embodiments of the present application.

[0079] The fourth aspect of the embodiment of the present application also provides a computer-readable storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the server-unaware information extraction method described in the first aspect of the embodiment of the present application are implemented.

[0080] The fifth aspect of the embodiments of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor implements the steps in the server-unaware information extraction method described in the first aspect of the embodiments of the present application.

[0081] A server-unaware information extraction method provided in an embodiment of the present application is applied to a server-unaware information extraction device, the method comprising: obtaining an original data file and an information extraction template name; performing standardization preprocessing on the original data file and converting it into a standard format file; retrieving a target template from a database according to the information extraction template name, the target template comprising at least: a field, an explanatory text for each field, and remark information for each field; obtaining one or more matching candidate information extraction functions for each field in the target template; determining a target information extraction function from the candidate information extraction functions; generating an information extraction task according to the target information extraction function of each field; and executing the information extraction task using the standard format file to obtain an information extraction result.

[0082] The specific beneficial effects are:

[0083] On the one hand, the embodiment of the present application designs a target template, which is a standardized expression of the information extraction result. The target template includes at least: fields, explanatory text for each field, and remark information for each field. Users can select different templates according to actual needs, and convert the information extraction process of a complete data file into an independent extraction process for multiple fields, thereby eliminating the problem that information extraction algorithms for different modal information are difficult to reuse, realizing standardized execution of information extraction, and improving the versatility of information extraction.

[0084] On the other hand, the embodiment of the present application decomposes the information extraction algorithm into information extraction functions for different fields, and decomposes the complete execution process of an information extraction algorithm into the sequential execution process of multiple information extraction functions (target information extraction functions), so as to utilize the target information extraction function to extract the required fields from the standard format file, thereby realizing the information extraction function as the minimum granularity, supporting the reuse of information extraction algorithms, shielding the heterogeneity of the implementation methods and operating environments of the information extraction algorithms, and improving the versatility of information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0086] Figure 1 It is a flow chart of the steps of a server-unaware information extraction method provided in an embodiment of the present application;

[0087] Figure 2 It is a structural schematic diagram of a server-unaware information extraction device provided in an embodiment of the present application;

[0088] Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0090] The goal of general information extraction is to automatically extract structured information from unstructured text. This information may include but is not limited to text entity structure, relationship structure between entities, and multi-emotion structure. Taking entity relationship information extraction as an example, given the sentence "In 1997, user A served as the general manager of company B", an information extraction system should be able to recognize that the sentence belongs to an "employment" event, the trigger word of the event is "served", the argument structure is "user A" (subject), "company B" (object) and "1997" (event); the three entities are company B (company), user A (person), 1997 (time); and there is also a relationship, that is, "user A" works for "company B". General information extraction is a key task in knowledge graph construction and natural language understanding.

[0091] Specifically, the input of general information extraction is a specific extraction requirement and the text to be extracted, and the output is a structured knowledge structure. For example, for the following input text: "In 1997, user A served as the general manager of company B", the extraction requirements are person (Person) and work relationship (Work for), and the final extraction result is (person, user A, work relationship, general manager, company B). Different extraction requirements will cause the general information extraction model to extract different new structured knowledge.

[0092] However, in the field of information extraction, a large number of research works mainly propose a set of information extraction algorithms for a specific field, and implement information extraction in a one-by-one manner, which leads to poor versatility and fails to achieve the purpose of better information extraction. There are also some information extraction algorithms that can recognize multimodal information inputs, but they do not treat multiple inputs equally and do not achieve the versatility of output results. Specifically, relevant information extraction methods mainly include: sequence labeling methods, text block classification methods, and reading comprehension methods. However, these methods are usually designed for specific tasks, resulting in the following problems: (1) Specific architectures need to be used for different information extraction tasks. Developers need to design and develop specific architectures for many different information extraction tasks, settings, and scenarios, which is a lot of work; (2) Multiple independent models need to be trained for different data sets. Learning isolated models for different data and tasks severely limits knowledge sharing between related tasks and similar scenarios; (3) Multiple different data sources need to be labeled for different scenarios. Building data sets and knowledge sources specifically for specific information extraction tasks is both expensive and time-consuming. The low versatility of these task-specific information extraction solutions greatly hinders the rapid development and adaptation of information extraction systems.

[0093] In view of the above problems, the embodiment of the present application proposes a server-unaware information extraction method to improve the versatility of multimodal information extraction tasks. The information extraction method provided by the embodiment of the present application is described in detail below through some embodiments and their application scenarios in conjunction with the accompanying drawings.

[0094] The first aspect of the embodiment of the present application provides a server-unaware information extraction method, which is applied to an information extraction device, referring to Figure 1 , Figure 1 A flow chart of the steps of a server-unaware information extraction method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0095] Step S101, obtaining the original data file and the information extraction template name.

[0096] Specifically, the server receives the original data file and the information extraction template name input by the user through the non-perception interface. In this embodiment, the original data file is a file used for information extraction, which can be at least one of a text data file, an audio data file, and a video data file. When uploading the original data file, the user selects the corresponding information extraction template name according to his actual needs, that is, selects the template based on which the information extraction is based.

[0097] Server-aware computing is an emerging technology in the field of cloud computing, which realizes the cloud computing paradigm of function as a service (task). In the server-aware computing scenario, users use high-level programming languages ​​to write business logic functions on the server-aware computing platform and specify trigger events. On this basis, the information extraction device of server-aware computing will timely schedule and execute the corresponding function task (i.e., information extraction function) in response to the trigger event, shielding the user from the underlying system and hardware details. Users do not need to allocate specific computing resources and storage resources specifically for the requested information extraction task, and do not need to specify the working node for executing the information extraction task. The information extraction device of server-aware computing will automatically build a running environment for the task, including creating a container and loading code and related libraries, and then execute the information extraction task, which greatly improves the ease of use.

[0098] Step S102, performing standardization preprocessing on the original data file and converting it into a standard format file.

[0099] Specifically, there are multiple possible formats of the original data file. For example, for the original data file of text type, it can be any one of word format file, pdf format file, and txt format file. If a set of information extraction algorithms are customized for each format of the original data file, the storage cost will be too expensive. If the original data is not preprocessed, the same information extraction algorithm (such as information extraction algorithm A) is used to extract information, such as extracting the title of the text, and a special information extraction algorithm needs to be stored for each original file format. For example, an information extraction algorithm A-1 is stored for word format files, an information extraction algorithm A-2 is stored for pdf format files, and an information extraction algorithm A-3 is stored for txt format files, which will cause excessive redundancy of the storage algorithm.

[0100] In this embodiment, the original data file is parsed and converted into a standard format file. In order to be able to identify original data files of different format types, improve the versatility of information extraction, and realize the reuse of information extraction algorithms, the embodiment of the present application proposes to first perform standardized preprocessing on the original data file, that is, to perform a series of data type format conversions on the original data file. After reading the original data file, the type of the file is obtained according to the suffix identification, and then accessed. According to its file format type, a specific standardized function is selected to convert the original data file into a unified standard mode, that is, a standard format file. Exemplarily, for text-type original files, they are uniformly converted into files in txt format, in which string strings are stored.

[0101] Step S103, extracting a template name according to the information, and retrieving a target template in a database, wherein the target template at least includes: fields, explanatory text of each field, and remark information of each field.

[0102] In this embodiment, according to the name of the information extraction template input by the user, a template search is performed in the template file table in the database to obtain a target template. The main goal of information extraction is to automatically extract structured information from unstructured text, that is, information extraction for different fields. The templates can be pre-stored in the corresponding database, each template includes one or more three-dimensional tuples, each three-dimensional tuple includes: the name of the field, the explanatory text of the field and the remark information of the field, and the remark information of the field is used to indicate whether the field is a required field. Exemplarily, referring to Table 1, Table 1 shows a target template.

[0103] Field Name Explanatory text Remarks Keywords Extract 5 keywords from the article Required author Extract the author of the article Required name Extract the title of the article Required

[0104] Table 1

[0105] As shown in Table 1, the target template is a simple template for information extraction from papers. The template contains three triples, including the names of three fields, "keywords", "authors" and "titles", as well as the meanings of the three fields. Each field can be represented by a string, and these three fields must be filled in. Using the template shown in Table 1, keywords, authors and titles can be extracted from papers (original data files). Therefore, the user selects the corresponding template, inputs the name of the selected information extraction template into the information extraction device, and retrieves the corresponding target template from the database, so as to use the target template to extract the information (structured information) of the required fields from the original data file, and realize the standardization of the information extraction results.

[0106] Step S104: for each field in the target template, obtain one or more matching candidate information extraction functions.

[0107] Specifically, after the target template required for this information extraction task is determined through step S103, a possible matching information extraction algorithm is obtained for each field in the target template (the name of the information extraction function matching the field is retrieved in the database) to define this information extraction task. The embodiment of the present application selects the information extraction function matched by each field in the target template, so as to use the function as the minimum granularity. Compared with the application of a complete information extraction algorithm (often composed of multiple functions) in the related art, it supports the reuse of the information extraction algorithm and improves the versatility of the information extraction algorithm.

[0108] In a possible implementation, the step S104, for each field in the target template, obtaining one or more matching candidate information extraction functions, includes:

[0109] Step S1041, using a word embedding model, converting the explanatory text of the field in the target template into a first word vector.

[0110] Step S1042, using the word embedding model, converting the function description text of each information extraction function in the algorithm function library into a second word vector.

[0111] Step S1043: Calculate the matching value between the first word vector and each of the second word vectors respectively by using a cosine similarity algorithm.

[0112] Step S1044: determine the one or more information extraction functions with the highest matching values ​​as the candidate information extraction functions.

[0113] In order to find the information extraction function that matches the field, the embodiment of the present application utilizes the pre-trained word embedding model Word2Vec algorithm to convert the explanatory text of the field in the target template into the first word vector, and convert the functional description text of the information extraction function into the second word vector. The explanatory text is used to represent the meaning of the field and to explain it in detail. The functional description text is used to describe the specific function of the information extraction function, for example, which type of field the information extraction function can be used to extract. Then, the matching value between each second word vector and the first word vector is calculated by the cosine similarity algorithm, and the information extraction function with a sufficiently high matching value is determined as the candidate information extraction function of the field. Optionally, a matching value threshold is pre-set, and the information extraction function with a matching value higher than the matching value threshold is determined as the candidate information extraction function of the field. Optionally, the matching value is sorted from high to low, and the k information extraction functions with the highest matching value are determined as the candidate information extraction functions of the field. Thus, each field in the target template is traversed to determine one or more candidate information extraction functions matched by each field.

[0114] In a possible implementation, the remark information of each field indicates whether the field is a mandatory field or a non-mandatory field, and the step S104, for each field in the target template, obtaining one or more matching candidate information extraction functions, further includes:

[0115] Step S1045, when the remark information indicates that the field is a required field and there is no candidate information extraction function matching the field in the algorithm function library, a prompt message is sent to the user to terminate the current information extraction.

[0116] Step S1046, when the remark information indicates that the field is a non-mandatory field and there is no candidate information extraction function matching the field in the algorithm function library, ignore the field and continue to search for a candidate information extraction function matching the next field.

[0117] In this embodiment, the remark information of each field indicates whether the field is a required field or a non-required field. When the remark information indicates that the field is a required field, through steps S1041-S1044, it is calculated that all matching values ​​are not high (for example, all are lower than the preset matching value threshold), that is, it is determined that there is no candidate information extraction function matching the field in the algorithm function library. Since the field is a required field, a prompt message is sent to the user, and the prompt message contains information about the field to prompt the user to select another information extraction algorithm to extract the field.

[0118] When the remark information indicates that the field is a non-mandatory field, through steps S1041-S1044, all the matching values ​​calculated are not high (for example, they are all lower than the preset matching value threshold), that is, it is determined that there is no candidate information extraction function matching the field in the algorithm function library. Since the field is a non-mandatory field, information extraction of the field can be omitted, and the field is skipped, and steps S1041-S1044 are re-executed for the next field in the target template until the candidate information extraction functions of all fields in the target template are determined.

[0119] In a possible implementation, the word embedding model is trained by the following steps:

[0120] Step S201, obtaining a training sample set, wherein the training sample set includes: a plurality of sample pairs consisting of an explanatory text of a sample field and a function description text of a sample information extraction function, and a label for each of the sample pairs, wherein the label is used to indicate whether the sample field matches the sample information extraction function.

[0121] Step S202: Select a target sample pair from the training sample set.

[0122] Step S203: input the explanatory text of the sample field in the target sample pair into the initialization model to obtain a first sample word vector.

[0123] Step S204: input the function description text in the target sample pair into the initialization model to obtain a second sample word vector.

[0124] Step S205: Calculate the matching value between the first sample word vector and the second sample word vector by using a cosine similarity algorithm.

[0125] Step S206: Calculate the loss function value according to the matching value and the label of the sample pair.

[0126] Step S207: update the parameters of the initialization model according to the loss function value.

[0127] Step S208, reselecting the target sample pair from the training sample set to train the initialization model until the loss function value converges, thereby obtaining a trained word embedding model.

[0128] Step S105: determining a target information extraction function from the candidate information extraction functions.

[0129] Specifically, an information extraction task is the execution of several information extraction functions. For each field, a target information extraction function is determined from the candidate information extraction functions that match it, and is used to execute the information extraction task corresponding to the field.

[0130] In a possible implementation, the information extraction device includes a display module, and the step S105 of determining a target information extraction function from the candidate information extraction functions includes:

[0131] Step S1051, displaying the name of the candidate information extraction function and the function function description text through the display module.

[0132] Step S1052: receiving instruction information sent by the user, wherein the instruction information includes a target information extraction function selected by the user from the candidate information extraction functions.

[0133] In this embodiment, the information extraction function can be manually configured by the user. Specifically, the field name of each field and the function description text of the candidate information extraction function corresponding to the field are sent to the user, or displayed on the display module, and the user manually selects through the display module. The user selects a function from the candidate information extraction function as the target information extraction function according to personal preference, generates indication information, and the information extraction device determines the target information extraction function of each field according to the indication information, thereby completing the configuration of the information extraction task.

[0134] In a possible implementation, the step S105 of determining a target information extraction function from the candidate information extraction functions includes:

[0135] Step S1053: determine the candidate information extraction function with the highest matching value as the target information extraction function.

[0136] Specifically, the information extraction device may automatically configure the information extraction function, and based on the matching value corresponding to each candidate information extraction function, preferentially select the candidate information extraction function with the highest matching value as the target information extraction function for executing the information extraction task.

[0137] Step S106: Generate an information extraction task according to the target information extraction function of each field.

[0138] Specifically, each field to be extracted is determined according to the target template, and an information extraction task is generated according to the target information extraction function matched by each field. The content of the information extraction task includes at least: a standard format file, a target template, and a target information extraction function selected for each field in the target template.

[0139] Step S107, using the standard format file to execute the information extraction task and obtain the information extraction result.

[0140] In a possible implementation, the step S107, using the standard format file to perform the information extraction task to obtain the information extraction result, includes:

[0141] Step S1071, using the standard format file as a parameter, accessing the target information extraction functions in the information extraction task one by one, and obtaining the return result of each field;

[0142] Step S1072, integrating the returned results of all fields into the information extraction result.

[0143] In this embodiment, when executing the information extraction task, the selected target information extraction algorithm function is accessed one by one, the parameter is the standard format file, and the return value is the information extraction result of the field. The return result of each field is recorded, and finally the complete result of the information extraction is returned to the user, that is, an information data stream containing all fields in the target template is returned.

[0144] In a possible implementation, the target information extraction function is stored in a serverless system, and the step S1071 accesses the target information extraction function in the information extraction task one by one to obtain the return result of each field, including:

[0145] Sending an HTTP request to the Serverless system to access the target information extraction function in the Serverless system, wherein the HTTP request at least includes the standard format file;

[0146] An HTTP response returned by the Serverless system is received, wherein the HTTP response includes: a return result of a field corresponding to the target information extraction function.

[0147] In this embodiment, the Serverless system is accessed based on the HTTP protocol to achieve information extraction. Specifically, Serverless is a cloud computing architecture model in which application developers can write and deploy code without having to manage underlying server resources. In the traditional server model, developers need to manage server configuration, scalability, and maintenance issues. In the Serverless architecture, these tasks are automatically handled by the cloud service provider. The Serverless architecture also has the advantages of elastic scalability, server-free management, high availability, fast development speed, and high flexibility.

[0148] The embodiment of the present application abstracts each information extraction algorithm into a function and stores it in the algorithm function library of the Serverless system. When executing step S104, the target information extraction function is retrieved and determined from the algorithm function library in the Serverless system for each field in the target template, and an information extraction task is generated. When executing the information extraction task, the information extraction device interacts with the Serverless platform based on the HTTP protocol, accesses the information extraction algorithm in the Serverless system through the function name, the HTTP request contains a standard format file, and the HTTP response contains the result of this information extraction. The data parsing and information extraction device based on the Serverless system can achieve a high degree of versatility in information extraction, that is, it can achieve the versatility of the algorithm extraction function, so that personalized customization functions can be realized according to user needs.

[0149] The embodiment of the present application designs a target template to realize the standardized expression of the information extraction result, and the target template includes at least: fields, explanatory texts of each field, and remarks of each field. Users can select different templates according to actual needs, and convert the information extraction process of a complete data file into an independent extraction process for multiple fields, thereby eliminating the problem that the information extraction algorithm of different modal information is difficult to reuse, realizing standardized execution of information extraction, and improving the versatility of information extraction. In addition, the embodiment of the present application disassembles the information extraction algorithm into information extraction functions for different fields, and disassembles the complete execution process of an information extraction algorithm into the sequential execution process of multiple information extraction functions (target information extraction functions), thereby using the target information extraction function to extract the required fields from the standard format file, realizing the information extraction function as the minimum granularity, supporting the reuse of the information extraction algorithm, shielding the implementation mode of the information extraction algorithm, and the heterogeneity of the operating environment, and improving the versatility of information extraction.

[0150] The second aspect of the embodiment of the present application further provides a server-unaware information extraction device, referring to Figure 2 , Figure 2 A schematic diagram of the structure of a server-unaware information extraction device is shown. Figure 2 As shown, the device comprises:

[0151] File acquisition module, used to obtain the original data file and information extraction template name;

[0152] A standardization preprocessing module, used for performing standardization preprocessing on the raw data file and converting it into a standard format file;

[0153] A template retrieval module is used to extract a template name according to the information, and retrieve a target template in a database, wherein the target template includes at least: fields, explanatory text of each field, and remark information of each field;

[0154] A candidate information extraction function acquisition module, used to acquire one or more matching candidate information extraction functions for each field in the target template;

[0155] A target information extraction function configuration module, used to determine a target information extraction function from the candidate information extraction functions;

[0156] A task generation module, used to generate an information extraction task according to the target information extraction function of each field;

[0157] The execution module is used to use the standard format file to execute the information extraction task and obtain the information extraction result.

[0158] In a possible implementation, the candidate information extraction function acquisition module includes:

[0159] A first word vector conversion submodule, configured to convert the explanatory text of the field in the target template into a first word vector by using a word embedding model;

[0160] A second word vector conversion submodule, used to convert the function description text of each information extraction function in the algorithm function library into a second word vector by using the word embedding model;

[0161] A matching value calculation submodule, used to calculate the matching value between the first word vector and each of the second word vectors respectively by using a cosine similarity algorithm;

[0162] The candidate information extraction function determination module is used to determine one or more information extraction functions with the highest matching values ​​as the candidate information extraction functions.

[0163] In a possible implementation, the target information extraction function configuration module includes:

[0164] A display submodule, used for displaying the name of the candidate information extraction function and the function function description text through the display module;

[0165] The first determination submodule is used to receive indication information sent by a user, wherein the indication information includes a target information extraction function selected by the user from the candidate information extraction functions.

[0166] In a possible implementation, the target information extraction function configuration module includes:

[0167] The second determination submodule is used to determine the candidate information extraction function with the highest matching value as the target information extraction function.

[0168] In a possible implementation, the execution module includes:

[0169] A first execution submodule is used to use the standard format file as a parameter, access the target information extraction functions in the information extraction task one by one, and obtain the return result of each field;

[0170] The second execution submodule is used to integrate the returned results of all fields into the information extraction result.

[0171] In a possible implementation manner, the target information extraction function is stored in a Serverless system, and the first execution submodule includes:

[0172] A first execution unit is configured to send an HTTP request to the Serverless system to access the target information extraction function in the Serverless system, wherein the HTTP request includes at least the standard format file;

[0173] The second execution unit is used to receive the HTTP response returned by the Serverless system, wherein the HTTP response includes: the return result of the field corresponding to the target information extraction function.

[0174] In a possible implementation, the device further includes a model training module, configured to obtain the word embedding model through training by the following steps:

[0175] Acquire a training sample set, the training sample set comprising: a plurality of sample pairs consisting of an explanatory text of a sample field and a function description text of a sample information extraction function, and a label of each sample pair, the label being used to indicate whether the sample field matches the sample information extraction function;

[0176] Selecting a target sample pair from the training sample set;

[0177] Inputting the explanatory text of the sample field in the target sample pair into the initialization model to obtain a first sample word vector;

[0178] Inputting the function description text in the target sample pair into the initialization model to obtain a second sample word vector;

[0179] Calculating a matching value between the first sample word vector and the second sample word vector by using a cosine similarity algorithm;

[0180] Calculating a loss function value according to the matching value and the label of the sample pair;

[0181] According to the loss function value, updating the parameters of the initialization model;

[0182] Reselect the target sample pair from the training sample set to train the initialization model until the loss function value converges, thereby obtaining a trained word embedding model.

[0183] In a possible implementation manner, the remark information of each field indicates whether the field is a mandatory field or a non-mandatory field, and the candidate information extraction function acquisition module further includes:

[0184] A prompt information sending submodule is used to send a prompt information to the user to terminate the current information extraction when there is no candidate information extraction function matching the field in the algorithm function library when the field indicated by the remark information is a required field;

[0185] The search submodule is used to ignore the field when the remark information indicates that the field is a non-mandatory field and there is no candidate information extraction function matching the field in the algorithm function library, and continue to search for a candidate information extraction function matching the next field.

[0186] The present application also provides an electronic device, referring to Figure 3 , Figure 3 Schematic diagram of an electronic device proposed in an embodiment of the present application. Figure 3 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. A computer program is stored in the memory 110. The computer program can be run on the processor 120, thereby implementing the steps in the server-unaware information extraction method disclosed in the embodiment of the present application.

[0187] An embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the server-unaware information extraction method disclosed in the embodiment of the present application are implemented.

[0188] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor implements the steps of the server-unaware information extraction method disclosed in the embodiment of the present application.

[0189] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0190] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0191] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0193] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0194] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0195] The above is a detailed introduction to a server-unaware information extraction method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A server-unaware information extraction method, characterized in that: The method for extracting information without server perception includes: Get the original data file and information extraction template name; Performing standardization preprocessing on the raw data file and converting it into a standard format file; Extracting a template name according to the information, and retrieving a target template in a database, wherein the target template includes at least: fields, explanatory text of each field, and remark information of each field; For each field in the target template, obtaining one or more matching candidate information extraction functions; Determining a target information extraction function from the candidate information extraction functions; Generate an information extraction task according to the target information extraction function of each field; The standard format file is used to execute the information extraction task and obtain an information extraction result.

2. The server-unaware information extraction method according to claim 1, characterized in that: The step of obtaining one or more matching candidate information extraction functions for each field in the target template includes: Using a word embedding model, converting the explanatory text of the field in the target template into a first word vector; Using the word embedding model, convert the function description text of each information extraction function in the algorithm function library into a second word vector; Calculate the matching value between the first word vector and each of the second word vectors using a cosine similarity algorithm; The one or more information extraction functions with the highest matching values ​​are determined as the candidate information extraction functions.

3. The server-unaware information extraction method according to claim 1, characterized in that: The information extraction device includes a display module, and determining a target information extraction function from the candidate information extraction functions includes: Displaying the name of the candidate information extraction function and the function description text through the display module; Instruction information sent by a user is received, where the instruction information includes a target information extraction function selected by the user from the candidate information extraction functions.

4. The server-unaware information extraction method according to claim 1, characterized in that: Determining a target information extraction function from the candidate information extraction functions includes: The candidate information extraction function with the highest matching value is determined as the target information extraction function.

5. The server-unaware information extraction method according to claim 1, characterized in that: The step of using the standard format file to perform the information extraction task and obtain the information extraction result includes: Using the standard format file as a parameter, accessing the target information extraction function in the information extraction task one by one to obtain the return result of each field; The returned results of all fields are integrated into the information extraction result.

6. The server-unaware information extraction method according to claim 5, characterized in that: The target information extraction function is stored in the Serverless system, and the target information extraction function in the information extraction task is accessed one by one to obtain the return result of each field, including: Sending an HTTP request to the Serverless system to access the target information extraction function in the Serverless system, wherein the HTTP request includes at least the standard format file; Receive an HTTP response returned by the Serverless system, wherein the HTTP response includes: a return result of a field corresponding to the target information extraction function.

7. The server-unaware information extraction method according to claim 2, characterized in that: The word embedding model is trained through the following steps: Obtaining a training sample set, the training sample set comprising: a plurality of sample pairs consisting of an explanatory text of a sample field and a function description text of a sample information extraction function, and a label for each of the sample pairs, the label being used to indicate whether the sample field matches the sample information extraction function; Selecting a target sample pair from the training sample set; Inputting the explanatory text of the sample field in the target sample pair into the initialization model to obtain a first sample word vector; Inputting the function description text in the target sample pair into the initialization model to obtain a second sample word vector; Calculating a matching value between the first sample word vector and the second sample word vector using a cosine similarity algorithm; Calculating a loss function value according to the matching value and the label of the sample pair; According to the loss function value, the parameters of the initialization model are updated; Reselect the target sample pairs from the training sample set to train the initialization model until the loss function value converges, thereby obtaining a trained word embedding model.

8. The server-unaware information extraction method according to claim 2, characterized in that: The remark information of each field indicates whether the field is a required field or a non-required field, and obtaining one or more matching candidate information extraction functions for each field in the target template further includes: In the case where the remark information field is a required field, if there is no candidate information extraction function matching the field in the algorithm function library, a prompt message is sent to the user to terminate the information extraction; In the case that the remark information representation field is a non-mandatory field, when there is no candidate information extraction function matching the field in the algorithm function library, the field is ignored and the search continues for a candidate information extraction function matching the next field.