Information extraction method, device, equipment and storage medium
By combining model identification and standard vocabulary rule verification, order information in emails is extracted, and the problem of insufficient information extraction accuracy in the prior art is solved, achieving a more efficient information extraction effect.
Patent Information
- Application Number
- CN202111520172.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-12-13
AI Technical Summary
The existing email extraction methods have limitations in accuracy and cannot effectively extract information in diverse email content, resulting in inaccurate information extraction.
Combining model identification and standard vocabulary rule verification, by obtaining order data, using preset identification models to extract subject information, and based on standard vocabulary checksum correction, triple information of order data is finally generated.
Improve the accuracy and accuracy of information extraction, allowing users to quickly obtain key information in emails and save query time.
Smart Images

Figure CN114154480B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to an information extraction method, device, equipment and storage medium. Background Art
[0002] With the development of Internet technology, more and more goods are purchased through online orders, such as delivering order information through emails. For example, when a user places an order for a batch of goods on a platform, the order information is delivered via email.
[0003] The product information and arrival date in the order information are very important product data. When users want to check the product information and arrival date of related products in the email, they often need to open the email and search manually, which is very inconvenient for users. Therefore, the automatic information extraction technology of email content came into being.
[0004] In the existing email extraction methods, information is mainly extracted by writing rules, etc., but the extracted information has limitations and low accuracy. Due to the diversity of email content, it cannot meet the requirements of information extraction in any form. Therefore, how to improve the extraction accuracy of email content information has become a problem that needs to be solved urgently. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide an information extraction method, apparatus, device and storage medium, which simultaneously combines model recognition and standard vocabulary rule verification to extract order information and improve extraction accuracy.
[0006] A first aspect of an embodiment of the present application provides an information extraction method, comprising: obtaining order data corresponding to a query instruction; inputting the order data into a preset recognition model, and outputting subject matter information in the order data; verifying the subject matter information based on a standard vocabulary to obtain verified subject matter information; and generating triple information of the order data based on the verified subject matter information.
[0007] In one embodiment, the query instruction carries identification information of the target order; obtaining the order data corresponding to the query instruction includes: when the query instruction is received, extracting the order content corresponding to the identification information in a preset order library; parsing the order content to obtain text data of the target order, and using the text data as the order data.
[0008] In one embodiment, the steps of establishing the preset recognition model include: obtaining a sample order data set; converting the sample order data set into a predetermined standard format; labeling the sample subject matter information in the sample order data set in the standard format; and using the labeled sample order data set to train a neural network model to obtain the preset recognition model.
[0009] In one embodiment, the subject matter information includes: subject matter identification text and the text position of the identification text in the order data; the verification processing of the subject matter information based on the standard vocabulary to obtain the verified subject matter information includes: determining whether there is target standard data identical to the identification text in the standard vocabulary; when the target standard data does not exist in the standard vocabulary, correcting the identification text based on the text position to obtain the verified subject matter information.
[0010] In one embodiment, before determining whether there is target standard data identical to the identification text in the standard vocabulary, the method further includes: detecting character information at the boundary of the identification text, deleting non-text symbols at the boundary of the identification text, and obtaining a corrected identification text.
[0011] In one embodiment, the correction processing is performed on the identification text based on the text position to obtain the verified target object information, including: when the target standard data does not exist in the standard vocabulary, selecting target candidate data from the standard vocabulary whose similarity with the identification text is greater than a preset threshold; judging whether the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data; when the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data, using the target candidate data as the verified target object information.
[0012] In one embodiment, the correction processing of the identification text based on the text position to obtain the verified target object information also includes: when the spelling order of the target candidate data is different from the spelling order of the specified interval of the text position in the order data, the text content is extended along the text position boundary in the order data until a space symbol is encountered, and the text content obtained after the expansion and the new text position corresponding to the text content are used as the verified target object information.
[0013] In one embodiment, the method further includes: updating the verified target object information into the standard vocabulary.
[0014] In one embodiment, the target object information includes: a target object identification and date information corresponding to the target object; the generating of triple information of the order data based on the verified target object information includes: taking the target object identification and the date information as two entities respectively, and taking the type label and date label of the target object as the relationship between the two entities, to generate triple information of the order data.
[0015] The second aspect of an embodiment of the present application provides an information extraction device, comprising: an acquisition module, used to acquire order data corresponding to a query instruction; an identification module, used to input the order data into a preset recognition model, and output the subject matter information in the order data; a verification module, used to verify the subject matter information based on a standard vocabulary to obtain the verified subject matter information; and a generation module, used to generate triple information of the order data based on the verified subject matter information.
[0016] In one embodiment, the query instruction carries identification information of the target order; the acquisition module is used to: when receiving the query instruction, extract the order content corresponding to the identification information in the preset order library; parse the order content to obtain text data of the target order, and use the text data as the order data.
[0017] In one embodiment, it further includes: establishing a module for: obtaining a sample order data set;
[0018] The sample order data set is converted into a predetermined standard format; the sample subject matter information in the sample order data set in the standard format is labeled; and a neural network model is trained using the labeled sample order data set to obtain the preset recognition model.
[0019] In one embodiment, the subject matter information includes: subject matter identification text and the text position of the identification text in the order data; the verification module is used to: determine whether there is target standard data identical to the identification text in the standard vocabulary; when the target standard data does not exist in the standard vocabulary, correct the identification text based on the text position to obtain the verified subject matter information.
[0020] In one embodiment, before determining whether there is target standard data identical to the identification text in the standard vocabulary, the method further includes: detecting character information at the boundary of the identification text, deleting non-text symbols at the boundary of the identification text, and obtaining a corrected identification text.
[0021] In one embodiment, the correction processing is performed on the identification text based on the text position to obtain the verified target object information, including: when the target standard data does not exist in the standard vocabulary, selecting target candidate data from the standard vocabulary whose similarity with the identification text is greater than a preset threshold; judging whether the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data; when the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data, using the target candidate data as the verified target object information.
[0022] In one embodiment, the correction processing of the identification text based on the text position to obtain the verified target object information also includes: when the spelling order of the target candidate data is different from the spelling order of the specified interval of the text position in the order data, the text content is extended along the text position boundary in the order data until a space symbol is encountered, and the text content obtained after the expansion and the new text position corresponding to the text content are used as the verified target object information.
[0023] In one embodiment, it further includes: an updating module, configured to update the verified target object information into the standard vocabulary.
[0024] In one embodiment, the target object information includes: a target item identifier and date information corresponding to the target item; the generation module is used to: take the target item identifier and the date information as two entities respectively, and take the type label and date label of the target item as the relationship between the two entities, to generate triple information of the order data.
[0025] A third aspect of an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the method of the first aspect of an embodiment of the present application and any one of its embodiments.
[0026] The fourth aspect of the embodiments of the present application provides a non-transitory electronic device readable storage medium, including: a program, which, when executed by an electronic device, enables the electronic device to execute the method of the first aspect of the embodiments of the present application and any one of its embodiments.
[0027] The information extraction method, apparatus, device and storage medium provided in the present application first processes the order data corresponding to the query instruction using a recognition model to obtain the subject matter information in a unified format, then verifies the subject matter information output by the recognition model based on a standard vocabulary, and generates triple information of the order data based on the verified subject matter information. In this way, the order information is extracted by combining model recognition and standard vocabulary rule verification at the same time, thereby improving the accuracy of information extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0029] Figure 1 A schematic diagram of the structure of an electronic device according to an embodiment of the present application;
[0030] Figure 2A A schematic diagram of a flow chart of an information extraction method according to an embodiment of the present application;
[0031] Figure 2B A schematic diagram of email content parsing according to an embodiment of the present application;
[0032] Figure 3 A schematic diagram of a flow chart of an information extraction method according to an embodiment of the present application;
[0033] Figure 4 A schematic diagram of the structure of an information extraction device according to an embodiment of the present application. DETAILED DESCRIPTION
[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the present application, the terms "first", "second" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0035] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 1 A processor is taken as an example. The processor 11 and the memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11, and the instructions are executed by the processor 11 so that the electronic device 1 can execute all or part of the process of the method in the following embodiment, so as to extract the order information by combining the model recognition and the standard word library rule verification, and improve the information extraction accuracy.
[0036] In one embodiment, the electronic device 1 may be a mobile phone, a tablet computer, a laptop computer, a desktop computer, or a large computing system composed of multiple computer devices.
[0037] Please see Figure 2A , which is an information extraction method of an embodiment of the present application, the method can be Figure 1The electronic device 1 shown in the figure can be used to perform the method, and can be applied to the information extraction scenario of order data with emails as the information carrier, so as to extract the order information by combining model recognition and standard word library rule verification to improve the accuracy of information extraction. The method includes the following steps:
[0038] Step 201: Obtain order data corresponding to the query instruction.
[0039] In this step, the query instruction may be input by the user, for example, the user may enter it through the interactive interface of the terminal, and the query instruction may include identification information of the specified order data, such as the name. The order data may be carried in a variety of ways, for example, the order data may be transmitted via email, and the query instruction may include the name of the email to be queried. The order data may be pre-stored in a preset order library.
[0040] In one embodiment, step 201 may include: when receiving a query instruction, extracting order content corresponding to the identification information in a preset order library, parsing the order content, obtaining text data of the target order, and using the text data as order data.
[0041] In this step, a large amount of searchable order data is pre-stored in the preset order library, and the order data can be in the form of emails or documents. Assuming that the query instruction entered by the user carries the identification information of the target order, such as the email name of the target order, the target order content corresponding to the email name is first searched in the preset order library, and then the order content is parsed to obtain the text data of the order, and the text data is used as the order data of the target order. In this way, no matter what form the order content is, it can be processed into unified text data, so that information extraction is no longer limited by the order form.
[0042] In one embodiment, if Figure 2B As shown, taking an order in the form of an email as an example, the user enters the name of the email to be queried, and the order content is the email content containing the target order. When parsing the order content, the content type of the email is first determined, and the content is parsed based on the email content type. The email content types can include the following:
[0043] 1. If there is only text, input the email content into the text parsing module to obtain the corresponding text content.
[0044] 2. If there is a table in the body of the email, the table part will be input into the table parsing module to convert the table into text format.
[0045] 3. If the email has an attachment, the type of the attachment is determined. If the attachment is a word document, the word parser is used for parsing; if it is an excel document, the excel parser is used for parsing; if it is a pdf document, the pdf parser is used for parsing; if it is a picture, the ocr (optical character recognition) picture parsing tool is used to extract information; if it is a text file, it is directly transferred to the text parsing module. Through the above method, the email content of different formats is uniformly converted into text content. Email content extraction is no longer limited to the email type, which expands the scope of use of email information extraction.
[0046] Step 202: Input the order data into a preset recognition model, and output the subject matter information in the order data.
[0047] In this step, the subject matter may be a commodity, an ordinary article, etc. Different subject matters may be selected based on different scenarios. In actual scenarios, emails related to the subject matter generally contain various attribute information of the subject matter, such as name, category, etc. If the subject matter is a commodity, the order data may be the order information of the commodity delivered in the form of email content, such as an email order form for a batch of cultural and creative products, which generally contains information such as the category, name, quantity, and arrival date of the commodity.
[0048] The preset recognition model can be a recognition model based on a neural network. Assuming that the subject matter is a commodity, taking order data in the form of an email as an example, since the commodity information and its arrival date are strongly correlated, a neural network model can be trained based on the regularity of commodity information and arrival dates in previous emails to obtain a preset recognition model. The email content is first uniformly converted into a text format through step 201, and then the email content in text format is input into the preset recognition model, and the information such as the commodity information and arrival date in the email can be output.
[0049] In one embodiment, before inputting the text-formatted email content into the preset recognition model, the text-formatted email content may be standardized and converted into a predetermined standard format so that the preset recognition model can more accurately extract the target object information.
[0050] In one embodiment, before step 202, the step of establishing a preset recognition model may further include: obtaining a sample order data set. Converting the sample order data set into a predetermined standard format. Labeling the sample subject information in the sample order data set in the standard format. Using the labeled sample order data set to train a neural network model to obtain a preset recognition model.
[0051] In this step, the sample order data set can be a set of multiple purchase order email content of the subject matter, which is uniformly converted into a text format order data set after being processed by a parser. First, the order data set is standardized into a predetermined standard format, such as a unified text typesetting format, and data cleaning and deduplication are performed. Then, the sample order data set in the standard format is annotated, such as annotating the subject matter information involved in each text and the label corresponding to the high-subject matter information. The label can be a label that characterizes the category of the subject matter. Finally, the labeled sample order data set is used to train the neural network model to obtain the preset recognition model.
[0052] Taking the example that the subject matter can be a commodity, the input of the preset recognition model is the text and label of the commodity information, and the output result of the preset recognition model is the text and label of the queried subject matter information and the position information of the text of the subject matter information in the input text.
[0053] In one embodiment, the model training can adopt a bidirectional LSTM+CRF network architecture. Taking product information as an example, the pre-processed product information and labels are used as the input data of the network architecture, and the data is iteratively trained to minimize the loss function loss and reach the set F1 value on the test set. The model training process ends. When predicting, after inputting text information, the model can obtain whether there is product information in the input text, and the label and the location information corresponding to the product information through some processing of the label with the highest prediction probability.
[0054] Step 203: Verify the target object information based on the standard word library to obtain verified target object information.
[0055] In this step, the subject information output by the preset recognition model may be inaccurate, such as inaccurate word segmentation of the email content, which may cause the recognition result to contain incomplete information, thereby affecting the final information extraction accuracy. Therefore, on the basis of model recognition, the rules can be verified, that is, the subject information is verified based on the standard vocabulary to obtain the verified subject information. The standard vocabulary presets the relevant standard format information of the subject matter. Taking the commodity as the subject matter as an example, the standard vocabulary can include the name, order quantity, arrival date and other information of a completed order. The standard vocabulary can be obtained based on different completed order data statistics. Therefore, by verifying the results of model recognition through the standard vocabulary, the accuracy of information extraction can be further guaranteed.
[0056] Step 204: Generate triplet information of order data based on the verified subject matter information.
[0057] In this step, based on the verified subject information, the corresponding triple information is extracted, which allows users to more clearly check the content of interest in the email, avoiding the situation where the user has to read the entire email to obtain relevant subject information when the email content is very large. Taking the commodity as the subject matter, in actual scenarios, an order email for a commodity may contain a lot of content, and even have many attachments. If the user only wants to query the name and arrival date of the commodity, it is obviously too time-consuming and laborious to obtain the information by reading the entire email content. If the user only needs to enter the name of the query email, the triple information related to the commodity information and its arrival date in the email can be automatically returned, which will greatly save the user's query time.
[0058] The above information extraction method first converts the email content corresponding to the query instruction into a unified text format through a parser, and then processes the order data in text format using a recognition model to obtain target object information in a unified format. The target object information output by the recognition model is then verified based on a standard vocabulary, and triple information of the order data is generated based on the verified target object information. In this way, the unified conversion into a text format makes it possible for email information extraction to no longer be limited by the type of email content. At the same time, the order information is extracted by combining model recognition and standard vocabulary rule verification, thereby improving the accuracy of information extraction.
[0059] Please see Figure 3 , which is an information extraction method of an embodiment of the present application, the method can be Figure 1 The electronic device 1 shown in the figure can be used to perform the method, and can be applied to the information extraction scenario of order data with emails as the information carrier, so as to extract the order information by combining model recognition and standard word library rule verification to improve the accuracy of information extraction. The method includes the following steps:
[0060] Step 301: Acquire order data corresponding to the query instruction. For details, refer to the description of step 201 in the above embodiment.
[0061] Step 302: Input the order data into a preset recognition model, and output the subject matter information in the order data. For details, refer to the description of step 202 in the above embodiment.
[0062] Step 303: Determine whether there is target standard data identical to the identified text in the standard word library. If yes, proceed to step 308; otherwise, proceed to step 304.
[0063] In this step, the object information output by the preset recognition model may include: the target object identification text and the text position of the identification text in the order data. Taking the target object as a commodity as an example, assuming that the target object identification is the commodity name, the target object information output by the preset recognition model also includes the text position of the commodity name in the input text, and the text position may be a coordinate position. When the target object information is verified based on the standard vocabulary, it can be first determined whether there is target standard data identical to the identification text in the standard vocabulary. Taking the commodity as an example, the identification text is the text of the commodity name. Each commodity can have a standard vocabulary, and different standard vocabulary can be maintained through label matching. When judging, the text of the commodity name can be input into the standard vocabulary of the commodity information. If there is no target standard data identical to the text of the commodity name in the standard vocabulary, step 304 is entered, otherwise, step 308 is entered if there is.
[0064] In one embodiment, before step 303, the process may further include: detecting character information at the boundary of the mark text, deleting non-text symbols at the boundary of the mark text, and obtaining a corrected mark text.
[0065] In actual scenarios, the identification text recognized by the preset recognition model may not be accurate enough. For example, some similar non-text symbols may be used as the content of the identification text, which will cause the verification based on the standard vocabulary to be inaccurate. For example, in scenario 1:
[0066] The user enters the email name and the email content is parsed as follows:
[0067] -CSSA MONTE VLCC LOAD MONGSTAD / DISCHARGE INDIA LAYCAN 20-21 / 08-REMARKRPLC
[0068] The parser first parses the above email content into order data in text format. After the preset recognition model recognizes, -CSSA MONTE is the product name text (i.e., identification text), and the text position information is (0, 12). The text of the arrival date is 20-21 / 08, and the text position information is (57, 65). The label of -CSSA MONTE is PRD, and the label of 20-21 / 08 is DAT. The output results of the preset recognition model are: (-CSSA MONTE, PRODUCT, 0, 12) and (20-21 / 08, DATA, 57, 65). Among them, the product name text -CSSA MONTE contains a non-text symbol "-". If -CSSAMONTE is directly matched and verified with the standard word library, since the real product name in the standard word library is CSSA MONTE, the standard word library cannot match the same target standard data as -CSSA MONTE.
[0069] The above error is caused by the output result of the preset recognition model. In order to avoid the above error causing error to the final information extraction result, the output result of the preset recognition model can be preprocessed before matching and verifying with the standard word library. Based on the text position of the identification text, the character information at the boundary of the identification text is detected, and the non-text symbols at the boundary of the identification text are deleted to obtain the corrected identification text. For example, the product name is truncated by the position information, and the "-" is removed to obtain CSSA MONTE. Finally, the tuple of the product name is (CSSA MONTE, PRODUCT, 2, 12). Obviously, since CSSA MONTE is already in the word library table, that is, the standard word library has the target standard data, then step 308 can be entered.
[0070] Step 304: Select target candidate data from the standard word library whose similarity with the identified text is greater than a preset threshold.
[0071] In this step, when the target standard data does not exist in the standard word library, it is necessary to correct the identification text based on the text position to obtain the verified target object information. For example, for scenario 2:
[0072] The user enters the email name to be queried, and the corresponding email content is parsed as follows:
[0073] 09-12 DELTA APOLLONIA 319 15 22.52 VADINAR 17-11 DELTA
[0074] First, the above email content is parsed into order data in text format, and then the preset recognition model is used to identify the text of DELTA APOLLON as the product name, and the text position information is (9, 22), the text of the arrival date is 17-11, and the text position information is (61, 66), the label of DELTA APOLLON is PRD, and the label of 17-11 is DAT. The output results of the preset recognition model are recorded as: (DELTA APOLLON, PRODUCT, 9, 22) and (17-11, DATA, 61, 66). The first item in each tuple is matched and checked through the standard vocabulary. If the word DELTA APOLLON does not exist in the standard vocabulary, the standard vocabulary can be further searched to select the target candidate data whose similarity with the text DELTA APOLLON of the product name is greater than the preset threshold. There may be multiple candidate data with a similarity greater than the preset threshold, and the one with the largest similarity value is selected from the multiple candidate data as the target candidate data.
[0075] Step 305: Determine whether the spelling order of the target candidate data is the same as the spelling order of the specified text position interval in the order data. If yes, proceed to step 306; otherwise, proceed to step 307.
[0076] In this step, for the selected target candidate data, although the similarity between the selected target candidate data and the text DELTA APOLLON of the product name is greater than the preset threshold, the spelling order may be different, especially for English-like texts, different spelling orders of the same letters will indicate different meanings. Therefore, the target candidate data can be extracted from the words in the standard vocabulary. Assuming that the target candidate data is DELTA APOLLONIA, within the interval between the starting position and the ending position of the text DELTA APOLLON of the product name, the target candidate data DELTA APOLLONIA is slid on the text DELTAAPOLLON of the text product name to make a judgment, and then the letter-by-letter judgment is made whether the spelling order of the letters within the specified interval between the starting position and the ending position of the text DELTAAPOLLON is the same as that of the target candidate data. Here, it can be obtained that the spelling order of the target candidate data DELTA APOLLONIA is the same as that of the text DELTA APOLLON of the product name within the specified interval, and then step 306 can be entered.
[0077] Step 306 : Use the target candidate data as the verified target object information. Then proceed to step 308 .
[0078] In this step, when the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data, for example, the spelling order of the target candidate data DELTA APOLLONIA in the above scenario 2 is the same as the text DELTA APOLLON of the product name within the specified interval, then the target candidate data DELTA APOLLONIA can be used as the verified object information.
[0079] Step 307: Extend the text content along the text position boundary in the order data until a space symbol is encountered, and use the text content obtained after the extension and the new text position corresponding to the text content as the verified subject matter information. Go to step 308.
[0080] In this step, when the spelling order of the target candidate data is different from the spelling order of the specified interval of the text position in the order data, or the target candidate data that meets the standard is not screened out from the standard vocabulary in step 304, it means that the standard vocabulary may not have pre-saved the standard data of the subject matter of this query. At this time, the identification text can be corrected based on the text position of the identification text of the subject matter and the original order data. For example, the identification text output by the preset recognition model may be incomplete, and then it can be completed.
[0081] Taking scenario 2 as an example, assuming that the identification text of the subject matter is the text of the product name DELTA APOLLON, and the text position is (9, 22), the text position boundary 22 can be located in the original order data as N, and the text content is continued to be extended along the boundary. If the last digit is the letter I, the product name is extended one digit backward, and the above extension is repeated in sequence, ending with a space symbol as a separator. If a space symbol is encountered, the process stops, and the finally expanded text content DELTAAPOLLONIA and its new text position (9, 24) are used as the verified subject matter information, and the verified subject matter information is obtained as (DELTA APOLLONIA, PRODUCT, 9, 24).
[0082] In one embodiment, it is assumed that the subject matter information includes: the target item identification and the date information corresponding to the target item. It should be noted that in the above scenarios 1 and 2, the verification process is described by taking the product name as the triple of the target item identification. For the verification process of the date information corresponding to the target item, please refer to the verification process of the target item identification. For example, the triple information corresponding to the date of arrival of the product can also be verified in the same way. Since there are multiple dates in the original email sentence, directly searching for the date through the rules has only a 50% chance of finding it right in a sentence. The preset recognition model can identify the approximate position of the date, and more accurate results can be obtained through standard vocabulary verification. I will not go into details here.
[0083] Step 308: The target item identification and date information in the subject matter information are respectively taken as two entities, and the type label and date label of the target item are taken as the relationship between the two entities, to generate triple information of the order data.
[0084] In this step, it is assumed that after matching the identification text of the subject matter in each triplet in the standard word library, a completely matching target standard data is found, such as CSSA MONTE in scenario 1 is already in the word library table, or after the verification process of step 307 and step 306, complete subject matter information is obtained, and the subject matter information includes: target item identification and date information corresponding to the target item. Taking the commodity as the subject matter as an example, the subject matter information can include the commodity identification and the commodity arrival date, and the triplet information can be (commodity identification, commodity-date, arrival date), so that the user can clearly check the content of interest in the email at a glance.
[0085] In one embodiment, the date information can be formatted and converted into a unified predetermined date format. For example, in scenario 1, the date information is correctly identified and does not need to be corrected. The original date 20-21 / 08 is actually the abbreviation of two dates. It only needs to convert 20-21 / 08 into the standard format 2021-08-20 / 2021-08-21. The final output triple is (CSSA MONTE, PRD-DAT, 2021-08-20 / 2021-08-21)
[0086] For example, in scenario 2, the date 17-11 is converted to the standard format 2021-11-17, and the final output triple is (DELTA APOLLONIA, PRD-DAT, 2021-11-17).
[0087] Step 309: Update the verified target object information into the standard word library.
[0088] In this step, for the object information that is not matched to the target standard data or target candidate data in the standard vocabulary, it means that the object information is not registered in the standard vocabulary. In order to enrich the standard vocabulary, the object information can be added to the corresponding standard vocabulary, and the standard vocabulary can be updated to further promote the subsequent improvement of the accuracy of information extraction.
[0089] Please see Figure 4 , which is an information extraction device 400 of an embodiment of the present application, which can be applied to Figure 1 The electronic device 1 shown can be applied to the information extraction scenario of order data with email as the information carrier, so as to extract the order information by combining model recognition and standard word library rule verification to improve the information extraction accuracy. The device includes: an acquisition module 401, an identification module 402, a verification module 403 and a generation module 404, and the principle relationship of each module is as follows:
[0090] The acquisition module 401 is used to acquire the order data corresponding to the query instruction.
[0091] The recognition module 402 is used to input the order data into a preset recognition model and output the subject matter information in the order data.
[0092] The verification module 403 is used to verify the target object information based on the standard word library to obtain verified target object information.
[0093] The generating module 404 is used to generate triple information of the order data based on the verified subject matter information.
[0094] In one embodiment, the query instruction carries identification information of the target order. The acquisition module 401 is used to: upon receiving the query instruction, extract the order content corresponding to the identification information in the preset order library, parse the order content, obtain the text data of the target order, and use the text data as the order data.
[0095] In one embodiment, it further includes: establishing module 405, which is used to: obtain a sample order data set.
[0096] The sample order data set is converted into a predetermined standard format. The sample subject information in the sample order data set in the standard format is labeled. The labeled sample order data set is used to train the neural network model to obtain a preset recognition model.
[0097] In one embodiment, the subject information includes: the subject identification text and the text position of the identification text in the order data. The verification module 403 is used to determine whether there is target standard data identical to the identification text in the standard word library. When the target standard data does not exist in the standard word library, the identification text is corrected based on the text position to obtain the verified subject information.
[0098] In one embodiment, before determining whether there is target standard data identical to the identification text in the standard vocabulary, the method further includes: detecting character information at the boundary of the identification text, deleting non-text symbols at the boundary of the identification text, and obtaining a corrected identification text.
[0099] In one embodiment, the identification text is corrected based on the text position to obtain the verified subject information, including: when the target standard data does not exist in the standard word library, select the target candidate data whose similarity with the identification text is greater than a preset threshold from the standard word library. Determine whether the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data. When the spelling order of the target candidate data is the same as the spelling order of the specified interval of the text position in the order data, the target candidate data is used as the verified subject information.
[0100] In one embodiment, the identification text is corrected based on the text position to obtain the verified target object information, which also includes: when the spelling order of the target candidate data is different from the spelling order of the text position specified interval in the order data, the text content is expanded along the text position boundary in the order data until a space symbol is encountered, and the text content obtained after the expansion and the new text position corresponding to the text content are used as the verified target object information.
[0101] In one embodiment, the method further includes: an updating module 406, configured to update the verified target object information into the standard word library.
[0102] In one embodiment, the subject information includes: a target item identifier and date information corresponding to the target item. The generation module 404 is used to: use the target item identifier and date information as two entities, and use the type label and date label of the target item as the relationship between the two entities to generate triple information of the order data.
[0103] For a detailed description of the above information extraction device 400, please refer to the description of the relevant method steps in the above embodiment.
[0104] The embodiment of the present invention also provides a non-transitory electronic device readable storage medium, including: a program, when it is run on an electronic device, the electronic device can execute all or part of the process of the method in the above embodiment. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memory.
[0105] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. An information extraction method, characterized in that: include: Get the order data corresponding to the query instruction; Inputting the order data into a preset recognition model, and outputting the subject matter information in the order data; Verifying the subject matter information based on a standard word library to obtain verified subject matter information; Generating triplet information of the order data based on the verified subject matter information; The subject matter information includes: subject matter identification text and the text position of the identification text in the order data; the subject matter information is verified based on the standard word library to obtain the verified subject matter information, including: Determining whether there is target standard data identical to the identified text in the standard word library; When the target standard data does not exist in the standard word library, the identification text is corrected based on the text position to obtain the verified target object information; The step of correcting the identification text based on the text position to obtain the verified target object information includes: When the target standard data does not exist in the standard word library, selecting target candidate data whose similarity with the identification text is greater than a preset threshold from the standard word library; Determining whether the spelling order of the target candidate data is the same as the spelling order of the text position specified interval in the order data; When the spelling order of the target candidate data is the same as the spelling order of the text position specified interval in the order data, the target candidate data is used as the verified target object information.
2. The method according to claim 1, characterized in that The query instruction carries identification information of the target order; the step of obtaining order data corresponding to the query instruction includes: When receiving the query instruction, extracting the order content corresponding to the identification information in the preset order library; The order content is parsed to obtain text data of the target order, and the text data is used as the order data.
3. The method according to claim 1, characterized in that The steps of establishing the preset recognition model include: Get the sample order dataset; Converting the sample order data set into a predetermined standard format; Annotating sample subject matter information in the sample order data set in a standard format; The labeled sample order data set is used to train a neural network model to obtain the preset recognition model.
4. The method according to claim 1, characterized in that Before determining whether there is target standard data identical to the identification text in the standard word library, the method further includes: The character information at the boundary of the marking text is detected, and the non-text symbols at the boundary of the marking text are deleted to obtain the corrected marking text.
5. The method according to claim 1, characterized in that The step of correcting the identification text based on the text position to obtain the verified target object information further includes: When the spelling order of the target candidate data is different from the spelling order of the specified interval of the text position in the order data, the text content is extended along the text position boundary in the order data until a space symbol is encountered, and the text content obtained after the expansion and the new text position corresponding to the text content are used as the verified object information.
6. The method according to claim 5, characterized in that Also includes: The verified target object information is updated into the standard word library.
7. The method according to claim 1, characterized in that The subject matter information includes: a target item identifier and date information corresponding to the target item; the triplet information of the order data generated based on the verified subject matter information includes: The target item identifier and the date information are respectively taken as two entities, and the type label and the date label of the target item are taken as the relationship between the two entities, to generate triple information of the order data.
8. An information extraction device, characterized in that: include: The acquisition module is used to obtain the order data corresponding to the query instruction; An identification module, used to input the order data into a preset identification model and output the subject matter information in the order data; A verification module, used for verifying the subject information based on a standard word library to obtain verified subject information; A generating module, configured to generate triplet information of the order data based on the verified subject matter information; The subject matter information includes: subject matter identification text and the text position of the identification text in the order data; the subject matter information is verified based on the standard word library to obtain the verified subject matter information, including: Determining whether there is target standard data identical to the identified text in the standard word library; When the target standard data does not exist in the standard word library, the identification text is corrected based on the text position to obtain the verified target object information; The step of correcting the identification text based on the text position to obtain the verified target object information includes: When the target standard data does not exist in the standard word library, selecting target candidate data whose similarity with the identification text is greater than a preset threshold from the standard word library; Determining whether the spelling order of the target candidate data is the same as the spelling order of the text position specified interval in the order data; When the spelling order of the target candidate data is the same as the spelling order of the text position specified interval in the order data, the target candidate data is used as the verified target object information.
9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7.
10. A non-transitory electronic device readable storage medium, characterized in that: The invention comprises: a program, which, when executed by an electronic device, causes the electronic device to execute the method according to any one of claims 1 to 7.
