Method, device, equipment, medium and product for extracting entity

By constructing formal rules and noise character insertion rules, the problems of inserting noise characters in the middle of target entities and rich context diversity are solved, and efficient and accurate target entity extraction is achieved, especially for identifying fixed-pattern entities such as mobile phone numbers in massive text on the Internet.

CN114722800BActive Publication Date: 2025-10-03BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210346916.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-02
Publication Date
2025-10-03
Estimated Expiration
2042-04-02

AI Technical Summary

Technical Problem

Existing information extraction technologies cannot effectively handle scenarios where noise characters are randomly inserted in the middle of target entities and where the context of target entities is rich in diversity, resulting in unsatisfactory extraction results.

Method used

Construct formal rules and noise character insertion rules, traverse the characters in the text, filter out characters that meet the rules and splice them into target entities, and extract target entities from the text through formal rules and noise character insertion rules.

Benefits of technology

In scenarios where noise characters are inserted in the middle of the target entity string and the context is rich in diversity, high-accuracy and high-recall target entity extraction is achieved, with a test accuracy of 98.2% and a recall of 99.6%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114722800B_ABST
    Figure CN114722800B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method, apparatus, device, medium and product for extracting entities, and relates to the field of computer technology, and in particular to information extraction technology in natural language processing. The specific implementation scheme is: receiving a text to be recognized, wherein the text includes a target original entity with noise characters inserted; selecting a formal rule and a noise character insertion rule corresponding to the target original entity; traversing the text according to the formal rule and the noise character insertion rule, screening characters for splicing the target entity, and using the character string spliced ​​together as the target entity extracted from the text. The present disclosure traverses the text through the noise character insertion rule and the formal rule constructed according to the target entity, and gradually splices the character string with noise characters inserted inside the target entity and in the context into the target character, so as to extract the target entity with noise characters inserted from the text.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to information extraction technology in natural language processing. Background Art

[0002] Information extraction (IE) technology involves extracting valuable structured features or information from unstructured data. The unstructured data involved in information extraction includes text, images, audio, and video. With the rapid development and widespread adoption of digital technology and mobile internet, client users continue to generate massive amounts of text, and many applications require extracting target entities from this vast amount of text. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device, medium, and product for extracting entities.

[0004] According to one aspect of the present disclosure, a method for extracting an entity is provided, comprising: receiving a text to be recognized, the text including a target original entity with noise characters inserted therein; determining and selecting a formal rule and a noise character insertion rule corresponding to the target original entity; traversing the text according to the formal rule and the noise character insertion rule, screening characters for splicing the target entity, and using the string spliced ​​together by the characters as the target entity to be extracted from the text.

[0005] According to another aspect of the present disclosure, a device for extracting an entity is provided, comprising: a determination unit for receiving a text to be recognized, wherein the text includes a target original entity with noise characters inserted therein, and further for selecting a formalization rule and a noise character insertion rule corresponding to the target original entity; a first extraction unit for traversing the text according to the formalization rule and the noise character insertion rule, screening characters for splicing the target entity, and using the character string spliced ​​together by the characters as the target entity to be extracted from the text.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a method.

[0007] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method.

[0008] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method when executed by a processor.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a flow chart of a method for extracting entities according to an embodiment of the present disclosure;

[0012] Figure 2 is a flow chart of a method for determining formal rules provided according to an embodiment of the present disclosure;

[0013] Figure 3 is a flow chart of a method for extracting a target entity with inserted noise characters from a text according to an embodiment of the present disclosure;

[0014] Figure 4 This is a flow chart of a method for determining based on the target entity string length and formal rules provided in an embodiment of the present disclosure;

[0015] Figure 5 is a flow chart of a method for matching a current character with a formal rule according to an embodiment of the present disclosure;

[0016] Figure 6 is a processing flow chart after updating the noise character variable according to an embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of extracting a target entity phone number provided in accordance with an embodiment of the present disclosure;

[0018] Figure 8 is a block diagram of a device for extracting entities according to an exemplary embodiment;

[0019] Figure 9 It is a block diagram of an electronic device used to implement the method for extracting entities according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] The present disclosure is applied to scenarios where target entities are extracted from text. In related art, methods for extracting target entities from text include two categories. The following embodiments briefly describe the two methods for extracting target entities in related art.

[0022] The first category is the pattern matching method based on rule strategies. In this method, domain experts observe and summarize text data to construct regular expressions that can formally describe the internal and contextual patterns of the target entity. When executing the pattern matching method of rule strategies, regular expressions are used to extract the target entity from the text. That is, the string in the text is compared with the regular expression representing the target entity. If there is a string in the text that matches the regular expression, the string is extracted. The extracted string is used as the target entity. From the above matching process, it can be seen that the first category of methods is suitable for scenarios where the diversity of the contextual patterns of the target entity is low and the characters that make up the target entity are continuous and complete.

[0023] The second category involves supervised learning-based generalized entity extraction modeling methods. This method requires manually constructing a training set of labeled sequences, which is then used to train a model to identify target entities within text. This second category of methods falls within the field of named entity recognition (NER) within natural language processing (NLP). Named entities in a narrow sense can include names of people, organizations, dates, addresses, and so on. Broadly speaking, named entities refer to any textual object that needs to be extracted. For example, train numbers, departure stations, medication names, disease names, and so on can be extracted from text. Currently, supervised learning-based generalized entity extraction modeling methods have been applied to named entity recognition. However, these applications can only identify a small number of narrowly defined named entities, and when applied to real-world text, the extraction results achieved are suboptimal. Therefore, the second category of methods is suitable for extracting target entities from complex text patterns. Combining deep learning methods with sequence annotation methods generally achieves high extraction results and requires less rigidity in the target entity patterns. However, these second-category methods rely excessively on large amounts of manually annotated entity sequence corpora.

[0024] Compared to the second type of methods, the first type of methods has the advantages of not requiring manual data annotation, high target entity extraction efficiency, and high extraction stability and accuracy. However, the first type of methods is only suitable for scenarios with low data diversity and only extracting a few fixed target entities.

[0025] In summary, the usage scenarios of the first and second methods are different, and the problems and shortcomings they bring are also different. However, with the development of the Internet, client users continue to generate massive amounts of text data with rich text diversity and strong feature concealment, which increases the difficulty of extracting target entities from text. For example, the text is "Welcome to the logistics special line from a certain place to a certain place (departing every day in 2020) 186 See XXX8 Manager He 1235 (same number on WeChat) / / 130XXX8 He 0009 Direct access to the highway". There is at least one of the following difficulties in extracting the target entity phone number from the sample text using the above-mentioned first and second methods.

[0026] 1. The target entity is discontinuous. For various reasons, the client user inserts noise characters at random positions in the string that constitutes the target entity, causing the fixed pattern of the target entity to be destroyed.

[0027] 2. The target entity context is rich in diversity. This means that the target entity context introduces a large number of variant words, abbreviations, special symbols, etc., which destroys the recognition pattern on both sides of the target entity.

[0028] In view of the above difficulties, the target entity recognition methods based on regular expressions and sequence labeling technology cannot solve them. That is, the existing target entity extraction methods in the relevant technology cannot cope with the scenarios where random noise characters are inserted at random positions in the middle of the target entity and the target entity context recognition pattern is destroyed. In the text "Welcome to the logistics special line from a certain place to a certain place (departing every day in 2020) 186 See XXX8 Manager He 1235 (same number on WeChat) / / 130XXX8 He 0009 Direct access to the highway", the noise characters referred to in this disclosure are "See", "Manager He", and "He" inserted into the entity mobile phone number. It can be understood that the noise characters in this disclosure are not connectors, but characters that do not affect the user's recognition of the entity, but can prevent automatic detection and are inconsistent with the type of the entity.

[0029] In view of this, the present disclosure provides a method for extracting entities, which constructs formal rules based on the target entity to be extracted. According to the formal rules and the noise character insertion rules, the characters in the text are traversed, the characters used to splice the target entity are filtered from the text, and the filtered characters are spliced ​​into the target entity. In this way, the target entity with the noise characters inserted in the text is extracted. Traversing the characters in the text through the formal rules constructed by the present disclosure can effectively solve the problem of difficulty in accurately extracting entities in scenarios where noise characters are inserted in the middle of the entity string and the entity context is rich in diversity.

[0030] The following embodiments will illustrate a method for extracting entities provided by the present disclosure in conjunction with the accompanying drawings.

[0031] Figure 1 is a flow chart of a method for extracting entities according to an embodiment of the present disclosure; Figure 1 As shown, the present disclosure provides a method for extracting entities, including the following steps S101-S103.

[0032] In step S101 , a text to be recognized is received, wherein the text includes a target original entity with noise characters inserted therein.

[0033] The text to be recognized in the present disclosure may be sent by other devices connected to the device that executes the entity extraction method, or it may be locally stored by the device that executes the entity extraction method. In order to distinguish the target entity with noise characters inserted and the target entity without noise characters inserted, the present disclosure uses the target original entity to represent the target entity with noise characters inserted. It can be understood that the purpose of the present disclosure is to extract the target entity that does not include noise characters when noise characters interfere. The target entity extracted in the present disclosure may be an entity with a fixed pattern. Specifically, the target entity with a fixed pattern with noise characters inserted is extracted. For example, mobile phone numbers, driver's license file numbers, identity card numbers and unified social credit codes, etc. It should be noted that the present disclosure does not limit the text to be recognized to include the target original entity. Whether the text to be recognized includes the target original entity can be determined through the present disclosure. If the target entity is extracted through the present disclosure, it means that the text to be recognized includes the target original entity. If the target entity is not extracted through the present disclosure, it means that the text to be recognized does not include the target original entity.

[0034] In step S102 , a formalization rule and a noise character insertion rule corresponding to the target original entity are selected.

[0035] After the target original entity is known, the present disclosure selects the formal rules and noise character insertion rules corresponding to the target original entity from the pre-constructed formal rules and noise character insertion rules.

[0036] The formal rules in the present disclosure can be constructed based on target entity samples without inserting noise characters. Before applying the present disclosure, formal rules can be pre-constructed based on target entity samples. The way in which the present disclosure constructs formal rules is to analyze the patterns of massive target entity samples and determine the constraint rules between the characters that make up the target entity. Formal rules are constructed based on the constraint rules between the characters of the target entity. For example, massive mobile phone number samples, driver's license file number samples, identity card number samples and unified social credit code samples are analyzed to construct corresponding formal rules.

[0037] In one embodiment, a distributed crawler can be used to crawl publicly available postings in internet communities or forums, thereby accumulating massive amounts of text data containing target entity samples and target original entity samples, providing data support for constructing formal rules and noise character insertion rules. However, this disclosure does not limit the method for accumulating text data, and text data can be accumulated based on actual circumstances. It should be noted that this disclosure crawls text data with the consent and permission of the user.

[0038] The present disclosure performs a statistical analysis on the acquired text data and summarizes the rules for inserting noise characters in the target original entity samples. According to the rules for inserting noise characters in the target original entity samples, the noise character insertion rules are determined. For example, the noise character insertion rule is to limit the number of noise characters randomly inserted in the target entity. If too many noise characters are inserted inside the target entity, the user will not be able to accurately identify the entity information. However, in order to prevent the program from automatically detecting, it is necessary to insert an appropriate amount of noise characters into the target entity. Therefore, in order to strike a balance between making it difficult for the automated program to recognize easily and allowing the audience to accurately identify the entity, the client user inserts a certain number of noise characters into the target entity.

[0039] In step S103, the text is traversed according to the formalization rules and the noise character insertion rules, characters used for concatenating the target entity are screened, and the character string concatenated by the characters is used as the target entity extracted from the text.

[0040] Parse the formalized rules and traverse the characters in the text. Match the characters in the text against the formalized rules. During the matching process, filter the characters used to concatenate the target entity according to the noise character insertion rules. Concatenate the characters into a string representing the target entity, and use the concatenated string as the target entity to be extracted from the text. By executing steps S101-S103, the target entity can be accurately extracted in scenarios where noise characters are randomly inserted into the target entity string and the entity context is diverse.

[0041] The present disclosure traverses the text through noise character insertion rules and formalized rules constructed according to the target entity, and filters out characters that meet the formalized rules and noise character insertion rules in the target original entity within the target entity and the context, and gradually splices the characters into a character string representing the target character, so as to extract the target entity with the inserted noise character from the text.

[0042] In order to extract the target entity of the inserted noise character, the present disclosure needs to construct a formal rule based on the target entity. Figure 2 and concrete examples to illustrate the process of constructing formal rules. Figure 2 is a flow chart of a method for determining formal rules according to an embodiment of the present disclosure; Figure 2 As shown, the process of determining formal rules provided by the present disclosure includes the following steps S201-S202.

[0043] In step 201 , the target entity sample is analyzed to determine the character set allowed to appear at each character position of the target entity sample and the constraint relationship between each character position.

[0044] The massive target entity samples acquired in advance are analyzed, and the character set allowed to appear at each character position of the target entity is constructed according to the character position, and the dependency constraint relationship between each character position is analyzed.

[0045] In step 202, a formal rule is constructed based on the set of characters allowed to appear at each character position and the constraint relationship between each character position.

[0046] In order to better understand the process of constructing formal rules, this disclosure takes the target entity mobile phone number as an example to illustrate the process of constructing formal rules.

[0047] By analyzing a large number of mobile phone numbers, it can be concluded that only the number 1 is allowed to appear in the first character position of a mobile phone number (formal rule 0:'1' is constructed).

[0048] The second character position is only allowed to contain any number between 3 and 9 (formal rule 1: '3-9').

[0049] The number allowed in the third character position depends on the number in the second character position.

[0050] If the number that appears in the second character position is 3, 4, 5, 8 or 9, any number between 0 and 9 is allowed to appear in the third character position.

[0051] If the number that appears in the second position is 6, the numbers that can appear in the third character position are 1, 2, 4, 5, 6, and 7.

[0052] If the number that appears in the second position is 7, then any number between 0 and 8 is allowed to appear in the third character position.

[0053] The numbers allowed to appear in other character positions after the mobile phone number are all from 0 to 9.

[0054] From the above analysis, it can be seen that if the character string of the target entity does not conform to the above formal rules of mobile phone numbers, it means that the character string is not a mobile phone number entity.

[0055] In one embodiment, to ensure the simplicity of formal rule construction and reduce the number of characters required for manual input, abbreviations are used to represent sets of numbers. For example, '1-5' represents the set of numbers (1, 2, 3, 4, 5). '13-6' represents the set of numbers (1, 3, 4, 5, 6). The following embodiments of the present disclosure illustrate formal rules constructed using abbreviations.

[0056]

[0057]

[0058] By constructing entity formalization rules through this disclosure, we can accurately, concisely, and efficiently represent a target entity with a fixed pattern, providing a basis for extracting the target entity with inserted noise characters. Furthermore, through automated parsing of entity formalization rules, we can convert manually constructed rules into programming language dictionary data structures, effectively supporting the use of formalization rules in subsequent target entity extraction steps.

[0059] After determining the formalization rules and noise character insertion rules, the present disclosure traverses the text to be recognized according to the formalization rules and noise character insertion rules, extracts the target entity of the inserted noise character from the text to be recognized, and the following embodiments are combined with the attached Figure 3 ,The target entity of inserting noise characters from text is explained based on the formal ,rules and noise character insertion rules. Figure 3 is a flow chart of a method for extracting target entities with inserted noise characters from text according to an embodiment of the present disclosure; Figure 3 As shown, the present disclosure provides a method of traversing a text according to formalized rules and noise character insertion rules, screening characters for splicing a target entity, and using the string spliced ​​from the characters as the target entity extracted from the text, including the following steps S301-S303.

[0060] In step S301, the characters in the text are traversed to determine the type of the current character and the target entity string length of the target entity variable.

[0061] The characters in the text are traversed, with the currently traversed character being referred to as the current character. The present disclosure requires performing the following judgment on each character. The type of the current character is determined, and whether the current character type matches the type of the target entity character is determined. Subsequent processes are executed based on the judgment result. The target entity variable in the present disclosure is used to store the characters selected for forming the target entity. These selected characters are the characters that meet the formalized rules for forming the target entity. When determining the characters that may form the target entity, the characters that meet the formalized rules need to be stored. Ultimately, the string concatenated from all characters that meet the formalized rules is the extracted target entity. Therefore, the present disclosure sets a target entity variable to temporarily store characters that meet the formalized rules. During the judgment process, the target entity string length in the target entity variable is used as a basis to determine whether the current character meets the formalized rules. When the first character is traversed, the target entity variable does not store any characters that meet the formalized rules; therefore, the target entity string length is zero. During the traversal process, if a character that meets the formalized rules is found, the target entity string length is determined based on the number of characters that meet the formalized rules stored in the target entity variable.

[0062] In step S302a, if the type of the current character matches the target type of the target entity, the current character is stored in the target entity variable or the target variable is initialized according to the target entity string length and formal rules, and traversal continues to the next character.

[0063] If the current character type is int and the target entity's target type is also int, then the current character type matches the target entity's target type. Based on the target entity's string length and formal rules, the current character is stored in the target entity variable or the target variable is initialized, and then traversal continues to the next character.

[0064] In step S302b, if the type of the current character does not match the target type of the target entity, the noise character variable used to record the number of noise characters is updated, and based on the updated noise character variable and the entity string length, the target variable is initialized or the characters stored in the target entity variable are concatenated into a string representing the target entity and stored in the target entity list, and traversal continues to the next character.

[0065] If the type of the current character is char, and the target type of the target entity is int, the type of the current character does not match the target type of the target entity. The noise character insertion rule in the present disclosure is that the number of noise characters is less than the noise character variable threshold. When the type of the current character does not match the target type of the target entity, the noise character variable used to record the number of noise characters is updated according to the preset rules. For example, the preset rule is that when the type of the current character does not match the target type of the target entity, the noise character variable is accumulated by 1. Based on the updated noise character variable and the length of the entity string, the target variable is initialized. Or based on the updated noise character variable and the length of the entity string, the characters stored in the target entity variable are spliced ​​into a target entity, and the spliced ​​target entity is stored in the target entity list. Then continue to traverse the next character. The target variable in the present disclosure includes a target entity variable and a noise character variable.

[0066] In step S303 , until all characters in the text are traversed, the target entities stored in the target entity list are taken as target entities extracted from the text.

[0067] In the present disclosure, in order to traverse all target entities for inserting noise characters in a text, the target entities extracted during the traversal process are stored in a target entity list to ensure the normal execution of subsequent processes. After traversing the characters in the text, the target entities stored in the target entity list are used as the target entities for inserting noise characters extracted from the text.

[0068] The present invention traverses the characters in the text, performs formal rule matching on the current character that matches the target type of the target entity, and stores the characters that meet the formal rule in the target entity variable, so that the characters in the target entity variable can be concatenated to obtain the target entity. When the current character does not match the target type of the target entity, the noise character variable is updated. Based on the updated noise character variable, it is determined whether to store the target entity stored in the target entity variable in the target entity list. In this way, the target entity inserted with the noise character can be extracted from the text, thereby improving the recognition accuracy and recall rate of the target entity.

[0069] This disclosure is combined with the attached Figure 4 , describes the process of storing the current character into the target entity variable or initializing the target variable according to the target entity string length and formal rules. Figure 4 is a flow chart of a method for determining based on the target entity string length and formal rules according to an embodiment of the present disclosure; Figure 4 As shown, the process provided by the present disclosure for storing the current character into the target entity variable or initializing the target variable according to the target entity string length and formal rules includes the following steps S401-S402.

[0070] In step S401 , the relationship between the target entity character string length and the target entity character string length threshold is determined.

[0071] In step S402a, if the target entity character string length is greater than or equal to the target entity character string length threshold, the target entity variable and the noise character variable are initialized.

[0072] In the present disclosure, if the target entity string length is greater than or equal to the target entity string length threshold, it means that a string that meets the target entity string length threshold has been stored in the target entity variable. If the current character is added to the target entity variable, it is obvious that it does not meet the pattern of the target entity variable. Therefore, when the target entity string length is greater than or equal to the target entity string length threshold, the target entity variable and the noise character variable are initialized to facilitate traversal of subsequent texts. For example, the target entity is a mobile phone number, that is, the target entity string length threshold is 11. If the target entity string length is greater than or equal to 11, it means that 11 digits have been stored in the target entity variable. If the current character is stored in the target entity variable again, 12 digits are stored in the target entity variable. Obviously, it does not meet the requirements of the target entity mobile phone number. In this case, the target entity variable and the noise character variable need to be initialized.

[0073] In step S402b, if the target entity string length is less than the target entity string length threshold, the current character is matched with the formal rule, and based on the matching result, the current character is stored in the target entity variable or the target variable is initialized.

[0074] In this disclosure, if the target entity string length is less than the target entity string length threshold, it means that the characters already stored in the target entity variable are insufficient to form the target entity. In this case, it is necessary to match the current character with the formal rules and, based on the matching result, store the current character in the target entity variable or initialize the target variable.

[0075] Through the matching results of the target entity string length and the target entity string length threshold, as well as the matching results of the formalized rule and the current character disclosed in the present invention, it is determined whether to store the current character into the target entity variable, in preparation for extracting the target entity with the inserted noise character.

[0076] On the basis of any of the above embodiments, the present disclosure is combined with the attached Figure 5 , describes the process of matching the current character with the formal rules and storing the current character in the target entity variable or initializing the target variable based on the matching result. Figure 5 is a flow chart of a method for matching a current character with a formal rule according to an embodiment of the present disclosure; Figure 5As shown, the process provided by the present disclosure of matching the current character with the formalized rule and storing the current character into the target entity variable or initializing the target variable based on the matching result includes the following steps S501-S503.

[0077] In step S501, the target position of the target entity character to be stored in the target entity is determined according to the length of the target entity character string.

[0078] Based on the target entity string length, add 1 to the target entity string length to obtain the target position of the target entity character to be stored in the target entity. For example, if the target entity string length is 5, then the target entity character to be stored is the 6th character, that is, the target position of the character to be stored in the target entity is the 6th character.

[0079] In step S502 , a character set at a target position is determined in the dictionary data structure.

[0080] The dictionary data structure is obtained by parsing the formalized rules. The character set in this disclosure refers to a set of characters that are allowed to appear at the target position. It should be noted that the characters that are allowed to appear at the target position can also be stored in other forms besides the set.

[0081] In one embodiment, if the formalized rules use customized abbreviation rules, the high-level programming language program cannot directly recognize the digital set, so the formalized rules of the target entity need to be parsed and converted into a specific data structure that can be recognized by the high-level programming language.

[0082] Continuing with the above example, in this embodiment, the formal rules of the target entity mobile phone number are converted into the following Python dictionary data structure.

[0083] {'0':['1'],'1':['3','4','5','6','7','8','9'],'2':{'3':['0','1','2','3','4','5','6','7','8','9'],'4':['0','1','2','3','4','5','6','7','8','9'],'5':['0','1','2','3','4','5','6','7','8 ','9'],'6':['1','2','3','4','5','6','7'],'7':['0','1','2','3','4','5','6','7','8'],'8':['0','1','2','3','4','5','6','7','8'],'9':['0','1','2','3','4','5','6','7','8','9']},'3':[' 0','1','2','3','4','5','6','7','8','9'],'4':['0','1','2','3','4','5','6','7','8','9'],'5':['0','1','2','3','4','5','6','7','8','9'],'6':['0','1','2','3','4','5','6','7','8','9'],'7' :['0','1','2','3','4','5','6','7','8','9'],'8':['0','1','2','3','4','5','6','7','8','9'],'9':['0','1','2','3','4','5','6','7','8','9'],'10':['0','1','2','3','4','5','6','7','8','9']}

[0084] It should be noted that the dictionary data structure in the embodiment of the present disclosure is only for the purpose of illustrating the present disclosure and should not be construed as limiting the present disclosure. Formalized rules can be represented in other forms besides the dictionary data structure, such as key-value pairs.

[0085] In step S503a, if there is a character matching the current character in the character set, the current character is stored in the target entity variable.

[0086] Match the current character with the characters in the character set. If the current character matches any character in the character set, it means that the current character can be used to concatenate the target entity, that is, the current character is stored in the target entity variable.

[0087] In step S503b, if there is no character matching the current character in the character set, the target entity variable and the noise character variable are initialized.

[0088] If the current character does not match any character in the character set, it means that the current character cannot be used to concatenate the target entity. In this case, the target entity variable and the noise character variable need to be initialized. The reason for initializing the target entity is that the current character already matches the character type of the target entity, which eliminates the possibility that the current character is a noise character. However, if there is no character in the character set that matches the current character, it means that the character stored in the target entity variable is not a character that constitutes the target entity, so the target entity variable and the noise character variable need to be initialized to facilitate subsequent traversal.

[0089] When the current character matches the formal rule, the current character is stored in the target entity variable, and when the current character does not match the formal rule, the target variable is initialized to facilitate the extraction of the target entity with the inserted noise character.

[0090] The above embodiment illustrates the process when the type of the current character matches the target type of the target entity. The following embodiment will illustrate the process when the type of the current character does not match the target type of the target entity with reference to the accompanying drawings.

[0091] Figure 6 is a processing flow chart after updating the noise character variable according to an embodiment of the present disclosure; Figure 6 As shown, the process provided by the present disclosure for initializing a target variable or concatenating the characters stored in the target entity variable into a string representing the target entity and storing it in a target entity list based on the updated noise character variable and the entity string length includes the following steps S601-S602.

[0092] In step S601 , the relationship between the entity character string length and the target entity character string length threshold, and the relationship between the updated noise character variable and the noise character variable threshold are determined.

[0093] The purpose of determining the relationship between the entity string length and the target entity string length threshold in this disclosure is to store the target entity stored in the target entity variable in the target entity list when the entity string length is equal to the target entity string length threshold. The purpose of determining the relationship between the updated noise character variable and the noise character variable threshold is to determine the subsequent process based on the relationship between the updated noise character variable and the noise character variable threshold when the entity string length is less than the target entity string length threshold.

[0094] In step S602a, if the entity string length is equal to the target entity string length threshold, the characters stored in the target entity variable are concatenated into a string representing the target entity and stored in the target entity list, and the target entity variable and the noise character variable are initialized.

[0095] If the entity string length is equal to the target entity string length threshold, the characters stored in the target entity variable can be concatenated into a complete target entity. In this case, the characters stored in the target entity variable need to be concatenated into a string and the concatenated string stored as the target entity in the target entity list. After the target entity stored in the target entity variable is stored in the target entity list, the target entity variable and the noise character variable are initialized to facilitate traversal of the next character.

[0096] In step S602b, if the entity string length is less than the target entity string length threshold, and the updated noise character variable is greater than the noise character variable threshold, the target entity variable and the noise character variable are initialized.

[0097] If the entity string length is less than the target entity string length threshold, it is necessary to further determine the relationship between the updated noise character variable and the noise character variable threshold. If the updated noise character variable is greater than the noise character variable threshold, it means that the noise character variable has exceeded the set noise character variable threshold. That is, it is necessary to initialize the target entity variable and the noise character variable to judge the next character. In addition to the above situation, if the entity string length is less than the target entity string length threshold, and the updated noise character variable is less than or equal to the noise character variable threshold. When the entity string length is less than the target entity string length threshold, and the updated noise character variable is less than or equal to the noise character variable threshold, it means that the operation of initializing the target entity variable and initializing the noise character variable has not been triggered, and the next character can be judged based on the current target entity variable and the noise character variable.

[0098] Through the relationship between the entity string length and the target entity string length threshold, as well as the relationship between the updated noise character variable and the noise character variable threshold, it is determined that when the type of the current character does not match the target type of the target entity, the target entity stored in the target entity variable is stored in the target entity list or the target variable is initialized, so as to extract the target entity with the inserted noise character from the text.

[0099] Based on any of the above embodiments, the present disclosure determines the text to be recognized, including: acquiring the text to be recognized, preprocessing the text, and determining the preprocessed text as the text to be recognized.

[0100] In the present disclosure, the process of preprocessing a file involves removing connectors from the target entity. This ensures the simplicity and accuracy of subsequent target entity recognition. Connectors that need to be removed include "*," "-," " / ," and spaces. For example, if the text to be recognized is "Yinghua Live Streaming Guild Registration Contact Number 188-XXX4-2222," the text will be preprocessed and output as "Yinghua Live Streaming Guild Registration Contact Number 188XXX42222." In one embodiment, the above text preprocessing process is performed using Python's string module.

[0101] The present disclosure preprocesses the text to ensure the simplicity and accuracy of subsequent target entity recognition.

[0102] On the basis of any of the above embodiments, in order to more comprehensively identify the target entity from the text, the present disclosure extracts the target entity from the text according to a pre-constructed regular expression. The regular expression in the present disclosure is constructed according to the target entity.

[0103] The present disclosure constructs a regular expression of the target entity to perform entity extraction, and can extract the target entity without inserting noise characters from the text. Regular expressions can match fixed patterns of character strings, and the target entity expressed by the regular expression can be extracted for the text that meets the regular expression. For example, a date can be represented as a concatenation of the numerical values ​​of year, month, and day, where the year, month, and day are represented by 4, 1 or 2, 1 or 2 digits respectively, and are connected by several types of connectors. The present disclosure is aimed at massive text data, and by constructing a regular expression of the target entity to extract the target entity, it can achieve stable and efficient extraction of target entities with fixed patterns. It should be noted that the present disclosure does not limit the order of executing regular expressions to extract target entities and executing formal rules to extract target entities. The two complement each other to improve the comprehensiveness of extracting target entities from text.

[0104] This disclosure enables accurate and efficient extraction of target entities in scenarios where noise characters are inserted into entity strings and the entity contexts are diverse. A practical application system built based on this disclosure's technical solution has achieved an accuracy rate of 98.2% and a recall rate of 99.6% for extracting several target entities.

[0105] In order to more clearly illustrate the present disclosure, the following embodiment of the present disclosure extracts the mobile phone number entity disclosed by the merchant in the Internet community advertisement post. The text of the community advertisement post is "Welcome to the logistics line from a certain place to a certain place (departing every day in 2020) 186 to see XXX8 Manager He 1235 (same number on WeChat) / / 130XXX8 He 0009 direct access to the highway".

[0106] In this example, two stages are used: identifying complete entities based on regular expressions and identifying entities with inserted noise characters based on formal rules. Among them, the main function of the regular expression recognition stage is to recognize complete entities in the text through regular expressions. This process cannot recognize entities with inserted noise characters. The string traversal process based on formal rules can effectively identify target entities in scenarios where noise characters are inserted inside the entity and in the context. By combining the two stages, it is possible to achieve accurate recognition of target entities in the above complex scenarios in a simple and efficient manner. The following embodiments will be combined with the attached Figure 7 The two-stage process of target entity extraction is explained. Figure 7 is a flowchart of extracting the target entity phone number provided according to an embodiment of the present disclosure; Figure 7 As shown, the extraction process includes the following steps S701-S707.

[0107] In step S701, the text to be recognized is obtained and preprocessed.

[0108] In this embodiment, the pre-processing process includes deleting the connector inserted in the mobile phone number in the text.

[0109] In step S702 , target entity extraction is performed using regular expressions, and the extracted target entity is returned.

[0110] The mobile phone number recognition regular expression constructed in this embodiment is as follows:

[0111] (.*)(13[0-9]\d{8}|14[0-9]\d{8}|15[0-9]\d{8}|16[124567]\d{8}|17[0-8]\d{8}|18[0-9]\d{8}|19[0-9]\d{8})(?!\d.*)

[0112] It should be noted that this regular expression is not the only regular expression for mobile phone number identification and is only for illustration.

[0113] Using the regular expression constructed above, the text is matched against the regular expression. If a match is successful, the specified string is extracted as the target entity string. In the regular expression for the mobile phone number above, the string extracted from the second group is the recognized mobile phone number.

[0114] In step S703, the formalized rules are parsed into a dictionary data structure, a target entity string length threshold and a noise character variable threshold are set, and the target entity variable, the noise character variable and the target entity list are initialized.

[0115] In step S704, the text is traversed from left to right to determine whether the current character is a number.

[0116] In step S705, if the current character is a number, the entity character string is spliced ​​or the target entity variable and the noise character variable are initialized according to the dictionary data structure.

[0117] Calculate the target entity string length (i.e., the length of the current mobile phone number). If the current mobile phone number length is 11, set the target entity variable to an empty string and the noise character variable to 0. If the current mobile phone number length is less than 11, determine whether the current character is allowed to appear in the current mobile phone number character position based on the formal rules for mobile phone numbers. If it meets the formal rules for mobile phone numbers, concatenate the current character to the target entity variable. Otherwise, set the target entity variable to an empty string and set the noise character variable to 0.

[0118] In step S706, if the current character is not a number, based on the updated noise character variable and the length of the mobile phone number, the target entity that has been spliced ​​in the target entity variable is stored in the target entity list or the target variable is initialized.

[0119] First, the noise character variable is increased by 1. If the current mobile phone number length is 11, a mobile phone number recognition is completed, the mobile phone number is inserted into the target entity list, i.e., the mobile phone number list, and the target entity variable and the noise character variable are initialized. If the current mobile phone number length is less than 11, it is determined whether the noise character variable is greater than the noise character variable threshold. If the noise character variable is greater than the noise character variable threshold, the target entity variable and the noise character variable are initialized. In the present disclosure, the noise character variable threshold can be set to 5.

[0120] In step S707, after traversing the text characters, the concatenated target entity in the target entity list is returned.

[0121] In this embodiment, when all text characters are traversed, the mobile phone number in the target entity list is returned.

[0122] In the above embodiment of the present disclosure, the method for extracting entities provided by the present disclosure is described by taking the target entity mobile phone number as an example. In addition to the target entity being a mobile phone number, the target entity can also be other entities with rules. For example, the name of a drug. The systematic naming of a drug generally consists of "stereoconfiguration + substituent + basic skeleton + functional group". The present disclosure can be used to extract drug names with random noise characters (such as Latin characters) inserted into the drug names.

[0123] Based on the same concept, an embodiment of the present disclosure also provides a device for extracting an entity.

[0124] It is understandable that the device for extracting entities provided by the embodiments of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of the various examples disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiments of the present disclosure.

[0125] Figure 8 FIG. 1 is a block diagram of a device for extracting a target entity according to an exemplary embodiment. Figure 8 The device 800 includes a determining unit 801 and a first extracting unit 802.

[0126] The determination unit 801 is used to receive a text to be recognized, which includes a target original entity with noise characters inserted therein, and is also used to select formal rules and noise character insertion rules corresponding to the target original entity; the first extraction unit 802 is used to traverse the text according to the formal rules and noise character insertion rules, screen characters used to splice the target entity, and use the character string spliced ​​together as the target entity extracted from the text.

[0127] The determination unit 801 is used to: analyze the target entity sample, determine the set of characters that are allowed to appear at each character position of the target entity sample, and the constraint relationship between each character position; and construct a formal rule based on the set of characters that are allowed to appear at each character position and the constraint relationship between each character position.

[0128] In one embodiment, the first extraction unit 802 is used to: traverse the characters in the text, determine the type of the current character, and the target entity string length of the target entity variable, the target entity variable is used to store the characters screened out for splicing the target entity; if the type of the current character matches the target type of the target entity, then according to the target entity string length and the formalization rules, the current character is stored in the target entity variable or the target variable is initialized, and the traversal continues to the next character; if the type of the current character does not match the target type of the target entity, the noise character variable used to record the number of noise characters is updated, and based on the updated noise character variable and the entity string length, the target variable is initialized or the characters stored in the target entity variable are spliced ​​into a string representing the target entity and stored in the target entity list, the target variable includes the target entity variable and the noise character variable, and the traversal continues to the next character; until all the characters in the text are traversed, the target entity stored in the target entity list is used as the target entity extracted from the text.

[0129] In one embodiment, the first extraction unit 802 is further used to: if the target entity string length is greater than or equal to the target entity string length threshold, initialize the target entity variable and the noise character variable; if the target entity string length is less than the target entity string length threshold, match the current character with the formal rule, and store the current character in the target entity variable or initialize the target variable based on the matching result.

[0130] In one embodiment, the first extraction unit 802 is further used to: determine the target position of the target entity character to be stored in the target entity based on the length of the target entity string; determine the character set at the target position in the dictionary data structure, and the dictionary data structure is obtained based on parsing the formal rules; if there is a character matching the current character in the character set, the current character is stored in the target entity variable; if there is no character matching the current character in the character set, the target entity variable and the noise character variable are initialized.

[0131] In one embodiment, the first extraction unit 802 is also used to: if the entity string length is equal to the target entity string length threshold, then the characters stored in the target entity variable are spliced ​​into a string representing the target entity and stored in the target entity list, and the target entity variable and the noise character variable are initialized; if the entity string length is less than the target entity string length threshold, and the updated noise character variable is greater than the noise character variable threshold, then the target entity variable and the noise character variable are initialized.

[0132] In one embodiment, the determining unit 801 is configured to: obtain text to be recognized; preprocess the text, and determine the preprocessed text as the text to be recognized.

[0133] In one embodiment, the apparatus 800 further includes: a second extraction unit 803, configured to extract a target entity from the text according to a pre-constructed regular expression, where the regular expression is constructed according to the target entity.

[0134] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0135] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0136] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0137] Figure 9A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0139] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 901 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the method of extracting entities. For example, in some embodiments, the method of extracting entities can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method of extracting entities described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the method of extracting entities by any other appropriate means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0146] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0147] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0148] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for extracting an entity, comprising: receiving a text to be recognized, wherein the text includes a target original entity with noise characters inserted therein; Selecting a formal rule and a noise character insertion rule corresponding to the target original entity; Traversing the text according to the formalization rule and the noise character insertion rule, screening characters for splicing a target entity, and using a string spliced ​​from the characters as the target entity extracted from the text; comprising: Traversing the characters in the text, determining the type of the current character and the target entity string length of a target entity variable, wherein the target entity variable is used to store the characters selected for concatenating the target entity; If the type of the current character matches the target type of the target entity, then according to the target entity string length and the formalization rule, the current character is stored in the target entity variable or the target variable is initialized, and traversal continues to the next character; If the type of the current character does not match the target type of the target entity, then the noise character variable used to record the number of noise characters is updated, and based on the updated noise character variable and the length of the entity string, the target variable is initialized or the characters stored in the target entity variable are concatenated into a string representing the target entity and stored in the target entity list, the target variable including the target entity variable and the noise character variable, and the next character is traversed. Until all characters in the text are traversed, the target entity stored in the target entity list is used as the target entity extracted from the text.

2. The method according to claim 1, wherein The formal rules are constructed in the following way: Analyze the target entity sample to determine a set of characters allowed to appear at each character position of the target entity sample, and a constraint relationship between each character position; Formalized rules are constructed based on the set of characters allowed to appear at each character position and the constraint relationship between each character position.

3. The method according to claim 1, wherein The step of storing the current character in the target entity variable or initializing the target variable according to the target entity character string length and the formalization rule includes: If the target entity string length is greater than or equal to the target entity string length threshold, initializing the target entity variable and the noise character variable; If the target entity string length is less than the target entity string length threshold, the current character is matched with the formal rule, and based on the matching result, the current character is stored in the target entity variable or the target variable is initialized.

4. The method according to claim 3, wherein: Matching the current character with the formalized rule and storing the current character in the target entity variable or initializing the target variable based on the matching result includes: Determining a target position of the target entity character to be stored in the target entity according to the length of the target entity character string; determining a character set at the target position in a dictionary data structure, the dictionary data structure being obtained based on parsing the formalized rule; If there is a character matching the current character in the character set, storing the current character in the target entity variable; If there is no character matching the current character in the character set, the target entity variable and the noise character variable are initialized.

5. The method according to claim 1, wherein Initializing a target variable or concatenating characters stored in the target entity variable into a character string representing a target entity and storing the character string in a target entity list based on the updated noise character variable and the entity character string length includes: If the entity string length is equal to the target entity string length threshold, concatenating the characters stored in the target entity variable into a string representing the target entity and storing it in the target entity list, and initializing the target entity variable and the noise character variable; If the length of the entity character string is less than the target entity character string length threshold, and the updated noise character variable is greater than the noise character variable threshold, the target entity variable and the noise character variable are initialized.

6. The method according to any one of claims 1 to 5, wherein: The receiving of the text to be recognized comprises: Get the text to be recognized; The text is preprocessed, and the preprocessed text is determined as the text to be recognized.

7. The method according to claim 6, further comprising: A target entity is extracted from the text according to a pre-built regular expression, wherein the regular expression is built according to the target entity.

8. A device for extracting an entity, comprising: a determination unit, configured to receive a text to be recognized, the text including a target original entity with noise characters inserted therein, and to select a formalization rule and a noise character insertion rule corresponding to the target original entity; A first extraction unit is configured to traverse the text according to the formalization rule and the noise character insertion rule, screen characters for concatenating a target entity, and use a string concatenated with the characters as the target entity to be extracted from the text; and includes: Traversing the characters in the text, determining the type of the current character and the target entity string length of a target entity variable, wherein the target entity variable is used to store the characters selected for concatenating the target entity; If the type of the current character matches the target type of the target entity, then according to the target entity string length and the formalization rule, the current character is stored in the target entity variable or the target variable is initialized, and traversal continues to the next character; If the type of the current character does not match the target type of the target entity, then the noise character variable used to record the number of noise characters is updated, and based on the updated noise character variable and the length of the entity string, the target variable is initialized or the characters stored in the target entity variable are concatenated into a string representing the target entity and stored in the target entity list, the target variable including the target entity variable and the noise character variable, and the next character is traversed. Until all characters in the text are traversed, the target entity stored in the target entity list is used as the target entity extracted from the text.

9. The device according to claim 8, wherein The determining unit is configured to: Analyze the target entity sample to determine a set of characters allowed to appear at each character position of the target entity sample, and a constraint relationship between each character position; Formalized rules are constructed based on the set of characters allowed to appear at each character position and the constraint relationship between each character position.

10. The device according to claim 8, wherein The first extraction unit is further configured to: If the target entity string length is greater than or equal to the target entity string length threshold, initializing the target entity variable and the noise character variable; If the target entity string length is less than the target entity string length threshold, the current character is matched with the formal rule, and based on the matching result, the current character is stored in the target entity variable or the target variable is initialized.

11. The device according to claim 10, wherein The first extraction unit is further configured to: Determining a target position of the target entity character to be stored in the target entity according to the length of the target entity character string; determining a character set at the target position in a dictionary data structure, the dictionary data structure being obtained based on parsing the formalized rule; If there is a character matching the current character in the character set, storing the current character in the target entity variable; If there is no character matching the current character in the character set, the target entity variable and the noise character variable are initialized.

12. The device according to claim 8, wherein The first extraction unit is further configured to: If the entity string length is equal to the target entity string length threshold, concatenating the characters stored in the target entity variable into a string representing the target entity and storing it in the target entity list, and initializing the target entity variable and the noise character variable; If the length of the entity character string is less than the target entity character string length threshold, and the updated noise character variable is greater than the noise character variable threshold, the target entity variable and the noise character variable are initialized.

13. The device according to any one of claims 8 to 12, wherein: The determining unit is configured to: Get the text to be recognized; The text is preprocessed, and the preprocessed text is determined as the text to be recognized.

14. The apparatus according to claim 13, further comprising: The second extraction unit is configured to extract a target entity from the text according to a pre-constructed regular expression, wherein the regular expression is constructed according to the target entity.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Geological named entity extraction method and device

    CN111079436A