Entity recognition method, device, equipment and storage medium based on semantic analysis

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to an entity recognition method, device, equipment and storage medium based on semantic analysis. The method comprises: obtaining a corpus to be recognized, matching the corpus to be recognized in a suffix dictionary, recording all suffix word indexes, and obtaining a suffix word index set. The corpus to be recognized between two adjacent suffix word indexes in the suffix word index set is segmented and extracted to obtain a substring text, the substring text is matched in a candidate dictionary, all candidate word indexes are recorded, and the target candidate word is combined with the adjacent suffix words one by one to obtain a plurality of entity words. The present invention defines the starting boundary of the entity word based on a prefix dictionary, a suffix dictionary and a candidate dictionary, and the matching process is efficient, the search difficulty is small, the recognition range is wide, the portability is strong, and new words and entity words containing special symbols can be recognized to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the field of speech semantics technology in the field of artificial intelligence, and in particular to an entity recognition method, device, equipment and storage medium based on semantic parsing. Background Art

[0002] Named entity recognition (NER) is a basic task in natural language processing. Its purpose is to identify entity words with special meanings in text corpus: such as place names, job occupations or organizational names. It is an essential technical component in system tasks such as intent recognition, knowledge graph construction, information extraction and retrieval.

[0003] Unlike English, Chinese has no spaces as word boundary markers, nor does it have uppercase and lowercase letters to help with word segmentation. In addition, the word usage is varied and the word range is complex, which increases the difficulty of entity recognition. However, the NER recognition methods in the prior art generally include the following categories:

[0004] 1. Construct rule templates in advance and match text with templates to find entities: Manually written rules are required, which requires a lot of linguistic knowledge and vertical field background knowledge, so the scalability and portability are poor;

[0005] 2. Entity recognition based on dictionary: Entity words that may exist in the text are matched in the dictionary. In order to ensure the recognition accuracy, a large-scale dictionary needs to be constructed, which leads to low search efficiency and inability to recognize new words or unfamiliar words outside the dictionary.

[0006] 3. Methods based on statistical machine learning and deep learning: Based on the features learned by artificial construction or neural networks, the model is combined to identify entities in the corpus. However, this method requires a large amount of training data to be labeled in advance, and the model training time is long, it is highly dependent on parameter settings, and the model interpretability is poor.

[0007] Therefore, whether it is using rule templates or methods based on dictionaries or statistical machine learning and deep learning, each has its own significant shortcomings, which restricts the application of NER recognition. Summary of the invention

[0008] In view of this, it is necessary to provide an entity recognition method, device, equipment and storage medium based on semantic parsing to address the problems of existing named entity recognition, such as high recognition difficulty and poor scalability.

[0009] An entity recognition method based on semantic parsing, comprising:

[0010] Obtain a corpus to be recognized, match the corpus to be recognized in a suffix dictionary, and if a target suffix word is matched, record the suffix word indexes of all target suffix words in the corpus to be recognized, sort and merge all suffix word indexes to obtain a suffix word index set;

[0011] Segment and extract the corpus to be recognized corresponding to two adjacent suffix word indexes in the suffix word index set to obtain substring text, match the substring text in a candidate dictionary, and if a target candidate word is matched, record the candidate word indexes of all target candidate words in the corpus to be recognized, and store all target candidate words in a data structure;

[0012] All target candidate words in the data structure are merged to obtain a plurality of merged target candidate words, and the target candidate words are combined one by one with the suffix words adjacent to the rear of the corresponding candidate word index to obtain a plurality of entity words.

[0013] In a possible design, the step of obtaining the corpus to be recognized includes:

[0014] Establishing a connection with the business system and receiving a push request sent by the business system;

[0015] The data in the push request is parsed by a preset parsing method to obtain the corpus to be recognized.

[0016] In a possible design, the sorting and merging of all suffix word indexes includes:

[0017] In all suffix indexes, check whether there are completely overlapping index intervals. If so, merge them into the suffix index with the largest interval.

[0018] Find whether there are partially overlapping index intervals. If so, merge them into the same suffix index, and the index start bit of the suffix index is the minimum value in the partially overlapping index interval, and the index end bit of the suffix index is the maximum value in the partially overlapping index interval;

[0019] Check whether there are adjacent index intervals. If so, merge them into the same suffix index, and the index start bit of the suffix index is the minimum value in the adjacent index interval, and the index end bit of the suffix index is the maximum value in the adjacent index interval.

[0020] In a possible design, the substring text is matched in a candidate dictionary. If a target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure, including:

[0021] Matching the substring text in a candidate dictionary;

[0022] If the target candidate word is matched, the target candidate word is stored in a data structure, and the candidate word index of the target candidate word in the corpus to be recognized is recorded, and it is determined whether the total length of all matched target candidate words is equal to the length of the initial substring text. If they are equal, the match is completed; if they are not equal, the target candidate word and all characters after the target candidate word are removed from the initial substring text to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued;

[0023] If the target candidate word is not matched, the substring text is matched in the prefix dictionary. If the target prefix word is matched, the prefix word index of the target prefix word in the corpus to be recognized is recorded and the matching is ended; if the target prefix word is not matched, it is determined whether there is one character left in the substring text. If there is one character left, the character is removed from the initial substring text to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued. If there is not one character left, the first character of the initial substring text is removed to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued.

[0024] In a possible design, the substring text is matched in a candidate dictionary. If a target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure, including:

[0025] The substring text is matched in the prefix dictionary. If the target prefix word is matched, the matching is terminated. If the target prefix word is not matched, the step of matching the substring text in the candidate dictionary is executed.

[0026] In a possible design, all the target candidate words in the data structure are merged to obtain a plurality of target candidate words after the merger, including:

[0027] In the candidate word indexes corresponding to all target candidate words, find out whether there are partially overlapping index intervals. If so, merge them into the same candidate word index, and the index start bit of the candidate word index is the minimum value in the partially overlapping index interval, and the index end bit of the candidate word index is the maximum value in the partially overlapping index interval;

[0028] Check whether there are adjacent index intervals. If so, merge them into the same candidate word index, and the index start bit of the candidate word index is the minimum value in the adjacent index interval, and the index end bit of the candidate word index is the maximum value in the adjacent index interval.

[0029] In a possible design, after combining the target candidate word with the suffix words adjacent to the rear of the corresponding candidate word index one by one to obtain multiple entity words, the method further includes:

[0030] The entity words are searched one by one in the preset data to be pushed, and it is determined whether the data to be pushed contains matching target data. If matching target data is contained, the target data is extracted and returned to the business system. If matching target data is not contained, an error prompt is returned.

[0031] An entity recognition device based on semantic parsing, comprising:

[0032] A suffix matching module is used to obtain a corpus to be recognized, match the corpus to be recognized in a suffix dictionary, and if a target suffix is ​​matched, record the suffix indexes of all target suffixes in the corpus to be recognized, sort and merge all suffix indexes to obtain a suffix index set;

[0033] A candidate word matching module is used to segment and extract the corpus to be recognized between two adjacent suffix word indexes in the suffix word index set to obtain a substring text, match the substring text in a candidate dictionary, and if a target candidate word is matched, record the candidate word indexes of all target candidate words in the corpus to be recognized, and store all target candidate words in a data structure;

[0034] An entity word module is determined, which is used to merge all target candidate words in the data structure to obtain multiple target candidate words after the merger, and to combine the target candidate words with the adjacent suffix words behind the corresponding candidate word index one by one to obtain multiple entity words.

[0035] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned entity recognition method based on semantic parsing.

[0036] A storage medium storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the above-mentioned entity recognition method based on semantic parsing.

[0037] The entity recognition method, device, equipment and storage medium based on semantic analysis include obtaining a corpus to be recognized, matching the corpus to be recognized in a suffix dictionary, if the target suffix word is matched, recording the suffix word indexes of all target suffix words in the corpus to be recognized, sorting and merging all suffix word indexes to obtain a suffix word index set; segmenting and extracting the corpus to be recognized between two adjacent suffix word indexes in the suffix word index set to obtain a substring text, matching the substring text in a candidate dictionary, if the target candidate word is matched, recording the candidate word indexes of all target candidate words in the corpus to be recognized, and storing all target candidate words in a data structure; merging all target candidate words in the data structure to obtain a plurality of target candidate words after merging, combining the target candidate words with the suffix words adjacent to the corresponding candidate word index one by one, to obtain a plurality of entity words. The present invention defines the starting boundary of entity words by using a prefix dictionary, a suffix dictionary and a candidate dictionary, and determines entity words after matching the main content of the corpus to be recognized in these dictionaries respectively. Because of the combined matching method, only a small prefix dictionary, suffix dictionary and candidate dictionary need to be maintained to complete entity recognition. The matching process is highly efficient, easy to find, has a wide recognition range, strong portability, and can recognize new words and entity words containing special symbols to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0039] Figure 1 is a flow chart of an entity recognition method based on semantic parsing in one embodiment of the present invention;

[0040] Figure 2 It is a structural diagram of an entity recognition device based on semantic parsing in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0043] Figure 1 is a flow chart of an entity recognition method based on semantic parsing in one embodiment of the present invention, such as Figure 1 As shown, a method for entity recognition based on semantic parsing includes the following steps:

[0044] Step S1, matching suffix words: obtaining a corpus to be recognized, matching the corpus to be recognized in a suffix dictionary, and if a target suffix word is matched, recording the suffix word indexes of all target suffix words in the corpus to be recognized, sorting and merging all suffix word indexes to obtain a suffix word index set.

[0045] The suffix dictionary in this step is a dictionary that is pre-built and set. There can be multiple suffix dictionaries, which can be classified by industry or type. For example, taking the occupation entity word as an example, the suffix dictionary contains suffix words such as "teacher", "teacher", "education", and "worker". This step needs to search the suffix dictionary to find out whether the corpus to be identified contains the suffix words in the suffix dictionary. When searching, string matching and other methods can be used to match whether the corpus to be identified contains the target suffix words. If the target suffix words are contained, the target suffix words also need to be suffixed. Indexing. Taking a text sentence as an example, the suffix word index corresponding to multiple target suffix words {idx_1, idx_2, ... idx_n} is recorded, where idx_n is an index interval [idx_n_start, idx_n_end], indicating the index interval of the nth target suffix word, and idx_n_start and idx_n_end respectively represent the index start position and index end position of the nth target suffix word in the text sentence.

[0046] For example, for the occupational word "stainless steel smelting worker" in the corpus to be identified, it is matched in the suffix dictionary. Since the suffix dictionary contains "worker", the target suffix word "worker" is finally found in the occupational word "stainless steel smelting worker". Since in the occupational word "stainless steel smelting worker", when the index starts from 0, the 5th to 6th positions are "worker", the suffix word index corresponding to this target suffix word "worker" is recorded: [5, 6].

[0047] In one embodiment, the corpus to be recognized in step S1 is obtained in the following manner:

[0048] Establish a connection with the business system, receive the push request sent by the business system, parse the data in the push request through the preset parsing method, and obtain the corpus to be recognized.

[0049] The business system can be a business system with push data. The user inputs the corpus to be identified through the push interface of the business system. For example, "stainless steel smelting worker" is entered in the occupation field. The system obtains this occupation field through the push interface of the business system. The business system can also be a system for intent recognition, knowledge graph construction, information extraction or retrieval. After establishing a connection with the business system and achieving communication, the push request is sent to the system. This step obtains the push request sent by the business system after establishing a connection with the business system. Usually, the push request has a preset format. Therefore, the push request sent by the business system can be parsed by regular expressions and other parsing methods, and the occupation field "stainless steel smelting worker" is directly obtained from the request. This occupation field "stainless steel smelting worker" is the corpus to be identified.

[0050] The corpus to be recognized in this embodiment can be directly obtained from the push request of the external business system. The external business system only needs to establish a connection with the entity recognition system of this step to communicate, so that the entity recognition system has good scalability.

[0051] It is possible to find multiple suffix words in the corpus to be recognized, so there will be multiple suffix word indexes. These suffix word indexes are a kind of index interval, which may be adjacent, non-adjacent, or overlapping. When matching suffix words in the suffix dictionary, the reverse maximum matching method is adopted. When searching forward for matches, the suffix word indexes will be arranged from back to front. For example, suffix word indexes [7, 11], [1, 5], [2, 3], etc. will be found. In order to facilitate subsequent calculations, these suffix word indexes need to be sorted. When sorting all the suffix word indexes found, the rule of sorting the suffix word indexes in ascending order of the index starting position size is used. For example, the three indexes mentioned above are sorted as [1, 5], [2, 3], [7, 11].

[0052] In one embodiment, in step S1, all suffix word indexes are sorted and merged, including:

[0053] Step S101, merging completely overlapping indexes: among all suffix word indexes, searching whether there are completely overlapping index intervals, if so, merging them into a suffix word index of the largest interval.

[0054] Completely overlapping suffix indexes are merged into a maximum interval index, for example, suffix index [1, 5] and suffix index [2, 3] are merged into suffix index [1, 5].

[0055] Step S102, merging partially overlapping indexes: searching whether there are partially overlapping index intervals, and if so, merging them into the same suffix index, with the index start bit of the suffix index being the minimum value in the partially overlapping index interval, and the index end bit of the suffix index being the maximum value in the partially overlapping index interval.

[0056] For partially overlapping suffix indexes, they are merged into the same interval index. The minimum value is selected as the starting position of the index, and the maximum value is selected as the ending position. For example, the suffix index [1, 6] and the suffix index [4, 11] are merged into the suffix index [1, 11].

[0057] Step S103, merging adjacent indexes: searching whether there are adjacent index intervals, if so, merging them into the same suffix index, and the index start bit of the suffix index is the minimum value in the adjacent index interval, and the index end bit of the suffix index is the maximum value in the adjacent index interval.

[0058] Adjacent suffix indexes are merged into the same interval index. The minimum value is selected as the start position of the index and the maximum value is selected as the end position. For example, the suffix index [1, 6] and the suffix index [7, 11] are merged into the suffix index [1, 11].

[0059] For suffix indexes other than the above three cases, all index intervals are retained, for example, suffix index [1, 3] and suffix index [7, 11] are still saved as suffix index [1, 3] and suffix index [7, 11].

[0060] This embodiment integrates related index intervals together by the above-mentioned centralized suffix index merging method, so as to facilitate the subsequent segmentation and extraction of relatively complete and accurate substring texts according to the suffix indexes.

[0061] Step S2, matching candidate words: segment and extract the corpus to be recognized corresponding to two adjacent suffix word indexes in the suffix word index set to obtain substring text, and match the substring text in the candidate dictionary. If the target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure.

[0062] The candidate dictionary in this step is a pre-built and set dictionary. There can be multiple candidate dictionaries, which can be classified by industry or type. For example, taking the occupation entity word as an example, the candidate dictionary contains candidate words such as "maintenance", "mathematics", "Chinese", "English", "stainless steel", and "metallurgy".

[0063] The segmentation and extraction in this step refers to removing the suffix words in the corpus to be recognized from the corpus, and only obtaining the text between two adjacent suffix words, which is recorded as the substring text. When the content of the corpus to be recognized is long, there may be multiple suffix words. After segmentation and extraction, there are multiple substring texts. In this step, multiple substring texts are matched in the candidate dictionary respectively to obtain multiple target candidate words.

[0064] For example, the corpus to be identified is the occupation word "stainless steel smelting worker". Through step 1), "worker" is obtained as the suffix word. In this step, "worker" is removed from the corpus, and finally "stainless steel smelting" is obtained as the substring to be identified. All words in "stainless steel smelting" are searched in the candidate dictionary to match whether they contain any target candidate word that is the same as that in the candidate dictionary.

[0065] When searching and matching the identified substring text in the candidate dictionary, the principle of the maximum matching algorithm is adopted, specifically: the substring text is segmented, and then compared with the candidate dictionary. If it contains a certain word, the candidate word index is recorded, otherwise, by adding or reducing a character, the comparison continues until there is one character left, and the search is terminated if the string cannot be segmented and is not in the candidate dictionary. This step preferably uses the forward maximum matching algorithm, and the existing forward maximum matching method is modified before matching.

[0066] In one embodiment, in step S2, the substring text is matched in the candidate dictionary. If a target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure, including:

[0067] Step S201, matching: matching the substring text in the candidate dictionary.

[0068] According to the principle of the forward maximum matching algorithm, in this step, when matching the substring text with the candidate dictionary, the initial substring text is first matched in the candidate dictionary. The initial substring text is the substring text after segmentation and extraction of the corpus to be identified between two adjacent suffix word indexes.

[0069] Step S202, first judgment: if the target candidate word is matched, the target candidate word is stored in a data structure, and the candidate word index of the target candidate word in the corpus to be recognized is recorded, and it is judged whether the total length of all matched target candidate words is equal to the length of the initial substring text, if they are equal, the match is completed; if not equal, the target candidate word and all characters after the target candidate word are removed from the initial substring text to obtain a new substring text, and continue to execute step S201;

[0070] When the target candidate word is matched, the length of the initial substring text is used as the basis to determine whether all matches are completed. If all matches are not completed, the matched target candidate word needs to be removed from the initial substring text before matching.

[0071] Step S203, second judgment: if the target candidate word is not matched, the substring text is matched in the prefix dictionary. If the target prefix word is matched, the prefix word index of the target prefix word in the corpus to be recognized is recorded and the matching is ended; if the target prefix word is not matched, it is determined whether there is one character left in the substring text. If there is one character left, the character is removed from the initial substring text to obtain a new substring text, and step S201 is continued. If there is not one character left, the first character of the initial substring text is removed to obtain a new substring text, and step S201 is continued.

[0072] When the target candidate word is not matched, it is necessary to match the substring text with the prefix dictionary, and determine whether to end the match through the prefix dictionary. The prefix dictionary in this step is a pre-built and set dictionary. There can be multiple prefix dictionaries, which can be classified by industry or type. For example, taking the occupation entity word as an example, the prefix dictionary contains prefix words such as "engaged in", "studying", and "I am". For example, the corpus to be identified is the occupation word "I am engaged in stainless steel smelting", from which the candidate words "smelting" and "stainless steel" are identified. When the word "engaged in" is searched, it is not matched in the candidate dictionary, so it is searched in the prefix dictionary. Since "engaged in" exists in the prefix dictionary, the match is ended.

[0073] The substring text of this embodiment is matched in the candidate dictionary. For example, the corpus to be identified is the occupation word "stainless steel smelting worker", and the target suffix word is "worker". The target suffix word is cut out from it, and the initial substring text is "stainless steel smelting". Assume that the candidate dictionary only contains two words "stainless steel" and "smelting". Using the matching method of this embodiment, the word "stainless steel smelting" will be searched in the candidate dictionary first. If the target candidate word is not matched, "rust steel smelting" and "steel smelting" will continue to be searched until the first target candidate word "smelting" is matched, and the corresponding candidate word index [3, 4] is recorded; then the word "smelting" is cut out from the initial substring text "stainless steel smelting", and the new substring text becomes "stainless steel". Continue to use the matching method of this embodiment, further search and match the new substring in the candidate dictionary, and finally obtain the second target candidate word "stainless steel" and its candidate word index [0, 2]. The first target candidate word "smelting" and the second target candidate word "stainless steel" are combined into "stainless steel smelting", which has the same length as the initial substring to be identified "stainless steel smelting", thus completing the entire search and matching process.

[0074] The above matching process can also be performed by first matching the prefix dictionary and then matching the candidate dictionary:

[0075] Step S211, matching: matching the substring text in the prefix dictionary, if matching the target prefix word, then the matching ends; if not matching the target prefix word, then matching the substring text in the candidate dictionary.

[0076] Step S212, first judgment: if the target candidate word is matched, the target candidate word is stored in the data structure, and the candidate word index of the target candidate word in the corpus to be recognized is recorded, and it is judged whether the total length of all matched target candidate words is equal to the length of the initial substring text, if they are equal, the match is completed; if not equal, the target candidate word and all characters after the target candidate word are removed from the initial substring text to obtain a new substring text, and continue to execute step S211;

[0077] Step S213, second judgment: if the target candidate word is not matched, determine whether there is one character left in the substring text. If there is one character left, remove the character from the initial substring text to obtain a new substring text, and continue to execute step S211. If there is not one character left, remove the first character of the initial substring text to obtain a new substring text, and continue to execute step S211.

[0078] Through the two different matching methods of this embodiment, relatively accurate target candidate words can be obtained. For example, for the corpus waiting to be recognized of "engage in physics education", "engage in mathematics education", and "engage in Chinese education", the target suffix word "education" is obtained through step 1), and then through the matching method of step 2), the target candidate words "mathematics" / "physics" / "Chinese" and the target prefix word "engage in" are finally obtained.

[0079] Step S3, determine the entity words: merge all the target candidate words in the data structure to obtain multiple target candidate words after the merger, and combine the target candidate words with the adjacent suffix words behind the corresponding candidate word index one by one to obtain multiple entity words.

[0080] There may be partially overlapping or adjacent candidate word indexes among the multiple target candidate words obtained in step S2. For example, for the corpus to be recognized "I am engaged in stainless steel smelting work", the candidate word index of the target candidate word "smelting" is [3, 4], and the candidate word index of the target candidate word "stainless steel" is [0, 2]. However, the target candidate words "stainless steel" and the target candidate word "smelting" are adjacent, so they should be merged and finally the target candidate word "stainless steel smelting" is more accurate and reasonable. Finally, "stainless steel smelting" is combined with the target suffix word "work" at the back to finally obtain the entity word "stainless steel smelting work".

[0081] In one embodiment, in step S3, all target candidate words in the data structure are merged to obtain a plurality of target candidate words after the merge process, including:

[0082] Step S301, merging partially overlapping indexes: among the candidate word indexes corresponding to all target candidate words, search whether there are partially overlapping index intervals. If so, merge them into the same candidate word index, and the index starting position of the candidate word index is the minimum value in the partially overlapping index interval, and the index ending position of the candidate word index is the maximum value in the partially overlapping index interval.

[0083] The merging method of this step is similar to that of step S102, that is: for partially overlapping candidate word indexes, they are merged into the same interval index, the minimum value is selected for the starting position of the index, and the maximum value is selected for the ending position. For example, the candidate word index [1, 6] and the candidate word index [4, 11] are merged into the candidate word index [1, 11].

[0084] Step S302, merging adjacent indexes: searching whether there are adjacent index intervals, if so, merging them into the same candidate word index, and the index start bit of the candidate word index is the minimum value in the adjacent index interval, and the index end bit of the candidate word index is the maximum value in the adjacent index interval.

[0085] The merging method of this step is similar to that of step S103, that is: for adjacent candidate word indexes, they are merged into the same interval index, the minimum value is selected as the starting position of the index, and the maximum value is selected as the ending position. For example, the candidate word index [1, 6] and the candidate word index [7, 11] are merged into the candidate word index [1, 11].

[0086] For candidate word indexes other than the above two cases, all index intervals are retained, for example, candidate word index [1, 3] and candidate word index [7, 11] are still saved as candidate word index [1, 3] and candidate word index [7, 11].

[0087] This embodiment integrates related index intervals together by the above-mentioned centralized merging of candidate word indexes, so as to facilitate subsequent combination of candidate word indexes and their adjacent suffix word indexes to ultimately obtain an accurately completed entity word.

[0088] In one embodiment, after step S3, the method further includes:

[0089] Search the entity words one by one in the preset data to be pushed, and determine whether the data to be pushed contains matching target data. If it contains matching target data, extract the target data and return it to the business system. If it does not contain matching target data, return an error prompt.

[0090] The pushed data of this embodiment contains multiple keywords corresponding to entity words. For example, when the data to be pushed is a project product, the project product contains multiple keywords, such as suitable for the crowd, suitable occupations, etc., and the entity words obtained in this step are matched in the suitable occupations. When matching, a fuzzy search method can be used. As long as a partial name in the entity word is included, the match can be considered successful, and the data to be pushed is the target data. If there are multiple matching target data, all target data will be pushed. When pushing the target data, if the corpus to be identified is obtained from the push interface in step 1), the target data will also be displayed through the push interface. If the corpus to be identified is obtained from the business system in step 1), the target data will be returned to the corresponding business system through the established connection.

[0091] In one embodiment, after step S3, the obtained entity words may be checked and corrected:

[0092] After step 3), multiple entity words are obtained, and two adjacent entity words may be merged into one. For example, the entity words obtained are "construction industry" and "stair plasterer". The index positions of the two entity words in the corpus to be identified are adjacent, and they can be further merged into the entity word "construction industry stair plasterer". Therefore, after multiple entity words are identified, the candidate word index and suffix word index of any entity word are first merged into an entity word index, and then the entity word index is searched for adjacent entity word indexes. If they are adjacent, the two entity word indexes are merged into one entity word index.

[0093] After step 3), multiple entity words are obtained. According to the candidate dictionary of the entity words and the industry corresponding to the suffix dictionary, the entity word content of the entity word is corrected: search whether the entity word contains the preset ambiguous words. If it contains ambiguous words, find the corresponding correction words to be added in the preset ambiguous word industry mapping table, and add the correction words in brackets after the ambiguous words. For example, the entity word "Apple Sales Worker" is identified from the corpus to be identified in the electronics industry. Since it is searched in the dictionary of the electronics industry, the "apple" here refers to the brand, not the fruit, so it can be further corrected to "Apple (device) Sales Worker".

[0094] After step 3), multiple entity words are obtained, and they are compared with pre-tagged entity words. If there are differences, the differences are extracted and expanded into the corresponding dictionary. The tagged entity words can also be segmented, and the frequency of each phrase after segmentation can be counted using algorithms such as term frequency-inverse document frequency (TF-IDF), and prefixes, candidate words, and suffixes can be extracted to expand the corresponding dictionary.

[0095] The entity words extracted in this embodiment can be applied to the insurance information processing process. By establishing a connection with the insurance business system, the policyholder information is received. This policyholder information is the corpus to be identified. Through this embodiment, multiple entity words are finally identified. These entity words are the policyholder's occupational information, so as to feedback insurance suggestions related to the occupation or push insurance products.

[0096] The entity words extracted in this embodiment can also be used in the process of constructing a knowledge graph. By establishing a connection with the knowledge graph processing system and receiving the occupation field sent by the knowledge graph system, this occupation field is the corpus to be identified. Through this embodiment, multiple entity words are finally identified and these entity words are fed back to the knowledge graph processing system. These entity words received by the knowledge graph are the occupation attributes of a certain entity. These occupation attributes can be linked to the knowledge graph of a certain entity to enrich the knowledge graph.

[0097] The entity recognition method based on semantic analysis in this embodiment defines the starting boundary of entity words in the corpus to be recognized through the pre-established candidate dictionary, prefix dictionary and suffix dictionary. Subsequently, only three dictionaries of smaller scale need to be maintained to improve entity recognition. The scale of the three dictionaries is one tenth of the dictionaries used in traditional recognition methods, which makes the search and matching process of the corpus to be recognized extremely efficient, easy to find, highly portable, and easy to maintain.

[0098] In one embodiment, a semantic parsing-based entity recognition device is proposed, such as Figure 2 As shown, including:

[0099] The suffix matching module 10 is used to obtain the corpus to be recognized, match the corpus to be recognized in the suffix dictionary, and if the target suffix is ​​matched, record the suffix indexes of all the target suffixes in the corpus to be recognized, sort and merge all the suffix indexes to obtain a suffix index set;

[0100] The candidate word matching module 20 is used to segment and extract the corpus to be recognized between two adjacent suffix word indexes in the suffix word index set to obtain a substring text, match the substring text in the candidate dictionary, and if a target candidate word is matched, record the candidate word indexes of all target candidate words in the corpus to be recognized, and store all target candidate words in a data structure;

[0101] The entity word determination module 30 is used to merge all target candidate words in the data structure to obtain multiple target candidate words after the merger, and to combine the target candidate words with the adjacent suffix words behind the corresponding candidate word index one by one to obtain multiple entity words.

[0102] In one embodiment, a computer device is proposed, including a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the processor implements the steps of the entity recognition method based on semantic parsing in the above-mentioned embodiments when executing the computer-readable instructions.

[0103] In one embodiment, a storage medium storing computer-readable instructions is proposed, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps in the entity recognition method based on semantic parsing in the above-mentioned embodiments. The storage medium may be a non-volatile storage medium.

[0104] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0105] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-mentioned embodiments only express some exemplary embodiments of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A method for entity recognition based on semantic parsing, characterized in that: include: Obtain a corpus to be recognized, match the corpus to be recognized in a suffix dictionary, and if a target suffix word is matched, record the suffix word indexes of all target suffix words in the corpus to be recognized, sort and merge all suffix word indexes to obtain a suffix word index set; The corpus to be recognized corresponding to two adjacent suffix word indexes in the suffix word index set is segmented and extracted to obtain a substring text. The segmentation and extraction is to remove the suffix words in the corpus to be recognized from the corpus, and only obtain the text between the two adjacent suffix words, which is recorded as the substring text. When the content of the corpus to be recognized is long and there are multiple suffix words, there are multiple substring texts after segmentation and extraction. The substring texts are matched in the candidate dictionary. If the target candidate word is matched, the candidate word indexes of all the target candidate words in the corpus to be recognized are recorded, and all the target candidate words are stored in a data structure; Merging all target candidate words in the data structure to obtain a plurality of target candidate words after the merger, and combining the target candidate words with the suffix words adjacent to the rear of the corresponding candidate word index one by one to obtain a plurality of entity words; Among them, the sorting and merging of all suffix word indexes include: searching among all suffix word indexes whether there are completely overlapping index intervals, and if so, merging them into a suffix word index of the largest interval; searching whether there are partially overlapping index intervals, and if so, merging them into the same suffix word index, and the index start bit of the suffix word index is the minimum value in the partially overlapping index interval, and the index end bit of the suffix word index is the maximum value in the partially overlapping index interval; searching whether there are adjacent index intervals, and if so, merging them into the same suffix word index, and the index start bit of the suffix word index is the minimum value in the adjacent index interval, and the index end bit of the suffix word index is the maximum value in the adjacent index interval.

2. The entity recognition method based on semantic parsing according to claim 1 is characterized in that: The step of obtaining the corpus to be recognized includes: Establishing a connection with the business system and receiving a push request sent by the business system; The data in the push request is parsed by a preset parsing method to obtain the corpus to be recognized.

3. The entity recognition method based on semantic parsing according to claim 1 is characterized in that: The substring text is matched in the candidate dictionary. If a target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure, including: Matching the substring text in a candidate dictionary; If the target candidate word is matched, the target candidate word is stored in a data structure, and the candidate word index of the target candidate word in the corpus to be recognized is recorded, and it is determined whether the total length of all matched target candidate words is equal to the length of the initial substring text. If they are equal, the match is completed; if they are not equal, the target candidate word and all characters after the target candidate word are removed from the initial substring text to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued; If the target candidate word is not matched, the substring text is matched in the prefix dictionary. If the target prefix word is matched, the prefix word index of the target prefix word in the corpus to be recognized is recorded and the matching is ended; if the target prefix word is not matched, it is determined whether there is one character left in the substring text. If there is one character left, the character is removed from the initial substring text to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued. If there is not one character left, the first character of the initial substring text is removed to obtain a new substring text, and the step of matching the substring text in the candidate dictionary is continued.

4. The entity recognition method based on semantic parsing according to claim 1 is characterized in that: The substring text is matched in the candidate dictionary. If a target candidate word is matched, the candidate word indexes of all target candidate words in the corpus to be recognized are recorded, and all target candidate words are stored in a data structure, including: Match the substring text in the prefix dictionary. If the target prefix word is matched, the matching is terminated. If the target prefix word is not matched, the substring text is matched in the candidate dictionary. If the target candidate word is matched, the target candidate word is stored in a data structure, and the candidate word index of the target candidate word in the corpus to be recognized is recorded, and it is determined whether the total length of all matched target candidate words is equal to the length of the initial substring text. If they are equal, the match is completed; if they are not equal, the target candidate word and all characters after the target candidate word are removed from the initial substring text to obtain a new substring text, and the step of matching the substring text in the prefix dictionary is continued; If the target candidate word is not matched, determine whether there is one character left in the substring text. If there is one character left, remove the character from the initial substring text to obtain a new substring text, and continue to execute the step of matching the substring text in the prefix dictionary. If there is not one character left, remove the first character of the initial substring text to obtain a new substring text, and continue to execute the step of matching the substring text in the prefix dictionary.

5. The entity recognition method based on semantic parsing according to any one of claims 1 to 4, characterized in that: The step of merging all target candidate words in the data structure to obtain a plurality of target candidate words after merging includes: In the candidate word indexes corresponding to all target candidate words, find out whether there are partially overlapping index intervals. If so, merge them into the same candidate word index, and the index start bit of the candidate word index is the minimum value in the partially overlapping index interval, and the index end bit of the candidate word index is the maximum value in the partially overlapping index interval; Check whether there are adjacent index intervals. If so, merge them into the same candidate word index, and the index start bit of the candidate word index is the minimum value in the adjacent index interval, and the index end bit of the candidate word index is the maximum value in the adjacent index interval.

6. The entity recognition method based on semantic parsing according to claim 2 is characterized in that: After combining the target candidate words with the suffix words adjacent to the rear of the corresponding candidate word index one by one to obtain multiple entity words, the method further includes: The entity words are searched one by one in the preset data to be pushed, and it is determined whether the data to be pushed contains matching target data. If matching target data is contained, the target data is extracted and returned to the business system. If matching target data is not contained, an error prompt is returned.

7. An entity recognition device based on semantic analysis, characterized in that: include: A suffix matching module is used to obtain a corpus to be recognized, match the corpus to be recognized in a suffix dictionary, and if a target suffix is ​​matched, record the suffix indexes of all target suffixes in the corpus to be recognized, sort and merge all suffix indexes to obtain a suffix index set; A candidate word matching module is used to segment and extract the corpus to be identified corresponding to two adjacent suffix word indexes in the suffix word index set to obtain a substring text. The segmentation and extraction is to remove the suffix words in the corpus to be identified from the corpus, and only obtain the text between the two adjacent suffix words, which is recorded as the substring text. When the content of the corpus to be identified is long and there are multiple suffix words, there are multiple substring texts after segmentation and extraction. The substring texts are matched in the candidate dictionary. If the target candidate word is matched, the candidate word indexes of all the target candidate words in the corpus to be identified are recorded, and all the target candidate words are stored in a data structure; Determine an entity word module, which is used to merge all target candidate words in the data structure to obtain multiple target candidate words after the merger, and combine the target candidate words with the suffix words adjacent to the rear of the corresponding candidate word index one by one to obtain multiple entity words; The sorting and merging of all suffix word indexes includes: searching among all suffix word indexes whether there are completely overlapping index intervals, and if so, merging them into a suffix word index of the largest interval; Check whether there are partially overlapping index intervals. If so, merge them into the same suffix word index, and the index start bit of the suffix word index is the minimum value in the partially overlapping index interval, and the index end bit of the suffix word index is the maximum value in the partially overlapping index interval; check whether there are adjacent index intervals. If so, merge them into the same suffix word index, and the index start bit of the suffix word index is the minimum value in the adjacent index interval, and the index end bit of the suffix word index is the maximum value in the adjacent index interval.

8. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the entity recognition method based on semantic parsing as claimed in any one of claims 1 to 6.

9. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the entity recognition method based on semantic parsing as claimed in any one of claims 1 to 6.

Images