Entity disambiguation method, entity disambiguation device, storage medium and electronic device

Through the matching of slot information and conflicting sentence patterns under the target intention, the intelligent robot's ability to disambiguate the entity of short texts is improved, and the problem of difficult dissolution of Chinese short text ambiguity is solved, and more efficient user command understanding is achieved.

CN114595696BActive Publication Date: 2024-11-19GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210209046.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-03
Publication Date
2024-11-19
Estimated Expiration
2042-03-03

AI Technical Summary

Technical Problem

In natural language processing, it is difficult to understand the text context and resolve entity ambiguity of short texts, especially short Chinese texts have poor physical disambiguation effects due to language characteristics, which makes it impossible for intelligent robots to accurately understand user instructions.

Method used

By determining the target intention of the statement to be identified, a first dictionary tree containing slot information and a second dictionary tree with conflicting sentence patterns are obtained, a candidate slot of a keyword is determined using the first dictionary tree, and a combination is made into a first candidate Sentence pattern, when the first candidate sentence pattern is not unique, the second dictionary tree is used to match to obtain the second candidate sentence pattern to achieve entity disambiguation.

Benefits of technology

It improves the accuracy and efficiency of entity disambiguation, combines slot information and conflicting sentences, reduces labor and time costs, and is suitable for intelligent robots to understand user instructions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114595696B_ABST
    Figure CN114595696B_ABST
Patent Text Reader

Abstract

The present disclosure provides an entity disambiguation method, an entity disambiguation device, a storage medium and an electronic device, and relates to the field of natural language processing technology. The entity disambiguation method comprises: determining the target intent corresponding to the sentence to be recognized, and obtaining a first dictionary tree and a second dictionary tree of the target intent, the first dictionary tree containing slot information, and the second dictionary tree containing conflicting sentence patterns; using the first dictionary tree to determine the candidate slots corresponding to the keywords in the sentence to be recognized; obtaining a first candidate sentence pattern by combining the candidate slots corresponding to each of the keywords; in response to the number of the first candidate sentence patterns being greater than 1, matching the first candidate sentence patterns using the second dictionary tree to obtain a second candidate sentence pattern; obtaining the target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern. The present disclosure improves the accuracy and efficiency of entity disambiguation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to an entity disambiguation method, an entity disambiguation device, a computer-readable storage medium, and an electronic device. Background Art

[0002] In natural language processing, since a word may have multiple meanings, entities are ambiguous, which makes it difficult to recognize and understand semantics. For example, in an intelligent robot scenario, users input commands to electronic devices (such as speakers and mobile phones) through voice interaction and other methods, and need to understand the semantics of the commands in order to provide corresponding services. However, the text of the commands is usually short, and there is a high possibility of ambiguity, which makes it impossible for intelligent robots to accurately understand the user's commands. In related technologies, the accuracy of entity disambiguation needs to be improved.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those skilled in the art. Summary of the invention

[0004] The present disclosure provides an entity disambiguation method, an entity disambiguation device, a computer-readable storage medium, and an electronic device to improve the accuracy of entity disambiguation at least to a certain extent.

[0005] According to a first aspect of the present disclosure, a method for entity disambiguation is provided, comprising: determining a target intent corresponding to a sentence to be recognized, and obtaining a first dictionary tree and a second dictionary tree of the target intent, wherein the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns; using the first dictionary tree to determine candidate slots corresponding to keywords in the sentence to be recognized; obtaining a first candidate sentence pattern by combining the candidate slots corresponding to the keywords; in response to the number of the first candidate sentence patterns being greater than 1, matching the first candidate sentence patterns with the second dictionary tree to obtain a second candidate sentence pattern; and obtaining a target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern.

[0006] According to a second aspect of the present disclosure, an entity disambiguation device is provided, comprising: a dictionary tree acquisition module, configured to determine a target intent corresponding to a sentence to be recognized, and to acquire a first dictionary tree and a second dictionary tree of the target intent, wherein the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns; a candidate slot determination module, configured to determine candidate slots corresponding to keywords in the sentence to be recognized using the first dictionary tree; a first candidate sentence pattern determination module, configured to obtain a first candidate sentence pattern by combining the candidate slots corresponding to the keywords; a second candidate sentence pattern determination module, configured to match the first candidate sentence pattern using the second dictionary tree in response to the number of the first candidate sentence patterns being greater than 1, to obtain a second candidate sentence pattern; and a target entity output module, configured to obtain a target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern.

[0007] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the entity disambiguation method of the first aspect and its possible implementation methods are implemented.

[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the entity disambiguation method of the above-mentioned first aspect and its possible implementation method by executing the executable instructions.

[0009] The technical solution disclosed in this disclosure has the following beneficial effects:

[0010] First, determine the target intent corresponding to the sentence to be recognized, and obtain the first dictionary tree containing slot information and the second dictionary tree containing conflicting sentences under the target intent; then, use the first dictionary tree to determine the candidate slots corresponding to the keywords in the sentence to be recognized, and combine the candidate slots corresponding to different keywords to obtain the first candidate sentence; next, when the first candidate sentence is not unique, use the second dictionary tree to match the first candidate sentence to obtain the second candidate sentence; finally, obtain the target entity corresponding to the sentence to be recognized according to the second candidate sentence. On the one hand, this scheme combines the slot information and conflicting sentences under the target intent, and can effectively determine the slots and resolve ambiguities for the keywords in the sentence to be recognized, thereby improving the accuracy of entity disambiguation. On the other hand, the candidate slots are determined by matching the sentence to be recognized with the first dictionary tree, and the second candidate sentence is determined by matching the first candidate sentence with the second dictionary tree to achieve ambiguity resolution, taking advantage of the high efficiency of dictionary tree search, which can improve the efficiency of entity disambiguation. On the other hand, in this solution, both the first dictionary tree and the second dictionary tree can be constructed based on a small amount of information under a specific intention, without investing a lot of manpower and time costs, so the solution is highly practical.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram showing a system architecture in this exemplary embodiment;

[0014] Figure 2 A flowchart of an entity disambiguation method in this exemplary embodiment is shown;

[0015] Figure 3 A flowchart showing matching of sentences to be recognized by a first dictionary tree in this exemplary embodiment is shown;

[0016] Figure 4 A schematic diagram showing the extraction of keywords and determination of candidate slots for a sentence to be recognized in this exemplary embodiment;

[0017] Figure 5 A flowchart showing matching of the first candidate sentence pattern by the second dictionary tree in this exemplary embodiment is shown;

[0018] Figure 6 A schematic diagram showing matching of the first candidate sentence pattern by the second dictionary tree in this exemplary embodiment;

[0019] Figure 7 A schematic flow chart of an entity disambiguation method in this exemplary embodiment is shown;

[0020] Figure 8 A schematic diagram showing the structure of an entity disambiguation device in this exemplary embodiment is shown;

[0021] Fig. 9 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. These embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, the known technical solutions are not shown or described in detail to avoid obscuring the various aspects of the present disclosure.

[0023] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0024] In the case of short texts, due to the defects of short texts such as serious colloquialism and lack of contextual content, it is more difficult to understand the text context and resolve entity ambiguity. In addition, due to the characteristics of the Chinese language itself, compared with English, sentences usually have multiple word segmentation methods, which increases the ambiguity of entities. The relevant technology has a poor effect on entity disambiguation for Chinese short texts.

[0025] In view of one or more of the above problems, an exemplary embodiment of the present disclosure first provides an entity disambiguation method for outputting a target entity corresponding to a sentence to be recognized, where the target entity may be a real entity of the sentence to be recognized, thereby helping to achieve natural language understanding. The entity disambiguation method in this exemplary embodiment may include other related processing for achieving entity disambiguation, such as entity recognition, entity linking, entity unification, etc.

[0026] Combine the following Figure 1 The system architecture and application scenarios of the operating environment of this exemplary embodiment are exemplarily described.

[0027] Figure 1A schematic diagram of a system architecture is shown, and the system architecture 100 may include a terminal 110 and a server 120. The terminal 110 may be a smart phone, a tablet computer, a personal computer, etc., and the terminal 110 may receive a user input or a specified sentence to be recognized. The server 120 may generally refer to a background system that provides entity disambiguation related services, and may be a server or a cluster formed by multiple servers. The terminal 110 and the server 120 may be connected via a wired or wireless communication link to exchange data.

[0028] In one embodiment, the user inputs a sentence to be recognized to the terminal 110, such as the user can wake up the intelligent robot on the terminal 110 and input a voice command, and the terminal 110 converts the voice command into a text-formatted sentence to be recognized. Then, the terminal 110 can send the sentence to be recognized to the server 120, and the server 120 obtains the target entity corresponding to the sentence to be recognized by executing the above-mentioned entity disambiguation method, and can return the target entity to the terminal 110, so that the terminal 110 responds to the user's voice command according to the target entity. Therefore, the execution subject of the above-mentioned entity disambiguation method can be the server 120.

[0029] In one implementation, this exemplary implementation may also be implemented independently based on the terminal 110. For example, after acquiring the sentence to be recognized input by the user, the terminal 110 executes the above entity disambiguation method to obtain the target entity corresponding to the sentence to be recognized.

[0030] In one implementation, this exemplary implementation may also be implemented independently based on the server 120. For example, the server 120 may obtain the sentence to be recognized from the background database, and obtain the target entity corresponding to the sentence to be recognized by executing the above entity disambiguation method.

[0031] As can be seen from the above, in this exemplary embodiment, the entity disambiguation method may be executed by the terminal 110 or the server 120. This disclosure does not limit this.

[0032] Combine the following Figure 2 The entity disambiguation method in this exemplary embodiment is described. Figure 2 An exemplary process of the entity disambiguation method is shown, which may include the following steps S210 to S250:

[0033] Step S210, determining the target intent corresponding to the sentence to be recognized, and obtaining a first dictionary tree and a second dictionary tree of the target intent, wherein the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns;

[0034] Step S220, using the first dictionary tree to determine candidate slots corresponding to the keywords in the sentence to be recognized;

[0035] Step S230, obtaining a first candidate sentence pattern by combining candidate slots corresponding to the keywords;

[0036] Step S240, in response to the number of the first candidate sentences being greater than 1, matching the first candidate sentences using the second dictionary tree to obtain a second candidate sentence;

[0037] Step S250, obtaining a target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern.

[0038] In the above entity disambiguation method, first, the target intent corresponding to the sentence to be recognized is determined, and the first dictionary tree containing slot information and the second dictionary tree containing conflicting sentences under the target intent are obtained; then, the candidate slots corresponding to the keywords in the sentence to be recognized are determined by using the first dictionary tree, and the candidate slots corresponding to different keywords are combined to obtain the first candidate sentence; next, when the first candidate sentence is not unique, the first candidate sentence is matched by using the second dictionary tree to obtain the second candidate sentence; finally, the target entity corresponding to the sentence to be recognized is obtained according to the second candidate sentence. On the one hand, this scheme combines the slot information and conflicting sentences under the target intent, and can effectively determine the slots and resolve ambiguities for the keywords in the sentence to be recognized, thereby improving the accuracy of entity disambiguation. On the other hand, the candidate slots are determined by matching the sentence to be recognized with the first dictionary tree, and the second candidate sentence is determined by matching the first candidate sentence with the second dictionary tree to achieve ambiguity resolution, which takes advantage of the high efficiency of dictionary tree search and can improve the efficiency of entity disambiguation. On the other hand, in this solution, both the first dictionary tree and the second dictionary tree can be constructed based on a small amount of information under a specific intention, without investing a lot of manpower and time costs, so the solution is highly practical.

[0039] Below Figure 2 Each step in the procedure is described in detail.

[0040] refer to Figure 2 In step S210, the target intent corresponding to the sentence to be recognized is determined, and a first dictionary tree and a second dictionary tree of the target intent are obtained, the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns.

[0041] The sentence to be recognized is a sentence that needs entity disambiguation. Generally, the purpose of entity disambiguation is to determine the target entity corresponding to the sentence to be recognized, and the target entity can be used to accurately understand the semantics of the sentence to be recognized. Therefore, the entity disambiguation method in this exemplary embodiment can be used to perform semantic recognition on the sentence to be recognized.

[0042] The present disclosure does not limit the source of the sentence to be recognized, for example, it can be a query sentence input by the user in any way, or it can be a sentence obtained from the Internet, such as a conversation sentence in a forum, a sentence in an article, a sentence extracted from a video, etc.

[0043] Intent refers to the purpose of a sentence. For example, the sentence to be recognized is "play Stephen Chow's The Mermaid", and its corresponding target intent can be "accurately play multimedia". The present disclosure does not limit the specific method of intent recognition. In one embodiment, a machine learning model for intent prediction (or classification) can be pre-trained, and the sentence to be recognized can be input into the model to obtain the result of intent prediction, that is, the target intent.

[0044] In this exemplary embodiment, the first dictionary tree and the second dictionary tree can be pre-established for different intentions. The first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentences. The slot information may include slots corresponding to different words, and the slots can be used to represent the attributes of words from a semantic level. For example, under the intention of "accurately playing movies and TV shows", the slots corresponding to the word "Stephen Chow" may include [director] and [actor], and the slots corresponding to the word "Mermaid" may include [movie_name], [tv_name], and [song_name]. Conflicting sentences refer to sentences that conform to people's daily language habits, which can be formed by the context of slots and slots. For example, {play [director]'s [movie_name]} is a conflicting sentence, while {play [director]'s [song_name]} is not a conflicting sentence. The former conforms to the habit of daily speaking, while the latter does not.

[0045] Assume that a domain A has N intents and M slots. i represents the i-th intention, A i Intention exists M i slots, represented as a set S i ={[slot 1 ]、[slot 2 ], …, [slot Mi ]},M i ≥0. Different intentions can share slots, for example, intention A i With intention A j Shared slots 2 ], indicating [slot 2 ] is intention A i The slot is also the intention A j Therefore, M=Len(S1 ∪S 2 ∪…∪S N ). Different intents may not share conflicting sentences, that is, a conflicting sentence can belong to only one intent. The slots contained in different conflicting sentences are generally not exactly the same.

[0046] When establishing the first dictionary tree, the correspondence between words and slots under a certain intent may be obtained first, and the first dictionary tree of the intent may be obtained by representing such correspondence in the form of a tree diagram.

[0047] In one embodiment, a basic dictionary tree of a field (such as a vertical field) to which a certain intent belongs can be obtained. The basic dictionary tree can be a trie tree (a prefix tree) generated by traversing the dictionary of the field. The basic dictionary tree contains the prefix relationship between different words in the field. By adding the slot information under the intent to the basic dictionary tree, a first dictionary tree is obtained. For example, each leaf node in the basic dictionary tree represents a complete word, and the slot corresponding to the word can be added to the leaf node, thereby completing the addition of the correspondence between the word and the slot. After adding all the correspondences between the word and the slot, the first dictionary tree is obtained.

[0048] When establishing the second dictionary tree, a conflicting sentence set under a certain intent may be obtained first, which includes all conflicting sentence sets under the intent, which may be obtained by gradual accumulation or by extracting sentence information from a large number of actual user sentences. The second dictionary tree of the intent is obtained by representing the conflicting sentence sets in the form of a tree diagram.

[0049] In one implementation, a basic dictionary tree of a field to which a certain intent belongs can be obtained, and a second dictionary tree can be obtained by adding conflicting sentences under the intent to the basic dictionary tree. For example, the conflicting sentence can be regarded as a string, and a node corresponding to the end character of a conflicting sentence can be found in the basic dictionary tree, and the conflicting sentence can be added to the node, thereby completing the addition of the conflicting sentence. After all conflicting sentences are added, the second dictionary tree is obtained.

[0050] In one implementation, after all conflicting sentences under a certain intention are obtained, a dictionary tree (such as a trie tree) of all strings may be established with each conflicting sentence as a string to obtain a second dictionary tree.

[0051] In one embodiment, the first dictionary tree and the second dictionary tree may be the same dictionary tree, which includes slot information and conflicting sentences. For example, a basic dictionary tree for a field to which an intent belongs may be obtained, and a comprehensive dictionary tree may be obtained by adding slot information and conflicting sentences to the basic dictionary tree. The comprehensive dictionary tree is both the first dictionary tree and the second dictionary tree.

[0052] In one embodiment, the first dictionary tree and the second dictionary tree may also be two subtrees of the above-mentioned comprehensive dictionary tree. For example, the first dictionary tree may be obtained by extracting the branch containing the node containing the slot information from the comprehensive dictionary tree, and the second dictionary tree may be obtained by extracting the branch containing the node containing the conflicting sentence from the comprehensive dictionary tree.

[0053] In one embodiment, at least one of the first dictionary tree and the second dictionary tree may be a DAT (Double-Array Trie). The DAT may be represented by a state transition (base) array and a verification (check) array, and string matching may be achieved through these two arrays. Exemplarily, the first dictionary tree may include a first state transition array and a first verification array, and the second dictionary tree may include a second state transition array and a second verification array.

[0054] In one embodiment, at least one of the first dictionary tree and the second dictionary tree can be a DAT (hereinafter referred to as ACDAT) based on an AC automaton (a matching method based on a trie tree). The AC automaton adds a fallback (fail) array and an output (output) array on the basis of the trie tree to achieve string matching. In this exemplary embodiment, the AC automaton is combined with the DAT into an ACDAT, and the trie tree in the AC automaton can be replaced by the state transition array and the verification array in the DAT, that is, the ACDAT can be embodied as four arrays: a state transition array, a verification array, a fallback array, and an output array. In this way, the negative impact caused by the space complexity of the trie tree and the performance consumption of the hash function can be improved, and efficient state matching transfer, matching judgment on a single state, and matching judgment of a complete string can be achieved, which has the advantages of both the AC automaton and the DAT, and is conducive to improving the efficiency and accuracy of string matching. Exemplarily, the first dictionary tree may include a first state transition array, a first verification array, a first fallback array, and a first output array, and the second dictionary tree may include a second state transition array, a second verification array, a second fallback array, and a second output array.

[0055] Continue to refer Figure 2 In step S220, the first dictionary tree is used to determine the candidate slots corresponding to the keywords in the sentence to be recognized.

[0056] Keywords can be information related to the target intent or of interest to the user in the sentence to be identified. Each keyword can correspond to one or more slots. When a keyword corresponds to multiple slots, it usually means that the semantics of the keyword under the target intent are not unique (that is, there is ambiguity), so these slots are called candidate slots, and a target slot can be determined from them later to obtain the semantics of the keyword in the sentence to be identified, thereby achieving entity disambiguation for the keyword. When a keyword corresponds to only one slot, it usually means that the semantics of the keyword under the target intent are unique, and the slot is both a candidate slot and a target slot.

[0057] The present disclosure does not limit the method of extracting keywords from the sentence to be recognized, and may include any one or more combinations of the following:

[0058] Keyword extraction based on a dictionary. The dictionary can be a dictionary in the field to which the target intent belongs. The words in the dictionary are searched in the sentence to be recognized, and the words found are extracted as keywords. For example, the sentence to be recognized can be pattern matched by the basic dictionary tree in the field to which the target intent belongs to extract the keywords. This method can achieve exhaustive enumeration of keywords.

[0059] Keyword extraction based on rules, where the rules can be embodied as regular expressions, searches for substrings that match the regular expressions in the sentences to be identified, thereby obtaining the keywords in the sentences to be identified. This method can extract more complex keywords (such as keywords that are actually phrases).

[0060] Model-based keyword extraction. The model is usually for a certain category of words, such as names of people, places, etc. By training the corresponding model, keywords of a specific category in the sentence to be identified based on the model can be extracted. This method has a certain generalization ability and can extract keywords that have not appeared before.

[0061] In one embodiment, the first dictionary tree can be obtained by adding slot information to the basic dictionary tree, so that the first dictionary tree includes information about the dictionary in the field to which the target intent belongs, which can be used to extract keywords. The above-mentioned determination of the candidate slot corresponding to the keyword in the sentence to be recognized using the first dictionary tree can include the following steps:

[0062] Matching the substring in the sentence to be recognized with the node of the first dictionary tree, and when the match is successful, determining the substring as a keyword substring;

[0063] The keyword in the sentence to be recognized is determined according to the keyword substring, and the candidate slot corresponding to the keyword is determined according to the slot information corresponding to the node matching the keyword substring.

[0064] The sentence to be recognized is regarded as a string, and any continuous characters in the string form a substring. The first dictionary tree can be used to perform pattern matching on the sentence to be recognized. If a substring in the sentence to be recognized successfully matches a node in the first dictionary tree, it means that the substring is the word corresponding to the node, and the substring is marked as a keyword substring. Furthermore, the keywords in the sentence to be recognized can be determined based on the keyword substring. For example, the keyword substring can be extracted as a keyword. And the candidate slot corresponding to the keyword can be determined based on the slot information corresponding to the node matching the keyword substring.

[0065] For example, the sentence to be identified is "play cross-era bubbles", the substring "cross-era" successfully matches the node 15 of the first dictionary tree, and "cross-era" is used as the keyword substring to obtain the keyword <cross-era>. The slot information corresponding to node 15 includes slots [album_name] (album name), [song_name], and the candidate slots corresponding to the keyword <cross-era> include [album_name], [song_name]. The substring "bubble" successfully matches the node 27 of the first dictionary tree, and "bubble" is used as the keyword substring to obtain the keyword <bubble>. The slot information corresponding to node 27 includes slots [album_name], [song_name], and the candidate slots corresponding to the keyword <bubble> include [album_name], [song_name].

[0066] Through the above method, keyword extraction and candidate slot determination can be combined into one step (eg, called slot extraction) for execution, thereby improving efficiency and response speed.

[0067] In one implementation, the above-mentioned determining the keywords in the sentence to be recognized based on the keyword substring may include the following steps:

[0068] In response to the existence of intersections among the plurality of keyword substrings, the keyword substrings are screened according to their lengths so that there is no intersection among the screened keyword substrings.

[0069] Among them, there is an intersection between different keyword substrings, indicating that different keywords are in an overlapping or inclusive relationship. For example, the sentence to be recognized is "play Happy Birthday", and the keyword substrings "release", "birthday", and "happy birthday" are obtained through matching. "Release" and "birthday" and "happy birthday" are all in an overlapping relationship, and there is an intersection "birth", "birthday" and "happy birthday" are in an inclusive relationship, and there is an intersection "birthday". The keyword substring with the longest length "Happy Birthday" can be filtered out, and the keyword <Happy Birthday> is extracted, while the keywords <release> and <birthday> are not extracted. This avoids more ambiguity caused by the intersection between keyword substrings.

[0070] In actual applications, more complex relationships may occur, such as the situation where keyword substrings A and B intersect, B and C intersect, but A and C do not intersect. In this regard, after finding all keyword substrings, different combinations can be performed to obtain multiple keyword substring sequences, where the keyword substrings in each sequence do not intersect, and then the keyword substring sequence with the largest total length (the sum of the lengths of all keyword substrings in the sequence) is selected, and the keywords in the sentence to be identified are obtained from the keyword substrings in the sequence.

[0071] Thus, a combination of local matching and global longest matching is achieved, which is conducive to extracting keywords that are more accurate and more in line with the context of the sentence to be recognized.

[0072] In one embodiment, extracting keywords and determining candidate slots may be performed as two independent steps. For example, keywords in the sentence to be recognized may be extracted based on rules or models, and then the candidate slots corresponding to the keywords may be determined using the first dictionary tree.

[0073] In one implementation, the first dictionary tree may be an ACDAT, comprising a first state transition array, a first verification array, a first fallback array, and a first output array. Figure 3 As shown, the above matching of the substring in the sentence to be recognized with the node of the first dictionary tree may include the following steps S310 to S340:

[0074] Step S310, determining the current to-be-matched node corresponding to the current to-be-matched substring in the to-be-recognized sentence according to the first state transition array;

[0075] Step S320, verifying whether the current substring to be matched and the current node to be matched successfully according to the first verification array;

[0076] Step S330, when the current substring to be matched and the current node to be matched are not matched successfully, the current substring to be matched and the current node to be matched are updated according to the first fallback array;

[0077] Step S340, when the current substring to be matched successfully matches the current node to be matched, if it is determined according to the first output array that the current node to be matched meets the output condition, the current substring to be matched is output as the keyword substring, and the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched; if it is determined according to the first output array that the current node to be matched does not meet the output condition, the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched.

[0078] When matching begins, the first character in the sentence to be recognized is usually used as the initial current substring to be matched, and according to the first state transfer array, it is transferred from the root node of the first dictionary tree to the current node to be matched corresponding to the current substring to be matched. In the matching process, the sentence to be recognized is usually matched character by character. For example, in step S340, when the current substring to be matched successfully matches the current node to be matched, regardless of whether the keyword substring is output, the current substring to be matched is added with the next character to obtain a new current substring to be matched. For example, assuming that the current substring to be matched "birthday" in the sentence to be recognized "play happy birthday" is matched successfully, after adding the next character, the current substring to be matched is updated to "happy birthday". In step S310, the first state transfer array can be used to quickly transfer to the next node based on the previous node. For example, based on the node s1 (s1 represents the node identifier, which can be a subscript in the array) that matches the substring "birthday", the first state transition array is used to calculate base[s1]+code(fast) (representing the encoding corresponding to the character "fast", such as ASCII code) to obtain the identifier of the node t1 corresponding to "Happy Birthday", which is equivalent to updating the current node to be matched from node s1 to node t1.

[0079] After each node transfer, the first verification array can be used to verify whether the parent node of the current node to be matched is the previous node of the current node to be matched, thereby verifying the correctness of the transfer between nodes. For example, after transferring from node s1 to node t1, it can be verified whether the check value of node t1 is node s1. If the parent node of the current node to be matched is the previous node, it means that the node transfer is correct and the current substring to be matched successfully matches the current node to be matched. If the parent node of the current node to be matched is not the previous node, it means that the node transfer is incorrect and the current substring to be matched does not match the current node to be matched successfully.

[0080] In the event of an unsuccessful match, the current substring to be matched and the current node to be matched can be updated according to the first fallback array. For example, assuming that the current substring to be matched "Happy Birthday" is not matched successfully with the current node to be matched t1, the node r1 pointed to by the fallback pointer of node t1 is found according to the first fallback array, and both satisfy r1=fail(t1). The r1 node is used as the current node to be matched, and the string corresponding to the r1 node is used as the current substring to be matched. If the parent node of the r1 node is the root node, the string corresponding to the r1 node is "fast", that is, the current substring to be matched is "fast". It can be seen that fast fallback can be achieved through the first fallback array, and the matched prefix information is fully utilized to avoid excessive fallback.

[0081] Generally, the current substring to be matched and the current node to be matched obtained after the first fallback array is backed off can be matched successfully, so there is no need to verify through step S320. It can be assumed that the current substring to be matched and the current node to be matched are matched successfully, and then step S340 is executed.

[0082] In step S340, it is determined whether the current node to be matched meets the output condition according to the first output array. Among them, the first output array can indicate whether the current node to be matched is a leaf node (in the first dictionary tree, the leaf node can have child nodes). If so, the output condition is met. If the current node to be matched meets the output condition, it indicates that the current substring to be matched is a pattern string in the first dictionary tree, and it is output as a keyword substring, and the current substring to be matched is updated. If the current node to be matched does not meet the output condition, it indicates that the current substring to be matched is not a pattern string in the first dictionary tree, and only the current substring to be matched is updated without outputting the keyword substring.

[0083] It should be understood that Figure 3 The steps shown can be a loop process. When the last character of the current substring to be matched is the last character in the sentence to be recognized, the current character to be matched can no longer be updated by adding the next character. At this time, the current substring to be matched and the current node to be matched can be updated according to the first fallback array until the current node to be matched pointed to by the first fallback array is the root node, and the entire matching process ends.

[0084] Figure 4 The schematic diagram of extracting keywords and determining candidate slots for the sentence to be recognized is shown. The sentence to be recognized is: I want to listen to "Seven-Li Xiang" from Jay Chou's album Seven-Li Xiang. The target intent of the sentence to be recognized is determined to be "accurately play multimedia". Using the first dictionary tree under the target intent, the keywords <Jay Chou>, <Seven-Li Xiang>, <Seven-Li Xiang> are extracted, and it is determined that the candidate slots corresponding to <Jay Chou> include [actor], [singer], the candidate slots corresponding to the former <Seven-Li Xiang> include [album_name], [song_name], and the candidate slots corresponding to the latter <Seven-Li Xiang> also include [album_name], [song_name].

[0085] Continue to refer Figure 2 In step S230, the first candidate sentence pattern is obtained by combining the candidate slots corresponding to the keywords.

[0086] Since the number of candidate slots corresponding to a keyword may be greater than 1, it means that the semantics of the keyword is not unique, and the semantics of the sentence to be recognized can be regarded as a combination of the semantics of the keywords. Therefore, by combining the candidate slots corresponding to each keyword, a first candidate sentence pattern for representing the semantics of the sentence to be recognized (the semantics here may not be the exact semantics, but can be called candidate semantics) can be obtained.

[0087] In one embodiment, the keywords in the sentence to be recognized can be replaced with their corresponding candidate slots to obtain the first candidate sentence pattern. For example, the result of keyword extraction and candidate slot determination for the sentence to be recognized "play Stephen Chow's Mermaid" is: keyword <Stephen Chow>, corresponding to the candidate slots [director] and [actor], keyword <Mermaid>, corresponding to the candidate slots [movie_name] and [tv_name]. Thus, the first candidate sentence pattern corresponding to the sentence to be recognized is obtained, as shown below:

[0088] First candidate sentence 1: Play [movie_name] by [director];

[0089] First candidate sentence 2: Play [tv_name] of [director];

[0090] First candidate sentence 3: Play [movie_name] by [actor];

[0091] First candidate sentence 4: Play [tv_name] of [actor].

[0092] It can be seen that after different combinations of candidate slots corresponding to keywords in different positions, the first candidate sentence is obtained. Assume that a keywords are extracted from the sentence to be recognized, and they are denoted as w 1 、w 2 ,…,w a , the corresponding number of candidate slots is k 1 , k 2 , …, k a , then the number of the first candidate sentences can be k 1 k 2 …k a .

[0093] Continue to refer Figure 2 In step S240, in response to the number of the first candidate sentences being greater than 1, the first candidate sentences are matched using the second dictionary tree to obtain the second candidate sentences.

[0094] If the number of the first candidate sentence pattern is 1, the semantics of the sentence to be identified is unique. Generally speaking, when the number of candidate slots corresponding to each keyword in the sentence to be identified is 1, the number of the first candidate sentence pattern is 1. In other words, when the semantics of each keyword is unique, the semantics of the sentence to be identified is unique. In this case, step S240 can be skipped, and the target entity corresponding to the sentence to be identified is obtained according to the first candidate sentence, or the first candidate sentence is used as the second candidate sentence and step S250 is executed.

[0095] If the number of the first candidate sentences is greater than 1, the semantics of the sentence to be recognized is not unique, and the first candidate sentences are matched using the second dictionary tree to determine a sentence that conforms to daily language habits and obtain a second candidate sentence.

[0096] In one implementation, the above-mentioned filtering out the second candidate sentence pattern from the first candidate sentence pattern by using the second dictionary tree may include the following steps:

[0097] The first candidate sentence is matched with a node of the second dictionary tree, and a second candidate sentence is determined according to the successfully matched first candidate sentence and at least one of the conflicting sentences corresponding to the node.

[0098] Among them, when matching the first candidate sentence pattern with the nodes of the second dictionary tree, a complete match can be performed, that is, the complete first candidate sentence pattern is matched with a node of the second dictionary tree, or a partial match can be performed, that is, the clause pattern in the first candidate sentence pattern is matched with a node of the second dictionary tree. If the match is successful, it means that the conflict sentence pattern corresponding to the node can match the first candidate sentence pattern, that is, the first candidate sentence pattern conforms to or contains the conflict sentence pattern, and it can be considered that the first candidate sentence pattern conforms to daily language habits. Furthermore, the second candidate sentence pattern is determined based on the first candidate sentence pattern conforming to at least one of the conflicting sentence patterns. The second candidate sentence pattern can be the first candidate sentence pattern, it can also be a conflicting sentence pattern, or it can be a sentence pattern that is deformed based on the first candidate sentence pattern or the conflicting sentence pattern.

[0099] In one embodiment, the above-mentioned matching of the first candidate sentence with the node of the second dictionary tree and determining the second candidate sentence according to the successfully matched first candidate sentence and at least one of the conflicting sentences corresponding to the node may include the following steps:

[0100] Matching the clause formula in the first candidate sentence formula with the nodes of the second dictionary tree, and when the match is successful, determining the clause formula as a matched clause formula, and determining the conflicting sentence formula corresponding to the node matching the matched clause formula as a matched conflicting sentence formula;

[0101] A second candidate sentence pattern is determined according to at least one of the matched sub-sentence pattern and the matched conflicting sentence pattern.

[0102] Among them, the sub - sentence pattern in the first candidate sentence pattern can be any segment of the first candidate sentence pattern. For example, the first candidate sentence pattern can be regarded as a string. It should be noted that a slot in the first candidate sentence pattern can be regarded as a character. For example, for the first candidate sentence pattern 1: Play the [movie_name] of [actor], including a total of 5 characters: play, the, [actor], of, [movie_name]. The sub - sentence pattern can be any substring among them.

[0103] The second dictionary tree can be used to perform pattern matching on the first candidate sentence pattern. If a certain sub - sentence pattern in the first candidate sentence pattern matches a certain node in the second dictionary tree successfully, it indicates that this sub - sentence pattern conforms to the conflict sentence pattern corresponding to this node. Mark this sub - sentence pattern as the matched sub - sentence pattern, and mark the conflict sentence pattern corresponding to this node as the matched conflict sentence pattern. There is a matching relationship between this matched sub - sentence pattern and this matched conflict sentence pattern. Furthermore, the second candidate sentence pattern can be determined according to at least one of the matched sub - sentence pattern and the matched conflict sentence pattern. For example, the matched sub - sentence pattern or the matched conflict sentence pattern can be determined as the second candidate sentence pattern, or the matched sub - sentence pattern or the matched conflict sentence pattern can be transformed to obtain the second candidate sentence pattern, and so on.

[0104] In one implementation, the second dictionary tree can be ACDAT, including a second state transition array, a second verification array, a second back - off array, and a second output array. Refer to Figure 5 As shown, when matching the sub - sentence pattern in the first candidate sentence pattern with the nodes in the second dictionary tree, when the match is successful, determining the sub - sentence pattern as the matched sub - sentence pattern can include the following steps S510 to S540:

[0105] Step S510: Determine the currently to - be - matched node corresponding to the currently to - be - matched sub - sentence pattern in the first candidate sentence pattern according to the second state transition array;

[0106] Step S520: Verify whether the currently to - be - matched sub - sentence pattern matches the currently to - be - matched node according to the second verification array;

[0107] Step S530: When the currently to - be - matched sub - sentence pattern does not match the currently to - be - matched node, update the currently to - be - matched sub - sentence pattern and the currently to - be - matched node according to the second back - off array;

[0108] Step S540: When the current sub - sentence pattern to be matched successfully matches the current node to be matched, if it is determined according to the second output array that the current node to be matched meets the output condition, output the current sub - sentence pattern to be matched as the matched sub - sentence pattern, and add the next character of the current sub - sentence pattern to the current sub - sentence pattern to update the current sub - sentence pattern; if it is determined according to the second output array that the current node to be matched does not meet the output condition, add the next character of the current sub - sentence pattern to the current sub - sentence pattern to update the current sub - sentence pattern.

[0109] At the start of the matching, usually the first character in the first candidate sentence pattern is used as the initial current sub - sentence pattern to be matched, and according to the second state transition array, transfer from the root node of the second dictionary tree to the current node to be matched corresponding to this current sub - sentence pattern. During the matching process, usually, each character of the first candidate sentence pattern is matched one by one. For example, in step S540, when the current sub - sentence pattern to be matched successfully matches the current node to be matched, regardless of whether the matched sub - sentence pattern is output or not, the current sub - sentence pattern is added with the next character to obtain a new current sub - sentence pattern. For example, assume that the current sub - sentence pattern "play [actor]" in the first candidate sentence pattern "play [actor]'s [movie_name]" matches successfully. After adding the next character, the current sub - sentence pattern is updated to "play [actor]'s". In step S510, through the second state transition array, it is possible to quickly transfer from the previous node to the next node. For example, based on the node s2 (s2 represents the identifier of the node, which can be the subscript in the array) that matches the sub - sentence pattern "play [actor]", calculate base[s2]+code('s') (where code('s') represents the encoding corresponding to the character's', such as the ASCII code) through the second state transition array to obtain the identifier of the node t2 corresponding to "play [actor]'s", which is equivalent to updating the current node to be matched from node s2 to node t2.

[0110] After each transfer of the node, the second verification array can be used to verify whether the parent node of the current node to be matched is the previous node that jumps to the current node to be matched, so as to verify the correctness of the transfer between nodes. For example, after transferring from node s2 to node t2, it can be verified whether the check value of node t2 is node s2. If the parent node of the current node to be matched is the previous node, it means that the current node transfer is correct and the current sub - sentence pattern successfully matches the current node to be matched. If the parent node of the current node to be matched is not the previous node, it means that the current node transfer is incorrect and the current sub - sentence pattern does not successfully match the current node to be matched.

[0111] In the case of unsuccessful matching, the current sub-pattern to be matched and the current node to be matched can be updated according to the second fallback array. For example, assume that the matching between the current sub-pattern to be matched "play [actor]'s" and the current node to be matched t2 is unsuccessful. According to the second fallback array, find the node r2 pointed to by the fallback pointer of node t2, where r2 = fail(t2). Take node r2 as the current node to be matched, and take the string corresponding to node r2 as the current sub-pattern to be matched. If the parent node of node r2 is the root node, the string corresponding to node r2 is "'s", that is, the current sub-pattern to be matched is "'s". Thus, it can be seen that the second fallback array can achieve fast fallback, make full use of the prefix information that has been matched, and avoid excessive fallback.

[0112] Generally, the current sub-pattern to be matched and the current node to be matched obtained after fallback according to the second fallback array can be successfully matched. Therefore, it is not necessary to verify through step S520, and it can be defaulted that the current sub-pattern to be matched and the current node to be matched are successfully matched, and then step S540 is executed.

[0113] In step S540, it is determined whether the current node to be matched meets the output condition according to the second output array. Among them, the second output array can indicate whether the current node to be matched is a leaf node (in the second dictionary tree, a leaf node can have child nodes). If so, it meets the output condition. If the current node to be matched meets the output condition, it indicates that the current sub-pattern to be matched is a pattern string in the second dictionary tree, output it as the matched sub-pattern, and update the current sub-pattern to be matched. If the current node to be matched does not meet the output condition, it indicates that the current sub-pattern to be matched is not a pattern string in the second dictionary tree, and only update the current sub-pattern to be matched without outputting the matched sub-pattern.

[0114] It should be understood that Figure 5 The steps shown can be a loop process. When the last character of the current sub-pattern to be matched is the last character in the first candidate pattern, it is no longer possible to update the current character to be matched by adding the next character. At this time, the current sub-pattern to be matched and the current node to be matched can be updated according to the second fallback array until the current node to be matched pointed to by the second fallback array is the root node, and the entire matching process ends.

[0115] Figure 6A schematic diagram of matching the first candidate sentence pattern through the second dictionary tree is shown. In the second dictionary tree, each node has an identifier, such as 0, 1, 2, etc., which represents a corresponding state, where 0 is the root node. There is a character between two adjacent nodes. For example, node 0 is transferred to node 1 through the character "I", and the transfer can be achieved according to the second state transfer array. The second fallback array is used to reflect the node pointed to by the fallback pointer of the node, which is represented by the dotted line at the node in the figure. Some nodes have corresponding conflicting sentences. The figure shows that nodes 7, 10, 15, and 18 correspond to conflicting sentences 1, 2, 3, and 4, respectively. Assuming that the sentence to be recognized is "I want to listen to the Seven Mile Fragrance in Jay Chou's album Seven Mile Fragrance", according to Figure 4 , multiple first candidate sentences can be obtained, one of which is "I want to listen to [song_name] from [singer]'s album [album_name]". The first candidate sentence is matched in the second dictionary tree to obtain two matched clauses "I want to listen to [singer]'s album [album_name]" and "I want to listen to [song_name] from [singer]'s album [album_name]", and two corresponding matched conflicting sentences {I want to listen to [singer]'s album [album_name]} and {I want to listen to [singer]'s album [album_name] [song_name]} are obtained.

[0116] Depend on Figures 3 to 6 As can be seen from any one or more of the accompanying drawings, this exemplary embodiment can use ACDAT to implement keyword extraction and slot matching (i.e., determine candidate slots) of the sentence to be recognized, or to achieve matching of the first candidate sentence pattern with the conflicting sentence pattern. Table 1 lists the comparison between ACDAT and several other character matching methods. Among them, DAG (Directed Acyclic Graph) + DP (Dynamic Programming) is directed acyclic graph + dynamic programming, and FlashText is a character search and replacement algorithm.

[0117] Table 1

[0118]

[0119]

[0120] Although the time and memory usage of several methods for matching a single string are similar, the resources and performance consumption required to deploy the algorithm are different, and the time cost of rollback is different. ACDAT can combine the advantages of DAT's lightweight deployment and AC automaton's fast rollback. In addition, in most actual business scenarios, for keyword extraction and slot matching, local matching has a higher fault tolerance than the longest matching, but for long sentences with a large number of keywords, the longest matching has a relatively higher accuracy. Therefore, ACDAT's approach of mainly using local matching and supplemented by the longest matching has higher robustness.

[0121] In this paper, for the convenience of explanation, the symbol <> is added to the keyword, the symbol [] is added to the slot, and the symbol {} is added to the conflict sentence. These symbols can be omitted in the construction of ACDAT or the implementation of related algorithms to further save memory and improve efficiency.

[0122] In one implementation, the determining of the second candidate sentence pattern according to at least one of the matched clause patterns and the matched conflicting sentence patterns may include the following steps:

[0123] According to the length information of the matched clause formula or the length information of the matched conflicting sentence formula, a second candidate sentence formula is determined from the matched clause formula or the matched conflicting sentence formula.

[0124] Generally, for a first candidate sentence, if multiple matched sub-clauses or matched conflicting sentence patterns are matched therefrom, the matched sub-clause or matched conflicting sentence pattern with the largest length (i.e., the most characters) can be retained and determined as a second candidate sentence.

[0125] In addition, for different first candidate sentences, different matched sub-clauses or matched conflicting sentences are obtained, and the matched sub-clauses or matched conflicting sentences with larger lengths can be selected and determined as the second candidate sentences. For example, the number of slots (or keywords) in the second candidate sentence can be made consistent with the number of slots (or keywords) in the first candidate sentence.

[0126] For example, after extracting keywords and determining candidate slots for the sentence to be recognized "play Stephen Chow's The Mermaid", the first candidate sentence is obtained as follows:

[0127] First candidate sentence 1: Play [movie_name] by [director];

[0128] First candidate sentence 2: Play [tv_name] of [director];

[0129] First candidate sentence 3: Play [movie_name] by [actor];

[0130] First candidate sentence 4: Play [tv_name] of [actor].

[0131] The above four first candidate sentences are matched respectively using the second dictionary tree, and the matched clauses in the four first candidate sentences are obtained as follows:

[0132] Matched clause 1: [movie_name] of [director];

[0133] Matched clause 2: [tv_name] of [director];

[0134] Matched clause 3: play [actor];

[0135] Matched clause 4: Play [actor].

[0136] Since the number of slots (or keywords) in the first candidate sentence formulas 1-4 is 2, the clause formula with the number of slots (or keywords) of 2 is selected from the above matched clause formulas 1-4 as the second candidate sentence formula, so that the second candidate sentence formula is as follows:

[0137] Second candidate sentence 1: [movie_name] of [director];

[0138] Second candidate sentence 2: [director]'s [tv_name].

[0139] In one implementation, the above-mentioned matching the first candidate sentence pattern with the second dictionary tree to obtain the second candidate sentence pattern may further include the following steps:

[0140] In response to the number of first candidate sentence patterns that successfully match nodes in the second dictionary tree being 0, all first candidate sentence patterns are determined as second candidate sentence patterns.

[0141] Generally, when the first candidate sentence is fully matched with the nodes in the second dictionary tree, that is, when the entire first candidate sentence is matched with the nodes in the second dictionary tree, the first candidate sentence may not be matched with any node. Of course, when the second dictionary tree is used to partially match the first candidate sentence (such as pattern matching), that is, when the clause is matched with the nodes in the second dictionary tree, it may also be possible that any clause in the first candidate sentence does not match with any node, so it can be considered that the first candidate sentence does not match with any node.

[0142] If all the first candidate sentences are not matched successfully, it is difficult to filter the sentences through the second dictionary tree. In this case, all the first candidate sentences can be determined as second candidate sentences to be filtered through subsequent processing.

[0143] Continue to refer Figure 2 In step S250, the target entity corresponding to the sentence to be recognized is obtained according to the second candidate sentence pattern.

[0144] In this exemplary embodiment, the target entity corresponding to the sentence to be recognized may be a real entity that conforms to its semantics in the context of the sentence to be recognized, including but not limited to: the target slots of the keywords in the sentence to be recognized, that is, the slots corresponding to each keyword in the sentence to be recognized that are retained after disambiguation; the target semantics of the sentence to be recognized may be the target sentence pattern corresponding to the sentence to be recognized. Ideally, the target entity corresponding to the sentence to be recognized may be a unique entity, and the sentence to be recognized has unique semantics. However, the present disclosure does not limit this. For example, in some cases, the sentence to be recognized input by the user is itself ambiguous, and the target entity corresponding to the sentence to be recognized may include multiple entities.

[0145] If only one second candidate sentence is obtained after step S240, the candidate slot corresponding to each keyword in the second candidate sentence can be used as the target slot corresponding to each keyword to obtain the slot disambiguation result of each keyword, or the second candidate sentence can be used as the target sentence corresponding to the sentence to be recognized to obtain the semantic disambiguation result of the sentence to be recognized.

[0146] In one implementation, obtaining the target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern may include the following steps:

[0147] The candidate slots corresponding to the keywords are screened according to the second candidate sentence pattern, and the target entity corresponding to the sentence to be recognized is obtained according to the screened candidate slots.

[0148] The second candidate sentence is a filtered sentence. The candidate slots in the second candidate sentence are extracted, which are the slots after the initial candidate slots are filtered and are more in line with the context and semantics of the sentence to be recognized. Therefore, the target slot corresponding to each keyword in the sentence to be recognized can be determined based on these filtered candidate slots.

[0149] If the number of candidate slots corresponding to each keyword in all second candidate sentence patterns is 1, then the candidate slot corresponding to each keyword is used as the target slot.

[0150] However, there may be a situation where the number of candidate slots corresponding to one or more keywords is greater than 1. In this regard, in one embodiment, obtaining the target entity corresponding to the sentence to be recognized according to the screened candidate slots may include the following steps:

[0151] In response to the existence of at least one keyword, the number of the filtered candidate slots corresponding to the keyword is greater than 1, the filtered candidate slots are further filtered according to the priorities of the filtered candidate slots to obtain the target entity corresponding to the sentence to be recognized.

[0152] The priority of the candidate slots can be pre-configured by counting the user's language habits. For example, in the scenario of playing songs, it is more common for users to want to play a certain song than to want to play a certain album, so the priority of slot [song_name] can be set higher than that of slot [album_name]. In this way, when there are multiple candidate slots, the candidate slot with the highest priority can be used as the target slot. Combined with the above example, the following further explanation is given to obtain the above two second-post candidate sentence patterns:

[0153] Second candidate sentence 1: [movie_name] of [director];

[0154] Second candidate sentence 2: [director]'s [tv_name].

[0155] Therefore, we can determine that the keyword <Stephen Chow> corresponds to only one candidate slot [director], which is the target slot. The keyword <Mermaid> corresponds to two candidate slots [movie_name] and [tv_name]. Assuming that the priority of slot [movie_name] is higher than [tv_name], we further screen and obtain a candidate slot [movie_name], which is the target slot. Thus, the entity disambiguation of the keywords <Stephen Chow> and <Mermaid> is completed, which is equivalent to the entity disambiguation of the recognition sentence "Play Stephen Chow's Mermaid".

[0156] Of course, the present disclosure is not limited to the above-mentioned candidate slot priority method, and other methods can also be used to further screen the candidate slots. For example, the number of times each candidate slot corresponding to each keyword appears in the second candidate sentence can be counted, and the candidate slot with the largest number of times is used as the target slot. The target slot of each keyword in the sentence to be recognized may not be unique.

[0157] In one embodiment, the pre-configured information such as the first dictionary tree, the second dictionary tree, and the candidate slot priority can be hot-updated. For example, in actual application, new slot information is continuously collected, added to the first dictionary tree, and the first dictionary tree is updated, such as the first state transition array, the first verification array, the first fallback array, the first output array, etc. can be updated. Alternatively, new conflicting sentences are continuously collected, added to the second dictionary tree, and the second dictionary tree is updated, such as the second state transition array, the second verification array, the second fallback array, the second output array, etc. can be updated. Alternatively, as the number of collected user sentences increases, the candidate slot priority is re-counted and determined, and so on.

[0158] Figure 7 The schematic process of the entity disambiguation method in this exemplary embodiment is shown, including:

[0159] Step S701, determine the target intent corresponding to the sentence to be recognized, and obtain the first dictionary tree and the second dictionary tree under the target intent.

[0160] Step S702: extract the keyword w from the sentence to be recognized using the first dictionary tree 1 、w 2 ,…,w a , and determine the candidate slots corresponding to each keyword. The candidate slots include slot 1 、slot 2 、…、slot b , each keyword can correspond to one or more candidate slots.

[0161] Step S703, combining candidate slots corresponding to different keywords to obtain a first candidate sentence pattern corresponding to the sentence to be recognized, including syn 1 ,syn 2 ,…,syn c .

[0162] Step S704, using the second dictionary tree to match the first candidate sentence pattern, to obtain conflicting sentence patterns that match the clause patterns in the first candidate sentence pattern, including syn' 1 ,syn' 2 ,…,syn' d .

[0163] Step S705: Screen the candidate slots corresponding to each keyword according to the second candidate sentence pattern. 1 The corresponding filtered candidate slots include slot' 11 、slot' 12 Wait, w 2 The corresponding filtered candidate slots include slot'21 、slot' 22 Wait, w a The corresponding filtered candidate slots include slot' a1 、slot' a2 wait.

[0164] Step S706: further screen the selected candidate slots corresponding to each keyword according to the candidate slot priority to obtain the target slot corresponding to each keyword. 1 The corresponding target slot is slot' 1 , w 2 The corresponding target slot is slot' 2 , w a The corresponding target slot is slot' a , thereby completing the entity disambiguation for each keyword in the sentence to be identified.

[0165] The exemplary embodiment of the present disclosure also provides an entity disambiguation device. Figure 8 As shown, the entity disambiguation device 800 may include:

[0166] The dictionary tree acquisition module 810 is configured to determine the target intent corresponding to the sentence to be recognized, and acquire a first dictionary tree and a second dictionary tree of the target intent, the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns;

[0167] A candidate slot determination module 820 is configured to determine a candidate slot corresponding to a keyword in a sentence to be recognized by using the first dictionary tree;

[0168] The first candidate sentence pattern determination module 830 is configured to obtain a first candidate sentence pattern by combining candidate slots corresponding to the keywords;

[0169] The second candidate sentence determination module 840 is configured to match the first candidate sentence using the second dictionary tree to obtain the second candidate sentence in response to the number of the first candidate sentence being greater than 1;

[0170] The target entity output module 850 is configured to obtain a target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern.

[0171] In one implementation, the above-mentioned determining the candidate slots corresponding to the keywords in the sentence to be recognized by using the first dictionary tree includes:

[0172] Matching the substring in the sentence to be recognized with the node of the first dictionary tree, and when the match is successful, determining the substring as a keyword substring;

[0173] The keyword in the sentence to be recognized is determined according to the keyword substring, and the candidate slot corresponding to the keyword is determined according to the slot information corresponding to the node matching the keyword substring.

[0174] In one embodiment, the above-mentioned determining the keywords in the sentence to be recognized according to the keyword substring further includes:

[0175] In response to the existence of intersections among the plurality of keyword substrings, the keyword substrings are screened according to their lengths so that there is no intersection among the screened keyword substrings.

[0176] In one implementation, the first dictionary tree further includes a first state transition array, a first verification array, a first fallback array, and a first output array.

[0177] The above-mentioned matching of the substring in the sentence to be recognized with the node of the first dictionary tree, when the match is successful, determining the substring as a keyword substring, includes:

[0178] Determine the current to-be-matched node corresponding to the current to-be-matched substring in the to-be-recognized sentence according to the first state transition array;

[0179] Verify whether the current substring to be matched and the current node to be matched successfully according to the first verification array;

[0180] When the current substring to be matched and the current node to be matched are not matched successfully, the current substring to be matched and the current node to be matched are updated according to the first fallback array;

[0181] When the current substring to be matched successfully matches the current node to be matched, if it is determined according to the first output array that the current node to be matched meets the output condition, the current substring to be matched is output as the keyword substring, and the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched; if it is determined according to the first output array that the current node to be matched does not meet the output condition, the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched.

[0182] In one implementation, the above-mentioned matching the first candidate sentence pattern with the second dictionary tree to obtain the second candidate sentence pattern includes:

[0183] The first candidate sentence is matched with a node of the second dictionary tree, and a second candidate sentence is determined according to the successfully matched first candidate sentence and at least one of the conflicting sentences corresponding to the node.

[0184] In one embodiment, the above-mentioned matching of the first candidate sentence with the node of the second dictionary tree and determining the second candidate sentence according to the successfully matched first candidate sentence and at least one of the conflicting sentences corresponding to the node include:

[0185] Matching the clause formula in the first candidate sentence formula with the nodes of the second dictionary tree, and when the match is successful, determining the clause formula as a matched clause formula, and determining the conflicting sentence formula corresponding to the node matching the matched clause formula as a matched conflicting sentence formula;

[0186] A second candidate sentence pattern is determined according to at least one of the matched sub-sentence pattern and the matched conflicting sentence pattern.

[0187] In one implementation, the second dictionary tree further includes a second state transition array, a second verification array, a second fallback array, and a second output array.

[0188] The above-mentioned matching of the clause formula in the first candidate sentence formula with the node of the second dictionary tree, when the match is successful, determining the clause formula as a matched clause formula, includes:

[0189] Determine the current node to be matched corresponding to the current clause to be matched in the first candidate sentence according to the second state transition array;

[0190] Verify whether the current clause to be matched and the current node to be matched are matched successfully according to the second verification array;

[0191] When the current clause to be matched and the current node to be matched fail to match, the current clause to be matched and the current node to be matched are updated according to the second fallback array;

[0192] When the current clause to be matched successfully matches the current node to be matched, if it is determined according to the second output array that the current node to be matched meets the output condition, the current clause to be matched is output as the matched clause, and the next character of the current clause to be matched is added to the current clause to be matched to update the current clause; if it is determined according to the second output array that the current node to be matched does not meet the output condition, the next character of the current clause to be matched is added to the current clause to be matched to update the current clause.

[0193] In one embodiment, the second candidate sentence is determined based on at least one of the matched clauses and the matched conflicting sentences, including:

[0194] According to the length information of the matched clause formula or the length information of the matched conflicting sentence formula, a second candidate sentence formula is determined from the matched clause formula or the matched conflicting sentence formula.

[0195] In one implementation, obtaining the target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern includes:

[0196] The candidate slots corresponding to the keywords are screened according to the second candidate sentence pattern, and the target entity corresponding to the sentence to be recognized is obtained according to the screened candidate slots.

[0197] In one implementation, obtaining the target entity corresponding to the sentence to be recognized according to the filtered candidate slots includes:

[0198] In response to the existence of at least one keyword, the number of the filtered candidate slots corresponding to the keyword is greater than 1, the filtered candidate slots are further filtered according to the priorities of the filtered candidate slots to obtain the target entity corresponding to the sentence to be recognized.

[0199] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0200] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium, which can be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to cause the electronic device to perform the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. In an optional embodiment, the program product can be implemented as a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0201] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0202] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0203] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0204] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0205] The exemplary embodiment of the present disclosure further provides an electronic device, which may be, for example, the user terminal 110 and the server terminal 120 described above. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the entity disambiguation method in the exemplary embodiment by executing the executable instructions, such as executing Figure 2 method steps.

[0206] Reference below Fig. 9 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Fig. 9 The electronic device 900 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0207] like Fig. 9 As shown, the electronic device 900 may include: a processor 910 , a memory 920 , a bus 930 , an I / O (input / output) interface 940 , and a network adapter 950 .

[0208] The memory 920 may include a volatile memory, such as a RAM 921, a cache unit 922, and may also include a non-volatile memory, such as a ROM 923. The memory 920 may also include one or more program modules 924, such program modules 924 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a certain combination may include the implementation of a network environment. For example, the program module 924 may include the modules in the above-mentioned entity disambiguation device 800.

[0209] The bus 930 is used to realize the connection between different components of the electronic device 900, and may include a data bus, an address bus, and a control bus.

[0210] The electronic device 900 can communicate with one or more external devices 1000 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 940 .

[0211] The electronic device 900 can communicate with one or more networks through the network adapter 950. For example, the network adapter 950 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 950 can communicate with other modules of the electronic device 900 through the bus 930.

[0212] although Fig. 9 Not shown, other hardware and / or software modules may also be provided in the electronic device 900, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.

[0213] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0214] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0215] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An entity disambiguation method, It is characterized in that include: Determine the target intent corresponding to the sentence to be recognized, and obtain a first dictionary tree and a second dictionary tree of the target intent, wherein the first dictionary tree contains slot information, and the second dictionary tree contains conflicting sentence patterns; conflicting sentence patterns refer to sentence patterns that conform to people's daily language habits and are formed by slots and slot contexts; Determine candidate slots corresponding to keywords in the sentence to be recognized by using the first dictionary tree; The first candidate sentence pattern is obtained by combining the candidate slots corresponding to the keywords; In response to the number of the first candidate sentences being greater than 1, matching the first candidate sentences using the second dictionary tree to obtain second candidate sentences; A target entity corresponding to the sentence to be recognized is obtained according to the second candidate sentence pattern.

2. The method according to claim 1, It is characterized in that The determining, using the first dictionary tree, candidate slots corresponding to the keywords in the sentence to be recognized includes: Matching the substring in the sentence to be recognized with the node of the first dictionary tree, and when the match is successful, determining the substring as a keyword substring; The keyword in the sentence to be recognized is determined according to the keyword substring, and the candidate slot corresponding to the keyword is determined according to the slot information corresponding to the node matching the keyword substring.

3. The method according to claim 2, It is characterized in that The step of determining the keywords in the sentence to be identified according to the keyword substring further includes: In response to the existence of intersections among the plurality of keyword substrings, the keyword substrings are screened according to the lengths of the keyword substrings so that there are no intersections among the filtered keyword substrings.

4. The method according to claim 2, It is characterized in that The first dictionary tree also includes a first state transition array, a first verification array, a first fallback array, and a first output array; matching the substring in the sentence to be recognized with the node of the first dictionary tree, and when the match is successful, determining the substring as a keyword substring includes: Determine, according to the first state transition array, a current to-be-matched node corresponding to a current to-be-matched substring in the to-be-recognized sentence; Verify whether the current to-be-matched substring matches the current to-be-matched node successfully according to the first verification array; When the current substring to be matched and the current node to be matched fail to match, updating the current substring to be matched and the current node to be matched according to the first fallback array; When the current substring to be matched successfully matches the current node to be matched, if it is determined according to the first output array that the current node to be matched meets the output condition, the current substring to be matched is output as the keyword substring, and the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched; if it is determined according to the first output array that the current node to be matched does not meet the output condition, the next character of the current substring to be matched is added to the current substring to be matched to update the current substring to be matched.

5. The method according to claim 1, It is characterized in that The using the second dictionary tree to match the first candidate sentence pattern to obtain a second candidate sentence pattern includes: The first candidate sentence is matched with a node of the second dictionary tree, and the second candidate sentence is determined according to the successfully matched first candidate sentence and at least one of the conflicting sentences corresponding to the node.

6. The method according to claim 5, It is characterized in that The step of matching the first candidate sentence with the node of the second dictionary tree and determining the second candidate sentence according to at least one of the conflicting sentence patterns corresponding to the node and the first candidate sentence that is successfully matched comprises: Matching a clause formula in the first candidate sentence formula with a node in the second dictionary tree, and when the match succeeds, determining the clause formula as a matched clause formula, and determining a conflicting sentence formula corresponding to the node matching the matched clause formula as a matched conflicting sentence formula; The second candidate sentence pattern is determined according to at least one of the matched sub-sentence pattern and the matched conflicting sentence pattern.

7. The method according to claim 6, It is characterized in that The second dictionary tree further includes a second state transition array, a second verification array, a second fallback array, and a second output array; the matching of the clause formula in the first candidate sentence formula with the node of the second dictionary tree, when the match is successful, determining the clause formula as a matched clause formula, includes: Determine the current to-be-matched node corresponding to the current to-be-matched clause in the first candidate sentence according to the second state transition array; Verify whether the current clause to be matched and the current node to be matched are matched successfully according to the second verification array; When the current clause to be matched and the current node to be matched fail to match, updating the current clause to be matched and the current node to be matched according to the second fallback array; When the current clause to be matched successfully matches the current node to be matched, if it is determined according to the second output array that the current node to be matched meets the output condition, the current clause to be matched is output as the matched clause, and the next character of the current clause to be matched is added to the current clause to be matched to update the current clause; if it is determined according to the second output array that the current node to be matched does not meet the output condition, the next character of the current clause to be matched is added to the current clause to be matched to update the current clause.

8. The method according to claim 6, It is characterized in that The determining the second candidate sentence according to at least one of the matched sub-sentence and the matched conflicting sentence comprises: The second candidate sentence is determined from the matched sub-sentence or the matched conflicting sentence according to the length information of the matched sub-sentence or the length information of the matched conflicting sentence.

9. The method according to claim 1, It is characterized in that The step of obtaining a target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern includes: The candidate slots corresponding to the keyword are screened according to the second candidate sentence pattern, and the target entity corresponding to the sentence to be recognized is obtained according to the screened candidate slots.

10. The method according to claim 9, It is characterized in that Obtaining the target entity corresponding to the sentence to be recognized according to the filtered candidate slots includes: In response to the existence of at least one of the keywords, the number of the corresponding filtered candidate slots is greater than 1, the filtered candidate slots are further filtered according to the priorities of the filtered candidate slots to obtain the target entity corresponding to the sentence to be recognized.

11. An entity disambiguation device, It is characterized in that include: A dictionary tree acquisition module is configured to determine a target intent corresponding to a sentence to be recognized, and acquire a first dictionary tree and a second dictionary tree of the target intent, wherein the first dictionary tree includes slot information, and the second dictionary tree includes conflicting sentence patterns; conflicting sentence patterns refer to sentence patterns that conform to people's daily language habits and are formed by slots and slot contexts; a candidate slot determination module, configured to determine, by using the first dictionary tree, candidate slots corresponding to keywords in the sentence to be recognized; A first candidate sentence pattern determination module is configured to obtain a first candidate sentence pattern by combining candidate slots corresponding to the keywords; A second candidate sentence determination module is configured to, in response to the number of the first candidate sentences being greater than 1, match the first candidate sentences using the second dictionary tree to obtain a second candidate sentence; The target entity output module is configured to obtain the target entity corresponding to the sentence to be recognized according to the second candidate sentence pattern.

12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. An electronic device, It is characterized in that include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 10 by executing the executable instructions.