Entity Recognition Method, Apparatus, Computer-Readable Medium, and Electronic Device

Abstract

The present disclosure provides an entity recognition method, an entity recognition device, a computer-readable medium, and an electronic device, which relate to the technical field of natural language processing. The method includes: obtaining a plurality of original entity types corresponding to a sample sequence and an original entity annotation sample sequence in the sample sequence corresponding to the plurality of original entity types respectively; training an entity recognition network for at least one of the plurality of original entity types through the annotated sample sequence; and recognizing an entity in a conversation sequence input by a user according to the trained entity recognition network. This method can, to a certain extent, solve the problem of low semantic analysis efficiency of intelligent assistants. By training an entity recognition network that can target one or more entity types to recognize the conversation sequence input by a user, the entity recognition efficiency and the semantic analysis efficiency of the intelligent assistant are improved. Moreover, the intelligence level of the intelligent assistant is improved, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular, to an entity recognition method, an entity recognition device, a computer-readable medium, and an electronic device. Background Art

[0002] With the development of natural language processing technology, e-commerce platforms provide intelligent assistants for people to help them shop on the e-commerce platforms. The intelligent assistant can analyze the user's needs based on the conversation with the user, and then provide corresponding services according to the needs.

[0003] Generally, before analyzing the user's needs, the intelligent assistant needs to perform semantic analysis on the user's conversation, and determine the meaning represented by the user's conversation according to the entity words in the user's conversation. Specifically, when the intelligent assistant detects a user's conversation, it generally matches the conversation text with a corpus, and determines the meaning represented by the user's conversation according to the meaning of the corpus information corresponding to the entity words in the conversation text. However, if the corpus information corresponding to the entity words is not matched from the corpus, it is difficult to recognize the meaning of the conversation text, resulting in low semantic analysis efficiency of the intelligent assistant.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present disclosure is to provide an entity recognition method, an entity recognition device, a computer-readable medium, and an electronic device, which can at least solve the problem of low semantic analysis efficiency of the intelligent assistant to a certain extent. By training an entity recognition network that can recognize one or more entity types for the user input session sequence, the entity recognition efficiency and the semantic analysis efficiency of the intelligent assistant are improved, and the intelligence level of the intelligent assistant is improved, and the user experience is improved.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or be learned in part through the practice of the present disclosure.

[0007] The first aspect of the present disclosure provides an entity recognition method, which may include the following steps:

[0008] Obtain a plurality of original entity types corresponding to the sample sequence and an original entity annotation sample sequence corresponding to each of the plurality of original entity types in the sample sequence;

[0009] Train an entity recognition network for at least one of the plurality of original entity types through the annotated sample sequence;

[0010] Identify the entities in the conversation sequence input by the user according to the trained entity recognition network.

[0011] In an exemplary embodiment of the present disclosure, through multiple original entity types corresponding to the sample sequence and the original entity annotation sample sequence corresponding to each of the multiple original entity types in the sample sequence, the following steps may be included:

[0012] Collect the first sentences in multiple groups of historical user conversations as sample sequences respectively, and determine the user intents corresponding to the multiple groups of historical user conversations according to the content of the multiple groups of historical user conversations;

[0013] Determine multiple original entity types corresponding to the sample sequence and the original entities corresponding to each of the multiple original entity types in the sample sequence according to the user intent;

[0014] According to multiple original entity types and the original entity annotation sample sequence.

[0015] In an exemplary embodiment of the present disclosure, collecting the first sentences in multiple groups of historical user conversations as sample sequences respectively may include the following steps:

[0016] Collect the first sentences in multiple groups of historical user conversations respectively, screen the first sentences according to preset speech rules, and determine the screened first sentences as sample sequences.

[0017] In an exemplary embodiment of the present disclosure, determining multiple original entity types corresponding to the sample sequence and the original entities corresponding to each of the multiple original entity types in the sample sequence according to the user intent may include the following steps:

[0018] Determine the user intents corresponding to the sample sequences respectively according to the user intents corresponding to the multiple groups of historical user conversations;

[0019] Adjust the number of sample sequences corresponding to each type of user intent according to the preset user intent data distribution, and adjust the number of sample sequences in each sequence length section according to the preset sequence length distribution;

[0020] Determine multiple original entity types corresponding to the adjusted sample sequence according to the user intent, and determine the original entities corresponding to each of the multiple original entity types in the adjusted sample sequence.

[0021] In an exemplary embodiment of the present disclosure, training an entity recognition network for at least one of the multiple original entity types through the annotated sample sequence may include the following steps:

[0022] Determine the character-level feature vector and word-level feature vector of the annotated sample sequence;

[0023] Input the character-level feature vector and the word-level feature vector into the entity recognition network, and identify the entity corresponding to the sample sequence through the entity recognition network; wherein, the entity recognition network is used to perform entity recognition for at least one of multiple original entity types.

[0024] Update the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity.

[0025] In an exemplary embodiment of the present disclosure, determining the character-level feature vector and the word-level feature vector of the labeled sample sequence may include the following steps:

[0026] Determine the feature vector corresponding to each character in the labeled sample sequence as the character-level feature vector.

[0027] Perform word segmentation on the labeled sample sequence, and determine the feature vector corresponding to the word segmentation result as the word-level feature vector.

[0028] In an exemplary embodiment of the present disclosure, wherein: the entity recognition network includes a first entity recognition network for the first original entity type and a second entity recognition network for the second original entity type, and the multiple original entity types include the first original entity type and the second original entity type.

[0029] Inputting the character-level feature vector and the word-level feature vector into the entity recognition network and identifying the entity corresponding to the sample sequence through the entity recognition network may include the following steps:

[0030] Input the character-level feature vector and the word-level feature vector into the first entity recognition network and the second entity recognition network respectively, identify the first entity corresponding to the first original entity type through the first entity recognition network, and identify the second entity corresponding to the second original entity type through the second entity recognition network.

[0031] Updating the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity may include the following steps:

[0032] Calculate the first loss function between the first entity and the original entity corresponding to the first original entity type, and calculate the second loss function between the second entity and the original entity corresponding to the second original entity type.

[0033] Update the network parameters of the first entity recognition network according to the first loss function, and update the network parameters of the second entity recognition network according to the second loss function.

[0034] According to the second aspect of the present disclosure, there is provided an entity recognition device, including a sample annotation unit, a network training unit, and an entity recognition unit, wherein:

[0035] A sample annotation unit, configured to annotate a sample sequence through multiple original entity types corresponding to the sample sequence and original entities respectively corresponding to the multiple original entity types in the sample sequence;

[0036] A network training unit, configured to train an entity recognition network for at least one of the multiple original entity types through the annotated sample sequence;

[0037] An entity recognition unit, configured to recognize entities in a session sequence input by a user according to the trained entity recognition network.

[0038] In an exemplary embodiment of the present disclosure, the manner in which the sample annotation unit annotates the sample sequence through multiple original entity types corresponding to the sample sequence and original entities respectively corresponding to the multiple original entity types in the sample sequence may specifically be:

[0039] The sample annotation unit respectively collects the first sentences in multiple groups of historical user sessions as the sample sequence, and determines the user intents respectively corresponding to the multiple groups of historical user sessions according to the content of the multiple groups of historical user sessions;

[0040] The sample annotation unit determines multiple original entity types corresponding to the sample sequence and original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent;

[0041] The sample annotation unit annotates the sample sequence according to the multiple original entity types and the original entities.

[0042] In an exemplary embodiment of the present disclosure, the manner in which the sample annotation unit respectively collects the first sentences in multiple groups of historical user sessions as the sample sequence may specifically be:

[0043] The sample annotation unit respectively collects the first sentences in multiple groups of historical user sessions, filters the first sentences according to preset speech rules, and determines the filtered first sentences as the sample sequence.

[0044] In an exemplary embodiment of the present disclosure, the manner in which the sample annotation unit determines multiple original entity types corresponding to the sample sequence and original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent may specifically be:

[0045] The sample annotation unit determines the user intent corresponding to the sample sequence according to the user intents respectively corresponding to the multiple groups of historical user sessions;

[0046] The sample annotation unit adjusts the number of sample sequences corresponding to each type of user intent according to a preset user intent data distribution, and adjusts the number of sample sequences in each sequence length section according to a preset sequence length distribution;

[0047] The sample annotation unit determines multiple original entity types corresponding to the adjusted sample sequence according to the user's intention, and determines the original entities corresponding to the multiple original entity types in the adjusted sample sequence respectively.

[0048] In an exemplary embodiment of the present disclosure, the manner in which the network training unit trains an entity recognition network for at least one of the multiple original entity types through the annotated sample sequence may specifically be:

[0049] The network training unit determines the character-level feature vector and the word-level feature vector of the annotated sample sequence;

[0050] The network training unit inputs the character-level feature vector and the word-level feature vector into the entity recognition network, and recognizes the entity corresponding to the sample sequence through the entity recognition network; wherein, the entity recognition network is used to recognize entities for at least one of the multiple original entity types.

[0051] The network training unit updates the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity.

[0052] In an exemplary embodiment of the present disclosure, the manner in which the network training unit determines the character-level feature vector and the word-level feature vector of the annotated sample sequence may specifically be:

[0053] The network training unit determines the feature vector corresponding to each character in the annotated sample sequence as the character-level feature vector;

[0054] The network training unit performs word segmentation on the annotated sample sequence and determines the feature vector corresponding to the word segmentation result as the word-level feature vector.

[0055] In an exemplary embodiment of the present disclosure, wherein: the entity recognition network includes a first entity recognition network for the first original entity type and a second entity recognition network for the second original entity type, and the multiple original entity types include the first original entity type and the second original entity type;

[0056] The manner in which the network training unit inputs the character-level feature vector and the word-level feature vector into the entity recognition network and recognizes the entity corresponding to the sample sequence through the entity recognition network may specifically be:

[0057] The network training unit inputs the character-level feature vector and the word-level feature vector into the first entity recognition network and the second entity recognition network respectively, recognizes the first entity corresponding to the first original entity type through the first entity recognition network, and recognizes the second entity corresponding to the second original entity type through the second entity recognition network;

[0058] The way for the network training unit to update the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity can be specifically as follows:

[0059] The network training unit calculates a first loss function between the first entity and the original entity corresponding to the first original entity type, and calculates a second loss function between the second entity and the original entity corresponding to the second original entity type;

[0060] The network training unit updates the network parameters of the first entity recognition network according to the first loss function, and updates the network parameters of the second entity recognition network according to the second loss function.

[0061] According to the third aspect of the present disclosure, there is provided a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, it implements the entity recognition method as described in the first aspect of the above embodiments.

[0062] According to the fourth aspect of the present disclosure, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the entity recognition method as described in the first aspect of the above embodiments.

[0063] The technical solution provided by the present disclosure may include the following beneficial effects:

[0064] The technical solutions provided by some embodiments of the present disclosure can be applied to identify entities in a user session. In the technical solutions provided by the embodiments of the present disclosure, a sample sequence (such as a sample session) can be labeled with multiple original entity types (such as product words) corresponding to the sample sequence and the original entities (such as mobile phones) respectively corresponding to the multiple original entity types in the sample sequence; and, an entity recognition network for at least one of the multiple original entity types can be trained through the labeled sample sequence; and, entities in the session sequence input by the user can be identified according to the trained entity recognition network. According to the above-described solution, on the one hand, the present disclosure can, to a certain extent, solve the problem of low semantic analysis efficiency of intelligent assistants, by training an entity recognition network that can target one or more entity types to identify the session sequence input by the user, thereby improving the entity recognition efficiency and the semantic analysis efficiency of intelligent assistants; on the other hand, it can improve the intelligence level of intelligent assistants through the improvement of entity recognition efficiency and optimize the user experience; on the other hand, compared with the manually constructed corpus in the prior art, the present application can identify entities in the session sequence through the entity recognition network, reducing the labor cost.

[0065] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. Description of the Drawings

[0066] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts. In the drawings:

[0067] Figure 1 A flowchart showing a method for entity recognition according to an exemplary embodiment of the present disclosure;

[0068] Figure 2 A flowchart showing a method for sample sequence annotation according to an exemplary embodiment of the present disclosure;

[0069] Figure 3 A flowchart showing a method for training an entity recognition network according to an exemplary embodiment of the present disclosure;

[0070] Figure 4 A schematic diagram showing the extraction of character-level feature vectors according to an exemplary embodiment of the present disclosure;

[0071] Figure 5 A signal flow diagram of a feature extraction window according to an embodiment of the present disclosure is schematically shown;

[0072] Figure 6 A schematic diagram showing the extraction of word-level feature vectors according to an exemplary embodiment of the present disclosure;

[0073] Figure 7 A block diagram showing the architecture of an entity recognition network according to an exemplary embodiment of the present disclosure;

[0074] Figure 8 A schematic diagram showing the application of an entity recognition network according to an exemplary embodiment of the present disclosure;

[0075] Figure 9 A schematic diagram showing another application of an entity recognition network according to an exemplary embodiment of the present disclosure;

[0076] Figure 10 A schematic diagram showing the structure of an entity recognition system according to an exemplary embodiment of the present disclosure;

[0077] Figure 11 A block diagram showing the structure of an entity recognition device according to an exemplary embodiment of the present disclosure;

[0078] Figure 12 The structural schematic diagram of a computer system of an electronic device suitable for implementing an exemplary embodiment of the present disclosure is shown. Detailed implementation manners

[0079] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0080] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0081] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0082] The flowcharts shown in the drawings are only illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0083] Please refer to Figure 1 , Figure 1 The flow schematic diagram of an entity recognition method according to an exemplary embodiment of the present disclosure is shown, and the entity recognition method can be implemented by a server or a terminal device.

[0084] As Figure 1 shown, the entity recognition method according to an embodiment of the present disclosure includes the following steps S110, step S120, and step S130, where:

[0085] Step S110: Through a plurality of original entity types corresponding to the sample sequence and original entity annotation sample sequences respectively corresponding to the plurality of original entity types in the sample sequence.

[0086] Step S120: training an entity recognition network for at least one original entity type among the multiple original entity types through the labeled sample sequence.

[0087] Step S130: Identify entities in the conversation sequence input by the user according to the trained entity recognition network.

[0088] The following is a detailed description of each step:

[0089] In step S110 , the sample sequence is labeled by a plurality of original entity types corresponding to the sample sequence and original entities respectively corresponding to the plurality of original entity types in the sample sequence.

[0090] The sample sequence may be one or more, and the sample sequence includes the user's current needs. The sample sequence may be text information corresponding to the user session, text information corresponding to the intelligent assistant log, or historical search information corresponding to the search window crawled by the crawler program, etc., which is not limited in the embodiments of the present disclosure. The intelligent assistant is used to identify the semantic intent of the user input information, extract the user input information, and give corresponding personalized recommendations.

[0091] It should be noted that the intelligent assistant log is stored in the big data Hive table, which may include fields such as salesperson scene, channel number, current scene, device id (Identity document), input text, user personal identification number (PIN), user location, time, session id and context information, which are not limited in the embodiments of the present disclosure. Hive is a data warehouse of Hadoop, which is used to map structured data files stored on a distributed system (Hadoop Distributed File System, HDFS) into a table and provide query functions of the hsql database. The Hive framework can convert statements in the hsql database into MapReduce tasks to analyze files on HDFS. Among them, MapReduce is a programming model for integrating user-written business logic code and built-in default components into a complete distributed computing program, which runs concurrently on a hadoop cluster. ID is an abbreviation for various proprietary terms such as identity card identification number, account number, unique code, exclusive number, industrial design, national abbreviation, legal vocabulary, general account, decoder and software company.

[0092] In addition, a sample sequence can correspond to multiple original entity types, and each original entity type corresponds to one or more entities. It can also be understood that a piece of text information input by a user can correspond to multiple original entity types, and each entity type can correspond to one or more entities. The original entity type can be understood as a slot, and the entity can be understood as the slot value corresponding to the slot. A slot refers to the key information that the system needs to collect from the user, and the slot value is the specific key information expressed by the user. From a macroscopic perspective, slots include vertical slots (or progressive slots), parallel slots, and combined slots. Among them, vertical slots mean that the system sets multiple slots through a certain topic and guides the user to fill in the slots according to the set path. These slots are usually the key nodes for guiding the user to delve deeper into the topic. For example, when a user fills out a questionnaire. Parallel slots are usually mainly task-oriented conversations, and the number of slots required to complete the task is fixed. For example, when a user fills out ticket booking information, including time, flight, destination, departure place, etc. Combined slots include vertical slots and horizontal slots. Among them, horizontal slots do not determine the branch direction of the path but determine whether the path can be passed. From a microscopic perspective, slots include custom slots, thesaurus slots, and interface slots. Among them, custom slots can be custom score path options; thesaurus slots are a set of data taken from the database as options or used as matching items to verify the user's input; interface slots are equivalent to thesaurus slots.

[0093] Among them, the original entity type can be a product word, a brand word, a modifier, gender, price, etc. The product word represents the name of the commodity, that is, the central product word of the commodity. The corresponding entities can be "mobile phone", "laptop", "scarf", etc. The brand word represents the brand of the commodity, and the corresponding entities can be "Lao Wang Brand", "Lao Zhang Brand", "Lao Li Brand". The modifier represents the description of the commodity, that is, the modifier of the commodity, and the corresponding entities can be "red", "128G", "beauty function", etc. The entities corresponding to gender can be "male", "female", and the entities corresponding to price can be "3,000 yuan", "2,000 yuan", "1,000 yuan", etc. The embodiments of the present disclosure are not limited thereto. For example, the text information input by the user is "I want to buy a mobile phone of Lao Wang Brand, which is red, has 128G memory and costs about 3,000 yuan". The original entity types corresponding to this text information include product word, brand word, modifier, and price. Among them, the entity corresponding to the product word is "mobile phone", the entity corresponding to the brand word is "Lao Wang Brand", the entities corresponding to the modifier are "128G" and "red", and the entity corresponding to the price is "3,000".

[0094] Specifically, the method of labeling a sample sequence by multiple original entity types corresponding to the sample sequence and original entity annotation samples corresponding to the multiple original entity types in the sample sequence can be as follows: determining multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence; labeling tags for the multiple original entities in the sample sequence, where the labeled tags can be the original entity types to which the original entities belong; or modifying at least one of the font, font size, character color, and background color of the multiple original entities in the sample sequence to implement the labeling of the sample sequence. Implementing the above implementation method can obtain sample data for network training through the annotation of the original entities in the sample sequence.

[0095] Please refer to Figure 2 , Figure 2 FIG. shows a flowchart of a sample sequence annotation method according to an exemplary embodiment of the present disclosure. The sample sequence annotation method is to annotate the sample sequence by multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence, specifically including step S210 to step S230, where:

[0096] Step S210: Collect the first sentences in multiple groups of historical user sessions as sample sequences respectively, and determine the user intents corresponding to the multiple groups of historical user sessions according to the contents of the multiple groups of historical user sessions.

[0097] Step S220: Determine multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence according to the user intent.

[0098] Step S230: Annotate the sample sequence according to the multiple original entity types and the original entities.

[0099] The following is a detailed description of each step:

[0100] In step S210, the first sentences in multiple groups of historical user sessions are collected as sample sequences respectively, and the user intents corresponding to the multiple groups of historical user sessions are determined according to the contents of the multiple groups of historical user sessions.

[0101] Among them, each group of historical user sessions may include multiple text messages input by the user. For example, a group of historical user sessions may include "I want to buy a mobile phone", "a red mobile phone", and "a mobile phone with three cameras" input by the user. The first text message input by the user in each group of historical user sessions can be used as the above sample sequence. In addition, the user intent may be specific commodity query, order query, fuzzy discount query, specific discount query, after-sales service, full-site direct access, or unknown, which is not limited in the embodiments of the present disclosure.

[0102] Considering that the first sentence entered by the user may be entity-free text information such as "Hello" or "Are you there", etc., therefore, the above method of collecting the first sentences in multiple groups of historical user conversations as sample sequences can be: determining the first sentences that are not greetings in multiple groups of historical user conversations as sample sequences in the order of input time.

[0103] In addition, the method of determining the user intents corresponding to multiple groups of historical user conversations according to the content of multiple groups of historical user conversations can be: performing keyword matching on the content of multiple groups of historical user conversations, and determining the user intents corresponding to multiple groups of historical user conversations according to the business scenarios to which the matched keywords belong; among them, keywords can be understood as words with a relatively high output probability of the user in the corresponding business scenario. For example, the sentence "Has the old Ace mobile phone been delivered?" in the historical user conversation contains the keyword "delivered", and the business scenario to which this keyword belongs is the "order query business scenario". It should be noted that the business scenario to which the keyword belongs can be a specific commodity query business scenario, and the corresponding string in the program is ACT_COMMODITY; or, an order query business scenario, and the corresponding string in the program is ACT_ORDER; or, a fuzzy discount query business scenario, and the corresponding string in the program is ACT_DISCOUNT; or, a specific discount query business scenario, and the corresponding string in the program is ACT_SPECIFY_DISCOUNT; or, an after-sales service business scenario, and the corresponding string in the program is ACT_AFTER_SALES; or, a full-site direct access business scenario, and the corresponding string in the program is ACT_SHORTCUT; or, an unknown business scenario, and the corresponding string in the program is ACT_UNKNOWN. The embodiments of the present disclosure do not make limitations.

[0104] Specifically and optionally, collecting the first sentences in multiple groups of historical user conversations as sample sequences may include the following steps:

[0105] Collect the first sentences in multiple groups of historical user conversations respectively, screen the first sentences according to the preset speech rules, and determine the screened first sentences as sample sequences.

[0106] Among them, the preset speech rules are used to stipulate the screening conditions for the first sentences. For example, if the first sentence is "Hello", then this first sentence is screened out. The method of screening the first sentences according to the preset speech rules can be: performing regular matching between the first sentences and the screening conditions stipulated in the preset speech rules, and screening out the first sentences that match successfully.

[0107] It can be seen that implementing this optional implementation method can remove user inputs that do not contain important information through the screening of the first sentences (i.e., sample sequences), thereby improving the generalization ability of the entity recognition network obtained by training.

[0108] In step S220, multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence are determined according to the user intention.

[0109] In the embodiments of the present disclosure, there are multiple types of user intentions. Each user intention can correspond to multiple original entity types. The original entity types corresponding to different user intentions can be the same or different, which is not limited in the embodiments of the present disclosure. For example, if the user intention is order query and the sample sequence is "Has the old Wangpai mobile phone been delivered?", then under this user intention, the original entity types corresponding to the sample sequence are brand word and product word. The original entity corresponding to the brand word is "old Wangpai", and the original entity corresponding to the product word is "mobile phone".

[0110] Specifically and optionally, determining multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence according to the user intention may include the following steps:

[0111] Determine the user intentions corresponding to the sample sequences according to the user intentions corresponding to multiple groups of historical user conversations;

[0112] Adjust the number of sample sequences corresponding to each type of user intention according to the preset user intention data distribution, and adjust the number of sample sequences in each sequence length section according to the preset sequence length distribution;

[0113] Determine multiple original entity types corresponding to the adjusted sample sequence according to the user intention, and determine the original entities corresponding to the multiple original entity types in the adjusted sample sequence.

[0114] Among them, determining the user intentions corresponding to the sample sequences according to the user intentions corresponding to multiple groups of historical user conversations can be understood as that if there is a keyword in the content of a group of historical user conversations indicating that the group of historical user conversations corresponds to the target user intention, then the first sentence of the group of historical user conversations (i.e., the sample sequence) also corresponds to the target user intention.

[0115] In addition, for example, the preset user intention data distribution is as follows in the following table:

[0116] ACT_COMMODITY 2436 ACT_ORDER 1015 ACT_DISCOUNT 779 ACT_SPECIFY_DISCOUNT 597 ACT_AFTER_SALES 690 ACT_SHORTCUT 501

[0117] Based on the data shown in the above table, the way to adjust the number of sample sequences corresponding to each type of user intention according to the preset data distribution of user intentions can be: determine the data ratio between each type of user intention in the preset data distribution of user intentions (e.g., 2436:1015:779:597:690:501), and adjust the number of sample sequences in each type of user intention to the above data ratio. This can make the data distribution during the training process of the entity recognition network consistent with the real online data distribution, so as to improve the recognition accuracy of the entity recognition network.

[0118] Further, the way to adjust the number of sample sequences in each type of user intention to the above data ratio can be: determine whether the number of sample sequences in each type of user intention needs to be increased or decreased according to the above data ratio. If it needs to be decreased, randomly discard the sample sequences in this type of user intention; or discard the sample sequences in this type of user intention in the order of increasing string length. If it needs to be increased, supplement the sample sequences in this type of user intention with the corpus data in the intelligent assistant log or user corpus.

[0119] In addition, the way to adjust the number of sample sequences in each sequence length section according to the preset sequence length distribution can be: determine the data ratio corresponding to the preset sequence length distribution (e.g., 3:4:5), and adjust the ratio of the number of sample sequences in each sequence length section (e.g., 3 - 5 characters, 6 - 9 characters, and 10 - 15 characters) to this data ratio. Among them, there are multiple sequence length sections, and there is no intersection between each sequence length section. Each sequence length section is used to classify sample sequences of different lengths.

[0120] Further, the way to adjust the ratio of the number of sample sequences in each sequence length section to this data ratio can be: determine whether the number of sample sequences in each sequence length section needs to be increased or decreased according to this data ratio. If it needs to be decreased, randomly discard the sample sequences in this sequence length section; or discard the sample sequences in this sequence length section in the order of increasing string length. If it needs to be increased, supplement the sample sequences in this sequence length section with the corpus data in the intelligent assistant log or user corpus.

[0121] It can be seen that implementing this optional implementation method can balance the number of sample sequences based on each type of user intention and sequence length section, without making the number of sample sequences of a certain type of user intention or a certain length section too large, optimizing the ratio of the number of sample sequences between each type of user intention and the ratio of the number of sample sequences between each sequence length section, thereby improving the training effect of the entity recognition network and enhancing the recognition accuracy of the entity recognition network.

[0122] In step S230, according to multiple original entity types and the original entity annotation sample sequence.

[0123] For example, if the sample sequence is "I want to buy a mobile phone of the old Wangpai brand", according to the above implementation, in this sentence, "old Wangpai" is the entity corresponding to the original entity type of brand word, and "mobile phone" is the entity corresponding to the original entity type of product word. According to multiple original entity types and the original entity annotation sample sequence, it can be understood that "old Wangpai" is labeled as the original entity corresponding to the brand word, and "mobile phone" is labeled as the original entity corresponding to the product word.

[0124] It can be seen that implementing Figure 2 the sample sequence annotation method shown can enrich the training samples through the annotation of the sample sequence to achieve the entity recognition training for the entity recognition network.

[0125] In step S120, the entity recognition network for at least one of the multiple original entity types is trained through the annotated sample sequence.

[0126] Among them, the entity recognition network is used to recognize the entity content in the text information input by the user. This entity recognition network can recognize entities of one original entity type or entities of multiple original entity types, which is not limited in the embodiments of the present disclosure.

[0127] Please refer to Figure 3 , Figure 3 which shows a schematic flowchart of an entity recognition network training method according to an exemplary embodiment of the present disclosure. The entity recognition network training method is to train the entity recognition network for at least one of the multiple original entity types through the annotated sample sequence, specifically including step S310 to step S330, where:

[0128] Step S310: Determine the character-level feature vector and word-level feature vector of the annotated sample sequence.

[0129] Step S320: Input the character-level feature vector and word-level feature vector into the entity recognition network, and recognize the entity corresponding to the sample sequence through the entity recognition network; among them, the entity recognition network is used to perform entity recognition for at least one of the multiple original entity types.

[0130] Step S330: Update the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity.

[0131] The following will explain each step in detail:

[0132] In step S310, the character-level feature vectors and word-level feature vectors of the labeled sample sequence are determined.

[0133] Specifically, determining the character-level feature vectors and word-level feature vectors of the labeled sample sequence may include the following steps:

[0134] Determine the feature vector corresponding to each character in the labeled sample sequence as the character-level feature vector;

[0135] Perform word segmentation on the labeled sample sequence and determine the feature vector corresponding to the word segmentation result as the word-level feature vector.

[0136] Among them, the character-level feature vector represents the feature vector corresponding to each character in the sample sequence, and the word-level feature vector represents the feature vector corresponding to each word in the sample sequence. The method of determining the feature vector corresponding to each character in the labeled sample sequence as the character-level feature vector can be: splitting the sample sequence by character and determining the feature vector corresponding to each split character according to the vector mapping relationship. The method of performing word segmentation on the labeled sample sequence and determining the feature vector corresponding to the word segmentation result as the word-level feature vector can be: calculating the cosine distance between characters according to the feature vector corresponding to each character, determining characters with a cosine distance less than a preset distance as a word, and then obtaining multiple words corresponding to the sample sequence, and calculating the feature vector corresponding to each word according to the feature vector corresponding to each character. Among them, the vector mapping relationship is used to convert characters into vectors in the vector space.

[0137] Please refer to Figure 4 , Figure 4 which shows a schematic diagram of the extraction of character-level feature vectors according to an exemplary embodiment of the present disclosure. As Figure 4 shown, the sample sequence is "buy a mobile phone". Splitting the sample sequence into characters gives "buy", "a", "mobile", "phone". The extraction of the character-level feature vector can be obtained through a character-based feature extraction network. Through the vector mapping relationship in the feature extraction network, "buy", "a", "mobile", "phone" can be mapped to feature vectors x_c1, x_c2, x_c3, x_c4 respectively, as the vector representation of the first hidden layer and the input of the second hidden layer. Then, through the feature extraction window in the recurrent neural network, the feature vectors c_c1, c_c2, c_c3, c_c4 and feature vectors h_c1, h_c2, h_c3, h_c4 corresponding to the feature vectors x_c1, x_c2, x_c3, x_c4 are obtained respectively.

[0138] Specifically, among the feature vectors c_c1, c_c2, c_c3, c_c4 and the feature vectors h_c1, h_c2, h_c3, h_c4 corresponding to the feature vectors x_c1, x_c2, x_c3, x_c4 obtained through the feature extraction window in the temporal recurrent neural network, the ways of obtaining c_c1 and h_c1 corresponding to the feature vector x_c1, c_c2 and h_c2 corresponding to the feature vector x_c2, c_c3 and h_c3 corresponding to the feature vector x_c3, and c_c4 and h_c4 corresponding to the feature vector x_c4 through the feature extraction window are the same. The following takes obtaining c_c2 and h_c2 corresponding to the feature vector x_c2 through the feature extraction window as an example to illustrate the specific implementation manner:

[0139] According to the feature vector x_c2 corresponding to the current feature extraction window and the feature vector c_c1 output by the previous feature extraction window, calculate the candidate state information, the input weight of the candidate state information, the forgetting weight of the feature vector c_c1 of the previous feature extraction window, and the output weight of the feature vector c_c2 of the current feature extraction window; retain the feature vector c_c1 of the previous feature extraction window according to the forgetting weight to obtain the first intermediate state information; retain the candidate state information according to the input weight of the candidate state information to obtain the second intermediate state information; obtain the feature vector c_c2 of the current feature extraction window according to the first intermediate state information and the second intermediate state information; retain the feature vector c_c2 of the current feature extraction window according to the output weight of the feature vector c_c2 of the current feature extraction window to obtain the feature vector h_c2 of the current feature extraction window.

[0140] For further illustration, please refer to Figure 5 , Figure 5 which schematically shows the signal flow diagram of the feature extraction window according to an embodiment of the present disclosure. Specifically:

[0141] The forgetting gate is used to determine how much information to discard from the feature vector c_c1 in the previous feature extraction process. Therefore, the forgetting weight is used to represent the weight of the feature vector c_c1 in the previous feature extraction process that is not forgotten (i.e., can be retained); the forgetting weight can essentially be a weight matrix. Exemplarily, the feature vector x_c2 corresponding to the current feature extraction window and the feature vector h_c1 in the previous feature extraction process can be encoded through the activation function used to characterize the forgetting gate, and mapped to a value between 0 and 1 to obtain the forgetting weight of c_c1 in the previous feature extraction process. Among them, 0 means complete discard, and 1 means complete retention. For example, the forgetting weight f of the feature vector c_c1 in the previous feature extraction process can be calculated according to the following formula t :

[0142] f t = σ(Wf · [h t-1 , S t + b f )

[0143] Among them, h t-1 represents the feature vector h_c1 of the previous feature extraction process, S t represents the feature vector x_c2 corresponding to the current feature extraction window, σ represents the activation function Sigmod function, W f and b f represent the parameters of the Sigmod function in the forget gate, [h t-1 , S t means combining h t-1 and S t , and it can also be understood as concatenating h t-1 and S t . The output range of the Sigmod function is (0, 1). In a binary classification task, the sigmoid outputs the event probability.

[0144] The input gate is used to determine how much of the current input vehicle driving information is important and needs to be retained. Exemplarily, the feature vector x_c2 corresponding to the current feature extraction window and the feature vector h_c1 of the previous conversion process can be encoded through the activation function used to represent the input gate to obtain the candidate state information and the input weight of the candidate state information; among them, the input weight of the candidate state information is used to determine how much new information in the candidate state information can be added to the feature vector c_c2.

[0145] For example, the candidate state information can be calculated according to the following formula

[0146]

[0147] Among them, tanh represents the activation function as the hyperbolic tangent function, W c and b c represent the parameters of the tanh function in the input gate.

[0148] And the input weight i of the candidate state information can be calculated according to the following formula t :

[0149] i t = σ(W i · [h t-1 , S t + b i )

[0150] Among them, σ represents the activation function Sigmod function, W i and bi Represents the parameter of the Sigmod function in the input gate.

[0151] The output gate is used to determine which information should be included in the feature vector h_c2 output to the next feature extraction window. Exemplarily, the activation function used to characterize the output gate can be used to encode the feature vector x_c2 corresponding to the current feature extraction window and the feature vector h_c1 of the previous feature extraction process, to obtain the output weight of the feature vector c_c2 of the current feature extraction process. For example, the candidate state information o can be calculated according to the following formula t :

[0152] o t = σ(W o ·[h t-1 , S t +b o )

[0153] where σ represents the Sigmod function of the activation function, and W o and b o represent the parameters of the Sigmod function in the output gate.

[0154] The feature vector c_c1 of the previous feature extraction window is retained according to the forgetting weight to obtain the first intermediate state information. For example, the obtained first intermediate state information can be where C t-1 represents the feature vector c_c1 of the previous feature extraction process.

[0155] The candidate state information is retained according to the input weight of the candidate state information to obtain the second intermediate state information. For example, the obtained second intermediate state information can be

[0156] According to the first intermediate state information and the second intermediate state information, the feature vector c_c2 of the current feature extraction window is obtained. For example, the feature vector c_c2 of the current feature extraction process can be

[0157] The feature vector c_c2 of the current feature extraction window is retained according to the output weight of the feature vector c_c2 of the current feature extraction window to obtain the feature vector h_c2 of the current feature extraction window. For example, the feature vector h_c2 of the current feature extraction process can be

[0158] In addition, it should be noted that the algorithm applied in the character-based feature extraction network can be the LSTM_CRF algorithm, that is, an algorithm combining LSTM with LSTM. Among them, LSTM is a time recurrent neural network that can be used to solve the long-term dependence problem existing in the general Recurrent Neural Network (RNN). The core idea of LSTM is to retain the state within a certain period of time through a storage unit. Conditional Random Fields (CRF) is a conditional probability distribution model of another set of output random variables given a set of input random variables. Its characteristic is that it assumes that the output random variables form a Markov random field. In addition, the conditional random field is a discriminative model.

[0159] Please refer to Figure 6 , Figure 6 which shows a schematic diagram of the extraction of word-level feature vectors according to an exemplary embodiment of the present disclosure. As Figure 6 shown, the sample sequence is "buy a red second-hand mobile phone". The sample sequence is divided into words to obtain "buy", "red", "second-hand", and "mobile phone". The extraction of the word-level feature vectors can be obtained through a word-based feature extraction network. Through the feature vectors corresponding to each character in the word, the feature vectors x_w1, x_w2, x_w3, and x_w4 corresponding to the above words can be obtained, which are used as the vector representation of the first hidden layer and the input of the second hidden layer. Then, through the feature extraction window in the time recurrent neural network, the feature vectors c_w1, c_w2, c_w3, c_w4 and the feature vectors h_w1, h_w2, h_w3, h_w4 corresponding to the feature vectors x_w1, x_w2, x_w3, and x_w4 are obtained respectively.

[0160] Specifically, when obtaining the feature vectors c_w1, c_w2, c_w3, c_w4 and the feature vectors h_w1, h_w2, h_w3, h_w4 corresponding to the feature vectors x_w1, x_w2, x_w3, and x_w4 respectively through the feature extraction window in the time recurrent neural network, the methods of obtaining c_w1 and h_w1 corresponding to the feature vector x_w1, c_w2 and h_w2 corresponding to the feature vector x_w2, c_w3 and h_w3 corresponding to the feature vector x_w3, and c_w4 and h_w4 corresponding to the feature vector x_w4 through the feature extraction window are the same. The specific implementation manner is the same as that Figure 4 for obtaining c_c2 and h_c2 corresponding to the feature vector x_c2 through the feature extraction window in

[0161] In addition, it should be noted that the algorithm applied to the feature extraction network based on words can be the word+char+LSTM algorithm, where word is used to represent words and char is used to represent string types.

[0162] In step S320, the character-level feature vector and the word-level feature vector are input into the entity recognition network, and the entity corresponding to the sample sequence is recognized through the entity recognition network; wherein, the entity recognition network is used to perform entity recognition for at least one of multiple original entity types.

[0163] Among them, the manner of recognizing the entity corresponding to the sample sequence through the entity recognition network can be: recognizing the entity corresponding to the sample sequence through the entity recognition network in combination with the character-level feature vector and the word-level feature vector corresponding to the sample sequence. Please refer to Figure 7 , Figure 7 shows an architecture diagram of an entity recognition network according to an exemplary embodiment of the present disclosure. As Figure 7 shown, the input of the entity recognition network is "Nanjing City" determined from the corpus dictionary according to "Nan", "Jing", "Shi" and "Nan", "Jing", "Shi"; wherein, the feature vectors corresponding to "Nan", "Jing", "Shi" are x_c1, x_c2, x_c3 respectively. x_c1, x_c2, x_c3 can be used as the vector representation of the first hidden layer and the input of the second hidden layer, and then the feature vectors c_c1, c_c2, c_c3 and the feature vectors h_c1, h_c2, h_c3 corresponding to the feature vectors x_c1, x_c2, x_c3 are obtained through the feature extraction window in the time recurrent neural network. The feature extraction network based on the word-level feature vector can determine the feature vector x_w13 corresponding to "Nanjing City". Further, by combining the character-level feature vector and the word-level feature vector, the feature vector c_w13 can be determined, and then the entity "Nanjing City" can be recognized according to the feature vector c_w13. In addition, for the specific implementation manner of obtaining the feature vectors c_c1, c_c2, c_c3 and the feature vectors h_c1, h_c2, h_c3 corresponding to the feature vectors x_c1, x_c2, x_c3 through the feature extraction window in the time recurrent neural network, please refer to Figure 4 shown, the c_c2 and h_c2 corresponding to the feature vector x_c2 obtained through the feature extraction window are the same, which will not be elaborated here.

[0164] In addition, it should be noted that the lattice-structured LSTM model (lattice-structured LSTM, abbreviated as Lattice LSTM) can encode both the input character sequence and the hidden word information in the sequence that can match the dictionary. Lattice LSTM can encode both the character-level sequence information and the word information corresponding to the sequence for the model to call. Compared with character-level encoding, Lattice LSTM adds word information, enriching the semantic expression; compared with word-level encoding, Lattice LSTM can, to a certain extent, avoid the influence brought by word segmentation errors.

[0165] Please refer to Figure 8 , Figure 8 which shows an application schematic diagram of an entity recognition network according to an exemplary embodiment of the present disclosure. Combining Figure 7 with the architecture diagram and the specific implementation manner of the entity recognition network shown, the feature vectors c and the feature vectors h corresponding to "Nan", "Jing", "Shi", "Chang", "Jiang", "Da", and "Qiao" can be determined. According to the Lattice-structured LSTM model, the entities "Nanjing", "Nanjing City", "Mayor", "Yangtze River", "Yangtze River Bridge", and "Bridge" can be determined by using the feature vectors c, the feature vectors h, and the word-level feature vectors corresponding to "Nan", "Jing", "Shi", "Chang", "Jiang", "Da", and "Qiao". Please further refer to Figure 9 , Figure 9 which shows another application schematic diagram of an entity recognition network according to an exemplary embodiment of the present disclosure. Combining Figure 8 with the application schematic diagram of the entity recognition network shown and the embodiments of the present disclosure, the entities "Nanjing", "Nanjing City", "Mayor", "Yangtze River", "Yangtze River Bridge", and "Bridge" can be recognized from the characters "Nan", "Jing", "Shi", "Chang", "Jiang", "Da", and "Qiao" in the sample sequence "Nanjing Yangtze River Bridge". For the specific recognition method, please refer to the specific implementation manner above, which will not be elaborated here.

[0166] In step S330, the network parameters of the entity recognition network are updated through the loss function between the entity and the corresponding original entity.

[0167] Optionally, the entity recognition network includes a first entity recognition network for the first original entity type and a second entity recognition network for the second original entity type. The multiple original entity types include the first original entity type and the second original entity type; wherein, the first original entity type can be one or more, and the second original entity recognition type can also be one or more, which are not limited in the embodiments of the present disclosure;

[0168] Specifically, inputting the character-level feature vector and the word-level feature vector into the entity recognition network to recognize the entity corresponding to the sample sequence through the entity recognition network may include the following steps:

[0169] Input the character-level feature vector and the word-level feature vector into the first entity recognition network and the second entity recognition network respectively, recognize the first entity corresponding to the first original entity type through the first entity recognition network, and recognize the second entity corresponding to the second original entity type through the second entity recognition network;

[0170] Specifically, updating the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity may include the following steps:

[0171] Calculate the first loss function between the first entity and the original entity corresponding to the first original entity type, and calculate the second loss function between the second entity and the original entity corresponding to the second original entity type;

[0172] Update the network parameters of the first entity recognition network according to the first loss function, and update the network parameters of the second entity recognition network according to the second loss function.

[0173] For example, the first entity recognition network can be used to recognize brand words and product words of the first original entity type, and the second entity recognition network can be used to recognize brand words, product words, and modifier words of the second original entity type. By inputting the character-level feature vectors and word-level feature vectors of the sample sequence "purchase a red Laowangpai mobile phone" into the first entity recognition network and the second entity recognition network respectively, the first entity recognition network can recognize the brand word "Laowangpai" and the product word "mobile phone"; among them, "Laowangpai" and "mobile phone" are the first entities. And the second entity recognition network can recognize the brand word "Laowangpai", the product word "mobile phone", and the modifier word "red"; among them, "Laowangpai", "mobile phone", and "red" are the second entities. If the brand word recognized by the first entity recognition network is not "Laowangpai" and the product word recognized is not "mobile phone", then the cosine distance between the entity corresponding to the brand word recognized by the first entity recognition network and "Laowangpai", and the cosine distance between the entity corresponding to the product word and "mobile phone" can be calculated as the first loss function. The number of the first loss functions is one or more, which is not limited in the embodiments of the present disclosure. In addition, if the brand word recognized by the second entity recognition network is not "Laowangpai", the product word recognized is not "mobile phone", and the modifier word recognized is not "red", then the cosine distance between the entity corresponding to the brand word recognized by the second entity recognition network and "Laowangpai", the cosine distance between the entity corresponding to the product word and "mobile phone", and the cosine distance between the entity corresponding to the modifier word and "red" can be calculated as the second loss function. The number of the second loss functions can also be one or more, which is not limited in the embodiments of the present disclosure. Furthermore, the first entity recognition network can be adjusted according to the first loss function until the first loss function falls within a preset threshold range, and the second entity recognition network can be adjusted according to the second loss function until the second loss function falls within a preset threshold range. Through the above implementation manners, entity recognition networks applicable to different entity types can be trained, so that corresponding entity recognition networks can be used according to different requirements, improving the accuracy of entity recognition.

[0174] It can be seen that implementing Figure 3 the entity recognition network training method shown can perform entity recognition on the sample sequence by combining the feature vectors at the character level and the word level, improving the accuracy of entity recognition and the recognition effect of the entity recognition network.

[0175] Please refer to Figure 10 , Figure 10 which shows a schematic structural diagram of an entity recognition system according to an exemplary embodiment of the present disclosure. As Figure 10As shown in the figure, the entity recognition system may include a data preprocessing module 1010, a slot setting module 1020, an interface design module 1030, an entity recognition network 1040, an entity recognition module 1050, and an online application module 1060. Among them, the data preprocessing module 1010 may include a data collection sub-module 1011, a data filtering sub-module 1012, a data annotation sub-module 1013, and a data category balancing sub-module 1014. The slot setting module 1020 may include a product word slot 1021, a brand word slot 1022, a modifier slot 1023, and a gender and price slot 1024. The interface design module 1030 may include a skill interface design sub-module 1031, an intent interface design sub-module 1032, a slot interface design sub-module 1033, and an automation script sub-module 1034. The entity recognition network 1040 may include a feature vector extraction layer 1041 and a feature vector combination layer 1042. The entity recognition module 1050 may include a first entity recognition sub-module 1051, a second entity recognition sub-module 1052, a multi-task sharing sub-module 1053, and a parameter update sub-module 1054. The online application module 1060 may include an offline F1 metric evaluation sub-module 1061, an intent evaluation sub-module 1062, an entity recognition evaluation sub-module 1063, and an iterative optimization sub-module 1064.

[0176] Specifically, the first sentences in multiple groups of historical user sessions can be collected by the data collection sub-module 1011. Furthermore, the first sentences can be screened by the data filtering sub-module 1012 according to preset conversation rules, and the screened first sentences can be determined as sample sequences. The sample sequences can be text information corresponding to user sessions, text information corresponding to intelligent assistant logs, or historical search information corresponding to search windows crawled by a crawler program, etc., which are not limited in the embodiments of the present disclosure. Furthermore, the user intentions corresponding to the sample sequences can be determined by the data annotation sub-module 1013 according to the user intentions corresponding to multiple groups of historical user sessions, and the number of sample sequences corresponding to each type of user intention can be adjusted by the data category balancing sub-module 1014 according to the preset user intention data distribution, and the number of sample sequences in each sequence length section can be adjusted according to the preset sequence length distribution. Then, the data annotation sub-module 1013 determines multiple original entity types corresponding to the adjusted sample sequences according to the user intentions, and determines the original entities corresponding to the multiple original entity types in the adjusted sample sequences, and annotates the sample sequences according to the multiple original entity types and the original entities. Furthermore, slots can be set by the slot setting module 1020. The slots set by the slot setting module 1020 can include: a product word slot 1021, a brand word slot 1022, a modifier slot 1023, and a gender and price slot 1024. Furthermore, a skill interface SkillWrap corresponding to the user session can be designed by the skill interface design sub-module 1031, including two attributes: a skill id and a skill number code, and an intention interface IntentWrap corresponding to the user session can be designed by the intention interface design sub-module 1032, including three attributes: an intention id, an intention number code, and a cloud storage sentenceKey corresponding to the intention conversation, and a slot interface slotWrap corresponding to the user session can be designed by the slot interface design sub-module 1033, including three attributes: a slot id, a slot number code, and a cloud storage dictKey corresponding to the slot value, and the skill, intention, and corresponding data corresponding to the user session can be prepared by the automation script sub-module 1034, and then automated trigger training can be performed. The input parameter of the automation script sub-module 1034 is the skill SkillWrap and the set of intentions List under the skill <intentwrap>And the set List of slot values under the skill <slotwrap>, the output parameter is the sample sequence format required by the generation model. Furthermore, the feature vector corresponding to each character in the labeled sample sequence can be determined through the feature vector extraction layer 1041 as the character-level feature vector, and the labeled sample sequence can be segmented, and the feature vector corresponding to the segmentation result can be determined as the word-level feature vector. Furthermore, the character-level feature vector and the word-level feature vector can be combined through the feature vector combination layer 1042, so as to facilitate the first entity recognition sub-module 1051 to recognize the first entity corresponding to the sample sequence, and to facilitate the second entity recognition sub-module 1052 to recognize the second entity corresponding to the sample sequence; wherein, the entity types recognized by the first entity recognition sub-module 1051 and the second entity recognition sub-module 1052 are different. For example, the first entity recognition sub-module 1051 corresponds to the first original entity type, and the second entity recognition sub-module 1052 corresponds to the second original entity type. The multiple original entity types include the first original entity type and the second original entity type. The first original entity type can be one or more, and the second original entity recognition type can also be one or more. Therefore, the entities recognized by the first entity recognition sub-module 1051 and the second entity recognition sub-module 1052 can be the same or different, which is not limited in the embodiments of the present disclosure. In addition, the first entity recognition sub-module 1051 and the second entity recognition sub-module 1052 can share the feature vectors determined by the feature vector extraction layer 1041 and the feature vector combination layer 1042 through the multi-task sharing sub-module 1053. Furthermore, the first loss function between the first entity and the original entity corresponding to the first original entity type can be calculated according to the parameter update sub-module 1054, and the second loss function between the second entity and the original entity corresponding to the second original entity type can be calculated, and the network parameters of the first entity recognition network can be updated according to the first loss function, and the network parameters of the second entity recognition network can be updated according to the second loss function. Furthermore, the offline F1 metric evaluation sub-module 1061 can evaluate the offline F1 metric (i.e., F1-Measure, a comprehensive evaluation metric) of the trained entity recognition network. Among them, F-Measure is a statistic, also known as F-Score, and is also the weighted harmonic mean of precision and recall, which is used to evaluate the quality of a classification model. Furthermore, the accuracy of determining the user intention by the trained entity recognition network can also be evaluated through the intention evaluation sub-module 1062, and the recognition accuracy of the trained entity recognition network for entities can be evaluated through the entity recognition evaluation sub-module 1063, and the samples with poor quality in each round of evaluation can be optimized through the iterative optimization sub-module 1064, and the sample can be optimized according to the evaluation result to improve the recognition accuracy of the entity recognition network.

[0177] It can be seen that the implementation Figure 10 As shown, it can, to a certain extent, solve the problem of low semantic analysis efficiency of intelligent assistants. By training an entity recognition network that can target one or more entity types to recognize the conversation sequence input by the user, the entity recognition efficiency and the semantic analysis efficiency of the intelligent assistant can be improved. Moreover, it can improve the intelligence level of the intelligent assistant through the improvement of entity recognition efficiency and optimize the user experience. In addition, compared with the manually constructed corpus in the prior art, the present application can perform entity recognition on the conversation sequence through the entity recognition network, reducing the labor cost.

[0178] In step S130, the entity recognition network after training is used to recognize the entities in the conversation sequence input by the user.

[0179] For example, the conversation sequence input by the user is "I want to buy an old Wangpai mobile phone and a Lao Zhangpai mobile phone". The entity recognition network after training can recognize the entities "one", "old Wangpai", "mobile phone", and "Lao Zhangpai" in this conversation sequence. Among them, the conversation sequence can be the text information input by the user, or the text information obtained by recognizing the user's input audio, or other forms of text information, which is not limited in the embodiments of the present disclosure.

[0180] It can be seen that implementing Figure 1 the entity recognition method shown can, to a certain extent, solve the problem of low semantic analysis efficiency of intelligent assistants. By training an entity recognition network that can target one or more entity types to recognize the conversation sequence input by the user, the entity recognition efficiency and the semantic analysis efficiency of the intelligent assistant can be improved. Moreover, it can improve the intelligence level of the intelligent assistant through the improvement of entity recognition efficiency and optimize the user experience. In addition, compared with the manually constructed corpus in the prior art, the present application can perform entity recognition on the conversation sequence through the entity recognition network, reducing the labor cost.

[0181] Please refer to Figure 11 , Figure 11 which shows a structural block diagram of an entity recognition device according to an exemplary embodiment of the present disclosure. The entity recognition device includes a sample annotation unit 1101, a network training unit 1102, and an entity recognition unit 1103, where:

[0182] The sample annotation unit 1101 is configured to annotate the sample sequence with the corresponding multiple original entity types and the original entities respectively corresponding to the multiple original entity types in the sample sequence;

[0183] The network training unit 1102 is configured to train an entity recognition network for at least one of the multiple original entity types through the annotated sample sequence;

[0184] The entity recognition unit 1103 is configured to recognize entities in the session sequence input by the user according to the trained entity recognition network.

[0185] It can be seen that implementing Figure 11 the entity recognition device shown can, to a certain extent, solve the problem of low semantic analysis efficiency of the intelligent assistant. By training an entity recognition network that can target one or more entity types to recognize the session sequence input by the user, the entity recognition efficiency and the semantic analysis efficiency of the intelligent assistant can be improved; moreover, the intelligence level of the intelligent assistant can be improved through the improvement of the entity recognition efficiency, optimizing the user experience; and, compared with the manually constructed corpus in the prior art, the present application can perform entity recognition on the session sequence through the entity recognition network, reducing the labor cost.

[0186] As an exemplary embodiment, the specific manner in which the sample annotation unit 1101 annotates the sample sequence with multiple original entity types corresponding to the sample sequence and original entity annotations corresponding to the multiple original entity types in the sample sequence can be:

[0187] The sample annotation unit 1101 respectively collects the first sentences in multiple groups of historical user conversations as sample sequences, and determines the user intents corresponding to the multiple groups of historical user conversations according to the content of the multiple groups of historical user conversations;

[0188] The sample annotation unit 1101 determines multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence according to the user intent;

[0189] The sample annotation unit 1101 annotates the sample sequence according to the multiple original entity types and the original entities.

[0190] Among them, the specific manner in which the sample annotation unit 1101 respectively collects the first sentences in multiple groups of historical user conversations as sample sequences can be:

[0191] The sample annotation unit 1101 respectively collects the first sentences in multiple groups of historical user conversations, screens the first sentences according to the preset speech rules, and determines the screened first sentences as sample sequences. In this way, by screening the first sentences (i.e., the sample sequences), the user inputs that do not contain important information can be removed, thereby improving the generalization ability of the trained entity recognition network.

[0192] Among them, the specific manner in which the sample annotation unit 1101 determines multiple original entity types corresponding to the sample sequence and original entities corresponding to the multiple original entity types in the sample sequence according to the user intent can be:

[0193] The sample annotation unit 1101 determines the user intents corresponding to the sample sequences according to the user intents corresponding to the multiple groups of historical user conversations respectively;

[0194] The sample annotation unit 1101 adjusts the number of sample sequences corresponding to each type of user intention according to the preset user intention data distribution, and adjusts the number of sample sequences in each sequence length section according to the preset sequence length distribution;

[0195] The sample annotation unit 1101 determines multiple original entity types corresponding to the adjusted sample sequences according to the user intention, and determines the original entities corresponding to the multiple original entity types in the adjusted sample sequences. This can balance the number of sample sequences based on each type of user intention and sequence length section, preventing the number of sample sequences of a certain type of user intention or a certain length section from being excessive, optimizing the ratio of the number of sample sequences between different types of user intentions and the ratio of the number of sample sequences between different sequence length sections, thereby improving the training effect of the entity recognition network and enhancing the recognition accuracy of the entity recognition network.

[0196] It can be seen that implementing this exemplary embodiment can enrich the training samples through the annotation of sample sequences to achieve entity recognition training for the entity recognition network.

[0197] As another exemplary embodiment, the specific manner in which the network training unit 1102 trains the entity recognition network for at least one of the multiple original entity types through the annotated sample sequences can be:

[0198] The network training unit 1102 determines the character-level feature vector and word-level feature vector of the annotated sample sequences;

[0199] The network training unit 1102 inputs the character-level feature vector and word-level feature vector into the entity recognition network to identify the entities corresponding to the sample sequences through the entity recognition network; wherein, the entity recognition network is used to perform entity recognition for at least one of the multiple original entity types;

[0200] The network training unit 1102 updates the network parameters of the entity recognition network through the loss function between the entities and the corresponding original entities.

[0201] Among them, the specific manner in which the network training unit 1102 determines the character-level feature vector and word-level feature vector of the annotated sample sequences can be:

[0202] The network training unit 1102 determines the feature vector corresponding to each character in the annotated sample sequence as the character-level feature vector;

[0203] The network training unit 1102 performs word segmentation on the annotated sample sequences and determines the feature vector corresponding to the word segmentation result as the word-level feature vector.

[0204] Wherein: The entity recognition network includes a first entity recognition network for the first original entity type and a second entity recognition network for the second original entity type, and the multiple original entity types include the first original entity type and the second original entity type;

[0205] The way that the network training unit 1102 inputs the character-level feature vector and the word-level feature vector into the entity recognition network and recognizes the entity corresponding to the sample sequence through the entity recognition network can be specifically as follows:

[0206] The network training unit 1102 inputs the character-level feature vector and the word-level feature vector into the first entity recognition network and the second entity recognition network respectively, recognizes the first entity corresponding to the first original entity type through the first entity recognition network, and recognizes the second entity corresponding to the second original entity type through the second entity recognition network;

[0207] The way that the network training unit 1102 updates the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity can be specifically as follows:

[0208] The network training unit 1102 calculates a first loss function between the first entity and the original entity corresponding to the first original entity type, and calculates a second loss function between the second entity and the original entity corresponding to the second original entity type;

[0209] The network training unit 1102 updates the network parameters of the first entity recognition network according to the first loss function, and updates the network parameters of the second entity recognition network according to the second loss function.

[0210] It can be seen that implementing this exemplary embodiment can perform entity recognition on the sample sequence by combining the feature vectors at the character level and the word level, improving the accuracy of entity recognition and the recognition effect of the entity recognition network.

[0211] Since each functional module of the entity recognition device in the exemplary embodiment of the present disclosure corresponds to the steps in the exemplary embodiment of the above entity recognition method, for the details not disclosed in the embodiment of the device of the present disclosure, please refer to the embodiment of the above entity recognition method of the present disclosure.

[0212] Please refer to Figure 12 , Figure 12 which shows a schematic structural diagram of a computer system 1200 of an electronic device suitable for implementing an exemplary embodiment of the present disclosure. Figure 12 The shown computer system 1200 of the electronic device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0213] Such as Figure 12 As shown, computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage section 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0214] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read from it can be installed into the storage section 1208 as needed.

[0215] Specifically, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above functions defined in the system of the present application are executed.

[0216] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0217] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram can represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0218] The units involved in the embodiments described in the present disclosure may be implemented in software or in hardware, and the described units may also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0219] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device is caused to implement the entity recognition method as described in the above embodiments.

[0220] For example, the electronic device may implement as Figure 1 shown in: Step S110: Label the sample sequence with multiple original entity types corresponding to the sample sequence and original entity annotation corresponding to each of the multiple original entity types in the sample sequence; Step S120: Train an entity recognition network for at least one of the multiple original entity types through the labeled sample sequence; Step S130: Recognize the entities in the session sequence input by the user according to the trained entity recognition network.

[0221] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by multiple modules or units.

[0222] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0223] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0224] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.< / slotwrap> < / intentwrap>

Claims

1. An entity recognition method, characterized in that, The method includes: Annotating the sample sequence with multiple original entity types corresponding to the sample sequence and original entities respectively corresponding to the multiple original entity types in the sample sequence, including: respectively collecting the first sentences in multiple groups of historical user conversations as the sample sequence, and determining the user intents respectively corresponding to the multiple groups of historical user conversations according to the content of the multiple groups of historical user conversations; determining the multiple original entity types corresponding to the sample sequence and the original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent; annotating the sample sequence according to the multiple original entity types and the original entities; Training an entity recognition network for at least one of the multiple original entity types with the annotated sample sequence; Identifying entities in the conversation sequence input by the user according to the trained entity recognition network; Among them, determining the multiple original entity types corresponding to the sample sequence and the original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent, including: determining the user intent corresponding to the sample sequence respectively according to the user intents respectively corresponding to the multiple groups of historical user conversations; adjusting the number of sample sequences respectively corresponding to each type of user intent according to the preset user intent data distribution, and adjusting the number of sample sequences in each sequence length section according to the preset sequence length distribution; determining the multiple original entity types corresponding to the adjusted sample sequence according to the user intent, and determining the original entities respectively corresponding to the multiple original entity types in the adjusted sample sequence.

2. The method according to claim 1, characterized in that, Respectively collecting the first sentences in multiple groups of historical user conversations as the sample sequence, including: Respectively collecting the first sentences in the multiple groups of historical user conversations, screening the first sentences according to the preset speech rules, and determining the screened first sentences as the sample sequence.

3. The method according to claim 1, characterized in that, Training an entity recognition network for at least one of the multiple original entity types with the annotated sample sequence, including: Determining the character-level feature vector and word-level feature vector of the annotated sample sequence; Inputting the character-level feature vector and the word-level feature vector into the entity recognition network, and identifying the entity corresponding to the sample sequence through the entity recognition network; wherein, the entity recognition network is used for entity recognition for at least one of the multiple original entity types; Updating the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity.

4. The method according to claim 3, characterized in that, Determining the character-level feature vector and word-level feature vector of the annotated sample sequence, including: Determining the feature vector corresponding to each character in the annotated sample sequence as the character-level feature vector; Performing word segmentation processing on the annotated sample sequence, and determining the feature vector corresponding to the word segmentation result as the word-level feature vector.

5. The method according to claim 3, characterized in that, The entity recognition network includes a first entity recognition network for the first original entity type and a second entity recognition network for the second original entity type, and the first original entity type and the second original entity type are included in the multiple original entity types.

6. The method according to claim 5, characterized in that, Inputting the character-level feature vector and the word-level feature vector into the entity recognition network, and recognizing the entity corresponding to the sample sequence through the entity recognition network, including: Inputting the character-level feature vector and the word-level feature vector into the first entity recognition network and the second entity recognition network respectively, recognizing a first entity corresponding to the first original entity type through the first entity recognition network, and recognizing a second entity corresponding to the second original entity type through the second entity recognition network.

7. The method according to claim 5, characterized in that, Updating the network parameters of the entity recognition network through the loss function between the entity and the corresponding original entity, including: Calculating a first loss function between the first entity and the original entity corresponding to the first original entity type, and calculating a second loss function between the second entity and the original entity corresponding to the second original entity type; Updating the network parameters of the first entity recognition network according to the first loss function, and updating the network parameters of the second entity recognition network according to the second loss function.

8. An entity recognition device, characterized in that, The device includes: A sample annotation unit for annotating the sample sequence through multiple original entity types corresponding to the sample sequence and the original entities respectively corresponding to the multiple original entity types in the sample sequence, including: respectively collecting the first sentences in multiple groups of historical user sessions as sample sequences, and determining the user intents respectively corresponding to the multiple groups of historical user sessions according to the contents of the multiple groups of historical user sessions; determining multiple original entity types corresponding to the sample sequence and the original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent; annotating the sample sequence according to the multiple original entity types and the original entities. A network training unit for training an entity recognition network for at least one of the multiple original entity types through the annotated sample sequence; An entity recognition unit for recognizing the entity in the session sequence input by the user according to the trained entity recognition network; Wherein, determining multiple original entity types corresponding to the sample sequence and the original entities respectively corresponding to the multiple original entity types in the sample sequence according to the user intent includes: determining the user intent corresponding to the sample sequence respectively according to the user intents respectively corresponding to the multiple groups of historical user sessions; adjusting the number of sample sequences respectively corresponding to each type of user intent according to the preset user intent data distribution, and adjusting the number of sample sequences in each sequence length section according to the preset sequence length distribution; determining multiple original entity types corresponding to the adjusted sample sequence according to the user intent, and determining the original entities respectively corresponding to the multiple original entity types in the adjusted sample sequence.

9. A computer-readable medium, on which a computer program is stored, characterized in that, When the program is executed by a processor, it implements the entity recognition method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Comprising: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the entity recognition method according to any one of claims 1 to 7.

Images