Entity identification method and device in dialogue, equipment and storage medium
Through dialogue text coding and domain knowledge base matching weighting processing, the problem of low accuracy of film and television titles and character names recognition is solved, accurate recognition of new vocabulary is achieved, and user interaction experience is improved.
Patent Information
- Application Number
- CN202310574034.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-22
AI Technical Summary
When the existing naming entity recognition method is used to process film and television titles and character names, it is difficult to distinguish new vocabulary in film and television dramas from terms in daily dialogues, resulting in low recognition accuracy, especially in newly released film and television dramas with online terms and spoken language. When using words to make use of them, it is difficult to accurately identify entity words.
The feature vector is generated by encoding the dialogue text and matched with the domain knowledge base. The second feature vector is generated based on the preset weight weight weighting. The entity recognition model is input to generate entity recognition results. The domain knowledge base is introduced for interactive calculations to improve the recognition accuracy.
It improves the recognition ability and accuracy of the entity recognition model, can accurately identify new entity words that appear in the conversation, and improves the user interaction experience.
Smart Images

Figure CN119005186A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method, device, equipment and storage medium for entity recognition in a conversation. Background Art
[0002] Named Entity Recognition (NER) refers to identifying entity words with specific meanings in a given text, including names of people, places, organizations, proper nouns, etc. In the film and television entity recognition scenario, the entities to be identified are film and television titles or character names.
[0003] Current named entity recognition methods include using deep learning methods to represent the task as a multi-classification task or a sequence labeling task, and learning from data to perform named entity recognition.
[0004] With the development of big data technology and the exponential growth of resource data, a large number of film and television related entities have emerged in the field of video media resources, such as unstructured data such as film and television work names, actor names, and director names. New words such as Internet slang and colloquial terms often appear in many film and television titles. For example, "I'm a Little Confused" is both the name of the theme song and a common term in daily chat. When the aforementioned new words exist in newly released film and television dramas, it is difficult to distinguish them from chat scenes when identifying such entity words. Therefore, how to accurately extract named entities from conversations in response to the above situation is a technical problem that needs to be solved urgently. Summary of the invention
[0005] In order to solve the above technical problems, the present disclosure provides a method, apparatus, device and storage medium for entity recognition in a conversation.
[0006] In a first aspect, an embodiment of the present disclosure provides a method for identifying entities in a conversation, comprising:
[0007] Acquire a dialogue text, wherein the dialogue text includes at least one single sentence;
[0008] Encoding each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence;
[0009] Matching each single sentence with entity information in a domain knowledge base to determine a target entity contained in the single sentence from the domain knowledge base;
[0010] Weighting the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base to generate a second feature vector corresponding to each single sentence;
[0011] The second feature vector is input into an entity recognition model to generate an entity recognition result.
[0012] In a second aspect, an embodiment of the present disclosure provides a device for identifying entities in a conversation, including:
[0013] An acquisition module, used for acquiring a dialogue text, wherein the dialogue text includes at least one single sentence;
[0014] An encoding module, used for encoding each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence;
[0015] A matching module, used for matching each single sentence with entity information in a domain knowledge base, so as to determine a target entity contained in the single sentence from the domain knowledge base;
[0016] A generating module, configured to weight the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base, and generate a second feature vector corresponding to each single sentence;
[0017] The recognition module is used to input the second feature vector into the entity recognition model to generate an entity recognition result.
[0018] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the entity recognition method in the conversation described in the first aspect above.
[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the entity recognition method in the conversation described in the first aspect is implemented.
[0020] In a fifth aspect, an embodiment of the present disclosure provides a vehicle, comprising the entity recognition device in the conversation as described in the second aspect above, or the electronic device as described in the third aspect above.
[0021] Compared with the prior art, the technical solution provided by the disclosed embodiment has the following advantages: by encoding each single sentence in the dialogue text, a first feature vector corresponding to each single sentence is generated, each single sentence is matched with the entity information in the domain knowledge base to determine the target entity contained in the single sentence from the domain knowledge base, the target entity in the first feature vector is weighted according to the weight corresponding to the target entity, a second feature vector corresponding to each single sentence is generated, the second feature vector is input into the entity recognition model, and an entity recognition result is generated. Therefore, by introducing the entity information in the domain knowledge base for interactive calculation, it is possible to accurately identify and mine the entity words in the corresponding field in the user dialogue, improve the recognition ability and accuracy of the entity recognition model, and enhance the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0024] Figure 1 A flowchart of a method for identifying entities in a conversation provided by an embodiment of the present disclosure;
[0025] Figure 2 A flowchart of another entity recognition method in a conversation provided by an embodiment of the present disclosure;
[0026] Figure 3 A schematic diagram of the structure of a device for identifying entities in a conversation provided by an embodiment of the present disclosure;
[0027] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0030] The method of the disclosed embodiment can be applied to the voice interaction scenario between the user and the vehicle-mounted device. In practical applications, there are the following scenarios: the user inputs a voice command by voice interaction with the vehicle-mounted device, such as I want to watch movie XX, I want to listen to music, etc. The vehicle-mounted device recognizes the user's voice and then plays the corresponding movie, music, etc. At present, the deep learning method is used to represent the task as a multi-classification task or a sequence labeling task, and learn from the data to perform named entity recognition. With the rapid development of the Internet, there are many new words such as network terms and colloquial terms in the names of film and television titles and music. For example, the names of newly released film and television dramas contain the aforementioned new words. When identifying such entity words, it is difficult to distinguish them from the chat scene, and the recognition accuracy needs to be improved. Therefore, the entity recognition method in the dialogue provided in this embodiment improves the recognition ability and accuracy of the entity recognition model, has a certain recognition ability for new entity words that appear in the dialogue, and improves the user's interactive experience.
[0031] Figure 1 This is a flow chart of a method for entity recognition in a conversation provided in an embodiment of the present disclosure. The method provided in the embodiment of the present disclosure can be executed by an entity recognition device in a conversation. The device can be implemented in software and / or hardware and can be integrated on any electronic device with computing capabilities, such as a vehicle-mounted terminal.
[0032] like Figure 1 As shown, the entity recognition method in the conversation provided by the embodiment of the present disclosure may include:
[0033] Step 101, obtaining the conversation text.
[0034] In this embodiment, the user voice data is acquired through the voice collection device in the vehicle, and the user voice data is recognized through the pre-trained voice recognition model to generate text corresponding to the user voice data as the dialogue text. The dialogue text includes at least one single sentence. For example, the dialogue text "I want to watch a movie, and I want to see you" contains two single sentences "I want to watch a movie" and "I want to see you".
[0035] Step 102: Encode each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence.
[0036] In this embodiment, an encoder is used to encode the acquired dialogue text to obtain a feature vector corresponding to the dialogue text. Optionally, each single sentence in the dialogue text is input into a preset encoder to generate a feature vector for each single sentence as a first feature vector corresponding to the single sentence. The encoder can use a pre-trained language model to enhance the semantic learning of the single sentence text.
[0037] As an example, for a single sentence Q1 and a single sentence Q2, Q1 and Q2 are respectively input into an encoder for encoding processing to generate a feature vector U1 corresponding to Q1 and a feature vector U2 corresponding to Q2.
[0038] Among them, the specific single sentence can be determined from the dialogue text using existing methods, such as using punctuation marks.
[0039] Step 103 , matching each single sentence with entity information in the domain knowledge base, so as to determine the target entity contained in the single sentence from the domain knowledge base.
[0040] In this embodiment, the domain knowledge base includes entity information, which can be updated in real time or periodically. For example, taking music entity recognition as an example, the domain knowledge base can be a music entity library, and new music name entities are regularly added to the music entity library. The supplementation methods include manual input by users, network data crawling, etc., and no specific restrictions are made here.
[0041] As an example, for a single sentence "I want to listen to the theme song I'm a little confused", if the entity information in the music entity library includes I'm a little confused, then the entity is determined to be the target entity included in the single sentence.
[0042] Step 104 , weighting the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base, and generating a second feature vector corresponding to each single sentence.
[0043] In this embodiment, a corresponding preset weight is set for the entity information in the domain knowledge base and stored in the domain knowledge base. The preset weight can be determined based on domain prior knowledge, for example, the preset weight is calculated by certain rules according to the attributes of the entity information; in addition, the corresponding preset weight can also be set for the entity information in the domain knowledge base through other existing methods. By determining the preset weight corresponding to the target entity, the target entity in the single sentence is weighted based on the preset weight, and the second feature vector corresponding to the single sentence is generated. Thus, through interactive calculation methods such as weighted summation, the entity in the single sentence that hits the entity information in the domain knowledge base is given a weight, thereby enhancing the model's ability to understand the entity boundary.
[0044] Optionally, for each single sentence, the position of the target entity in the single sentence is used to obtain the original target entity sub-vector corresponding to the target entity in the first feature vector, and a weighted calculation is performed according to the preset weight corresponding to the target entity and the entity vector of the target entity in the domain knowledge base and the original target entity sub-vector to generate a weighted sub-vector, and the weighted sub-vector is used to replace the original target entity sub-vector in the first feature vector to obtain a second feature vector. As an example, a single sentence is segmented, and the single sentence is composed of segmentation one, segmentation two and segmentation three. Each segmentation has corresponding position information. According to the position information, the subvector corresponding to the segmentation in the first feature vector can be determined, wherein segmentation two is the target entity that hits the domain knowledge base, and the corresponding subvector two is determined in the first feature vector using the position information of segmentation two. In the same way, subvector one corresponding to segmentation one and subvector three corresponding to segmentation three can be determined, and subvector four is generated according to the preset weight corresponding to the target entity and the entity vector of the target entity in the knowledge base. Subvector one, subvector four and subvector three are spliced into a new vector according to the position information, and the new vector is used as the second feature vector; wherein, according to the preset weight corresponding to the target entity The entity vector of the target entity in the knowledge base generates a sub-vector four which can be calculated using K1*T1+K2*T2. More specifically, K1 and K2 are weights, wherein K2 is the preset weight of the second segmentation word in the domain knowledge base, and the sum of K1 and K2 is a preset value, for example, K1+K2=1, T1 is the entity vector (sub-vector two) of the second segmentation word at the corresponding position in the first feature vector, and T2 is the preset entity vector of the second segmentation word in the knowledge base. Optionally, when setting the weight, it can be determined by the similarity of T1 and T2 and normalized. For example, if the similarity is 0.8, K1 and K2 are 0.8 and 0.2 respectively. The similarity includes cosine similarity and Euclidean distance, so as to weight the target entity in the first feature vector according to the weight corresponding to the target entity.
[0045] Optionally, different entity information in the domain knowledge base can correspond to different weights. In a single-task scenario, if multiple entities in a single sentence hit the same domain knowledge base, that is, the number of target entities contained in a single sentence is multiple, then the maximum weight is determined from the multiple weights corresponding to the multiple target entities, and the target entity corresponding to the maximum weight in the first feature vector of the single sentence is weighted according to the maximum weight to generate a second feature vector corresponding to the single sentence. For example, taking the film and television entity library as an example, when a single sentence is matched with the entity information in the film and television entity library, if the number of film and television entities contained in the single sentence is matched, then the maximum weight is determined from the weights corresponding to the multiple film and television entities, and the film and television entities corresponding to the maximum weight in the first feature vector of the single sentence are weighted according to the maximum weight to generate a second feature vector corresponding to the single sentence.
[0046] Step 105: input the second feature vector into the entity recognition model to generate an entity recognition result.
[0047] In this embodiment, the input of the entity recognition model is the second feature vector of a single sentence, which introduces information from an external knowledge module, and the output of the entity recognition model is the entity recognized in the single sentence. The entity recognition model can be implemented based on a deep neural network.
[0048] In one embodiment of the present disclosure, the second feature vector is input into the entity recognition model to generate the entity recognition result, including: the second feature vector of each single sentence is input into the conditional random field for prediction, and the sequence labeling result of each single sentence is generated, the sequence labeling result is used to indicate whether each word in the single sentence belongs to the field to be recognized, and the sequence labeling result is determined from the single sentence as the word in the field to be recognized as the entity recognition result. Among them, the conditional random field (CRF) is a discriminative probability model, which is often used for the labeling of natural language text or biological sequences or the analysis of sequence data. The conditional random field is used to impose certain constraints on the recognized entity, and the entity contained in the current single sentence is parsed. The specific implementation process of the conditional random field in this embodiment will not be repeated.
[0049] According to the technical solution of the embodiment of the present disclosure, each single sentence in the dialogue text is encoded to generate a first feature vector corresponding to each single sentence, and each single sentence is matched with the entity information in the domain knowledge base to determine the target entity contained in the single sentence from the domain knowledge base, and the first feature vector is weighted according to the weight corresponding to the target entity to generate a second feature vector corresponding to each single sentence, and the second feature vector is input into the entity recognition model to generate an entity recognition result. Thus, by introducing the entity information in the domain knowledge base for interactive calculation, the model's ability to understand entity boundaries is enhanced, and the entity words in the corresponding field in the user dialogue can be accurately identified and mined, thereby improving the recognition ability and accuracy of the entity recognition model, having a certain recognition ability for new entity words appearing in the dialogue, and enhancing the user's interactive experience.
[0050] In practical applications, it is a common scenario for users to control the in-vehicle device to play movies and TV series through voice interaction. In this scenario, how to improve the recognition accuracy of movie entities becomes an important requirement. Based on the above embodiment, the following is an explanation in combination with the movie entity recognition scenario.
[0051] Figure 2 A flow chart of a model training process provided by an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the method includes:
[0052] Step 201, obtaining the conversation text.
[0053] In this embodiment, the dialogue text includes at least one single sentence. For example, the dialogue text "I want to watch a movie, and I want to see you" contains two single sentences "I want to watch a movie" and "I want to see you".
[0054] Step 202, input each single sentence in the dialogue text into a single sentence encoder, generate an initial feature vector for each single sentence, and concatenate the feature vectors of all the single sentences to obtain a target feature vector.
[0055] Step 203: input the target feature vector into a context encoder to generate a first feature vector corresponding to each single sentence.
[0056] As an example, for a single sentence Q1 and a single sentence Q2, Q1 and Q2 are respectively input into the single sentence encoder for encoding processing to generate an initial feature vector U1 corresponding to Q1 and an initial feature vector U2 corresponding to Q2. Then, U1 and U2 are concatenated according to the context relationship to construct a batch. Further, the batch is input into the context encoder to generate a first feature vector C1 corresponding to Q1 and a first feature vector C2 corresponding to Q2. Thus, the initial feature vector of the single sentence can be used to obtain the first feature vector containing context information, and the entity and sentence information can be learned from the dialogue semantics.
[0057] Among them, single-sentence encoders include convolutional neural networks (CNN), long short-term memory networks (LSTM), a sequence-to-sequence model based on self-attention mechanism (transformer), BERT (Bidirectional Encoder Representation from Transformers, a bidirectional encoder representation from Transformers, an unsupervised pre-trained language model for natural language processing tasks), and context encoders include gated recurrent units (GRU, one of the variants of recurrent neural networks RNN), bidirectional long short-term memory networks and other temporal encoders, or context encoding implemented using attention mechanism.
[0058] Step 204 , each single sentence is matched with the entity information in the small talk knowledge base and the entity information in the film and television entity base to determine the small talk entity and the film and television entity contained in the single sentence.
[0059] In this embodiment, the domain knowledge base may be a small talk knowledge base and a film and television entity base, and both the small talk knowledge base and the film and television entity base contain entity information. For example, for a single sentence "I want to watch movie XX", if the entity information in the small talk knowledge base includes XX, then the entity is determined to be a small talk entity contained in the single sentence, and if the entity information in the film and television entity base includes XX, then the entity is determined to be a film and television entity contained in the single sentence.
[0060] The following is an explanation of the film and television entity library. The entity information in the film and television entity library is updated according to the preset update conditions. When the update conditions are met, the candidate film and television entity is crawled from the preset film and television platform. If the candidate film and television entity is not in the film and television entity library, the candidate film and television entity is added to the film and television entity library.
[0061] As an example, the update condition can be updated when a new film or TV series is detected on a specified film or TV platform, or it can be updated periodically at a specified time interval. In this example, when crawling candidate film and TV entities from a preset film and TV platform, they can be obtained according to popularity ranking, such as obtaining film and TV entities of multiple film and TV series with higher popularity rankings, or they can be obtained in chronological order of update time, such as obtaining film and TV entities of multiple film and TV series with the most recent update time. Therefore, by updating the film and TV entity library and combining the subsequent matching and interactive calculation steps, the accuracy of film and TV entity recognition can be improved. For example, if the names of newly released film and TV series contain Internet terms and colloquial terms, the recognition ability of such entity words can be improved.
[0062] It should be noted that the domain knowledge base may contain multiple entity libraries or knowledge bases. The above-mentioned domain knowledge bases are the chat knowledge base and the film and television entity library, which are only used as examples for explanation and do not limit which entity libraries or knowledge bases the domain knowledge base is composed of.
[0063] Step 205: weight the chat entities and the film and television entities in the first feature vector according to the weights corresponding to the chat entities and the weights corresponding to the film and television entities, and generate a second feature vector corresponding to each single sentence.
[0064] In this embodiment, entity information in the chat knowledge base and entity information in the film and television entity library are introduced, and the entity vector of the aforementioned step, that is, the first feature vector, is interactively calculated, and the entity words of the sentence are weighted, so that the feature vector of a single sentence is updated to a feature vector containing chat features, the film and television entity library and context information. Thus, through interactive calculation methods such as weighted summation, weights are given to single sentences that hit entity information in the chat knowledge base and the film and television entity library, thereby enhancing the model's ability to understand the boundaries of film and television entities.
[0065] Among them, the entity information in the chat knowledge base and the entity information in the film and television entity base are respectively set with corresponding preset weights.
[0066] As an example, after determining the chat entities and / or film and television entities in a single sentence, the chat entities in the first feature vector are weighted according to the preset weights of the chat entities in the chat knowledge base, and / or the film and television entities in the first feature vector are weighted according to the preset weights of the film and television entities in the film and television entity library, and through interactive calculation methods such as weighted summation, the entity vectors of the chat entities in the chat knowledge base and the entity vectors of the film and television entities in the film and television entity library are integrated to generate a second feature vector.
[0067] As another example, an entity in a single sentence hits both the chat knowledge base and the film and television entity library. For this entity, a weight K2 is preset in the chat knowledge base, and a weight K3 is preset in the film and television entity library. The sum of K1, K2, and K3 is a preset value, and the preset value is 1, for example. The calculation result of the weighted word segmentation is K1*T1+K2*T2+K3*T3, where T1 is the entity vector of the word segmentation at the corresponding position in the first feature vector, T2 is the entity vector of the word segmentation in the chat knowledge base, and T3 is the entity vector of the word segmentation in the film and television entity library.
[0068] Step 206: input the second feature vector into the entity recognition model to generate an entity recognition result.
[0069] In this embodiment, the second feature vector of each single sentence is input into CRF for prediction, a sequence labeling result of each single sentence is generated, and the entity recognition result is determined according to the sequence labeling result.
[0070] For example, the single sentence encoder uses the pre-trained language model bert to learn the semantics of the single sentence text, the context encoder uses Bi-LSTM, and the bidirectional temporal encoder helps the model encode the context dialogue information. The introduced external knowledge module contains chat features and film and television entity library information. After interactive calculation methods such as weighted summation, the model's ability to understand the entity boundary is enhanced. The CRF layer imposes certain constraints on the recognized entities with external knowledge information, and parses to obtain the entity information contained in the current single sentence query. In this example, the dialogue text includes multiple single sentences Q1, Q2...Qn. The input of the single sentence encoder is Qi, and the output is Ui, that is, Ui=BERT(Qi). The input of the context encoder is Ui, and the output is Ci, that is, U=[U1+U2+...+Un], C=BiLSTM(U), C=[C1+C2+...+Cn]. The input of the external knowledge injection encoder is Ci, kb_chat and kb_entity, and the output is Oi, where kb_chat and kb_enti ty represents the weight, that is, Oi=KB_Moudle[Ci, kb_chat, kb_entity]. Further, the encoded sentence information is input into the CRF layer for prediction, and the category with the highest score is selected as the prediction result of the model. For example, for Q1 "I want to watch a movie", the sequence labeling result is O, O, O, B-type (type), I-type, and for Q2 "I just want to see you", the sequence labeling result is O, B-title (title), I-title, I-title.
[0071] In the disclosed embodiments, a deep learning-based method encodes single sentences and contexts in a conversation scenario, learns entity and sentence information from conversation semantics, and performs interactive calculations based on small talk and popular film and television entities, thereby realizing a new heat calculation method and introducing multi-way arbitration joint judgment, accurately identifying and mining entity words in the film and television field in user conversations, and being able to improve the recognition ability of a film and television entity recognition model, having a certain recognition ability for new film and television entity words that appear in conversations, and enhancing the user's interactive experience.
[0072] The disclosed embodiment also provides a device for identifying entities in a conversation.
[0073] Figure 3 A schematic diagram of a device for identifying entities in a conversation provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the entity recognition device in the conversation includes: an acquisition module 31, an encoding module 32, a matching module 33, a generation module 34, and an identification module 35.
[0074] The acquisition module 31 is used to acquire a dialogue text, wherein the dialogue text includes at least one single sentence.
[0075] The encoding module 32 is used to encode each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence.
[0076] The matching module 33 is used to match each single sentence with the entity information in the domain knowledge base to determine the target entity contained in the single sentence from the domain knowledge base.
[0077] The generating module 34 is used to weight the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base, and generate a second feature vector corresponding to each single sentence.
[0078] The recognition module 35 is used to input the second feature vector into an entity recognition model to generate an entity recognition result.
[0079] In one embodiment of the present disclosure, the encoding module 32 is specifically used to: input each single sentence in the dialogue text into a single sentence encoder to generate an initial feature vector for each single sentence; concatenate the initial feature vectors of all the single sentences to obtain a target feature vector; input the target feature vector into a context encoder to generate a first feature vector corresponding to each single sentence.
[0080] In one embodiment of the present disclosure, the domain knowledge base includes a small talk knowledge base and a film and television entity base, and the matching module 33 is specifically used to: match each single sentence with the entity information in the small talk knowledge base and the entity information in the film and television entity base, respectively, to determine the small talk entity and the film and television entity contained in the single sentence;
[0081] The generating module 34 is specifically used for weighting the chat entities and the film and television entities in the first feature vector according to the weight corresponding to the chat entity and the weight corresponding to the film and television entity.
[0082] In one embodiment of the present disclosure, the device also includes: an update module, which is used to crawl candidate film and television entities from a preset film and television platform when a preset update condition is met; if the candidate film and television entity is not in the film and television entity library, the candidate film and television entity is added to the film and television entity library.
[0083] In one embodiment of the present disclosure, the recognition module 35 is specifically used to: input the second feature vector of each single sentence into the conditional random field for prediction, and generate a sequence labeling result for each single sentence; and determine the words belonging to the specified field from the single sentence according to the sequence labeling result as the entity recognition result.
[0084] The entity recognition device in the conversation provided by the embodiment of the present disclosure can execute any entity recognition method in the conversation provided by the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not described in detail in the embodiment of the device of the present disclosure, reference can be made to the description in any method embodiment of the present disclosure.
[0085] Figure 4 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the electronic device includes one or more processors 401 and a memory 402 .
[0086] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
[0087] The memory 402 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 401 may run the program instructions to implement the method of the embodiment of the present disclosure above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0088] In one example, the electronic device may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). In addition, the input device 403 may also include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including determined distance information, direction information, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.
[0089] Of course, to simplify, Figure 4 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.
[0090] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present disclosure.
[0091] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0092] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present disclosure.
[0093] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0094] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0095] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for entity recognition in a conversation, characterized in that: include: Acquire a dialogue text, wherein the dialogue text includes at least one single sentence; Encoding each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence; Matching each single sentence with entity information in a domain knowledge base to determine a target entity contained in the single sentence from the domain knowledge base; Weighting the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base to generate a second feature vector corresponding to each single sentence; The second feature vector is input into an entity recognition model to generate an entity recognition result.
2. The method according to claim 1, characterized in that The step of encoding each single sentence in the conversation text to generate a first feature vector corresponding to each single sentence includes: Input each single sentence in the conversation text into a single sentence encoder to generate an initial feature vector for each single sentence; Concatenate the initial feature vectors of all single sentences to obtain the target feature vector; The target feature vector is input into a context encoder to generate a first feature vector corresponding to each single sentence.
3. The method according to claim 1, characterized in that The target entity in the first feature vector is weighted according to the preset weight corresponding to the target entity in the domain knowledge base to generate a second feature vector corresponding to each single sentence, including: For each single sentence, using the position of the target entity in the single sentence, obtain an original target entity subvector corresponding to the target entity in the first feature vector; Perform weighted calculation according to the preset weight corresponding to the target entity, the entity vector of the target entity in the domain knowledge base, and the original target entity subvector to generate a weighted subvector; The weighted subvector is used to replace the original target entity subvector in the first feature vector to obtain the second feature vector.
4. The method according to claim 1, characterized in that The domain knowledge base is a small talk knowledge base and a film and television entity base, and the matching of each single sentence with entity information in the domain knowledge base to determine the target entity contained in the single sentence from the domain knowledge base includes: Matching each single sentence with the entity information in the chat knowledge base and the entity information in the film and television entity base respectively to determine the chat entity and the film and television entity contained in the single sentence; The step of weighting the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base includes: The chat entity and the film and television entity in the first feature vector are weighted according to the weight corresponding to the chat entity and the weight corresponding to the film and television entity.
5. The method according to claim 4, characterized in that Also includes: When the preset update conditions are met, candidate film and television entities are crawled from the preset film and television platform; If the candidate film and television entity is not in the film and television entity library, the candidate film and television entity is added to the film and television entity library.
6. The method according to claim 1, characterized in that The step of inputting the second feature vector into an entity recognition model to generate an entity recognition result includes: The second feature vector of each single sentence is input into the conditional random field for prediction to generate the sequence labeling result of each single sentence; According to the sequence labeling result, words belonging to a specified field are determined from the single sentence as the entity recognition result.
7. A device for identifying entities in a conversation, characterized in that: include: An acquisition module, used for acquiring a dialogue text, wherein the dialogue text includes at least one single sentence; An encoding module, used for encoding each single sentence in the dialogue text to generate a first feature vector corresponding to each single sentence; A matching module, used for matching each single sentence with entity information in a domain knowledge base, so as to determine a target entity contained in the single sentence from the domain knowledge base; A generating module, configured to weight the target entity in the first feature vector according to the preset weight corresponding to the target entity in the domain knowledge base, and generate a second feature vector corresponding to each single sentence; The recognition module is used to input the second feature vector into the entity recognition model to generate an entity recognition result.
8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the entity recognition method in the conversation described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for entity recognition in a conversation described in any one of claims 1 to 6 is implemented.
10. A vehicle, characterized in that: include: The device as claimed in claim 7, or the electronic device as claimed in claim 8.
