Entity extraction method, device, apparatus and computer readable storage medium

Abstract

The application provides an entity extraction method, device and equipment and a computer readable storage medium. The method comprises: obtaining at least one character vector and at least one extended word vector contained in a text to be extracted; the at least one extended word vector contains at least one preset entity vector; the at least one preset entity vector is the vector information of the entity corresponding to the text to be extracted in a preset entity dictionary; performing encoding and decoding transformation based on the at least one character vector and the at least one extended word vector to obtain at least one target entity corresponding to the text to be extracted; and the at least one target entity is used to realize natural language processing of the text to be extracted. Through the application, the efficiency of entity extraction can be improved on the basis of ensuring the accuracy of entity extraction.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to an entity extraction method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, with the rise of Transformer technology based on self-attention mechanisms, the training process can be parallelized. The Bidirectional Encoder Representations from Transformers (BERT) model has gradually replaced the previous Long Short-Term Memory (LSTM) network and is widely used in Natural Language Processing (NLP) tasks. The BERT model is pre-trained on a massive corpus. When extracting entities from text, it extracts context-specific word vectors from the text and then uses a 12-layer network to perform Transformer encoding and decoding on these word vectors, outputting the entities in the text. Due to the complexity of the BERT model's structure, both the training and application phases are lengthy, resulting in long training times and significant time consumption for online inference based on real user interactions, thus reducing the efficiency of entity extraction. Summary of the Invention

[0003] This application provides an entity extraction method, apparatus, device, and computer-readable storage medium, which can improve the efficiency of entity extraction while ensuring the accuracy of entity extraction.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an entity extraction method, including:

[0006] Obtain at least one character vector and at least one extended word vector contained in the text to be extracted; the at least one extended word vector contains at least one preset entity vector; the at least one preset entity vector is the vector information of the entity corresponding to the text to be extracted in the preset entity dictionary;

[0007] Based on the at least one character vector and the at least one extended word vector, an encoding and decoding transformation is performed to obtain at least one target entity corresponding to the text to be extracted; the at least one target entity is used to implement natural language processing of the text to be extracted.

[0008] This application provides an entity extraction device, including:

[0009] The acquisition module is used to acquire at least one character vector and at least one extended word vector contained in the text to be extracted; the at least one extended word vector contains at least one preset entity vector; the at least one preset entity vector is the vector information of the entity corresponding to the text to be extracted in a preset entity dictionary;

[0010] The encoding / decoding transformation module is used to perform encoding / decoding transformation based on the at least one character vector and the at least one extended word vector to obtain at least one target entity corresponding to the text to be extracted; the at least one target entity is used to implement natural language processing of the text to be extracted.

[0011] In the above-described apparatus, the at least one extended word vector further includes: at least one word segmentation vector; the acquisition module is further configured to obtain the at least one character vector based on the vector information of each individual character in the at least one character contained in the text to be extracted; perform word segmentation processing on the text to be extracted to obtain the at least one word segmentation vector; perform relevance matching on the text to be extracted in the preset entity dictionary to obtain at least one preset entity information matched by the text to be extracted; each preset entity information in the at least one preset entity information includes at least one preset entity and a preset entity alias; and use the vector corresponding to the at least one preset entity information as the at least one preset entity vector.

[0012] In the above-described apparatus, the encoding / decoding transformation module is further configured to perform encoding / decoding transformation on the at least one character vector and the at least one extended word vector to obtain the at least one first entity; and use the at least one first entity as the at least one target entity; or, in the at least one character vector and the at least one extended word vector, obtain the boundary information corresponding to each character vector and each extended word vector; combine the boundary information to perform encoding / decoding transformation on the at least one character vector and the at least one extended word vector to obtain the at least one second entity; and use the at least one second entity as the at least one target entity.

[0013] In the above device, the boundary information includes: first boundary information corresponding to each extended word vector and second boundary information corresponding to each character vector; the acquisition module is further configured to obtain the first boundary information based on the positions of the first and last characters in each extended word vector; and use the position of each character vector as the second boundary information.

[0014] In the above apparatus, the encoding / decoding module is further configured to perform attention encoding on the at least one character vector and the at least one extended word vector to obtain a first attention encoding vector set; perform decoding prediction on the first attention encoding vector set to obtain a first position prediction sequence; the first position prediction sequence is used to indicate the position of the character belonging to the at least one first entity in the at least one character vector; and obtain the at least one first entity based on the first position prediction sequence.

[0015] In the above apparatus, the encoding / decoding module is further configured to: identify the at least one character vector and the at least one extended word vector respectively to obtain at least one encoding identifier; obtain the length of the vector to be encoded corresponding to each encoding identifier in the at least one encoding identifier according to the first boundary information and the second boundary information; perform attention encoding on the character vector or extended word vector corresponding to each encoding identifier in combination with the length of the vector to be encoded to obtain a second attention encoding vector set; truncate the second attention encoding vector set according to the length of the request statement in the text to be extracted to obtain a truncated encoding vector; perform decoding prediction on the truncated encoding vector to obtain a second position prediction sequence; the second position prediction sequence is a sequence composed of at least one second position prediction label; the at least one second position prediction label is used to indicate the position of the character belonging to the at least one second entity in the at least one character vector; and connect the character vectors indicated by the at least one second position prediction label at each position to obtain the at least one second entity.

[0016] In the aforementioned apparatus, the entity extraction device further includes an attention encoding / decoding transformation model. This model is used to combine the boundary information to perform encoding / decoding transformations on the at least one character vector and the at least one extended word vector to obtain the at least one second entity. The attention encoding / decoding transformation model is obtained by performing data mining processing on real interaction logs to obtain a training sample set, and then using the training sample set to train the initial attention encoding / decoding transformation model. The data mining processing includes at least one of log data mining, annotation replacement processing, and auxiliary synonym replacement processing.

[0017] This application provides an electronic device, including:

[0018] Memory, used to store executable instructions;

[0019] A processor, when executing executable instructions stored in the memory, implements the method provided in the embodiments of this application.

[0020] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the method provided in this application.

[0021] The embodiments of this application have the following beneficial effects:

[0022] In this embodiment, by fusing at least one extended word vector with at least one character vector together for encoding and decoding transformation, more potential word vector information from the text to be extracted can be introduced during the encoding process. This potential word vector information is then used to increase the accuracy of entity extraction, thereby achieving the goal of extracting at least one target entity from the text with fewer network layers and fewer processing steps, while simultaneously ensuring the accuracy of entity extraction. The method in this embodiment reduces the workload and time spent on network training and computation, achieving a balance between accuracy and efficiency, and ensuring improved efficiency in entity extraction while maintaining accuracy. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the current training corpus organization method for CRF++ feature engineering;

[0024] Figure 2 This is a schematic diagram illustrating the current process of entity extraction using the BERT model;

[0025] Figure 3 This is an optional architecture diagram of the entity extraction-based voice interaction system 100 provided in the embodiments of this application;

[0026] Figure 4 This is an optional structural diagram of the server 200 provided in an embodiment of this application;

[0027] Figure 5 This is an optional flowchart illustrating the entity extraction method provided in the embodiments of this application;

[0028] Figure 6 This is an optional flowchart illustrating the entity extraction method provided in the embodiments of this application;

[0029] Figure 7 This is an optional flowchart illustrating the entity extraction method provided in the embodiments of this application;

[0030] Figure 8 This is an optional flowchart illustrating the entity extraction method provided in the embodiments of this application;

[0031] Figure 9 This is a schematic diagram of a processing procedure for the entity extraction method provided in an embodiment of this application;

[0032] Figure 10 This is an optional effect diagram of the configuration interface of the preset entity dictionary provided in the embodiments of this application;

[0033] Figure 11 This is a schematic diagram of the network structure and application processing flow of the encoding / decoding transformation model provided in the embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0038] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0039] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0040] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0041] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0042] 2) Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0043] 3) Request statement: A short text request (query) entered by the user in the smart assistant, usually containing only one intention or expectation of the user. For example: "Play a song 2 by singer 1"; "I want to watch movie 3", etc.

[0044] 4) Entities: Fundamental concepts in Natural Language Processing (NLP), typically basic vocabulary within a specific domain. In task-oriented dialogue systems, they represent key information within the user's input query. For example, in a query like "Play a song by singer 1 (song 2)," the query itself expresses the user's intention to listen to a song (music.play). Entities like "sys.music.singer=singer 1" and "sys.music.song=song 2" are also designed to represent specific key information within the query, allowing subsequent services to use structured information obtained through semantic understanding to respond to the user's query. Entities are allowed to have aliases. For instance, "Liu Moumou" is an entity in the music domain representing a singer's name; "Mou Mou Ge," "Andy Liu," and the pinyin string corresponding to "Liu Moumou" can all be aliases. An entity may belong to multiple domains; therefore, "Liu Moumou" could also be an entity in the film and television domain, representing an actor's name.

[0045] 5) Skills, similar to apps, refer to one or more specific functions and services that can be provided to users through voice interaction. Different skills offer different services. For example, skills like music, weather, jokes, and news can provide users with functions such as listening to music, checking the weather, and telling jokes. In order to perform voice interaction and understanding, skills need to build the necessary dialogue models. In order to provide specific services, skills need to acquire and configure relevant services.

[0046] 6) Media Asset Entities: Similar to the definitions above, entities like `sys.music.song` in the `music` skill, `sys.video.film`, `sys.video.tvseries`, and `sys.video.cartoon` in the `video` skill, and `sys.fm.album` in the `fm` skill can all represent media information entities. Media information entities share certain similarities, their content overlaps, and user queries are similar, thus they can be defined as media asset entities.

[0047] 7) Entity Dictionary: For domain design experts working on task-oriented dialogues, when designing a new skill intent, they typically provide a set of entity instances, informing the boundaries and rules of the entity set involved in the new skill. This is crucial predefined feature information for entity extraction. Entity instances with similar characteristics can form entity libraries, such as a singer library or an actor library. An entity dictionary can contain at least one entity library.

[0048] 8) BERT Algorithm: A pre-trained language model proposed by Google in 2018, based on Transformers technology and considering contextual information, it is frequently used in general NLP tasks. In entity extraction scenarios, the pre-trained results of BERT are often used as the feature extraction part to optimize the performance of Named Entity Recognition (NER) models.

[0049] 9) Conditional Random Field (CRF) algorithm: Proposed by John Lafferty in 2001, it is commonly used in NLP scenarios such as word segmentation and entity extraction.

[0050] 10) Corpus. To clearly express their intentions, users will use common question formats, such as "How's the weather today?", "What's the temperature in Shenzhen tomorrow?", and "What's the air quality index today?". These sentences all convey a certain intention. These sentences are called the corpus.

[0051] 11) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0052] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0053] Currently, one approach to entity extraction in natural language processing involves manually constructing CRF++ feature engineering and implementing the CRF algorithm in C++ to achieve entity extraction. The execution flow of the CRF++ method is as follows: organizing the training corpus into... Figure 1The example form shown in the figure, where the first column is the character feature, the second column is the bi-gram feature, the third column is the词性 feature, the fourth column is the entity information feature, and the fifth column is the l prediction label predicted by the CRF++ feature engineering based on the data in the first four columns, marked using the BIO label system, where B represents begin, I represents inter, and O represents other.

[0054] Currently, another implementation of entity extraction technology, that is, the implementation of the BERT model, can be as Figure 2 shown. Figure 2 In it, the bottom layer is the feature extraction layer 20 of BERT. Its input terminals tok1, tok2, tokn are respectively the word IDs of each word in the current request statement (query), and [cls] is a preset dedicated token used to mark the start of the sentence. Here, in the scenario of entity recognition, the feature extraction layer 20 of BERT usually contains 6 encoding layers and 6 decoding layers, which are used to process the word vectors of each token, that is, the vector information of each single word, layer by layer, and output the 768-dimensional information of the word vector of each token and the 768-dimensional information of the [cls] part. Then, through the intermediate layer 21, based on the 768-dimensional information output by the feature extraction layer 20, the custom dictionary information of each token with 40 dimensions is concatenated (corresponding to Figure 1 the 4th column feature. For example, if the three characters "Wangqing Shui" exist in the entity dictionary of the song type, then for the three characters "Wang", "qing", and "Shui" respectively, there are features of B-dictionary & I-dictionary, and correspondingly, they can be transformed into 3 vectors with 40 dimensions), thus completing the process of concatenating the BERT output and the custom dictionary features, and inputting each concatenated token into the CRF decoding layer. Through the CRF decoding layer, considering both the vector information of each token and the transition matrix information of each prediction label (label), probability annotation is given to each token to predict the probability that each token is which character in the entity to be finally extracted. Furthermore, the entity to be finally extracted is obtained based on the probability annotation of each token.

[0055] It can be seen that the CRF++ feature engineering method in the current mainstream technical solutions is characterized by its inability to automatically construct feature engineering or its limited ability to automatically capture features. Users need to manually construct feature engineering based on the results of multiple trials to determine which features to use. Feature engineering is very time-consuming and requires a certain level of expertise from the model tuner during model development and tuning. Therefore, the efficiency of entity extraction using CRF++ feature engineering is relatively low. While introducing the BERT model into the CRF algorithm can simplify the workload of feature engineering, BERT processes individual characters and mainly examines the relationships between tokens, thus losing the relationships between word orders. Therefore, accuracy can only be ensured by increasing the number of layers. However, too many layers lead to low training and prediction efficiency, making the entire BERT model network computationally time-consuming and resulting in low entity extraction efficiency. In summary, current entity extraction methods are relatively inefficient and not suitable for widespread deployment in real-world high-concurrency, high-traffic scenarios.

[0056] This application provides an entity extraction method, apparatus, device, and computer-readable storage medium, which can improve the efficiency of entity extraction while ensuring accuracy. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart home devices (e.g., smart TVs, smart speakers, smart light bulbs, smart air conditioners, etc.), smart wearables, in-vehicle devices, robots, and various other types of user terminals, or as servers. The following describes exemplary applications when the device is implemented as a server.

[0057] See Figure 3 , Figure 3 This is an optional architecture diagram of the entity extraction-based voice interaction system 100 provided in the embodiments of this application. In order to support a voice interaction application, a terminal, such as a smart speaker 400, connects to the server 200 through a network 300. The network 300 can be a wide area network or a local area network, or a combination of both.

[0058] In an application scenario where a smart speaker is used to instruct a user to play a song, the user can initiate a voice command or request via the smart speaker 400 to "play a song by singer 1". Upon receiving the request, the smart speaker 400 forwards it to the server 200. The server 200 converts the request from speech to text to obtain the text to be extracted. It then acquires at least one character vector and at least one extended word vector contained within the text. The at least one extended word vector contains at least one preset entity vector. The at least one preset entity vector represents the vector information of the entity corresponding to the text in the preset entity dictionary. Encoding and decoding transformations are performed based on the at least one character vector and the at least one extended word vector to obtain at least one target entity corresponding to the text to be extracted. In some embodiments, the at least one target entity may contain a skill type entity and an intent type entity. The server 200 can identify the intent to play a song as a music skill based on the skill type entity and intent type entity in the at least one target entity, and extract the singer information as "singer 1" from the singer type entity in the at least one target entity, thereby achieving parsing of the request statement based on at least one target entity. Based on at least one identified target entity, server 200 retrieves the corresponding target song audio and target song information from music database 500. The target song information may include song images, album art, lyrics, and playback URL. Server 200 can reorganize the target song information into a streaming media card that can be played on a smart speaker with a screen, or into a natural human-computer interaction response suitable for playback on a screenless smart speaker. Finally, the song audio and song information are fed back to the smart speaker 400 and provided to the user.

[0059] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0060] See Figure 4 , Figure 4 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 4The server 200 shown includes at least one processor 410, memory 450, at least one network interface 420, and a user interface 430. The various components in server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 440.

[0061] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0063] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0064] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0065] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0066] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0067] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0068] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.).

[0069] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0070] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 4 An entity retrieval device 455 stored in memory 450 is shown. It may be software in the form of programs and plug-ins, including the following software modules: acquisition module 4551 and encoding module 4552. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0071] The functions of each module will be explained below.

[0072] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the entity extraction method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0073] The entity extraction method provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application.

[0074] See Figure 5 , Figure 5This is an optional flowchart illustrating the entity extraction method provided in the embodiments of this application, which will be combined with... Figure 5 The steps shown are explained.

[0075] S101. Obtain at least one character vector and at least one extended word vector contained in the text to be extracted; at least one extended word vector contains at least one preset entity vector; at least one preset entity vector is the vector information of the entity corresponding to the text to be extracted in the preset entity dictionary.

[0076] The entity extraction method provided in this application can be applied to artificial intelligence-based natural language processing scenarios such as voice interaction, speech synthesis, IoT interaction, system feedback, voice wake-up, and far-field speech recognition. The specific method can be selected according to the actual situation, and this application does not limit it.

[0077] In this embodiment, the text to be extracted can be text information input by the user into the electronic device, or text obtained by the speech-to-text conversion system through text recognition of the user's speech. The text to be extracted contains at least one character.

[0078] In this embodiment, the entity extraction device can perform word segmentation on the text to be extracted, and use the text to be extracted to perform relevance matching in a preset entity dictionary. The characters or words obtained from word segmentation, along with the vector information corresponding to at least one preset entity matched by the text to be extracted in the preset entity dictionary, such as semantic embedding information, are used as at least one character vector and at least one extended word vector. The at least one extended word vector contains at least one preset entity vector.

[0079] In this embodiment, the preset entity dictionary contains at least one preset entity. In some embodiments, the at least one preset entity may be a string of keywords, indicator words, directional words, positional words (such as the last character), and head words designed by linguistic experts for various natural language processing scenarios based on statistical information in specific scenarios. For example, the preset entity dictionary may contain common preset entities that can be directly used by various skills, such as frequently used quantifiers, dates, times, and regions; it may also contain preset entity vectors corresponding to specific skills, such as "phone number database," "room name," and "friend list." At least one preset entity vector represents the vector information of the entity corresponding to the text to be extracted in the preset entity dictionary.

[0080] S102. Based on at least one character vector and at least one extended word vector, perform encoding and decoding transformation to obtain at least one target entity corresponding to the text to be extracted; the at least one target entity is used to implement natural language processing of the text to be extracted.

[0081] In this embodiment of the application, the electronic device can perform encoding and decoding transformations on the text content based on at least one character vector and at least one extended word vector to obtain at least one target entity corresponding to the text to be extracted.

[0082] In some embodiments, by using at least one extended word vector, potential word groups and potential preset entities contained in the text to be extracted can be introduced during the encoding and decoding process. In this way, when the entity extraction device performs encoding and decoding transformations on at least one character vector and at least one extended word vector, such as transformer encoding and decoding transformations based on a multi-head self-attention mechanism, the encoded input information can be richer and more comprehensive. This allows the encoding and decoding process to more accurately identify the relationships between each character based on the potential segmentable word groups in the input text to be extracted and the corresponding preset entity vectors in the preset entity dictionary. This results in more accurate extraction of at least one target entity after encoding and decoding transformation, without requiring multi-layer network stacking to ensure accuracy in entity extraction.

[0083] In some embodiments, to further improve accuracy and avoid the resource consumption of multi-layer encoding / decoding processing, the entity extraction device can also obtain the boundary information of each character vector and each extended word vector from at least one character vector and at least one extended word vector. When performing encoding / decoding transformation on at least one character vector and at least one extended word vector, the lengths of the corresponding character vector and extended word vector can be obtained based on the boundary information corresponding to each character vector and each extended word vector. This allows for improved accuracy of the encoding / decoding transformation through richer input information, achieving accurate entity extraction with fewer processing steps, and obtaining at least one target entity.

[0084] In some embodiments, the boundary information may be the position of each character vector, and the boundary information of the vector indicated by the positions of the first and last characters in each extended word vector.

[0085] In this embodiment of the application, at least one target entity may include entities of different skill types. The electronic device or the downstream device for entity extraction can realize the natural language processing of the text to be extracted through at least one target entity.

[0086] In some embodiments, for certain demand scenarios with clear tasks, the demand dialogue carries a clear intent, expecting to return specific services, such as "I want XXX" or "Give me XXX". At least one target entity can contain an entity with task-type skills, and then the specific demand task can be realized based on the entity with task-type skills, such as reminders, alarm clocks, ride-hailing, booking air tickets, booking hotels, music playback, media control, etc.

[0087] In some embodiments, for question-and-answer scenarios—where there is no obvious intention in the dialogue and the user mainly wants to ask questions and expects answers—such as world record knowledge questions like "What is the highest mountain in the world?" or self-built personalized questions like "Where is your hometown?" or "What is your mother's name?", at least one target entity can contain an entity representing a task-oriented skill. Natural language processing methods can utilize entities representing task-oriented skills to achieve voice interaction in question-and-answer scenarios.

[0088] In some embodiments, for content such as audiobook scenarios, story listening scenarios, or news broadcast scenarios, at least one target entity may include an entity with content briefing skills specifically tailored for the content.

[0089] In some embodiments, for smart home scenarios, home devices and device modes can be controlled via voice dialogue. For example, for a smart light bulb, one could say "turn on the living room lights," "turn up the brightness," or "turn on sleep mode." At least one target entity can include an entity with smart home-related skills.

[0090] It is understood that in this embodiment, by fusing at least one extended word vector with at least one character vector for encoding and decoding transformation, more potential word vector information from the text to be extracted can be introduced during the encoding process. This potential word vector information is then used to increase the accuracy of entity extraction, thereby achieving the goal of extracting at least one target entity from the text with fewer network layers and processing steps, while simultaneously ensuring the accuracy of entity extraction. The method in this embodiment reduces the workload and time spent on network training and computation, achieving a balance between accuracy and efficiency. It ensures improved efficiency in entity extraction while maintaining accuracy, and is more practical in real-world high-traffic application scenarios.

[0091] In some embodiments, see Figure 6 , Figure 6 This is an optional flowchart illustrating the entity extraction method provided in this application embodiment. At least one extended word vector further includes: at least one word segmentation vector. Figure 5 The shown S101 can be implemented through S1011-S1014, which will be explained in conjunction with each step.

[0092] S1011. Among the at least one character contained in the text to be extracted, obtain at least one character vector based on the vector information of each individual character.

[0093] In some embodiments, the entity extraction device may divide the text to be extracted into individual characters and obtain a character vector based on each character, thereby obtaining at least one character vector.

[0094] Exemplarily, the character extraction device segments "I love China" into four single character vectors (unigrams) of "I", "love", "China", and "country", as at least one character vector.

[0095] S1012. Perform word segmentation on the text to be extracted to obtain at least one word segmentation vector.

[0096] In the embodiments of the present application, the entity extraction device can also perform word segmentation on the text to be extracted. For example, in binary word segmentation, each two characters from the beginning to the end of the sentence in the text to be extracted form a two-character word (bigram) to obtain at least one word segmentation vector; or, each three characters from the beginning to the end of the sentence in the text to be extracted form a word (trigram) to obtain at least one word segmentation vector. The specific segmentation method and segmentation level of word segmentation in the embodiments of the present application are not limited.

[0097] S1013. Use the text to be extracted to perform relevance matching in a preset entity dictionary to obtain at least one preset entity information matched by the text to be extracted; each preset entity information in the at least one preset entity information includes at least one of a preset entity and a preset entity alias.

[0098] In the embodiments of the present application, the entity extraction device can perform relevance matching on the at least one preset entity included in the preset entity dictionary using the text to be processed to determine whether the text to be processed contains potential preset entities, and further obtain at least one preset entity information matched by the text to be extracted. Among them, each preset entity information in the at least one preset entity information includes at least one of a preset entity and a preset entity alias.

[0099] In some embodiments, the method of relevance matching can be a suffix dictionary matching method or a forward maximum matching method, etc., and specific selection is made according to the actual situation, which is not limited in the embodiments of the present application.

[0100] S1014. Use the vectors corresponding to the at least one preset entity information as at least one preset entity vector.

[0101] In the embodiments of the present application, the entity extraction device uses the vector information corresponding to at least one preset entity information that matches the text to be extracted in the preset entity dictionary as at least one preset entity vector.

[0102] It is understood that, in the embodiments of this application, by matching the correlation between word segmentation processing and the preset entity dictionary, the entity extraction device can obtain and make full use of the potential word segments and potential preset entities in the text to be extracted, thereby making the input information of the encoding and decoding process richer and more comprehensive. While ensuring the accuracy of entity extraction, it simplifies the current multi-layer processing flow, reduces the computation time, and improves the efficiency of entity extraction.

[0103] In some embodiments, see Figure 7 , Figure 7 This is an optional flowchart illustrating the entity extraction method provided in an embodiment of this application. Figure 5 or Figure 6 The illustrated S102 can be implemented by executing any one of the methods in S201-S202 or S301-S303, which will be explained in conjunction with each step.

[0104] S201. Encode and decode at least one character vector and at least one extended word vector to obtain at least one first entity.

[0105] In this embodiment of the application, the entity extraction device can perform encoding and decoding transformations on at least one character vector and at least one extended word vector to obtain at least one first entity.

[0106] In some embodiments, S201 can be implemented by executing the process of S2011-S2013, which will be described in conjunction with each step.

[0107] S2011. Attention encoding is performed on at least one character vector and at least one extended word vector to obtain a first attention encoding vector set.

[0108] In this embodiment, the entity extraction device can organize at least one character vector and at least one extended word vector into a feature vector matrix in matrix form, and perform linear transformation on the feature vector matrix to generate three matrices: a request matrix Q (query), a key matrix K (key), and a value matrix V (value). Then, based on the request matrix Q and the key matrix K, an attention matrix is ​​obtained. Here, the attention matrix represents the probability distribution of attention weights. For example, each row of the attention matrix refers to the relevance probability of the character vector or extended word vector corresponding to the row number to other character vectors or extended word vectors. The entity extraction device can then use the attention weights contained in the attention matrix to weight the value matrix V, and then perform softmax normalization on the weighted result, so that the sum of the attention weights of each character vector or extended word vector to other character vectors or extended word vectors is 1, thus obtaining a first attention encoding vector set.

[0109] Here, each row of the value matrix V represents the mathematical expression of each character vector or each extended word vector. By using attention weights to perform a weighted linear combination of these mathematical expressions, each character vector or each extended word vector can contain information about all character vectors and extended word vectors in the current sentence of the text to be extracted.

[0110] In some embodiments, before using the attention matrix to weight the value matrix V, the entity extraction device may also perform standard normal distribution processing on the attention matrix to make the result after softmax normalization more stable.

[0111] S2012. Decode and predict the first attention-encoded vector set to obtain a first position prediction sequence; the first position prediction sequence is used to indicate the position of a character belonging to at least one first entity in at least one character vector.

[0112] In this embodiment of the application, the entity extraction device can decode and predict the attention-encoded vector set, predict the probability that each character vector belongs to a character at a certain position in at least one first entity in the at least one character vector contained in the text to be extracted, and obtain the first position prediction sequence according to the probability corresponding to each character vector.

[0113] In this embodiment, the first position prediction sequence is a sequence composed of first position prediction labels, used to indicate the position of a character belonging to at least one first entity in at least one character vector. In some embodiments, when using the BIO labeling system for annotation, the first position prediction sequence may include at least one start label (B-label) for marking the start character of each of the at least one first entity; the first position prediction sequence may also include at least one end label (E-label) for marking the end character of each of the at least one first entity; or, the first position prediction sequence may also include an intermediate label (I-label) for marking the intermediate character of each of the at least one first entity.

[0114] In some embodiments, the entity extraction device can decode and predict the first attention-encoded vector set using the CRF decoding method to obtain the first position prediction sequence. Other decoding methods can also be used, and the specific method selected depends on the actual situation. This application embodiment does not limit the specific method.

[0115] S2013. Based on the first position prediction sequence, at least one first entity is obtained.

[0116] S202, Use at least one first entity as at least one target entity.

[0117] In this embodiment of the application, the entity extraction device can combine character vectors according to the character vectors marked by each first position prediction label in the first position prediction sequence and the position order to obtain at least one first entity, and use at least one first entity as at least one target entity.

[0118] It is understood that, in the embodiments of this application, after introducing at least one extended word vector, the entity extraction device can extract at least one target entity from at least one character vector with fewer processing steps, without the need for multiple layers of encoding and decoding conversion, thereby improving the accuracy of entity extraction.

[0119] S301. In at least one character vector and at least one extended word vector, obtain the boundary information corresponding to each character vector and each extended word vector.

[0120] In this embodiment of the application, the entity extraction device can obtain the boundary information corresponding to each character vector and each extended word vector. For a single character vector, the boundary information can be the position where the character vector appears. For an extended word vector containing multiple characters, the boundary information can be the boundary of the extended word vector defined by the positions of the first character (i.e., the first character) and the last character (i.e., the last character).

[0121] In some embodiments, the entity extraction device can obtain first boundary information based on the positions of the first and last characters in each extended word vector; and use the position of each character vector as second boundary information.

[0122] S302. Combining boundary information, perform encoding and decoding transformations on at least one character vector and at least one extended word vector to obtain at least one second entity.

[0123] In this embodiment of the application, the entity extraction device can combine boundary information to further enrich the input information of the encoding and decoding transformation, and perform encoding and decoding transformation on at least one character vector and at least one extended word vector to obtain at least one second entity.

[0124] In some embodiments, see Figure 8 , Figure 8 This is an optional flowchart illustrating the entity extraction method provided in an embodiment of this application. Figure 7 The shown S302 can be achieved by executing the process of S3021-S3026, which will be explained in conjunction with each step.

[0125] S3021. Identify at least one character vector and at least one extended word vector respectively to obtain at least one encoded identifier.

[0126] In this embodiment of the application, the entity extraction device can identify each extended word vector and each character vector by means of ID or token, thereby obtaining at least one character vector and at least one encoded identifier corresponding to at least one extended word vector, and performing the next encoding and decoding transformation process according to at least one encoded identifier.

[0127] S3022. Based on the first boundary information and the second boundary information, obtain the length of the vector to be encoded corresponding to each encoding identifier in at least one encoding identifier.

[0128] In this embodiment of the application, in order to utilize the vector length corresponding to each ID or token during the encoding and decoding transformation process, the entity extraction device can obtain the length of the vector to be encoded corresponding to each encoded identifier based on the first boundary information and the second boundary information.

[0129] For example, when the encoded identifier corresponds to the first boundary information, that is, when the beginning and end positions in the boundary information are the same, the length of the vector to be encoded corresponding to the encoded identifier is 1. When the encoded identifier corresponds to the second boundary information, the entity extraction device can obtain the length of the vector to be encoded corresponding to the encoded identifier based on the total number of characters defined by the beginning and end characters. For example, when the second boundary information is [3,5], the length of the vector to be encoded is 3.

[0130] S3023. Based on the length of the vector to be encoded, perform attention encoding on the character vector or extended word vector corresponding to each encoding identifier to obtain the second attention encoding vector set.

[0131] In this embodiment of the application, during the encoding process, the entity extraction device can combine the length of the vector to be encoded to fully identify and reflect the relationship between word orders during the encoding process, and then perform attention encoding on the character vector or extended word vector corresponding to each encoding identifier to obtain a second attention encoding vector set.

[0132] In some embodiments, the entity extraction device can take the character vector or extended word vector corresponding to each encoded identifier and its corresponding first boundary information or second boundary information as a series of data to form an input vector matrix, and perform attention encoding on the input vector matrix to obtain a second attention encoding vector set.

[0133] S3024. Based on the length of the request statement in the text to be extracted, the second attention encoding vector set is truncated to obtain the truncated encoding vector.

[0134] In this embodiment of the application, in the voice command interaction scenario, the text to be extracted usually contains a request statement. The entity extraction device can truncate the second attention encoding vector set according to the length of the request statement to obtain the truncated encoding vector.

[0135] In this embodiment of the application, when the text to be extracted contains multiple request statements, the entity extraction device can process the multiple request statements multiple times. In each processing, the second attention encoding vector set is truncated according to the length of the current request statement to obtain the truncated encoding vector.

[0136] S3025. Decode and predict the truncated encoded vector to obtain a second position prediction sequence; the second position prediction sequence is a sequence composed of at least one second position prediction label; at least one second position prediction label is used to indicate the position of a character belonging to at least one second entity in at least one character vector.

[0137] In this embodiment of the application, the entity extraction device decodes and predicts the truncated encoding vector to obtain the probability that each character belongs to each position in the second entity, and performs prediction labeling based on the predicted probability of each character to obtain the second position prediction sequence.

[0138] Here, the second position prediction sequence is a sequence consisting of at least one second position prediction label, used to indicate the position of a character belonging to at least one second entity in at least one character vector.

[0139] S3026. Connect the character vectors indicated by at least one second position prediction label at each position.

[0140] In this embodiment of the application, since at least one second position prediction label has indicated the position of the character belonging to at least one second entity in at least one character vector, the entity extraction device can connect the character vectors indicated by at least one second position prediction label at each position to obtain at least one second entity.

[0141] In some embodiments, see Figure 9 , Figure 9It is a schematic diagram of a processing procedure of the entity extraction method provided by an embodiment of the present application. For "Chongqing Renhe Pharmacy", the current BERT model only uses the information of six single characters, namely, "重", "庆", "人", "和", "药", and "店", during the encoding and decoding transformation. However, the entity extraction device in the embodiment of the present application can introduce at least one extended word vector, introducing more phrases that "Chongqing Renhe Pharmacy" may appear, such as "重庆", "人和药店", "药店", etc., as well as the English aliases of each character vector and the extended word vector. Further, the embodiment of the present application also introduces the position information corresponding to each character vector and the extended word vector. Exemplarily, for the request statement "Chongqing Renhe Pharmacy", if the position of the character vector "人" is 3, then the first boundary information corresponding to the character vector "人" is [3, 3]; in the extended word vector "重庆", the position of the first character "重" is 1, and the position of the last character "庆" is 2, then the second boundary information corresponding to the extended word vector "重庆" is [1, 2]. The entity extraction device performs encoding and decoding transformation on each character vector and its corresponding first boundary information and each extended word vector and its corresponding second boundary information, and obtains a second position prediction sequence as shown in Figure 9 The B-LOC, I-LOC, and E-LOC are the second position prediction labels. Among them, B-LOC is used to indicate the starting character of a second entity, and E-LOC after B-LOC is used to indicate the ending character of the second entity, and I-LOC is used to indicate the intermediate characters in the second entity. According to Figure 9 the second position prediction sequence shown, the entity extraction device can extract two second entities, namely, "重庆" and "人和药店".

[0142] S303. Use at least one second entity as at least one target entity.

[0143] In the embodiment of the present application, the entity extraction device can use the at least one second entity obtained by introducing the boundary information as at least one target entity.

[0144] It can be understood that in the embodiment of the present application, by introducing the boundary information, the self-attention mechanism in the encoding and decoding process can better capture the relationship between word orders, so that on the basis of ensuring the accuracy of the encoding and decoding process, the multi-layer processing process can be reduced, and the efficiency of entity extraction can be improved.

[0145] In some embodiments, the execution process of S302 can be implemented by using an attention encoding and decoding transformation model, where the attention encoding and decoding transformation model can be obtained by training an initial attention encoding and decoding transformation model by using a training sample set mined from real interaction logs.

[0146] In some embodiments, for scenarios where deep learning requires a large number of training samples, the data mining processing method may include at least one of log data mining, annotation replacement processing, and auxiliary synonym replacement processing. Log data mining can extract representative interaction data as training samples based on the analysis and statistical processing of real interaction logs; annotation replacement processing can replace entity annotations in the training samples to obtain more training samples; auxiliary synonym replacement processing can use auxiliary synonym replacement on the corpus portion outside of entity annotations, such as replacing "TV series" with synonyms like "drama series" or "serial drama," to obtain more training samples. The entity extraction device can obtain a training sample set containing a large amount of real data through at least one of the above data mining processing methods to further optimize the performance of the encoding / decoding transformation model trained on the training sample set. Furthermore, obtaining the training sample set through automatic collection and data mining can further reduce the workload of sample collection and improve the efficiency of training the encoding / decoding transformation model.

[0147] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0148] In some embodiments, when the designer of the preset entity dictionary, such as a skills expert, designs the skills intent for real-world application scenarios, it can do so through methods such as... Figure 10 The configuration interface for the preset entity dictionary shown allows you to create and update the preset entity dictionary. Figure 10 The diagram illustrates an interface example for adding an entity to the animation entity type `sys.video.cartoon` within the `video` domain of the preset entity dictionary. Skill experts can import or manually add preset entity sets related to their skill intents using control 12 on configuration interface 10, or manually add them using control 11, according to their actual needs. They can also set the characters contained in the preset entities using control 13 and assign corresponding aliases to the preset entities using control 14, thus satisfying the diversity of entity representations.

[0149] In some embodiments, the network structure and application processing flow of the encoding / decoding transformation model provided in this application can be as follows: Figure 11 As shown. In some embodiments, a network model implemented using the PyTorch framework, comprising a transformer encoding layer 21 and a CRF decoding layer 22, can be used as the initial encoding-decoding transformation model. The entity extraction device can use collected entity corpora as training corpora and real user log data as test corpora. Then, the training and test corpora are labeled to obtain training and test sample sets for training the initial encoding-decoding transformation model, respectively. For example, the distribution of the number of training and test corpora can be as shown in Table 1, as follows:

[0150] Table 1

[0151]

[0152] The entity extraction device performs offline training on the initial encoding / decoding transformation model using training and testing sample sets. During each training iteration, for each training sample in the training sample set, it acquires at least one training character vector and at least one training extended word vector. The at least one extended word vector includes at least one training word segmentation vector obtained by segmenting the training sample, and at least one preset entity vector obtained by performing relevance matching of the training text in the aforementioned preset entity dictionary. The entity extraction device further acquires the boundary information corresponding to each training character vector and each training extended word vector in the at least one training character vector and at least one training extended word vector. This information, along with the corresponding training character vector or training extended word vector, is input into the encoding layer 21 for attention encoding. The decoding layer 22 then decodes and predicts the attention encoding results to obtain the training prediction position sequence. The entity extraction device can obtain at least one predicted entity based on the training predicted position sequence, compare the at least one predicted entity with the entity annotation of each training sample, obtain the training loss based on the comparison result, and then adjust the network parameters of the initial codec transformation model based on the training loss. In this way, iterative training is performed using the training sample set until the preset training cutoff condition is reached. The entity extraction result of the candidate codec transformation model obtained in the last training is verified using the test sample set. When all preset verification indicators are met, the training ends and the codec transformation model 20 is obtained.

[0153] In some embodiments, the entity extraction device can perform the above-described network training process offline. After obtaining the codec transformation model, the entity extraction device can convert the codec transformation model into a script mode suitable for online deployment, and then deploy and load the script mode codec transformation model into a natural language processing system in a real-world scenario to parse and process the text to be extracted in the real-world scenario, thereby realizing the entity extraction method in this embodiment.

[0154] Here, the entity extraction method provided in this application embodiment is compared with various indicators of entity extraction performance of Base1 models such as CRF++ feature engineering and Base2 models such as BERT models, as shown in Table 2 below:

[0155] Table 2

[0156]

[0157] One epoch is the process of training all training samples once. P-value represents the accuracy of entity extraction, R-value represents the recall of entity extraction, and F-value is the harmonic mean of the combined P and R values. In some embodiments, the F-value can be calculated using a formula... To obtain.

[0158] It is understandable that, compared to CRF++, the method in this embodiment significantly improves the overall F-score and has less online parsing time, thus improving the efficiency of entity extraction and facilitating large-scale deployment in practical engineering. Compared to BERT, the online parsing time is greatly reduced, achieving a balance between performance and time consumption. It effectively controls online parsing time while ensuring performance, making it practically significant in engineering implementation. Furthermore, the offline training time of this method is also relatively short. The entity extraction method in this embodiment can be fully deployed for high-volume interactive data in real-world scenarios, making it more practically valuable in engineering applications.

[0159] The following description continues to illustrate the exemplary structure of the entity extraction device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software module stored in the entity extraction device 455 in the memory 450 may include:

[0160] The acquisition module 4551 is used to acquire at least one character vector and at least one extended word vector contained in the text to be extracted; the at least one extended word vector contains at least one preset entity vector; the at least one preset entity vector is the vector information of the entity corresponding to the text to be extracted in the preset entity dictionary;

[0161] The encoding / decoding transformation module 4552 is used to perform encoding / decoding transformation based on the at least one character vector and the at least one extended word vector to obtain at least one target entity corresponding to the text to be extracted; the at least one target entity is used to implement natural language processing of the text to be extracted.

[0162] In some embodiments, the at least one extended word vector further includes: at least one word segmentation vector; the acquisition module 4551 is further configured to obtain the at least one character vector based on the vector information of each individual character in the at least one character contained in the text to be extracted; perform word segmentation processing on the text to be extracted to obtain the at least one word segmentation vector; perform relevance matching on the text to be extracted in the preset entity dictionary to obtain at least one preset entity information matched by the text to be extracted; each preset entity information in the at least one preset entity information includes at least one preset entity and a preset entity alias; and use the vector corresponding to the at least one preset entity information as the at least one preset entity vector.

[0163] In some embodiments, the encoding / decoding transformation module 4552 is further configured to perform encoding / decoding transformation on the at least one character vector and the at least one extended word vector to obtain the at least one first entity; and use the at least one first entity as the at least one target entity; or, in the at least one character vector and the at least one extended word vector, obtain the boundary information corresponding to each character vector and each extended word vector; combine the boundary information to perform encoding / decoding transformation on the at least one character vector and the at least one extended word vector to obtain the at least one second entity; and use the at least one second entity as the at least one target entity.

[0164] In some embodiments, the boundary information includes: first boundary information corresponding to each extended word vector and second boundary information corresponding to each character vector; the acquisition module 4551 is further configured to obtain the first boundary information based on the positions of the first and last characters in each extended word vector; and use the position of each character vector as the second boundary information.

[0165] In some embodiments, the encoding / decoding module 4552 is further configured to perform attention encoding on the at least one character vector and the at least one extended word vector to obtain a first attention encoding vector set; perform decoding prediction on the first attention encoding vector set to obtain a first position prediction sequence; the first position prediction sequence is used to indicate the position of the character belonging to the at least one first entity in the at least one character vector; and obtain the at least one first entity based on the first position prediction sequence.

[0166] In some embodiments, the encoding / decoding module 4552 is further configured to: identify the at least one character vector and the at least one extended word vector respectively to obtain at least one encoding identifier; obtain the length of the vector to be encoded corresponding to each encoding identifier in the at least one encoding identifier according to the first boundary information and the second boundary information; perform attention encoding on the character vector or extended word vector corresponding to each encoding identifier in combination with the length of the vector to be encoded to obtain a second attention encoding vector set; truncate the second attention encoding vector set according to the length of the request statement in the text to be extracted to obtain a truncated encoding vector; perform decoding prediction on the truncated encoding vector to obtain a second position prediction sequence; the second position prediction sequence is a sequence composed of at least one second position prediction label; the at least one second position prediction label is used to indicate the position of the character belonging to the at least one second entity in the at least one character vector; and connect the character vectors indicated by the at least one second position prediction label at each position to obtain the at least one second entity.

[0167] In some embodiments, the entity extraction device further includes an attention encoding / decoding transformation model, which is used to combine the boundary information to perform encoding / decoding transformation on the at least one character vector and the at least one extended word vector to obtain the at least one second entity; the attention encoding / decoding transformation model is obtained by performing data mining processing on real interaction logs to obtain a training sample set, and using the training sample set to train the initial attention encoding / decoding transformation model.

[0168] The data mining process includes at least one of log data mining, annotation replacement processing, and auxiliary synonym replacement processing.

[0169] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0170] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the entity extraction method described above in this application.

[0171] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 5-8 The method shown.

[0172] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0173] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0174] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0175] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0176] In summary, in this embodiment, by fusing at least one extended word vector with at least one character vector for encoding and decoding transformation, more potential word vector information from the text to be extracted can be introduced during the encoding process. This potential word vector information is then used to increase the accuracy of entity extraction, thereby achieving the goal of extracting at least one target entity from the text with fewer network layers and processing steps, while simultaneously ensuring accuracy. The method in this embodiment reduces the workload and time spent on network training and computation, achieving a balance between accuracy and efficiency. It ensures improved efficiency while maintaining accuracy in entity extraction. Compared to CRF++, the method in this embodiment shows a significant improvement in overall F-score and less online parsing time, thus improving entity extraction efficiency and facilitating large-scale deployment in practical engineering. Compared to BERT, the online parsing time is significantly reduced, achieving a balance between performance and time consumption. It effectively controls online parsing time while maintaining performance, making it practically significant in engineering implementation. Furthermore, the offline training time of this method is also relatively short. The entity extraction method in this application embodiment can be fully applied to high-volume interactive data in real-world scenarios, and has greater practical engineering value.

[0177] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An entity extraction method, characterized in that, include: Obtain at least one character vector and at least one extended word vector contained in the text to be extracted; The at least one character vector is obtained by taking the vector information of each individual character from the at least one character contained in the text to be extracted; the at least one extended word vector includes at least one preset entity vector and at least one word segmentation vector; the at least one preset entity vector is a vector of preset entity information corresponding to the text to be extracted in a preset entity dictionary; each preset entity information includes at least one preset entity and a preset entity alias; The at least one word segmentation vector is obtained by segmenting the text to be extracted; The text to be extracted is used to perform relevance matching in the preset entity dictionary to obtain at least one preset entity information that matches the text to be extracted; wherein, the relevance matching includes suffix number dictionary matching and positive maximum matching; The vector corresponding to the at least one preset entity information is used as the at least one preset entity vector; In the at least one character vector and the at least one extended word vector, the positions of the first and last characters in each extended word vector are used as first boundary information, and the position of each character vector is used as second boundary information; Based on the first boundary information and the second boundary information, the length of the vector to be encoded corresponding to each encoding identifier in at least one encoding identifier is determined; the encoding identifier is obtained by identifying the at least one character vector and the at least one extended word vector. Based on the length of the vector to be encoded, attention encoding is performed on the character vector or the extended word vector corresponding to each encoding identifier to obtain a second attention encoding vector set; Based on the length of the request statement in the text to be extracted, the second attention encoding vector set is truncated to obtain the truncated encoding vector; The truncated encoded vector is decoded and predicted to obtain a second position prediction sequence; the second position prediction sequence is a sequence composed of at least one second position prediction label; the at least one second position prediction label is used to indicate the position of the character belonging to the at least one second entity in the at least one character vector; By concatenating the character vectors indicated by the at least one second position prediction label at each position, at least one second entity is obtained as at least one target entity, which is used to perform natural language processing on the text to be extracted.

2. The method according to claim 1, characterized in that, The method further includes: The at least one character vector and the at least one extended word vector are encoded and decoded to obtain at least one first entity; and the at least one first entity is used as the at least one target entity.

3. The method according to claim 2, characterized in that, The process of encoding and decoding the at least one character vector and the at least one extended word vector to obtain at least one first entity includes: Attention encoding is performed on the at least one character vector and the at least one extended word vector to obtain a first attention encoding vector set; The first attention-encoded vector set is decoded and predicted to obtain a first position prediction sequence; the first position prediction sequence is used to indicate the position of the character belonging to the at least one first entity in the at least one character vector; Based on the predicted sequence from the first position, the at least one first entity is obtained.

4. The method according to claim 1, characterized in that, The method further includes: Using an attention encoding / decoding transformation model, combined with the first boundary information and the second boundary information, the at least one character vector and the at least one extended word vector are encoded / decoded to obtain the at least one second entity; the attention encoding / decoding transformation model is obtained by performing data mining processing on real interaction logs to obtain a training sample set, and the initial attention encoding / decoding transformation model is trained using the training sample set. The data mining process includes at least one of log data mining, annotation replacement processing, and auxiliary synonym replacement processing.

5. A solid extraction device, characterized in that, include: The acquisition module is used to acquire at least one character vector and at least one extended word vector contained in the text to be extracted; The at least one character vector is obtained by taking the vector information of each individual character from the at least one character contained in the text to be extracted; the at least one extended word vector includes at least one preset entity vector and at least one word segmentation vector; the at least one preset entity vector is a vector of preset entity information corresponding to the text to be extracted in a preset entity dictionary; each preset entity information includes at least one preset entity and a preset entity alias; The at least one word segmentation vector is obtained by segmenting the text to be extracted; The text to be extracted is used to perform relevance matching in the preset entity dictionary to obtain at least one preset entity information that matches the text to be extracted; wherein, the relevance matching includes suffix number dictionary matching and positive maximum matching; the vector corresponding to the at least one preset entity information is used as the at least one preset entity vector; The encoding / decoding transformation module is used to: take the first and last character positions in each extended word vector as first boundary information and the position of each character vector as second boundary information in the at least one character vector and the at least one extended word vector; determine the length of the vector to be encoded corresponding to each encoding identifier in the at least one encoding identifier according to the first boundary information and the second boundary information; the encoding identifier is obtained by identifying the at least one character vector and the at least one extended word vector; combine the length of the vector to be encoded, perform attention encoding on the character vector or the extended word vector corresponding to each encoding identifier to obtain a second attention encoding vector set; truncate the second attention encoding vector set according to the length of the request statement in the text to be extracted to obtain a truncated encoding vector; perform decoding prediction on the truncated encoding vector to obtain a second position prediction sequence; the second position prediction sequence is a sequence composed of at least one second position prediction label; the at least one second position prediction label is used to indicate the position of the character belonging to the at least one second entity in the at least one character vector; connect the character vectors indicated by the at least one second position prediction label at each position to obtain at least one second entity as at least one target entity, the at least one target entity being used to implement natural language processing of the text to be extracted.

6. The apparatus as claimed in claim 5, characterized in that, The encoding / decoding transformation module is also used for: The at least one character vector and the at least one extended word vector are encoded and decoded to obtain at least one first entity; and the at least one first entity is used as the at least one target entity.

7. The apparatus as claimed in claim 6, characterized in that, The encoding / decoding transformation module is also used for: Attention encoding is performed on the at least one character vector and the at least one extended word vector to obtain a first attention encoding vector set; decoding and prediction are performed on the first attention encoding vector set to obtain a first position prediction sequence; the first position prediction sequence is used to indicate the position of the character belonging to the at least one first entity in the at least one character vector; the at least one first entity is obtained according to the first position prediction sequence.

8. The apparatus as claimed in claim 5, characterized in that, It also includes an attention encoding / decoding transformation module, used for: Using an attention encoding / decoding transformation model, combined with the first boundary information and the second boundary information, the at least one character vector and the at least one extended word vector are encoded / decoded to obtain the at least one second entity; the attention encoding / decoding transformation model is obtained by performing data mining processing on real interaction logs to obtain a training sample set, and then using the training sample set to train the initial attention encoding / decoding transformation model; wherein, the data mining processing includes at least one of log data mining, annotation replacement processing, and auxiliary synonym replacement processing.

9. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the method of any one of claims 1 to 4 when executed by a processor.

11. A computer program product, characterized in that, It stores computer instructions for implementing the method described in any one of claims 1 to 4 when executed by a processor.

Images