Instruction data processing system and method based on multiple scenes

Abstract

The invention discloses an instruction data processing system and method based on multiple scenes. According to the instruction data processing system and method, a basic knowledge system is constructed based on preset entity definition data and entity relation data through a data system module; instruction input of a target object is obtained through a data recognition module, and recognition processing is performed on the instruction input based on the basic knowledge system to obtain a recognition result; according to the embodiment of the invention, multi-scene recognition and recognition in a single scene are adopted, then clear recognition and fuzzy recognition are adaptively determined, scene clustering, heuristic labeling, high-frequency words and sentences and generalization processing are utilized to realize the instruction recognition method, and by constructing the data system module and designing the multi-scene recognition module, the recognition efficiency is improved. The system can better understand diversified and complex user input, the accuracy of entity recognition is improved, and the method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-scenario based instruction data processing system and method. Background Art

[0002] When users interact with terminal devices, the user's command text is processed by natural language processing for entity recognition and segmentation, and then the processed text is sent to the corresponding device for intelligent recommendation or device control. Due to the diversity, ambiguity and noise interference of voice commands, the existing command recognition methods perform poorly in practical applications and can often only process some fixed templates or keywords, and cannot accurately understand the user's command intentions. This results in the platform being unable to push accurate results according to user needs, nor can it feedback accurate search results and accurately control smart home devices.

[0003] Among existing entity recognition methods, although the SOTA (State-of-the-Art) model has relatively good text recognition effects in general scenarios, its recognition accuracy is difficult to meet the high requirement of 90%-95% in complex and rich scenarios of interaction using terminal devices. At the same time, the model's judgment on the part of speech of words in instructions is particularly vague, resulting in poor recognition results. Summary of the invention

[0004] The present invention proposes a multi-scenario based instruction data processing system and method, aiming to solve one of the technical problems in the related art at least to a certain extent. The embodiments of the present invention can accurately implement instruction data processing.

[0005] On the one hand, an embodiment of the present invention provides an instruction data processing system based on multiple scenarios, the system comprising a data system module and a data identification module;

[0006] The data system module is used to construct a basic knowledge system based on preset entity definition data and entity relationship data; the basic knowledge system includes heuristic annotations and high-frequency words and sentences;

[0007] The data recognition module is used to obtain the command input of the target object, and to identify and process the command input based on the basic knowledge system to obtain the recognition result; the data recognition module includes:

[0008] A multi-scene recognition module is used to perform multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input;

[0009] The single scene recognition module is used to classify the command input based on the target scene and heuristic annotation to obtain the entity type of the command input; the entity type includes clear entity and fuzzy entity;

[0010] A clear recognition module, used for obtaining a first recognition result by matching high-frequency words and sentences based on the instruction input when the entity type is a clear entity;

[0011] The fuzzy recognition module is used to generalize the instruction input based on the pre-trained bidirectional long short-term memory network when the entity type is a fuzzy entity to obtain a second recognition result.

[0012] Optionally, the multi-scene recognition module performs multi-scene recognition on the command input through scene clustering to obtain a target scene corresponding to the command input, which is specifically used to perform the following operations:

[0013] Based on a preset clustering algorithm, a similarity result between the command input and the processing commands of multiple preset scenarios is obtained;

[0014] The target scene is determined in the preset scenes according to the similarity results.

[0015] Optionally, the single-scene recognition module classifies the instruction input based on the target scene and the heuristic annotation to obtain the entity type of the instruction input, which is specifically used to perform the following operations:

[0016] Focus the command input on the heuristic annotation of the target scene for entity information recognition;

[0017] When the instruction input includes preset specific entity information, the entity type of the instruction input is determined to be a clear entity; otherwise, the entity type of the instruction input is determined to be a fuzzy entity.

[0018] Optionally, when the entity type is a clear entity, the explicit recognition module obtains a first recognition result by matching high-frequency words and sentences based on the instruction input, and is specifically used to perform the following operations:

[0019] Based on a preset string search algorithm, the character matching positions of the instruction input and the high-frequency words and sentences are dynamically adjusted through preset character matching rules, so that the instruction input is matched in the high-frequency words and sentences to obtain a first recognition result.

[0020] Optionally, when the entity type is a fuzzy entity, the fuzzy recognition module generalizes the instruction input based on a pre-trained bidirectional long short-term memory network to obtain a second recognition result, which is specifically used to perform the following operations:

[0021] The feature information of the instruction input is extracted by using a bidirectional long short-term memory network; the feature information is obtained based on the context information link of the instruction input;

[0022] According to the feature information, a second recognition result is obtained through conditional random field processing; the second recognition result is obtained based on the dependency relationship obtained by learning the conditional random field according to the feature information.

[0023] Optionally, the system further includes a data backflow module;

[0024] The data reflux module is used to record and re-identify the instruction input when the data identification module fails to obtain an identification result based on the instruction input.

[0025] Optionally, the data reflux module records and re-identifies the command input, specifically for performing the following operations:

[0026] Log command input to a preset system log file;

[0027] Regularly export command inputs in system log files;

[0028] The command input exported from the system log is re-identified. When the re-identification process does not obtain an identification result, the command input is recorded in the warehouse.

[0029] On the other hand, an embodiment of the present invention provides a multi-scenario based instruction data processing method, which is applied to the aforementioned multi-scenario based instruction data processing system, and the method comprises the following steps:

[0030] The basic knowledge system is constructed based on the preset entity definition data and entity relationship data through the data system module; the basic knowledge system includes heuristic annotations and high-frequency words and sentences;

[0031] Obtaining the target object's command input through the data recognition module, and identifying and processing the command input based on the basic knowledge system to obtain a recognition result; obtaining the target object's command input through the data recognition module, and identifying and processing the command input based on the basic knowledge system to obtain a recognition result, including the following steps:

[0032] The multi-scene recognition module performs multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input;

[0033] Through the single scene recognition module, the entity type of the command input is obtained by classifying the entity of the command input based on the target scene and the heuristic annotation; the entity type includes clear entity and fuzzy entity;

[0034] When the entity type is a clear entity, a first recognition result is obtained by matching the high-frequency words and sentences based on the instruction input through the clear recognition module;

[0035] When the entity type is a fuzzy entity, the fuzzy recognition module generalizes the instruction input based on the pre-trained bidirectional long short-term memory network to obtain the second recognition result.

[0036] Optionally, the method further comprises the following steps:

[0037] When the data recognition module fails to obtain a recognition result based on the instruction input, the instruction input is recorded and re-recognized through the data reflux module.

[0038] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned instruction data processing method based on multiple scenarios.

[0039] On the other hand, an embodiment of the present invention provides a computer storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned instruction data processing method based on multiple scenarios.

[0040] The embodiment of the present invention constructs a basic knowledge system based on preset entity definition data and entity relationship data through a data system module; the basic knowledge system includes heuristic annotations and high-frequency words and sentences; the instruction input of the target object is obtained through a data recognition module, and the instruction input is recognized and processed based on the basic knowledge system to obtain a recognition result; the instruction input of the target object is obtained through a data recognition module, and the instruction input is recognized and processed based on the basic knowledge system to obtain a recognition result, including the following steps: the instruction input is recognized by the multi-scene recognition module through scene clustering to obtain the target scene corresponding to the instruction input; the instruction input is classified into entities based on heuristic annotations according to the target scene through a single-scene recognition module to obtain the entity type of the instruction input; the entity type includes clear entities and fuzzy entities; when the entity type is a clear entity, the first recognition result is obtained based on matching the instruction input in high-frequency words and sentences through the clear recognition module; when the entity type is a fuzzy entity, the instruction input is generalized based on a pre-trained bidirectional long short-term memory network to obtain a second recognition result. The embodiment of the present invention adopts multi-scene recognition and single-scene recognition, and then adaptively determines explicit recognition and fuzzy recognition, and uses scene clustering, heuristic labeling, high-frequency words and phrases, and generalization processing to implement the instruction recognition method. By constructing a data system module and designing a multi-scene recognition module, the system can better understand diverse and complex user inputs, thereby improving the accuracy of entity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0042] Figure 1 A schematic diagram of the module composition of the instruction data processing system based on multiple scenarios provided by an embodiment of the present invention;

[0043] Figure 2A schematic diagram of the specific execution logic of the multi-scene recognition module provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the specific execution logic of the recognition module in a single scene provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the specific execution logic of the fuzzy recognition module provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of the specific execution logic of the data return module provided in an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of the overall architecture of a multi-scenario instruction data processing system provided by an embodiment of the present invention;

[0048] Figure 7 A flow chart of the data backflow module provided by an embodiment of the present invention;

[0049] Figure 8 A schematic diagram of an implementation environment for a multi-scenario based instruction data processing method provided by an embodiment of the present invention;

[0050] Fig. 9 A flowchart of a method for processing instruction data based on multiple scenarios provided by an embodiment of the present invention;

[0051] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0055] To facilitate the understanding of the technical solution of the present invention, the proprietary technologies that may appear in the implementation of the present invention are first explained:

[0056] Natural language processing (NLP): An important direction in the fields of computer science and artificial intelligence, it studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. The research in this field involves natural language, that is, the language used by people in daily life, so it is also closely related to the study of linguistics, but there are important differences. Natural language processing is not a general study of natural language, but the development of computer systems that can effectively achieve natural language communication, especially the software systems therein, so it is a part of computer science. The goal of NLP is not only to convert speech into text, but also to understand the meaning of the text and extract valuable information from it. NLP can be used for various natural language processing tasks, such as sentiment analysis, text classification, machine translation, etc. In general, ASR focuses more on the conversion of speech to text, while NLP focuses more on the understanding of text and information extraction.

[0057] Entity: refers to an independent thing with clear characteristics that can be distinguished from other things. In the fields of information extraction, natural language processing, and knowledge graphs, the information used to describe these things is the entity. Entities can be abstract or concrete.

[0058] On the one hand, if Figure 1 As shown, an embodiment of the present invention provides an instruction data processing system 100 based on multiple scenarios, which may include a data system module 110 and a data identification module 120;

[0059] The data system module is used to construct a basic knowledge system based on preset entity definition data and entity relationship data; the basic knowledge system includes heuristic annotations and high-frequency words and sentences;

[0060] For example, in some specific implementations, building a data system through a data system module as the cornerstone of the technical solution requires defining a series of rules and steps to achieve this goal. The design of the data system needs to follow a certain logic and structure to ensure the stability and scalability of the system.

[0061] (1) Definition of entity:

[0062] Entity description: Define a description for each entity, including its name, type, attributes, behavior, etc.

[0063] Entity classification: Different types of entities need to be defined, such as channel entity, application entity, time entity, etc. Each entity type has its own scenario and usage.

[0064] Entity format: The format of the entity needs to be defined, including the entity's unique code (idcode), type code (tycode), and other possible fields. The unique code identifies the entity, and the type code is used to identify the type of entity instruction.

[0065] (2) Entity Relationship

[0066] Entity-to-entity mapping: Map entities to entities, that is, define which entities are synonymous with which entities, etc.

[0067] Specific implementation example: Define the entity description of "Peppa Pig" (name: Peppa Pig, type: children), entity classification (film and television), entity format (unique code: yh00001), entity relationship (Peppa Pig and Peppa Pig). Batch store entity data and entity relationship data. After the data system is built, the system successfully builds a basic knowledge system to provide basic services for subsequent modules of the system.

[0068] The data recognition module is used to obtain the command input of the target object, and to identify and process the command input based on the basic knowledge system to obtain the recognition result;

[0069] For example, in some specific implementations, the data recognition module 120 is a key component in the overall system architecture, which plays the role of the "brain" of the system, responsible for receiving, parsing and processing command inputs from users or other system components. These instructions may include various forms, such as text provided by the user, voice commands, or data streams transmitted by other systems. The data recognition module needs to be highly flexible and intelligent, and can accurately understand different forms of input and perform corresponding processing according to specific circumstances.

[0070] Among them, the data identification module includes:

[0071] A multi-scene recognition module is used to perform multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input;

[0072] It should be noted that in some embodiments, Figure 2As shown, the multi-scene recognition module performs multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input, which can be specifically used to perform the following operations: T1211, based on the preset clustering algorithm, obtain the similarity result between the command input and the processing instructions of multiple preset scenes; T1212, determine the target scene in the preset scenes according to the similarity result.

[0073] For example, in some specific implementations, the multi-scene recognition module may perform the following operations:

[0074] Users input personal instructions through terminal smart devices, for example: I want to listen to Zhou**'s songs, I want to check Liu**'s information, help me turn off the smart speaker. If a general model is used for entity data recognition, complex and diverse scenes will cause scene conflicts, thereby reducing the accuracy of entity recognition. Therefore, the system architecture has a multi-scene recognition module in front of the data recognition module, and through scene clustering, it distinguishes complex scenes and performs subsequent accurate recognition work. Common clustering algorithms include: K-means, AffinityPropagation, hierarchical clustering and other algorithms. This system uses the Affinity Propagation algorithm during implementation:

[0075] At the beginning of the algorithm, all attribution degrees are initialized to 0. First, the attraction r(i,k) sent from point i to the candidate cluster center point k is calculated, and the formula is as follows:

[0076] r(i,k)←s(i,k)-max k′s.t.k′≠k {a(i,k′)+s(i,k′)};

[0077] Among them, s(i,k) represents the similarity between two points, a(i,k′) represents the suitability of i to select point k as the cluster center, max(·) represents the maximum function, and k′ represents other points except the target point k. Then calculate the degree of belonging sent from the candidate cluster center point k to point i, the formula is as follows:

[0078]

[0079] Among them, r(k, k) represents the self-attraction, and min(·) represents the minimum function. Finally, the attraction information of each point is summed up, and a decision is made to determine whether the iteration should stop. The formula is as follows:

[0080] a(k,k)←∑ i′s.t.i′≠k max{0,r(i′,k)}};

[0081] Through iteration, the instructions are clustered into n scenarios. The similarity of instructions processed in the same scenario is high, while the similarity of instructions processed in different scenarios is low, which helps improve the subsequent recognition accuracy.

[0082] Specific implementation example: After inputting the target command (help me turn off the smart speaker), the system performs multi-scene recognition (smart home scene).

[0083] The single scene recognition module is used to classify the command input based on the target scene and heuristic annotation to obtain the entity type of the command input; the entity type includes clear entity and fuzzy entity;

[0084] It should be noted that in some embodiments, Figure 3 As shown, the single-scene recognition module classifies the instruction input based on the heuristic annotation according to the target scene to obtain the entity type of the instruction input, which can be specifically used to perform the following operations: T1221, focusing the instruction input on the heuristic annotation under the target scene to perform entity information recognition; T1222, when the instruction input includes preset specific entity information, determining that the entity type of the instruction input is a clear entity; otherwise, determining that the entity type of the instruction input is a fuzzy entity.

[0085] For example, in some specific implementations, the single scene recognition module may perform the following operations:

[0086] When focusing on a single scenario for entity recognition operations, it is crucial to first perform heuristic labeling. Operators periodically perform irregular sentence labeling work to provide the system with a large amount of sample data for training and optimizing models. In this way, the system can better understand and accurately identify various forms of instructions, improving the accuracy and coverage of entity recognition.

[0087] On the basis of heuristic annotation, the system can further classify the instructions to be parsed and identified. According to the characteristics and expression of the instructions, they are divided into two categories: entity-clear instructions and entity-fuzzy instructions. For entity-clear instructions, the instructions contain clear entity information, such as specific names, attributes or keywords. A clear recognition module can be constructed to directly identify and parse based on this information, determine the meaning of the instructions and perform corresponding operations. In this case, the task of entity recognition is relatively simple, and the system can complete the processing of instructions quickly and accurately. For entity-fuzzy instructions, there may be unclear entity information in the instructions, such as vague descriptions, generalized words or incomplete information. In this case, the system needs to adopt a more intelligent method for identification and parsing. The fuzzy recognition module is designed to conduct in-depth analysis of the instructions through context inference and other means, understand the user's intentions as much as possible and extract key information. When processing entity-fuzzy instructions, the system needs to be more flexible to deal with various situations, and improve the accuracy and robustness of recognition by continuously optimizing algorithms and models.

[0088] A clear recognition module, used for obtaining a first recognition result by matching high-frequency words and sentences based on the instruction input when the entity type is a clear entity;

[0089] It should be noted that in some embodiments, when the entity type is a clear entity, the clear recognition module obtains a first recognition result based on matching the instruction input in high-frequency words and sentences. Specifically, it can be used to perform the following operations: based on a preset string search algorithm, the character matching positions of the instruction input and high-frequency words and sentences are dynamically adjusted through preset character matching rules, so that the instruction input is matched in high-frequency words and sentences to obtain the first recognition result.

[0090] For example, in some specific implementations, the clear identification module may perform the following operations:

[0091] For instructions with clear sentence entities, the system needs to construct high-frequency entities and questions to parse them. High-frequency entities help improve recognition delay rate and accuracy. For example, high-frequency types include "movies", "TV series", "music", etc., and high-frequency prefix words include "I want to watch", "play", etc. Summarize and summarize high-frequency questions to form a standard model to provide a benchmark for subsequent matching. For example: "I want to watch Andy Lau's movie" is summarized as {prefix word} {name} {type}. After the high-frequency words and sentences are constructed, we use matching algorithms to perform matching work. Common matching algorithms include: AC automaton, KMP algorithm, Rabin-Karp algorithm, Boyer-Moore algorithm, etc.

[0092] The Boyer-Moore algorithm, also known as the Boyer-Moore string search algorithm (hereinafter referred to as the BM algorithm), is an efficient string matching algorithm used to find the occurrence position of a pattern string in a target string. Its core idea is to preprocess the pattern string and use the mismatch information of the symbol comparison to skip as many target symbols as possible, so as to quickly locate possible matching positions and reduce the number of comparisons.

[0093] The main idea of ​​the BM algorithm is to start matching from the end of the pattern string (the string of characters to be searched), and then skip some unnecessary comparisons based on the position of the unmatched characters in the pattern string. Specifically, the BM algorithm constructs two tables by preprocessing the pattern string:

[0094] Bad character rule table: used to record the last occurrence position of each character in the pattern string. During the matching process, when a mismatch occurs, the bad character rule table is used to determine whether to slide the pattern string to the right by the first preset position.

[0095] Good suffix rule table: records the matching length of each suffix substring in the pattern string. When the suffix substring of the pattern string matches a substring of the main string, the good suffix rule table can be used to decide to slide the pattern string to the right by the second preset position.

[0096] This system uses the Boyer-Moore algorithm in its implementation:

[0097] First, use the bad character rule to calculate, and according to the position of the character in the pattern string, move the pattern string to the right to the rightmost position of the character in the pattern string. The formula of the bad character rule is as follows:

[0098] s1=max(1,j-bc[text[i+j]]);

[0099] Among them, i is the position in the text string, j is the position in the pattern string, text[i+j] represents the target character, c in bc[c] represents the target character text[i+j], bc[c] represents the rightmost position of character c in the pattern string, and s1 represents the distance the pattern string moves to the right. Then, good suffix rule matching is performed to find a matching good suffix in the pattern string. The formula is as follows:

[0100] s2=max(1,m-suffix[j]);

[0101] Where m is the length of the pattern string, and suffix[j] represents the suffix substring ending at position j in the pattern string. Efficient string matching is achieved by comparing characters from right to left in the text string and adjusting the position of the pattern string using the information of good suffixes and bad characters.

[0102] The fuzzy recognition module is used to generalize the instruction input based on the pre-trained bidirectional long short-term memory network when the entity type is a fuzzy entity to obtain a second recognition result.

[0103] It should be noted that in some embodiments, Figure 4 As shown, when the entity type is a fuzzy entity, the fuzzy recognition module generalizes the instruction input based on the pre-trained bidirectional long short-term memory network to obtain a second recognition result, which is specifically used to perform the following operations: T1241, using the bidirectional long short-term memory network to extract the feature information of the instruction input; the feature information is obtained based on the context information link of the instruction input; T1242, according to the feature information, the second recognition result is obtained by conditional random field processing; the second recognition result is obtained based on the dependency relationship learned by the conditional random field according to the feature information.

[0104] For example, in some specific implementations, the fuzzy recognition module may perform the following operations:

[0105] User input is diverse and complex, which means that sentence entity ambiguity is often encountered in entity recognition tasks. To solve this problem, the system needs to have deep recognition capabilities and be able to better understand input instructions by connecting to the context. In addition, for ambiguous sentence entities, the system also needs to continuously improve its generalization recognition capabilities to improve recognition accuracy and robustness.

[0106] In order to achieve the construction of deep recognition capabilities, the system will adopt advanced natural language processing technology. Among them, bidirectional long short-term memory network and conditional random field are two powerful tools widely used in sequence labeling tasks. Bidirectional long short-term memory network can capture the contextual information of the input sequence and learn long-distance dependencies in the sequence. The conditional random field can model the dependencies between labels, further improving the accuracy of entity recognition. In the process of building entity recognition capabilities, the system will be trained with a large amount of labeled data, and feature extraction and model training will be performed in combination with contextual information. For fuzzy entities, the system will focus on the contextual information of the context, and improve the model's recognition ability for fuzzy entities by generalizing input instructions.

[0107] First, the system will build a bidirectional long short-term memory network to memorize long-distance (context) dependencies, which includes a forget gate, an input gate, and an output gate to control the flow of information and the update of memory. Specifically, for the user's input instruction X = (x1, x2, ..., x n ), and then capture the contextual information of the input sequence through the hidden states in the forward and backward directions. The formula is as follows:

[0108] i t =σ(W (i) x t +U (i) h t-1 +b (i) );

[0109] f t =σ(W (f) x t +U (f) h t-1 +b (f) );

[0110] o t =σ(W (o) x t +U (o) h t-1 +b (o) );

[0111]

[0112]

[0113] h t =o t ⊙tanh(c t );

[0114] where W(·), b(·) and U(·) represent network parameters, · includes i, f, o and c, x t Indicates the input command at the current moment, h t-1 represents the hidden information of the previous moment, σ(·) and tanh(·) represent the activation function and tangent function respectively. The backward calculation is consistent with the forward calculation. Finally, the hidden states in the two directions are linked to obtain the output result. The formula is as follows:

[0115]

[0116] where concat(·) represents the link function, and Represents the hidden information in two directions respectively. As a feature extractor, the network can better learn the feature representation of the input sequence. Then, the feature representation is input into the conditional random field to learn the dependency between labels to solve the problems of label bias and illegal label sequence. The formula is as follows:

[0117]

[0118] Where Y = (y1, y2, ..., y) is the label sequence, X = (x1, x2, ..., x n ) is the input sequence, Z(X) is the normalization factor, λ k is the weight parameter, f k is the characteristic function, and K is the number of characteristic functions. The loss function of the bidirectional long short-term memory network and the conditional random field includes the above two parts, and is jointly trained through the back propagation algorithm to maximize the likelihood probability of the model. Therefore, when the system inputs a relatively vague instruction, the fuzzy recognition module can accurately identify the entity.

[0119] Specific implementation example: After the target instruction (help me turn off the smart speaker) is identified as the corresponding scene (smart home scene), the corresponding result (action: turn off, device name: smart speaker) is given through the explicit recognition and fuzzy recognition modules.

[0120] In some optional embodiments, the system may further include a data reflow module; the data reflow module is used to record and re-recognize the instruction input when the data recognition module fails to obtain a recognition result based on the instruction input.

[0121] For example, in some specific implementations, in a production environment, the system uses a designed module to make corresponding judgments and processes for instructions with clear entities and vague entities to meet the needs of users. However, despite the careful design and optimization of the system, there are still some instructions that fail to parse. These instructions that fail to parse may be caused by various reasons, such as irregular input format, semantic ambiguity, data noise, etc.

[0122] In order to solve these instructions that failed to parse, the present invention also designs a data reflow module to achieve reprocessing of the instructions that failed to parse. As an important component of the system architecture, the data reflow module has automatic processing capabilities and can promptly detect and process instructions that failed to parse, so as to ensure stable operation of the system and a good user experience.

[0123] It should be noted that in some embodiments, Figure 5 As shown, the data reflow module records and re-identifies the command input, which can be used to perform the following operations:

[0124] T131. Record the command input to the preset system log file; T132. Regularly export the command input in the system log file; T133. Re-identify the command input exported from the system log. If the re-identification process does not obtain an identification result, the command input is recorded in the warehouse. The storage record is used to assist the management object in adjusting the basic knowledge system.

[0125] For example, in some specific implementations, the data reflow module may perform operations as follows:

[0126] Step 1: First, when the system finds that an instruction cannot be parsed correctly, it records it in the system log file.

[0127] Step 2: Then, the data return module periodically checks the system log file to find the parsing failure instructions recorded therein. Once the parsing failure instruction is found, the data return module will reprocess it.

[0128] Step 3: The reprocessing process includes re-parsing instructions, correcting errors, adjusting parameters, etc. (which can be customized by responding to the relevant control instructions of the management object) in the hope of successfully identifying and executing the instructions. If the reprocessing is successful, the processing results will be fed back to the user; if the reprocessing fails, the data reflux module will store and record high-frequency entities and high-frequency sentence patterns, thereby improving the system's knowledge system.

[0129] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0130] First of all, it should be noted that natural language processing is an important branch of artificial intelligence, and its purpose is to use computers to intelligently process natural language. Traditional natural language processing technology is mainly based on machine learning, such as support vector machine model (SVM), Markov model (Markov), conditional random field model (CRF), etc. However, there are some problems with traditional machine learning methods, such as over-reliance on the quality of the training set, the application effect of the training set in different fields will be quite different, and it cannot process higher-order and more abstract natural languages. In recent years, with the rapid development of deep learning technology, natural language processing technology based on deep learning has gradually become a research hotspot. Deep learning can automatically extract language features through neural networks, avoiding the tedious process of manually annotating training sets in traditional machine learning, and can process higher-order and more abstract natural languages. Natural language processing technology has been widely used in many fields, such as machine translation, information retrieval, intelligent question answering, text classification, sentiment analysis, etc.

[0131] Entity recognition is one of the core tasks of natural language processing (NLP), and its goal is to automatically identify entities in text, such as names of people, places, products, proper nouns, etc. In early NER research, researchers have tried many methods, such as SVM (support vector machine), decision tree, HMM (hidden Markov model) and CRF (conditional random field), among which the CRF method is considered to be a relatively successful one. In recent years, with the rise of deep learning technology, entity recognition methods based on neural networks have gradually become dominant. In addition, in order to cope with entity recognition tasks in big data scenarios, researchers have also proposed methods based on distributed platforms, thresholds, clustering, and a combination of two or more technologies. Entity recognition technology has a wide range of significance in practical applications. For example, in intelligent search systems, by performing entity recognition on search content, entity information related to the query can be provided, providing users with more accurate search results. In addition, entity recognition technology can also be applied to information extraction, machine translation, question-answering systems and other fields, providing strong support for the development of natural language processing technology.

[0132] The intelligent understanding platform is an advanced platform that integrates radio equipment and terminal equipment, aiming to provide users with a more efficient and natural interactive experience. The platform mainly serves smart device scenarios in vertical fields, including voice search for knowledge, news, storage information and other content, as well as business scenarios such as voice control of smart home devices. Through the intelligent understanding platform, users can easily search and view various content of interest on smart devices through voice, without tedious manual click operations. At the same time, the platform can also realize voice control of smart home devices, allowing users to control various devices in the home, such as lights, air conditioners, TVs, etc., through simple voice commands. At present, after the user's voice command is delivered to the intelligent understanding platform, the user's command is recognized and understood. However, due to the diversity, ambiguity and noise interference of voice commands, the existing command recognition methods perform generally in practical applications.

[0133] Therefore, in view of the shortcomings of existing related technologies, it is necessary to improve and optimize the existing entity recognition methods to better adapt to the needs of complex and rich scenarios and incremental data. Figure 6 As shown, the present invention provides an instruction data processing system based on multiple scenarios, which aims to improve the accuracy of entity recognition by building a detailed data knowledge system, designing a data recognition module suitable for multiple scenarios, and adding a data return module. The process steps of the overall architecture can be as follows:

[0134] Step 1: Data system module:

[0135] Building a data system as the cornerstone of the technical solution requires defining a series of rules and steps to achieve this goal. The design of the data system needs to follow certain logic and structure to ensure the stability and scalability of the system.

[0136] (1) Definition of entity:

[0137] Entity description: Define a description for each entity, including its name, type, attributes, behavior, etc.

[0138] Entity classification: Different types of entities need to be defined, such as channel entity, application entity, time entity, etc. Each entity type has its own scenario and usage.

[0139] Entity format: The format of the entity needs to be defined, including the entity's unique code (idcode), type code (tycode), and other possible fields. The unique code identifies the entity, and the type code is used to identify the type of entity instruction.

[0140] (2) Relationships between entities:

[0141] Entity-to-entity mapping: Map entities to entities, that is, define which entities are synonymous with which entities, etc.

[0142] Specific implementation example: Define the entity description of "Peppa Pig" (name: Peppa Pig, type: children), entity classification (film and television), entity format (unique code: yh00001), entity relationship (Peppa Pig and Peppa Pig). Batch store entity data and entity relationship data. After the data system is built, the system successfully builds a basic knowledge system to provide basic services for subsequent modules of the system.

[0143] Step 2: Data Identification Module

[0144] The data recognition module is a key component in the overall system architecture. It plays the role of the "brain" of the system and is responsible for receiving, parsing and processing command inputs from users or other system components. These commands may come in various forms, such as text provided by users, voice commands, or data streams transmitted by other systems. The data recognition module needs to be highly flexible and intelligent, and be able to accurately understand different forms of input and process them accordingly according to the specific situation.

[0145] Step 2-1: Multi-scene recognition module:

[0146] Users input personal instructions through terminal smart devices, for example: I want to listen to Zhou**'s songs, I want to check Liu**'s information, help me turn off the smart speaker. If a general model is used for entity data recognition, complex and diverse scenes will cause scene conflicts, thereby reducing the accuracy of entity recognition. Therefore, the system architecture has a multi-scene recognition module in front of the data recognition module, and through scene clustering, it distinguishes complex scenes and performs subsequent accurate recognition work. Common clustering algorithms include: K-means, Affinity Propagation, hierarchical clustering and other algorithms. This system uses the AffinityPropagation algorithm during implementation:

[0147] At the beginning of the algorithm, all attribution degrees are initialized to 0. First, the attraction degree sent from point i to the candidate cluster center point k is calculated, and the formula is as follows:

[0148] r(i, k)←s(i, k)-max k′s.t.k′≠k {a(i, k′)+s(i, k′)};

[0149] Where s(i, k) represents the similarity between two points, a(i, k′) represents the suitability of i to select point k as the cluster center, and max(·) represents the maximum function. Then the degree of belonging sent from the candidate cluster center point k to point i is calculated as follows:

[0150]

[0151] Among them, r(k, k) represents the self-attraction, and min(·) represents the minimum function. Finally, the attraction information of each point is summed up, and a decision is made to determine whether the iteration should stop. The formula is as follows:

[0152] a(k, k)←∑ i′s.t.i′≠k max{0,r(i′,k)}};

[0153] Through iteration, the instructions are clustered into n scenarios. The similarity of instructions processed in the same scenario is high, while the similarity of instructions processed in different scenarios is low, which helps improve the subsequent recognition accuracy.

[0154] Specific implementation example: After inputting the target command (help me turn off the smart speaker), the system performs multi-scene recognition (smart home scene).

[0155] Step 2-2: Single scene recognition module:

[0156] When focusing on a single scenario for entity recognition operations, it is crucial to first perform heuristic labeling. Operators periodically perform irregular sentence labeling work to provide the system with a large amount of sample data for training and optimizing models. In this way, the system can better understand and accurately identify various forms of instructions, improving the accuracy and coverage of entity recognition.

[0157] On the basis of heuristic annotation, the system can further classify the instructions to be parsed and identified. According to the characteristics and expression of the instructions, they are divided into two categories: entity-clear instructions and entity-fuzzy instructions. For entity-clear instructions, the instructions contain clear entity information, such as specific names, attributes or keywords. A clear recognition module can be constructed to directly identify and parse based on this information, determine the meaning of the instructions and perform corresponding operations. In this case, the task of entity recognition is relatively simple, and the system can complete the processing of instructions quickly and accurately. For entity-fuzzy instructions, there may be unclear entity information in the instructions, such as vague descriptions, generalized words or incomplete information. In this case, the system needs to adopt a more intelligent method for identification and parsing. The fuzzy recognition module is designed to conduct in-depth analysis of the instructions through context inference and other means, understand the user's intentions as much as possible and extract key information. When processing entity-fuzzy instructions, the system needs to be more flexible to deal with various situations, and improve the accuracy and robustness of recognition by continuously optimizing algorithms and models.

[0158] Step 2-2-1: Clearly identify the module:

[0159] For instructions with clear sentence entities, the system needs to construct high-frequency entities and questions to parse them. High-frequency entities help improve recognition delay rate and accuracy. For example, high-frequency types include "movies", "TV series", "music", etc., and high-frequency prefix words include "I want to watch", "play", etc. Summarize and summarize high-frequency questions to form a standard model to provide a benchmark for subsequent matching. For example: "I want to watch Andy Lau's movie" is summarized as {prefix word} {name} {type}. After the high-frequency words and sentences are constructed, we use matching algorithms to perform matching work. Common matching algorithms include: AC automaton, KMP algorithm, Rabin-Karp algorithm, Boyer-Moore algorithm, etc.

[0160] The Boyer-Moore algorithm, also known as the Boyer-Moore string search algorithm (hereinafter referred to as the BM algorithm), is an efficient string matching algorithm used to find the occurrence position of a pattern string in a target string. Its core idea is to preprocess the pattern string and use the mismatch information of the symbol comparison to skip as many target symbols as possible, so as to quickly locate possible matching positions and reduce the number of comparisons.

[0161] The main idea of ​​the BM algorithm is to start matching from the end of the pattern string (the string of characters to be searched), and then skip some unnecessary comparisons based on the position of the unmatched characters in the pattern string. Specifically, the BM algorithm constructs two tables by preprocessing the pattern string:

[0162] Bad character rule table: used to record the last occurrence position of each character in the pattern string. During the matching process, when a mismatch occurs, the bad character rule table is used to determine whether to slide the pattern string to the right by the first preset position.

[0163] Good suffix rule table: records the matching length of each suffix substring in the pattern string. When the suffix substring of the pattern string matches a substring of the main string, the good suffix rule table can be used to decide to slide the pattern string to the right by the second preset position.

[0164] This system uses the Boyer-Moore algorithm in its implementation:

[0165] First, use the bad character rule to calculate, and according to the position of the character in the pattern string, move the pattern string to the right to the rightmost position of the character in the pattern string. The formula of the bad character rule is as follows:

[0166] s = max(1,j-bc[text[i+j]]);

[0167] Among them, i is the position in the text string, j is the position in the pattern string, bc[c] indicates the rightmost position of character c in the pattern string, and s indicates the distance the pattern string moves to the right. Then the good suffix rule matching is performed to find the matching good suffix in the pattern string. The formula is as follows:

[0168] s=max(1,m-suffix[j]);

[0169] Where m is the length of the pattern string, and suffix[j] represents the suffix substring ending at position j in the pattern string. Efficient string matching is achieved by comparing characters from right to left in the text string and adjusting the position of the pattern string using the information of good suffixes and bad characters.

[0170] Step 2-2-2: Fuzzy recognition module:

[0171] User input is diverse and complex, which means that sentence entity ambiguity is often encountered in entity recognition tasks. To solve this problem, the system needs to have deep recognition capabilities and be able to better understand input instructions by connecting to the context. In addition, for ambiguous sentence entities, the system also needs to continuously improve its generalization recognition capabilities to improve recognition accuracy and robustness.

[0172] In order to achieve the construction of deep recognition capabilities, the system will adopt advanced natural language processing technology. Among them, bidirectional long short-term memory network and conditional random field are two powerful tools widely used in sequence labeling tasks. Bidirectional long short-term memory network can capture the contextual information of the input sequence and learn long-distance dependencies in the sequence. The conditional random field can model the dependencies between labels, further improving the accuracy of entity recognition. In the process of building entity recognition capabilities, the system will be trained with a large amount of labeled data, and feature extraction and model training will be performed in combination with contextual information. For fuzzy entities, the system will focus on the contextual information of the context, and improve the model's recognition ability for fuzzy entities by generalizing input instructions.

[0173] First, the system will build a bidirectional long short-term memory network to memorize long-distance (context) dependencies, which includes a forget gate, an input gate, and an output gate to control the flow of information and the update of memory. Specifically, for the user's input instruction X = (x1, x2, ..., x n ), and then capture the contextual information of the input sequence through the hidden states in the forward and backward directions. The formula is as follows:

[0174] i t =σ(W (i) x t +U (i) h t-1 +b(i) );

[0175] f t =σ(W (f) x t +U (f) h t-1 +b (f) );

[0176] o t =σ(W (o) x t +U (o) h t-1 +b (o) );

[0177]

[0178]

[0179] h t =o t ⊙tanh(c t );

[0180] Where W (·) and b (·) represents the network parameters, x t Indicates the input command at the current moment, h t-1 represents the hidden information of the previous moment, σ(·) and tanh(·) represent the activation function and tangent function respectively. The backward calculation is consistent with the forward calculation. Finally, the hidden states in the two directions are linked to obtain the output result. The formula is as follows:

[0181]

[0182] Where concat(·) represents the link function. As a feature extractor, the network can better learn the feature representation of the input sequence. Then, the feature representation is input into the conditional random field to learn the dependency between labels to solve the problems of label bias and illegal label sequence. The formula is as follows:

[0183]

[0184] Where Y = (y1, y2, ..., y) is the label sequence, X = (x1, x2, ..., x n ) is the input sequence, Z(X) is the normalization factor, λ k is the weight parameter, f kis the characteristic function, and K is the number of characteristic functions. The loss function of the bidirectional long short-term memory network and the conditional random field includes the above two parts, and is jointly trained through the back propagation algorithm to maximize the likelihood probability of the model. Therefore, when the system inputs a relatively vague instruction, the fuzzy recognition module can accurately identify the entity.

[0185] Specific implementation example: After the target instruction (help me turn off the smart speaker) is identified as the corresponding scene (smart home scene), the corresponding result (action: turn off, device name: smart speaker) is given through the explicit recognition and fuzzy recognition modules.

[0186] Step 3: Data reflux module:

[0187] In the production environment, the system uses the designed modules to judge and process the instructions with clear entities and ambiguous entities to meet the needs of users. However, despite the careful design and optimization of the system, there are still some instructions that fail to parse. These instructions may be caused by various reasons, such as irregular input format, semantic ambiguity, data noise, etc.

[0188] In order to solve these instructions that failed to parse, the present invention designs a data reflow module to achieve reprocessing of the instructions that failed to parse. As an important component of the system architecture, the data reflow module has automatic processing capabilities and can promptly detect and process instructions that failed to parse, so as to ensure stable operation of the system and a good user experience.

[0189] The workflow of the data reflux module is as follows: Figure 7 The flow chart shows:

[0190] Step 3-1: First, when the system finds that an instruction cannot be parsed correctly, it records it in the system log file.

[0191] Step 3-2: Then, the data return module periodically checks the system log file to find the parsing failure instructions recorded therein. Once the parsing failure instruction is found, the data return module will reprocess it.

[0192] Step 3-3: The reprocessing process includes re-parsing the instruction, correcting errors, adjusting parameters, etc., in the hope of successfully identifying and executing the instruction. If the reprocessing is successful, the processing result will be fed back to the user; if the reprocessing fails, the data reflux module will store and record high-frequency entities and high-frequency sentence patterns, thereby improving the system's knowledge system.

[0193] Through the design of the data reflux module, the system can timely discover and process the instructions that failed to parse, improving the fault tolerance and robustness of the system. At the same time, the data reflux module can also collect relevant information of the failed parse instructions, providing a useful reference for the optimization and improvement of the system. The data reflux module plays an important role in the system and provides strong support for the stability and reliability of the system.

[0194] In summary, in view of the poor accuracy of existing entity recognition methods in complex and rich scenarios, resulting in poor recognition effects, while existing methods are relatively good at processing static batch data, but when faced with incremental data, the performance is poor, and it is easy to cause the model to offset, affecting the recognition accuracy. The present invention proposes a command data processing system based on multiple scenarios, which solves the accuracy and scalability problems of current entity recognition methods by designing an overall architecture for command data recognition, including a data system, data recognition and data reflow modules. The present invention mainly describes the integration and interaction of user commands in the entity recognition architecture, and the solution provided is to be able to reasonably and accurately identify and understand user commands in multiple scenarios, so that the user's complex and diverse commands can be accurately executed.

[0195] Compared with the prior art, the present invention has at least the following beneficial effects:

[0196] 1. The technical solution proposed in the present invention has strong generalization ability. It adopts heuristic labeling, deep recognition and generalized recognition methods. By building a data system and designing a multi-scenario recognition module, the system can better understand diverse and complex user inputs, thereby improving the accuracy of entity recognition.

[0197] 2. The data reflux module designed in the present invention can timely discover and process instructions that fail to parse, ensure the stable operation of the system, and improve the user experience by continuously optimizing the system knowledge system.

[0198] 3. The present invention establishes a complete data knowledge system and a modular data identification system, providing a good foundation for system expansion and maintenance.

[0199] On the other hand, the embodiment of the present invention also provides an instruction data processing method based on multiple scenarios. It can be understood that the instruction data processing method based on multiple scenarios provided by the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various types of terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this.

[0200] like Figure 8 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 8 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0201] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0202] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0203] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0204] Based on the example Figure 8In the implementation environment shown, an embodiment of the present invention provides an instruction data processing method based on multiple scenarios. The following is explained using the example of the instruction data processing method based on multiple scenarios applied in the terminal 102. It can be understood that the instruction data processing method based on multiple scenarios can also be applied to the server 101.

[0205] Reference Fig. 9 , Fig. 9 The flowchart of the multi-scenario instruction data processing method applied to the terminal provided by the embodiment of the present invention, the execution subject of the multi-scenario instruction data processing method can be any of the aforementioned computer devices (including servers or terminals). Fig. 9 , the method may include the following steps:

[0206] S100, constructing a basic knowledge system based on preset entity definition data and entity relationship data through a data system module; the basic knowledge system includes heuristic annotations and high-frequency words and sentences;

[0207] S200, obtaining a command input of a target object through a data recognition module, and performing recognition processing on the command input based on a basic knowledge system to obtain a recognition result;

[0208] The step of obtaining the target object's command input through the data recognition module and performing recognition processing on the command input based on the basic knowledge system to obtain the recognition result may include the following steps:

[0209] The multi-scene recognition module performs multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input;

[0210] Through the single scene recognition module, the entity type of the command input is obtained by classifying the entity of the command input based on the target scene and the heuristic annotation; the entity type includes clear entity and fuzzy entity;

[0211] When the entity type is a clear entity, a first recognition result is obtained by matching the high-frequency words and sentences based on the instruction input through the clear recognition module;

[0212] When the entity type is a fuzzy entity, the fuzzy recognition module generalizes the instruction input based on the pre-trained bidirectional long short-term memory network to obtain the second recognition result.

[0213] In some embodiments, the method may further include the following steps:

[0214] When the data recognition module fails to obtain a recognition result based on the instruction input, the instruction input is recorded and re-recognized through the data reflux module.

[0215] It should be noted that the contents of the system embodiment of the present invention are applicable to the method embodiment, the process steps specifically implemented by the method embodiment are the same as those of the above system embodiment, and the beneficial effects achieved are also the same as those achieved by the above system. It should be understood that the functional logics implemented by the specific modules of each part of the above system embodiment all belong to the protection scope of the method embodiment of the present invention, and will not be described one by one here.

[0216] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned multi-scenario-based instruction data processing method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0217] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0218] like Fig.10 As shown, Fig.10 A specific example of the hardware structure of an electronic device 1000 according to an embodiment is shown. The electronic device 1000 includes:

[0219] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0220] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0221] Input / output interface 1003, used to implement information input and output;

[0222] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0223] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0224] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0225] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0227] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0228] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0229] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0230] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0231] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0232] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0233] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0234] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0235] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0236] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the solution of the present invention, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0237] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0238] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0239] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0240] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0241] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0242] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A multi-scenario based instruction data processing system, characterized in that: Including data system module and data identification module; The data system module is used to construct a basic knowledge system based on preset entity definition data and entity relationship data; the basic knowledge system includes heuristic annotations and high-frequency words and sentences; The data recognition module is used to obtain the command input of the target object, and to perform recognition processing on the command input based on the basic knowledge system to obtain a recognition result; The data identification module comprises: A multi-scene recognition module, used to perform the multi-scene recognition on the command input through scene clustering to obtain a target scene corresponding to the command input; A single scene recognition module, used for performing entity classification on the instruction input based on the heuristic annotation according to the target scene to obtain the entity type of the instruction input; the entity type includes a clear entity and a fuzzy entity; a clear recognition module, configured to obtain a first recognition result by matching the high-frequency words and sentences based on the instruction input when the entity type is the clear entity; The fuzzy recognition module is used to generalize the instruction input based on a pre-trained bidirectional long short-term memory network to obtain a second recognition result when the entity type is the fuzzy entity.

2. The multi-scenario based instruction data processing system according to claim 1, characterized in that: The multi-scene recognition module performs the multi-scene recognition on the command input through scene clustering to obtain the target scene corresponding to the command input, and is specifically used to perform the following operations: Obtaining similarity results between the command input and processing instructions of multiple preset scenarios based on a preset clustering algorithm; The target scene is determined in the preset scenes according to the similarity result.

3. The multi-scenario based instruction data processing system according to claim 1, characterized in that: The single-scene recognition module performs entity classification on the instruction input based on the heuristic annotation according to the target scene to obtain the entity type of the instruction input, which is specifically used to perform the following operations: Focusing the command input on the heuristic annotation under the target scene to perform entity information recognition; When the instruction input includes preset specific entity information, the entity type of the instruction input is determined to be the clear entity; otherwise, the entity type of the instruction input is determined to be the fuzzy entity.

4. The multi-scenario based instruction data processing system according to claim 1, characterized in that: When the entity type is the clear entity, the explicit recognition module obtains a first recognition result by matching the high-frequency words and sentences based on the instruction input, and is specifically used to perform the following operations: Based on a preset string search algorithm, the character matching positions of the instruction input and the high-frequency words and sentences are dynamically adjusted through preset character matching rules, so that the instruction input is matched in the high-frequency words and sentences to obtain a first recognition result.

5. The multi-scenario based instruction data processing system according to claim 1, characterized in that: When the entity type is the fuzzy entity, the fuzzy recognition module generalizes the instruction input based on a pre-trained bidirectional long short-term memory network to obtain a second recognition result, which is specifically used to perform the following operations: The feature information of the instruction input is extracted by using the bidirectional long short-term memory network; the feature information is obtained based on the context information link of the instruction input; According to the feature information, the second recognition result is obtained by conditional random field processing; the second recognition result is obtained based on the dependency relationship learned by the conditional random field according to the feature information.

6. The multi-scenario based instruction data processing system according to claim 1, characterized in that: The system also includes a data backflow module; The data reflow module is used to record and re-identify the instruction input when the data identification module fails to obtain an identification result based on the instruction input.

7. The multi-scenario based instruction data processing system according to claim 6, characterized in that: The data reflux module records and re-identifies the command input, and is specifically used to perform the following operations: Recording the command input into a preset system log file; Periodically exporting the command input in the system log file; The command input exported from the system log is re-identified, and when the re-identification process does not obtain an identification result, the command input is recorded in the warehouse.

8. A method for processing instruction data based on multiple scenarios, characterized in that: Applied to the multi-scenario based instruction data processing system according to any one of claims 1 to 7, the method comprises the following steps: A basic knowledge system is constructed based on preset entity definition data and entity relationship data through a data system module; the basic knowledge system includes heuristic annotations and high-frequency words and sentences; The step of obtaining the target object's command input through the data recognition module and performing recognition processing on the command input based on the basic knowledge system to obtain a recognition result comprises the following steps: Performing the multi-scene recognition on the command input by scene clustering through a multi-scene recognition module to obtain a target scene corresponding to the command input; Using a single scene recognition module, according to the target scene, and based on the heuristic annotation, entity classification is performed on the instruction input to obtain an entity type of the instruction input; the entity type includes a clear entity and a fuzzy entity; When the entity type is the clear entity, a first recognition result is obtained by matching the high-frequency words and sentences based on the instruction input through a clear recognition module; When the entity type is the fuzzy entity, the instruction input is generalized based on a pre-trained bidirectional long short-term memory network through the fuzzy recognition module to obtain a second recognition result.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to claim 8.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to claim 8 when executed by the processor.

Images