Text data processing method, neural network training method and related equipment

Abstract

This application discloses a method for processing text data, a method for training a neural network, and related equipment. This method can be used in the field of text data processing within the field of artificial intelligence. The method includes: acquiring text to be processed, which includes multiple characters; processing the text to be processed using a target model to obtain a prediction result, the prediction result indicating that the text to be processed should be split into multiple target character sets; the prediction result also includes multiple first labels, one first label indicating the semantics of one target character set, and multiple first labels used to determine the intent of the text to be processed. This method can split redup characters or modal particles into another target character set, enabling the understanding of the intent of the text to be processed even if reduplicated characters or modal particles exist throughout the entire text, thus providing a more generalizable natural language understanding method.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method for processing text data, a method for training neural networks, and related equipment. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0003] Utilizing models for natural language processing (NLP) is one of the important means of realizing artificial intelligence. Currently, the natural language understanding methods used include template matching. Specifically, the execution device can be configured with multiple template statements and the intent corresponding to each template statement. After obtaining the text to be processed by the user, the execution device can match it with multiple template statements. If a matching statement can be found, the intent of the text to be processed can be obtained; if a matching statement cannot be found, the matching fails.

[0004] However, the current template matching method is rather rigid. If the text input by the user is a similar expression to the template statement, such as having more reduplicated words or modal particles than the template statement, the matching will fail, and the intent of the text to be processed cannot be obtained. Therefore, a natural language understanding method with more generalization ability is urgently needed. Summary of the Invention

[0005] This application provides a text data processing method, a neural network training method, and related devices. A target character set may not include reduplicated characters or modal particles, that is, reduplicated characters or modal particles can be split into another target character set. Thus, even if reduplicated characters or modal particles exist in the entire text to be processed, the intention of the text to be processed can still be understood. In other words, a natural language understanding method with more generalization ability is provided.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] Firstly, embodiments of this application provide a method for processing text data, applicable to the field of text data processing in the field of artificial intelligence. An execution device acquires text to be processed, which includes one or more characters. The execution device processes the text to be processed using a target model to obtain a prediction result generated by the target model. This prediction result can be the final output of the target model or an intermediate result generated during processing. The prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes one or more first tags, where one of the aforementioned at least one first tag indicates the semantics of a target character set, and all first tags in the prediction result are used to determine the intent of the text to be processed.

[0008] In this implementation, the target model performs semantic understanding on the text to be processed, obtaining the prediction result output by the target model. The prediction result is used to indicate how to split the text to be processed into multiple target character sets. The prediction result also includes a first label that matches at least one target character set, so that the intent of the text to be processed can be reflected by all the matched first labels. Since this scheme matches labels to the characters in the text to be processed, a target character set may not include reduplicated words or modal particles, that is, reduplicated words or modal particles can be split into another target character set. Thus, even if reduplicated words or modal particles exist in the entire text to be processed, the intent of the text to be processed can still be understood, which provides a natural language understanding method with more generalization ability. Moreover, since the prediction result indicates how to split the entire text to be processed and also includes at least one label corresponding to the text to be processed, the prediction result carries richer information, which is beneficial to improving the accuracy of the intent understanding process of the text to be processed.

[0009] In one possible implementation of the first aspect, the multiple first tags include at least two-level tags, the at least two-level tags include parent tags and child tags, and there is a hierarchical relationship between the parent tags and child tags; the prediction result is also used to indicate the hierarchical relationship between the parent tags and child tags, that is, the prediction result is also used to indicate the number of levels corresponding to the at least two-level tags included in the multiple first tags, which level each first tag belongs to, and the correspondence between different tags.

[0010] In this implementation, multiple first tags include at least two-level tags, and the at least two-level tags include parent tags and child tags, with a hierarchical relationship between the parent tags and child tags; that is, the prediction results carry richer information, which is conducive to further improving the accuracy of the intent understanding process of the text to be processed.

[0011] In one possible implementation of the first aspect, there are N possible splitting methods corresponding to the text to be processed, where N is an integer greater than or equal to 1. The prediction result is used to indicate a target splitting method corresponding to the text to be processed. The target splitting method belongs to one of the N splitting methods. The prediction result is obtained based on the matching degree between multiple target character sets and multiple first tags. Further, the prediction result is obtained based on a first score corresponding to each target character set, which is used to indicate the matching degree between the target character set and the first tag. The target total score value corresponding to the target splitting method is the sum of at least one first score corresponding to the target splitting method. The target splitting method can be the splitting method with the highest target total score value among the N splitting methods; or, the target total score value corresponding to the target splitting method is greater than or equal to a first threshold.

[0012] In this implementation, the final output prediction result is used to indicate a target splitting method corresponding to the text to be processed. The prediction result is based on the matching degree between each target character set and the first label. The higher the matching degree between the target character set and the first label, the more accurately the first label can reflect the semantics of the target character set. Through the aforementioned scheme, at least one first label that can better reflect the intent of the text to be processed can be obtained.

[0013] In one possible implementation of the first aspect, the execution device processes the text to be processed using a target model to obtain a prediction result, including: the execution device matching each target character set with multiple strings in a first data set to determine the target string matching each target character set, and obtaining at least one second label corresponding to the target string from the first data set, where a string includes at least one character. Based on each target character set and at least one second label, the execution device matches each target character set with multiple labels in the first data set using a first model in the target model to obtain a first label matching each target character set. Further, based on each target character set and at least one second label, the training device can generate a matching degree between each target character set and each label in the first data set using the target model to obtain multiple second scores corresponding to each target character set, where one of the multiple second scores indicates the matching degree between each target character set and a label in the first data set; the training device determines the first label matching each target character set using the target model based on the multiple second scores.

[0014] In this embodiment, during the generation of the first tag matching the target character set, the target character set is matched with multiple strings in the first data set to determine at least one target string that matches the target character set. This then yields at least one second tag corresponding to the at least one target string. The target model then matches the target character set with multiple tags in the first data set to obtain the first tag matching the target character set. This means that when the strings matching the second tag in the first database are updated, the target model does not need to be retrained, avoiding the time cost of retraining the target model. Furthermore, it allows for timely modification of the strings in the first data set to adapt to changes in the actual application scenario, thus improving the accuracy of understanding the intent of the text being processed.

[0015] In one possible implementation of the first aspect, when at least one second label includes at least two second labels, after the executing device obtains at least one second label corresponding to the target string from the first data set, the method further includes: the executing device generating target indication information based on the text to be processed, the target character set, and at least two second labels, through a second model in the target model. The target indication information is used to indicate whether each second label matches or does not match the target character set. Specifically, the second model can be a binary classification model, where a first value indicates that the second label matches the target character set, and a second value indicates that the second label does not match the target character set. The executing device filters the at least two second labels based on the target indication information to obtain at least one filtered label. The at least one filtered label may include at least one second label, or, if the target indication information indicates that all second labels do not match the target character set, the at least one filtered label may include an empty label. The execution device matches the target character set with multiple tags in the first data set using a target model based on the target character set and the second tag. This includes: the execution device matching the target character set with multiple tags in the first data set using a target model based on the target character set and at least one filtered tag.

[0016] In this implementation, after obtaining at least one second tag corresponding to the target character set, the at least one second tag is further filtered according to the entire text to be processed to ensure that at least one filtered tag can satisfy the context of the entire text to be processed, thereby reducing the probability of matching the wrong second tag and improving the accuracy of understanding the intent of the text to be processed.

[0017] In one possible implementation of the first aspect, the target character set can be split into a first character and a second character. The execution device can obtain the first tag corresponding to the first character and the first tag corresponding to the second character respectively. Based on the first tag corresponding to the first character, the first tag corresponding to the second character, the target character set, and at least one second tag matched by the target character set, the first tag matching the target character set is determined.

[0018] In one possible implementation of the first aspect, after generating multiple second scores corresponding to the target character set based on the target character set and at least one second label using a target model, the execution device can obtain a sixth label corresponding to the second score with the highest value among the multiple second scores. The execution device determines whether the sixth label is the parent label of the first label corresponding to the first character based on a first data set, and / or determines whether the sixth label is the parent label of the first label corresponding to the second character based on the first data set. If the determination result is yes, the second score with the highest value among the multiple second scores is determined as the first score, and the sixth label is determined as the first label corresponding to the target character set. If the determination result is no, the execution device obtains a seventh label corresponding to the second highest value among the multiple second scores, and continues to perform the aforementioned operation on the second highest value among the multiple second scores until the first label corresponding to the target character set is obtained. Here, the first label corresponding to the target character set is the parent label of the first label corresponding to the first character, and / or, the first label corresponding to the target character set is the parent label of the first label corresponding to the second character.

[0019] In one possible implementation of the first aspect, the execution device can input a first label corresponding to a first character, a first label corresponding to a second character, a target character set, and at least one second label matched by the target character set into a first model of the target model. The first model in the target model generates a matching degree between the target character set and each label in the first data set, thereby obtaining multiple second scores corresponding to the target character set. The execution device can determine the second score with the highest value among the multiple second scores as the first score, and determine a fifth label corresponding to the second score with the highest value as the first label corresponding to the target character set.

[0020] Secondly, embodiments of this application provide a method for training a neural network, applicable to the field of text data processing in artificial intelligence. The method may include: a training device processing a target model to obtain a prediction result, wherein the text to be processed includes one or more characters, the prediction result indicating that the text to be processed is split into multiple first character sets, each first character set including at least one character, and the prediction result also including multiple first labels, one first label indicating the semantics of a target character set, and multiple first labels used to determine the predicted intent of the text to be processed; the training device training the target model according to a target loss function to obtain a trained target model. The target loss function indicates the similarity between the prediction result and the expected result corresponding to the text to be processed, the expected result corresponding to the text to be processed indicating that the text to be processed is split into multiple second character sets, each second character set including at least one character, and the expected result corresponding to the text to be processed also including multiple expected labels, one expected label indicating the semantics of a second character set, and multiple expected labels used to determine the correct intent of the text to be processed.

[0021] In one possible implementation of the second aspect, before the training device processes the text to be processed using the target model, the method further includes: the training device acquiring a target data subset, which includes a first subset and a second subset. The first subset includes a first string and a first expected label corresponding to the first string, and the second subset includes a second string and a second expected label corresponding to the second string. Based on the target data subset, the training device determines the text to be processed and the expected result corresponding to the text to be processed, wherein the text to be processed includes the first string and the second string, and the expected result includes the first expected label corresponding to the first string and the second expected label corresponding to the second string.

[0022] In this implementation, the text to be processed for training and the expected result corresponding to the text to be processed are generated in reverse from the target data subset. This ensures the divisibility of the text to be processed and the accuracy of the expected result corresponding to the text to be processed. In other words, it ensures that the text to be processed for training conforms to the processing logic of the target model, which is conducive to improving the fit between the text to be processed for training and the target model, and thus conducive to improving the accuracy of the target model after training.

[0023] In one possible implementation of the second aspect, a third score corresponding to the text to be processed satisfies a preset condition, and the third score is used to indicate the quality of the text to be processed. The preset condition can be that the third score is greater than or equal to a first preset threshold, or it can be that the third score is less than or equal to a second preset threshold. The specific form of the preset condition can be determined in conjunction with the specific generation method of the third score. Indicators used to evaluate the quality of the text to be processed include, but are not limited to, the fluency of candidate training texts or other types of evaluation indicators.

[0024] In this implementation, since the user inputs relatively fluent text into the target model during actual application, the third score corresponding to the text to be processed for training meets the preset conditions, thereby ensuring that the text to be processed for training is high-quality text. That is, the text to be processed for training is similar to the user's actual language habits, which is conducive to improving the quality of training data and also conducive to improving the accuracy of the target model after training.

[0025] The training device provided in the second aspect of the embodiments of this application can also execute the steps of the execution device in the various possible implementations of the first aspect. For the meaning of the terms, the specific implementation steps of the steps and the beneficial effects brought by each possible implementation in the second aspect of the embodiments of this application and the various possible implementations of the second aspect, please refer to the description in the various possible implementations of the first aspect, which will not be repeated here.

[0026] Thirdly, embodiments of this application provide a method for processing text data, applicable to the field of text data processing in the field of artificial intelligence. An electronic device acquires a target data subset, which includes a first subset and a second subset. The first subset includes a first string and a first expected label corresponding to the first string, and the second subset includes a second string and a second expected label corresponding to the second string. Based on the target data subset, the electronic device determines a second data set corresponding to a target model, wherein the target model is used for semantic understanding of the input text to be processed, and the second data set includes multiple target data subsets, used in the training phase and / or inference phase of the target model. The target data subset is used to indicate the expected result corresponding to the text to be processed, and the expected result is used to indicate splitting the text to be processed into at least two sets of strings, each set including the first string and the second string. The correct result also includes the first expected label corresponding to the first string and the second expected label corresponding to the second string.

[0027] In one possible implementation of the third aspect, the multiple first tags include at least two-level tags, each including a parent tag and child tags, with a hierarchical relationship between the parent and child tags. The method may further include: an electronic device obtaining a first modification operation corresponding to a subset of target data through a display interface; the first modification operation includes any one or more of the following operations: adding a parent tag, adding a child tag, deleting a parent tag, deleting a child tag, modifying tag content, or other operations.

[0028] In one possible implementation of the third aspect, the method may further include: the electronic device obtaining a second modification operation corresponding to the target data subset through a display interface, the second modification operation including any one or more of the following operations: deleting a string in the target data subset, updating the content of a string in the target data subset, or adding a new string to the target data subset.

[0029] In this implementation, users can easily modify the strings in the target data subset in a timely manner by combining the display interface with changes in the actual application scenario, which helps to improve the accuracy of understanding the intent of the text to be processed.

[0030] In one possible implementation of the third aspect, the method further includes: an electronic device acquiring a text to be processed, the text to be processed including multiple characters; processing the text to be processed through a target model to obtain a prediction result, wherein the prediction result is used to indicate splitting the text to be processed into multiple target character sets, each target character set including at least one character, and the prediction result also includes multiple first labels, one first label being used to indicate the semantics of a target character set, and multiple first labels being used to determine the intent of the text to be processed.

[0031] The electronic device provided in the third aspect of the embodiments of this application can also perform the steps of executing the device in the various possible implementations of the first aspect. For the meaning of the terms, the specific implementation steps of the steps, and the beneficial effects brought about by each possible implementation in the third aspect of the embodiments of this application, please refer to the description in the various possible implementations of the first aspect, which will not be repeated here.

[0032] Fourthly, embodiments of this application provide a text data processing apparatus, which can be used in the field of text data processing within the field of artificial intelligence. The text data processing apparatus may include: an acquisition module for acquiring text to be processed, the text to be processed including multiple characters; and a processing module for processing the text to be processed using a target model to obtain a prediction result, wherein the prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes multiple first labels, one first label indicating the semantics of a target character set, and multiple first labels used to determine the intent of the text to be processed.

[0033] The text data processing apparatus provided in the fourth aspect of the embodiments of this application can also perform the steps executed by the execution device in the various possible implementations of the first aspect. For the specific implementation steps of the fourth aspect of the embodiments of this application and the various possible implementations of the fourth aspect, as well as the beneficial effects brought about by each possible implementation, please refer to the description in the various possible implementations of the first aspect, which will not be repeated here.

[0034] Fifthly, embodiments of this application provide a neural network training apparatus that can be used in the field of text data processing within the field of artificial intelligence. The neural network training apparatus may include: a processing module, configured to process text to be processed using a target model to obtain a prediction result; the text to be processed includes one or more characters; the prediction result indicates that the text to be processed should be split into multiple first character sets, each first character set including at least one character; the prediction result also includes multiple first labels, one first label indicating the semantics of a target character set, and multiple first labels used to determine the predicted intent of the text to be processed; and a training module, configured to train the target model according to a target loss function to obtain a trained target model. The target loss function indicates the similarity between the prediction result and the expected result corresponding to the text to be processed; the expected result corresponding to the text to be processed indicates that the text to be processed should be split into multiple second character sets, each second character set including at least one character; the expected result corresponding to the text to be processed also includes multiple expected labels, one expected label indicating the semantics of a second character set, and multiple expected labels used to determine the correct intent of the text to be processed.

[0035] The neural network training apparatus provided in the fifth aspect of the present application can also execute the steps executed by the training device in the various possible implementations of the second aspect. For the specific implementation steps of the fifth aspect and the various possible implementations of the fifth aspect, as well as the beneficial effects brought about by each possible implementation, please refer to the description in the various possible implementations of the second aspect, which will not be repeated here.

[0036] Sixthly, embodiments of this application provide a text data processing apparatus, which can be used in the field of text data processing in the field of artificial intelligence. The text data processing apparatus may include: an acquisition module, configured to acquire a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, and the second subset including a second string and a second expected label corresponding to the second string; and a processing module, configured to determine a second data set corresponding to a target model based on the target data subset, wherein the target model is used for semantic understanding of the input text to be processed, the second data set including multiple target data subsets, and the second data set being used in the training phase and / or the inference phase of the target model. The target data subset is used to indicate the expected result corresponding to the text to be processed, the expected result being used to indicate splitting the text to be processed into at least two sets of strings, the at least two sets of strings including the first string and the second string, and the correct result also including the first expected label corresponding to the first string and the second expected label corresponding to the second string.

[0037] The text data processing apparatus provided in the sixth aspect of the embodiments of this application can also perform the steps executed by the execution device in the various possible implementations of the third aspect. For the specific implementation steps of the sixth aspect of the embodiments of this application and the various possible implementations of the sixth aspect, as well as the beneficial effects brought about by each possible implementation, refer to the description in the various possible implementations of the third aspect, which will not be repeated here.

[0038] In a seventh aspect, embodiments of this application provide a computer program product that, when the computer program is run on a computer, causes the computer to perform the text data processing method described in the first or third aspect above, or causes the computer to perform the neural network training method described in the second aspect above.

[0039] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the text data processing method described in the first or third aspect, or to perform the neural network training method described in the second aspect.

[0040] Ninthly, embodiments of this application provide an execution device that may include a processor and a memory coupled together. The memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the text data processing method described in the first aspect is implemented.

[0041] In a tenth aspect, embodiments of this application provide an electronic device that may include a processor and a memory coupled together. The memory stores program instructions, which, when executed by the processor, implement the neural network training method described in the second aspect above, or, when executed by the processor, implement the text data processing method described in the third aspect above.

[0042] Eleventhly, embodiments of this application provide a circuit system including a processing circuit, the processing circuit being configured to perform the text data processing method described in the first or third aspect above, or the processing circuit being configured to perform the neural network training method described in the second aspect above.

[0043] In a twelfth aspect, embodiments of this application provide a chip system including a processor for implementing the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for a server or communication device. This chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0044] Figure 1 A schematic diagram of the main framework of artificial intelligence provided in the embodiments of this application;

[0045] Figure 2 A system architecture diagram of a text data processing system provided in this application embodiment;

[0046] Figure 3 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0047] Figure 4 A schematic diagram of an interface for obtaining a subset of target data in a neural network training method provided in an embodiment of this application;

[0048] Figure 5 A schematic diagram of an interface for obtaining the first modification operation in the neural network training method provided in the embodiments of this application;

[0049] Figure 6 A schematic diagram of an interface for obtaining a second modification operation in the neural network training method provided in the embodiments of this application;

[0050] Figure 7 A schematic diagram of the prediction result in the training method of the neural network provided in the embodiments of this application;

[0051] Figure 8A schematic diagram of a process for generating multiple second scores corresponding to a first character set in the neural network training method provided in this application embodiment;

[0052] Figure 9 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0053] Figure 10 A flowchart illustrating a text data processing method provided in an embodiment of this application;

[0054] Figure 11 A flowchart illustrating a text data processing method provided in an embodiment of this application;

[0055] Figure 12 A schematic diagram of a text data processing apparatus provided in an embodiment of this application;

[0056] Figure 13 Another schematic diagram of the structure of the text data processing apparatus provided in the embodiments of this application;

[0057] Figure 14 A schematic diagram of a neural network training device provided in an embodiment of this application;

[0058] Figure 15 Another schematic diagram of the neural network training device provided in the embodiments of this application;

[0059] Figure 16 A schematic diagram of a text data processing apparatus provided in an embodiment of this application;

[0060] Figure 17 A schematic diagram of the structure of the execution device provided in the embodiments of this application;

[0061] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0062] Figure 19 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0063] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0064] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0065] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1 , Figure 1 The diagram illustrates a structural framework for artificial intelligence (AI). The framework is further elaborated below along two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that AI brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed through technological means) to the industrial ecosystem of the system.

[0066] (1) Infrastructure

[0067] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0068] (2) Data

[0069] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0070] (3) Data processing

[0071] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0072] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training of data by symbolizing and formalizing it.

[0073] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0074] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0075] (4) General ability

[0076] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0077] (5) Smart Products and Industry Applications

[0078] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, autonomous driving, and smart cities.

[0079] The embodiments of this application can be applied to natural language processing (NLP) in the field of artificial intelligence, specifically to various application scenarios requiring natural language understanding (NLU). For example, in the field of smart terminals, a user might input "Book a movie ticket for this afternoon" into their mobile phone, requiring the phone to perform semantic understanding on this text. As another example, in the field of smart homes, a user might input "Are there any recommended dinner recipes?" into their smart refrigerator, requiring the refrigerator to perform semantic understanding on this text.

[0080] As another example, in the field of autonomous driving, a user could input "Check the location of Xiaoming's phone number" into an autonomous vehicle. The autonomous vehicle would then need to perform semantic understanding on this text, and so on. The application scenarios of this application are not exhaustively listed here. In all the aforementioned application areas, artificial intelligence technology is required for natural language understanding; that is, in all these application areas, a more generalizable natural language understanding method is urgently needed.

[0081] Before providing a detailed description of the text data processing method provided in the embodiments of this application, let's first combine... Figure 2 This application describes the text data processing system provided in its embodiments. Please refer to [the relevant documentation] first. Figure 2 , Figure 2 A system architecture diagram of the text data processing system provided in the embodiments of this application is shown. Figure 2 In the text data processing system 200, there are execution devices 210, training devices 220, databases 230 and data storage systems 240, and the execution devices 210 are equipped with computing modules 211.

[0082] The database 230 stores a training data set, which includes multiple texts to be processed and the expected result corresponding to each text to be processed. The expected result is used to indicate that the text to be processed is split into at least one second character set, and each second character set includes at least one character. The expected result also includes at least one expected label corresponding to at least one second character set, and the at least one expected label is used to indicate the correct intent of the text to be processed.

[0083] Training device 220 generates target model / rule 201 and iteratively trains target model / rule 201 using training data set in database 230 to obtain trained target model / rule 201, which can also be called mature target model / rule 201. Furthermore, target model / rule 201 can be implemented using either a neural network or a non-neural network model; this embodiment does not impose any limitations.

[0084] The trained target model / rule 201 obtained by training device 220 can be applied to different systems or devices, such as mobile phones, tablets, laptops, virtual reality (VR) devices, intelligent robots, autonomous vehicles, or other types of devices. Execution device 210 can access data, code, etc., in data storage system 240, and can also store data, instructions, etc., in data storage system 240. Data storage system 240 can be located within execution device 210, or it can be an external storage device relative to execution device 210.

[0085] The calculation module 211 can perform semantic understanding on the input text to be processed through the target model / rule 201 to obtain the prediction result output by the target model / rule 201; wherein, the prediction result is used to indicate that the text to be processed is split into at least one target character set, each target character set includes at least one character, and the prediction result also includes at least one first label corresponding to at least one target character set, the at least one first label being used to reflect the intent of the text to be processed.

[0086] In this embodiment, since the characters in the text to be processed are matched with tags, a target character set may not include reduplicated characters or modal particles. That is, reduplicated characters or modal particles can be split into another target character set. Thus, even if reduplicated characters or modal particles exist in the entire text to be processed, the intent of the text to be processed can still be understood, providing a natural language understanding method with more generalization ability. Moreover, since the prediction result indicates how to split the entire text to be processed and also includes at least one tag corresponding to the text to be processed, the prediction result carries richer information, which is beneficial to improving the accuracy of the intent understanding process of the text to be processed.

[0087] In some embodiments of this application, please refer to Figure 2 The "user" can directly interact with the execution device 210, meaning the execution device 210 can directly display the prediction results output by the target model / rule 201 to the "user." It is worth noting that... Figure 2 This is merely a schematic diagram of one architecture of the text data processing system provided in this embodiment of the invention. The positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in some other embodiments of this application, the execution device 210 and the client device can also be separate independent devices. The execution device 210 is configured with an input / output (in / out, I / O) interface, and the execution device 210 interacts with the client device through the I / O interface.

[0088] Based on the above description, the specific implementation process of the training and inference stages of the text data processing method provided in the embodiments of this application will now be described.

[0089] I. Training Phase

[0090] In this embodiment, the training phase describes how the training device 220 uses the image data set in the database 230 to generate a mature neural network. For details, please refer to [link to specific documentation]. Figure 3 , Figure 3 This is a flowchart illustrating a neural network training method provided in an embodiment of this application. The neural network training method provided in an embodiment of this application may include:

[0091] 301. The training device acquires a target data subset, which includes at least one level of target label and a third string corresponding to the target label. The target label is used to indicate the semantics of the third string.

[0092] In some embodiments of the present application, before the training device trains the target model, multiple target data subsets can be obtained. Each target data subset includes at least one level of target labels, and the target data subset further includes a third string corresponding to the target label. The target label is used to indicate the semantics of the third string.

[0093] Among them, the target model is used to perform semantic understanding on the input text to be processed. The multiple target data subsets obtained can be used in the training stage and / or the inference stage of the target model. Further, the multiple target data subsets can be understood as "the multiple target data subsets obtained can be used to generate the training data of the target model".

[0094] A third string includes at least one character. The third string can specifically be a slot word, a conjunction, a modal particle or other types of words, etc.; the third string can be Chinese, English, Russian or other languages, etc., and should be flexibly determined in combination with the actual application scenario and is not limited here. As an example, for example, the third string can be "query", "check" or "look", etc.; as another example, for example, the third string can be "of", "and" or "with", etc.; as another example, for example, the third string can be "ah", "bar" or "la", etc.; as another example, for example, the third string can be "party" or "books", etc., and no exhaustive list is made here.

[0095] Further, in some cases, all the at least one target labels included in a target data subset are of the same level, and the target data subset further includes a third string corresponding to each target label. As an example, for example, a target data subset only includes the target label {query time}, and the third string corresponding to the target label {query time} can be "What time is it now?", "What time is it", "Check the time" or other strings, etc. It should be understood that the examples here are only for facilitating the understanding of what data can be included in a target data subset and are not used to limit the present solution.

[0096] In other cases, a target data subset includes at least two levels of labels, then the target data subset also needs to indicate the attribution relationship between different levels of labels; the at least two levels of target labels include a third label and a fourth label, and the target data subset further includes a third string corresponding to the third label. The third label is used to reflect the intention of the third string, and the fourth label is the parent label of the third label, that is, the third label is the sub-label of the fourth label.

[0097] As an example, for example, a target data subset can specifically be expressed as the following content:

[0098] IN_#{Query location} -> #{Query number} #{Location}

[0099] #{Search Number}->#{Search}#{Phone Number}

[0100] #{Query}->Query | Check | Ask | Take a look | ...

[0101] #{phone number}->phone number|mobile number|mobile number|number|…

[0102] #{Place of Origin}->Where is it from? |Where is it from? |Where is it from? |…

[0103] In this example, "->" represents the hierarchical relationship between different levels of tags, and "#{}" represents the tag name. The example above uses a target data subset containing three levels of tags: the tag "location query" is the parent tag of both "query number" and "location," and the tag "query number" is the parent tag of both "query" and "phone number." The third tag refers to the lowest level tag. The slot words "query," "check," ... "phone number," "mobile number," ... "where," and "where" shown in the example above are examples of multiple third strings included in the target data subset. The third tag reflects the intent of the third string. For example, the intent of the four third strings "query," "check," "ask," and "take a look" is all "query." That is, the target data subset also indicates the hierarchical relationship between different levels of tags. It should be understood that the examples here are only for ease of understanding what data a target data subset can include and are not intended to limit this scheme.

[0104] Optionally, if the target model is specifically designed to process text data within the target data domain, then each subset of target data can consist entirely of data within the target data domain. For example, if the target model is configured in a tutoring robot, meaning it's used for semantic understanding of text in the education domain, then the target data domain would be data from the education domain. It should be understood that this example is merely for illustrative purposes and is not intended to limit the scope of this solution.

[0105] The training device can also acquire multiple first data subsets. The concept of a first data subset is similar to that of a target data subset, except that the multiple third strings included in the target data subset all belong to the target data domain, while the multiple third strings included in the first data subset are not data from the target data domain. The third label corresponding to the third string in the first data subset can be the type of the third string, the type of the data domain in which the third string belongs, or other types of labels, etc. For example, the third label in the first data subset can be "noun," "verb," ​​or other labels, etc., but this is not exhaustive.

[0106] Regarding the acquisition method of the "target data subset", in one implementation, the training device can receive multiple target data subsets sent by other communication devices and store the received target data subsets locally.

[0107] In another implementation, the training device can be configured with a display interface, through which it can receive at least one target data subset input by the user; that is, the user can use the display interface to set several levels of labels and the attribution relationships between different levels of labels in a target data subset, and can also use the display interface to input the content of each label and the content of multiple third strings.

[0108] In another implementation, the training device can receive multiple subsets of target data sent by other communication devices, as well as at least one subset of target data input by the user through a display interface.

[0109] Optionally, the training device can also receive user settings for label categories through the display interface. The label category is used to indicate whether a label is a required label or an optional label.

[0110] Optionally, the training device can also receive user modification operations on a target subset of data through the display interface. These modification operations may include a first modification operation on a subset of data and / or a second modification operation on a third string. The first modification operation includes, but is not limited to, adding a parent label, adding a child label, deleting a parent label, deleting a child label, modifying label content, or other operations. The second modification operation includes, but is not limited to, adding, deleting, modifying the content of the third string, or other operations. The specific modifications can be flexibly configured according to the actual product form.

[0111] In this implementation, users can easily modify the strings in the target data subset in a timely manner by combining the display interface with changes in the actual application scenario, which helps to improve the accuracy of understanding the intent of the text to be processed.

[0112] For a more intuitive understanding of this solution, please refer to [link / reference]. Figures 4 to 6 , Figure 4 This is a schematic diagram of an interface for obtaining a subset of target data in the neural network training method provided in this application embodiment. (See also...) Figure 4 , Figure 4 The information displayed on the screen represents a subset of target data input by the user, such as... Figure 4As shown, a target data subset includes three levels of labels: the first level label is "location query"; the second level labels are "query number" and "location", with "location query" being the parent label of "query number" and "location"; and the third level labels are "query" and "number", with "query number" being the parent label of "query" and "number".

[0113] Figure 4 The document also indicates that the categories "Search Number," "Search," "Number," and "Location" are mandatory (i.e., ...). Figure 4 (Regarding the "Y" in the text), it should be noted that users can modify it. Figure 4 The content of the table is used to modify the category of a label; this should be understood. Figure 4 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0114] Continue reading Figure 5 , Figure 5 A schematic diagram of an interface for obtaining the first modification operation in the neural network training method provided in this application embodiment, combined with Figure 4 understand Figure 5 Users can delete the icon next to each label (i.e., Figure 5 The user can enter the input field (the trash can icon) to delete a label. To add a second-level label, click A1; to add a third-level label, click A2; or simply click the label text to modify its content. This should be understood... Figure 5 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0115] See also Figure 6 , Figure 6 A schematic diagram of an interface for obtaining the second modification operation in the neural network training method provided in this application embodiment, combined with Figure 4 and Figure 5 understand Figure 6 Users can add to the third string by clicking B1; they can delete from the third string by clicking B2; users can also directly click on the text of the third string to trigger modification operations. It should be understood that... Figure 6 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0116] 302. The training device acquires the text to be processed and the expected result corresponding to the text to be processed. The expected result is used to indicate that the text to be processed is split into at least one second character set, each second character set including at least one character. The expected result corresponding to the text to be processed also includes at least one expected label corresponding to at least one second character set. The at least one expected label is used to indicate the correct intent of the text to be processed.

[0117] In this embodiment of the application, before training the target model, the training device needs to obtain the text to be processed and the expected result corresponding to the text to be processed (that is, obtain training data).

[0118] Specifically, step 301 is an optional step. In one implementation, if step 301 is executed, and the multiple target data subsets obtained in step 301 are used to generate training data for the target model, then the training device can generate at least one text to be processed and the expected result corresponding to each text to be processed based on any one of the multiple target data subsets.

[0119] More specifically, in one implementation, a third score corresponding to the text to be processed used to train the target model satisfies preset conditions, and the third score is used to indicate the quality of the text to be processed.

[0120] The preset condition can be that the third score is greater than or equal to the first preset threshold, or it can be that the third score is less than or equal to the second preset threshold. The specific form of the preset condition can be determined in conjunction with the specific generation method of the third score. Indicators used to evaluate the quality of the text to be processed include, but are not limited to, the fluency of the candidate training text or other types of evaluation indicators.

[0121] In other words, after generating a candidate training text based on a subset of target data, the training device can also generate a third score corresponding to the candidate training text. The third score is used to indicate the quality of the candidate training text. The training device determines whether the third score corresponding to the candidate training text meets preset conditions. If the determination result is yes, the candidate training text is determined as the text to be processed for training the target model, and the expected result corresponding to the text to be processed is obtained based on the subset of target data. If the determination result is no, the candidate training text can be deleted, and the next candidate training text can be generated.

[0122] Furthermore, the training device can input candidate training texts into the first model to calculate the perplexity loss corresponding to the candidate training text, thereby obtaining the third score output by the first model corresponding to the candidate training text. As an example, the training device may generate six candidate training texts based on a subset of target data and obtain the third score for each candidate training text, as shown in the table below:

[0123] Candidate training texts Third rating Candidate training texts Third rating I want to tell you about picture books. 4.321032165179162 I want to tell you a picture book. 11.43761551051832 Take a look at this story 2.440600608831472 Reading together 14.794469739236522 You want to read this story with me? 6.343048812278409 We want to read a picture book together. 7.786327851748459

[0124] Table 1

[0125] Table 1 shows six third scores corresponding to six candidate training texts. Each third score represents the perplexity loss of that candidate training text. The smaller the third score of a candidate training text, the higher the quality of that candidate training text. Thus, a filtering operation can be performed based on the third score of each candidate training text. It should be understood that the examples in Table 1 are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0126] In this embodiment of the application, since the text input by the user to the target model is relatively fluent in the actual application of the target model, the third score corresponding to the text to be processed for training meets the preset conditions, thereby ensuring that the text to be processed for training is high-quality text, that is, the text to be processed for training is similar to the user's actual language habits, which is conducive to improving the quality of training data and also conducive to improving the accuracy of the target model after training.

[0127] In another implementation, the training device can directly generate the text to be processed for training the model based on the target data subset and obtain the expected result corresponding to the text to be processed. That is, each candidate training text is determined as the text to be processed for training the model, and the filtering operation is no longer performed based on the fluency of the candidate training text.

[0128] This describes the process of generating a text to be processed (or candidate training text) based on a subset of target data. Referring to the description in step 301, a subset of target data may include at least one level of target labels, and the subset of target data may also include a third string corresponding to the target labels, whereby the target labels are used to indicate the semantics of the third string.

[0129] Specifically, in one scenario, the target data subset contains at least two third tags, including a first expected tag and a second expected tag. The target data subset may include at least two target subsets, each corresponding to one of the at least two third tags. Each target subset includes a third tag and at least one third string corresponding to that third tag. For example, a target subset may include a third tag "query" and multiple third strings corresponding to the tag "query," including but not limited to "query," "check," "ask," or "take a look," etc., without exhaustive list.

[0130] At least two target subsets may include a first subset corresponding to a first expected label and a second subset corresponding to a second expected label. The first subset includes a first string and a first expected label corresponding to the first string, and the second subset includes a second string and a second expected label corresponding to the second string.

[0131] A text to be processed (or a candidate training text) may include a first string and a second string, and the expected result corresponding to the aforementioned text to be processed (or candidate training text) includes a first expected label corresponding to the first string and a second expected label corresponding to the second string.

[0132] As an example, a target data subset may include three target subsets corresponding to three third labels. The first subset includes the label "query" and the four strings "query", "check", "ask", and "take a look" corresponding to the label "query". The second subset includes the label "phone number" and the four strings "phone number", "mobile number", "mobile number", and "number" corresponding to the label "phone number". The third subset includes the label "location" and the four strings "where", "from where", "where", and "where" corresponding to the label "location".

[0133] The text to be processed based on the above target subset can be "Look up where the phone number is from", "Look up where the phone number is from", "Ask where the mobile phone number is from", or other forms of text to be processed. The expected result corresponding to the text to be processed includes three expected labels: "look up", "phone number", and "location". It should be understood that the example here is only for the convenience of understanding this solution and is not intended to limit this solution.

[0134] In this embodiment, the text to be processed for training and the expected result corresponding to the text to be processed are generated in reverse based on the target data subset. This ensures the divisibility of the text to be processed and the accuracy of the expected result corresponding to the text to be processed. In other words, it ensures that the text to be processed for training conforms to the processing logic of the target model, which is conducive to improving the fit between the text to be processed for training and the target model, and thus conducive to improving the accuracy of the target model after training.

[0135] In another scenario, if there is only one third label in the target data subset, the training device can select any one of the third strings corresponding to that target label as the text to be processed (or candidate training text) and use that target label as the expected result corresponding to that text to be processed (or candidate training text).

[0136] In another implementation, if step 301 is not executed, or if step 301 is executed but the multiple subsets of target data obtained in step 301 are not used to generate training data for the target model, then the training device may also pre-store a training data set, which includes multiple texts to be processed and the expected result corresponding to each text. The training device obtains the texts to be processed and the expected results corresponding to each text from the training data set.

[0137] 303. The training device performs semantic understanding on the text to be processed through the target model and obtains the prediction result output by the target model. The prediction result is used to indicate that the text to be processed is split into at least one first character set, each first character set includes at least one character, and the prediction result also includes at least one first label corresponding to at least one first character set. At least one first label is used to reflect the prediction intent of the text to be processed.

[0138] In this embodiment of the application, after the training device acquires the text to be processed, it performs semantic understanding of the text to be processed through the target model and obtains the prediction result output by the target model.

[0139] The prediction result is used to indicate that the text to be processed is split into at least one first character set, each first character set including at least one character. The prediction result also includes at least one first label corresponding to at least one first character set, and the at least one first label is used to reflect the prediction intent of the text to be processed.

[0140] Furthermore, at least one first tag may include only one level tag or at least two level tags. When at least one first tag includes at least two level tags, the at least two level tags may include parent tags and child tags. The prediction result is also used to indicate the affiliation relationship between parent tags and child tags. That is, the prediction result is also used to indicate the number of levels corresponding to the at least two level tags, which level each first tag belongs to, and the correspondence between different tags.

[0141] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of the prediction result in the training method of the neural network provided in the embodiments of this application. Figure 7 Taking the use of a tree structure to display the prediction results of the text to be processed as an example, such as... Figure 7 As shown, the text to be processed is "Check where the phone number is from". The prediction result of the text to be processed is used to show how the text to be processed is split into four sets of first characters: "Check", "phone number", "is", and "where". Figure 7 As shown, the prediction results for the text to be processed also include five primary labels: "location query," "query number," "location," "query," and "phone number." The "location query" label represents the semantics of the entire text; the "query number" label represents the semantics of "querying a phone number"; the "location" label represents the semantics of "where is it from"; the "query" label represents the semantics of "query"; and the "phone number" label represents the semantics of "phone number." Therefore, based on these five primary labels, the intent of the entire text to be processed can be understood. Figure 7 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0142] In this embodiment of the application, the prediction result may include at least two levels of labels, which may include parent labels and child labels. The prediction result is also used to indicate the attribution relationship between parent labels and child labels. That is, the prediction result carries richer information, which is conducive to further improving the accuracy of the intent understanding process of the text to be processed.

[0143] There are N possible splitting methods for the text to be processed, where N is an integer greater than or equal to 1. The prediction result indicates a target splitting method for the text to be processed, which is one of the N splitting methods. The prediction result is based on the matching degree between the first character set and the first label. Furthermore, the prediction result is based on the first score corresponding to each first character set, which indicates the matching degree between the first character set and the first label.

[0144] Furthermore, since there is a first score corresponding to each first character set, there is at least one first score corresponding to the text to be processed. The training device can obtain the target total score value corresponding to the target splitting method. The target total score value corresponding to the target splitting method is the sum of at least one first score corresponding to the target splitting method. The target splitting method can be the splitting method with the highest target total score value among N splitting methods; or, the target total score value corresponding to the target splitting method is greater than or equal to the first threshold.

[0145] As an example, if the text to be processed is "call Xiaoming", then there are several ways to split the text, including splitting it into "call Xiaoming" and "call", another way is "call", "Xiaoming" and "call", another way is "call Xiaoming", "call" and "call", another way is "call Xiaoming", "call" and "call", and so on. It should be understood that the examples here are only for the convenience of understanding the concept of splitting the text to be processed, and are not intended to exhaustively list all the ways to split the text to be processed.

[0146] In this embodiment of the application, the final output prediction result is used to indicate a target segmentation method corresponding to the text to be processed. The prediction result is obtained based on the first score corresponding to each first character set. The first score corresponding to the first character set is used to indicate the matching degree between the first character set and the first tag. The higher the matching degree between the first character set and the first tag, the more accurately the first tag can reflect the semantics of the first character set. Through the aforementioned scheme, at least one first tag that can better reflect the intent of the text to be processed can be obtained.

[0147] Specifically, since the text to be processed may be split into any form, the training device can extract the target string from the text, generate a first label that matches the target string, and obtain the first score of the first label that matches the target string. Here, the target string is a string of arbitrary length in the text to be processed. The training device can repeatedly perform the aforementioned operations to generate a first label and a first score that match all target strings corresponding to the text to be processed, thereby determining the prediction result corresponding to the text to be processed.

[0148] As an example, for instance, if the text to be processed is "What is Xiaoming's phone number", then the target string can be any one of the following: "Xiaoming", "Ming", "of", "phone", "number", "is", "how", "many", "Xiaoming", "Ming's", "of phone", "phone number", "number is", "is how", "how many", "Xiaoming's", "Ming's phone", "of phone number", "phone number", "number is", "number is how", "is how many", "Xiaoming's phone", "Ming's phone number", "of phone number", "phone number", "number is", "number is how", "is how many", "Xiaoming's phone number", "Ming's phone number", "Xiaoming's phone number", or "What is Xiaoming's phone number". For the omitted part, you can refer to the example part for understanding, and it will not be enumerated here.

[0149] Furthermore, regarding the process of obtaining the first label that matches a first character set (i.e., an example of the target string). The training device can match the first character set with multiple strings in the first data set to determine at least one target string that matches the first character set, and a string includes at least one character. The training device obtains at least one second label corresponding to at least one target string from the first data set; or, if there is no string in the first data set that matches the first character set, the training device can obtain an empty label corresponding to at least one target string.

[0150] Based on the first character set and at least one second label (or an empty label), the training device can match the first character set with multiple labels in the first data set through the target model to obtain the first label that matches the first character set. More specifically, based on the first character set and at least one second label (or an empty label), the training device can generate the matching degree between the first character set and each label in the first data set through the first model of the target model to obtain multiple second scores corresponding to the first character set, and a second score is used to indicate the matching degree between the first character set and a label in the first data set; the training device determines the first label that matches the first character set according to the multiple second scores through the target model, and obtains the first score of the first label that the first character set matches.

[0151] In this embodiment, during the generation of the first tag matching the first character set, the first character set is matched with multiple strings in the first data set to determine at least one target string that matches the first character set. This then yields at least one second tag corresponding to the at least one target string. The target model then matches the first character set with multiple tags in the first data set to obtain the first tag matching the first character set. This means that when the strings matching the second tag in the first database are updated, the target model does not need to be retrained, avoiding the time cost of retraining the target model. Furthermore, it allows for timely modification of the strings in the first data set to adapt to changes in the actual application scenario, thus improving the accuracy of understanding the intent of the text being processed.

[0152] The first data set may include multiple target data subsets, meaning that the target data subsets can be applied to the inference stage of the target model. Therefore, the meaning of "target string" is the same as the meaning of "third string" in step 301, and the meaning of "second label" is the same as the meaning of "third label" in step 301.

[0153] The first model may include an encoder and a decoder. As an example, the encoder may adopt a transformer structure, and the decoder may specifically adopt a multilayer perception (MLP).

[0154] Specifically, the training device can input a first character set and at least one second label (or an empty label) into the encoder to extract features and generate a vector representation corresponding to the first character set; the training device then inputs the vector representation corresponding to the first character set into the decoder to generate multiple second scores.

[0155] Optionally, after the training device obtains at least one second label corresponding to the first character set, after obtaining at least one second label corresponding to at least one target string from the first data set, the training device can also input the text to be processed, the first character set, and at least one second label into the second model in the target model to generate target indication information through the second model in the target model. The target indication information is used to indicate whether each second label matches or does not match the first character set. The training device filters at least one second label according to the target indication information to obtain at least one filtered label.

[0156] In this model, at least one filtered label can be represented as an empty label, and the second model can be represented as a binary classification model. When the target indication information is represented by the first value, it means the second label matches the first character set; when the target indication information is represented by the second value, it means the second label does not match the first character set. For example, the first value can be true, and the second value can be false; another example is that the first value can be 1, and the second value can be 0, etc. The specific representations of the first and second values ​​are not exhaustively listed here.

[0157] At least one filtered tag may include at least one second tag, or, if the target indication information indicates that all second tags do not match the first character set, at least one filtered tag may include an empty tag.

[0158] Correspondingly, the training device generates a matching degree between the first character set and each label in the first data set through the target model based on the first character set and at least one second label, so as to obtain multiple second scores corresponding to the first character set. This may include: the training device generating a matching degree between the first character set and each label in the first data set through the target model based on the first character set and at least one filtered label, so as to obtain multiple second scores corresponding to the first character set.

[0159] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 8 , Figure 8 This is a flowchart illustrating a method for generating multiple second scores corresponding to a first character set in a neural network training method provided in this application embodiment. D1. The training device matches the first character set with multiple strings in a first data set to determine at least one target string that matches the first character set, and obtains at least one second label corresponding to the at least one target string from the first data set. D2. Based on the text to be processed, the first character set, and at least two second labels, the training device generates target indication information through a target model. The target indication information is used to indicate whether each second label matches or does not match the first character set. D3. The training device filters at least two second labels based on the target indication information to obtain at least one filtered label. D4. Based on the first character set and at least one filtered label, the training device generates multiple second scores corresponding to the first character set through a target model. It should be understood that... Figure 8 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0160] In this embodiment of the application, after obtaining at least one second tag corresponding to the first character set, the at least one second tag is further filtered according to the entire text to be processed to ensure that at least one filtered tag can meet the context of the entire text to be processed, thereby reducing the probability of matching the wrong second tag and improving the accuracy of understanding the intent of the text to be processed.

[0161] Furthermore, the first character set can be split into a first character and a second character. For example, if the text to be processed is "find out where Xiaoming's phone number is from", and the first character set is "Xiaoming's phone number", then the first character can be "Xiaoming's" and the second character can be "phone number". It should be understood that this example is only for the convenience of understanding the relationship between the first character set, the first character, and the second character, and is not intended to limit this solution.

[0162] Optionally, the training device can obtain the first label corresponding to the first character and the first label corresponding to the second character, respectively. Based on the first label corresponding to the first character, the first label corresponding to the second character, the first character set, and at least one second label (or an empty label) matched by the first character set, the device determines the first label matching the first character set and obtains the first score of the first label matching the first character set. It should be noted that the first label corresponding to the first character and the first label corresponding to the second character can both be represented as empty labels or as labels with semantic meaning. The specific choice should be determined based on the actual application scenario, and no limitation is made here.

[0163] Specifically, in one implementation, the training device can input the first label corresponding to the first character, the first label corresponding to the second character, the first character set, and at least one second label (or an empty label) matched by the first character set into the first model of the target model. The first model in the target model generates the matching degree between the first character set and each label in the first data set, thus obtaining multiple second scores corresponding to the first character set. The training device can determine the second score with the highest value among the multiple second scores as the first score, and determine the fifth label corresponding to the highest value second score as the first label corresponding to the first character set.

[0164] In another implementation, after the training device generates multiple second scores corresponding to the first character set based on the first character set and the second label through the target model, it can obtain a sixth label corresponding to the second score with the highest value among the multiple second scores. The training device determines whether the sixth label is the parent label of the first label corresponding to the first character based on the first data set, and / or determines whether the sixth label is the parent label of the first label corresponding to the second character based on the first data set. If the determination result is yes, then the second score with the highest value among the multiple second scores is determined as the first score, and the sixth label is determined as the first label corresponding to the first character set.

[0165] If the judgment result is negative, the training device obtains a seventh label corresponding to the second highest value among multiple second scores. The training device determines whether the seventh label is the parent label of the first label corresponding to the first character based on the first data set, and / or determines whether the seventh label is the parent label of the first label corresponding to the second character based on the first data set. If the judgment result is positive, the second score with the highest value among multiple second scores is determined as the first score, and the seventh label is determined as the first label corresponding to the first character set.

[0166] If the judgment result is negative, the training device continues to perform the above operation on the third highest value among the multiple second scores until a first label corresponding to the first character set is obtained. Here, the first label corresponding to the first character set is the parent label of the first label corresponding to the first character, and / or, the first label corresponding to the first character set is the parent label of the first label corresponding to the second character.

[0167] 304. The training device trains the target model according to the target loss function, which is used to indicate the similarity between the predicted result and the expected result corresponding to the text to be processed.

[0168] In this embodiment, after the training device generates the prediction result corresponding to the text to be processed through the target model, it can generate the function value of the target loss function based on the prediction result and the expected result corresponding to the text to be processed, and update the weight parameters of the target model in reverse based on the function value of the target loss function to complete one training of the target model. The training device repeats steps 302 to 304 to complete the iterative training of the target model, thereby obtaining the trained target model, which can also be called the mature target model.

[0169] The target loss function is used to indicate the similarity between the predicted result and the expected result of the text to be processed. The meanings of the predicted and expected results can be found in steps 302 and 303 above, and will not be repeated here. For example, the target loss function can be an MLP loss function, a Hamming loss function, or other types of loss functions; however, no exhaustive list of specific forms of the target loss function will be provided here.

[0170] Furthermore, if the target model includes a first model and a second model, the second model can be a model that has undergone training. That is, during one training of the target model, the training device can update the weight parameters of the first model in the target model in reverse based on the function value of the target loss function, and no longer update the weight parameters of the second model in the target model.

[0171] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 9 , Figure 9 This is a flowchart illustrating a neural network training method provided in an embodiment of this application. In the data preparation stage, the training device acquires multiple target data subsets and obtains a first data set based on these subsets. The training device generates one or more candidate training texts based on each target data subset. The training device filters the one or more candidate training texts to obtain at least one text to be processed, and also generates the expected result corresponding to the aforementioned text to be processed based on the target data subsets. In the semantic understanding stage, the training device extracts a target string from the text to be processed and extracts at least one second label matching the target string from the first data set. Based on the target string and at least one second label, the training device generates a vector representation of the target string using the encoder in the target model. Based on the vector representation of the target string, the hierarchical relationships between labels at different levels in the first data set, and all third labels in the first data set, the training device generates a prediction result corresponding to the text to be processed using the decoder. The specific implementation of the aforementioned steps can be found in the descriptions of steps 301 to 303 above. It should be understood that... Figure 9 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0172] In this embodiment, a training method for the target model is provided, expanding the application scenarios of this solution. The trained target model performs semantic understanding on the text to be processed, obtaining the prediction result output by the target model. The prediction result indicates how to split the text to be processed into at least one first character set. The prediction result also includes a first label matching at least one first character set, so that the intent of the text to be processed can be reflected through all matched first labels. Since this solution matches labels to characters in the text to be processed, a first character set may not include reduplicated characters or modal particles, meaning that reduplicated characters or modal particles can be split into another first character set. Therefore, even if reduplicated characters or modal particles exist in the entire text to be processed, the intent of the text to be processed can still be understood, thus providing a more generalizable natural language understanding method. Furthermore, since the prediction result indicates how to split the entire text to be processed and also includes at least one label corresponding to the text to be processed, the prediction result carries richer information, which is beneficial to improving the accuracy of the intent understanding process of the text to be processed.

[0173] II. Reasoning Stage

[0174] In this embodiment, the inference phase describes how the execution device 210 uses the target model / rule 201 to perform semantic understanding to generate prediction results. For details, please refer to... Figure 10 , Figure 10 This is a flowchart illustrating a text data processing method provided in an embodiment of this application. The text data processing method provided in an embodiment of this application may include:

[0175] 1001. The execution device acquires a first data set, which includes multiple target data subsets. Each target data subset includes at least one level target label and a third string corresponding to the target label. The target label is used to indicate the semantics of the third string.

[0176] In this embodiment of the application, the execution device can receive a first data set sent by the training device and store the first data set locally. The meaning of the first data set can be found in [reference needed]. Figure 3 The description in the corresponding embodiment will not be repeated here. During the process of semantic understanding by the execution device through the target model, the training device can also update the first data set in the execution device, and correspondingly, the execution device can obtain the updated first data set.

[0177] 1002. The execution device acquires the text to be processed.

[0178] In this embodiment of the application, the execution device can receive text to be processed input by the user. The user can input the text to be processed by voice or text. The specific form of obtaining the text to be processed needs to be determined in conjunction with the specific product form of the execution device, and is not limited here.

[0179] 1003. The execution device performs semantic understanding on the text to be processed through the target model and obtains the prediction result output by the target model. The prediction result is used to indicate that the text to be processed is split into at least one target character set. Each target character set includes at least one character. The prediction result also includes at least one first label corresponding to at least one target character set. The at least one first label is used to reflect the intent of the text to be processed.

[0180] For a detailed implementation of step 1003 of the execution device in this embodiment, please refer to [reference needed]. Figure 3 The description of step 303 in the corresponding embodiment, Figure 10 The meaning of "target character set" in the corresponding embodiment is the same as... Figure 3 The meaning of "first character set" in the corresponding embodiment is similar and can be directly understood by referring to the above description, and will not be repeated here.

[0181] In this embodiment, the target model performs semantic understanding on the text to be processed, and obtains the prediction result output by the target model. The prediction result is used to indicate how to split the text to be processed into at least one target character set. The prediction result also includes a first label that matches at least one target character set, so that the intent of the text to be processed can be reflected by all the matched first labels. Since this solution matches labels to the characters in the text to be processed, a target character set may not include reduplicated words or modal particles, that is, reduplicated words or modal particles can be split into another target character set. Thus, even if reduplicated words or modal particles exist in the entire text to be processed, the intent of the text to be processed can still be understood, which provides a natural language understanding method with more generalization ability. Moreover, since the prediction result indicates how to split the entire text to be processed, and also includes at least one label corresponding to the text to be processed, the prediction result carries richer information, which is beneficial to improving the accuracy of the intent understanding process of the text to be processed.

[0182] This application also provides a method for processing text data. Please refer to [link to relevant documentation]. Figure 11 , Figure 11 This is a flowchart illustrating a text data processing method provided in an embodiment of this application. The text data processing method provided in an embodiment of this application may include:

[0183] 1101. The electronic device acquires a target data subset, which includes a first subset and a second subset. The first subset includes a first string and a first expected label corresponding to the first string, and the second subset includes a second string and a second expected label corresponding to the second string.

[0184] In this application embodiment, the concept of the target data subset and the specific implementation of the electronic device execution step 1101 can be found in [reference needed]. Figure 3 The description of step 301 in the corresponding embodiment will not be repeated here.

[0185] 1102. The electronic device determines a second data set corresponding to the target model based on the target data subset. The second data set includes multiple target data subsets. The target data subsets are used to indicate the correct result corresponding to the text to be processed. The correct result is used to indicate that the text to be processed is split into at least two sets of strings. The at least two sets of strings include a first string and a second string. The correct result also includes a first expected label corresponding to the first string and a second expected label corresponding to the second string.

[0186] 1103. The electronic device obtains a text to be processed and an expected result corresponding to the text to be processed based on a target data subset. The expected result is used to indicate that the text to be processed is split into at least one second character set, each second character set including at least one character. The expected result corresponding to the text to be processed also includes at least one expected label corresponding to at least one second character set. The at least one expected label is used to indicate the correct intent of the text to be processed.

[0187] 1104. The electronic device performs semantic understanding of the text to be processed through the target model and obtains the prediction result output by the target model. The prediction result is used to indicate that the text to be processed is split into at least one first character set, each first character set including at least one character. The prediction result also includes at least one first label corresponding to at least one first character set. At least one first label is used to reflect the prediction intent of the text to be processed.

[0188] 1105. The electronic device trains the target model according to the target loss function, which is used to indicate the similarity between the predicted result and the expected result corresponding to the text to be processed.

[0189] In this embodiment of the application, the concepts of each term in steps 1103 to 1105, as well as the specific implementation of steps 1103 to 1105 by the electronic device, can be found in the references. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0190] It should be noted that steps 1103 and 1105 are optional steps. If the electronic device is the execution device of the target model, steps 1103 and 1105 can be omitted. If the electronic device is the training device of the target model, steps 1103 and 1105 are mandatory steps.

[0191] In this embodiment of the application, a method for obtaining a subset of target data is also provided, which further expands the application scenarios of this solution.

[0192] To more intuitively understand the beneficial effects of the embodiments of this application, the following describes the beneficial effects of the embodiments of this application in conjunction with experimental data. The dataset used in the experiment is the publicly available dataset Task-oriented ParsingDataset, as shown in Table 2 below.

[0193] Voice control Social dialogue Template matching 39.8％ 81.3％ Examples of this application 93.8％ 88.7％

[0194] Table 2

[0195] Table 2 shows the accuracy rates when using the existing "template matching" method to understand the intent of the input text to be processed, and the accuracy rates when using the target model provided in this application to understand the intent of the input text to be processed. The "voice control" column means that the input text to be processed is used to control the volume, and the "social dialogue" column means that the input text to be processed is social language. Through the above comparison, it can be seen that the method provided in this application has strong generalization ability and can effectively improve the accuracy of the intent recognition process.

[0196] exist Figures 1 to 11 Based on the corresponding embodiments, in order to better implement the above-described solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 12 , Figure 12 This is a schematic diagram of a text data processing apparatus provided in an embodiment of this application. The text data processing apparatus 1200 may include: an acquisition module 1201, used to acquire text to be processed, the text to be processed including multiple characters; and a processing module 1202, used to process the text to be processed through a target model to obtain a prediction result, wherein the prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character, and the prediction result also includes multiple first labels, one first label being used to indicate the semantics of a target character set, and multiple first labels being used to determine the intent of the text to be processed.

[0197] In one possible design, multiple first tags include at least two levels of tags, each of which includes a parent tag and a child tag, with a hierarchical relationship between the parent and child tags.

[0198] In one possible design, there are N ways to split the text to be processed, where N is an integer greater than or equal to 1. The prediction result is used to indicate a target splitting method corresponding to the text to be processed. The target splitting method belongs to N splitting methods. The prediction result is obtained based on the matching degree between multiple target character sets and multiple first tags.

[0199] In one possible design, the processing module 1202 is specifically used to match each target character set with multiple strings in the first data set to determine the target string that matches each target character set, and to obtain at least one second tag corresponding to the target string from the first data set, wherein a string includes at least one character; based on each target character set and the second tag, each target character set is matched with multiple tags in the first data set through a target model to obtain a first tag that matches each target character set.

[0200] In one possible design, please refer to Figure 13 , Figure 13 This is another schematic diagram of the text data processing device provided in the embodiments of this application. The text data processing device 1200 further includes: a generation module 1203, used to generate target indication information through a target model based on the text to be processed, a target character set, and at least two second tags, wherein the target indication information is used to indicate whether each second tag matches or does not match the target character set; a filtering module 1204, used to filter at least two second tags according to the target indication information to obtain at least one filtered tag; and a processing module 1202, specifically used to match the target character set with multiple tags in a first data set through a target model based on the target character set and at least one filtered tag.

[0201] It should be noted that the information interaction and execution process between the modules / units in the text data processing device 1200 are based on the same concept as the above-mentioned method embodiments in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0202] This application also provides a neural network training device, please refer to... Figure 14 , Figure 14This is a schematic diagram of a neural network training device provided in an embodiment of this application. The neural network training device 1400 includes: a processing module 1401, used to process text to be processed through a target model to obtain a prediction result. The text to be processed includes one or more characters. The prediction result is used to indicate that the text to be processed is split into multiple first character sets, each first character set including at least one character. The prediction result also includes multiple first labels. One first label is used to indicate the semantics of a target character set, and the multiple first labels are used to determine the prediction intent of the text to be processed. A training module 1402 is used to train a target model according to a target loss function to obtain a trained target model. The target loss function is used to indicate the similarity between the prediction result and the expected result corresponding to the text to be processed. The expected result corresponding to the text to be processed is used to indicate that the text to be processed is split into multiple second character sets, each second character set including at least one character. The expected result corresponding to the text to be processed also includes multiple expected labels. One expected label is used to indicate the semantics of a second character set, and the multiple expected labels are used to determine the correct intent of the text to be processed.

[0203] In one possible design, multiple first tags include at least two levels of tags, each of which includes a parent tag and a child tag, with a hierarchical relationship between the parent and child tags.

[0204] In one possible design, please refer to Figure 15 , Figure 15 This is another schematic diagram of the neural network training device provided in the embodiments of this application. The neural network training device 1400 further includes: an acquisition module 1403, used to acquire a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, the second subset including a second string and a second expected label corresponding to the second string; and a determination module 1404, used to determine the text to be processed and the expected result corresponding to the text to be processed based on the target data subset, wherein the text to be processed includes a first string and a second string, and the expected result includes a first expected label corresponding to the first string and a second expected label corresponding to the second string.

[0205] In one possible design, a third score corresponding to the text to be processed satisfies preset conditions, and the third score is used to indicate the quality of the text to be processed.

[0206] It should be noted that the information interaction and execution process between the modules / units in the neural network training device 1400 are based on the same concept as the above-mentioned method embodiments in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0207] This application also provides a text data processing apparatus; please refer to [link / reference]. Figure 16 , Figure 16 This is a schematic diagram of a text data processing apparatus provided in an embodiment of this application. The text data processing apparatus 1600 includes: an acquisition module 1601, used to acquire a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, and the second subset including a second string and a second expected label corresponding to the second string; and a processing module 1602, used to determine a second data set corresponding to a target model based on the target data subset, wherein the target model is used for semantic understanding of the input text to be processed, the second data set including multiple target data subsets, and the second data set is used for the training phase and / or the inference phase of the target model. The target data subset is used to indicate the expected result corresponding to the text to be processed, and the expected result is used to indicate that the text to be processed is split into at least two sets of strings, the at least two sets of strings including the first string and the second string, and the correct result also includes the first expected label corresponding to the first string and the second expected label corresponding to the second string.

[0208] In one possible design, the acquisition module 1601 is also used to acquire the modification operation corresponding to the target data subset through the display interface. The modification operation includes any one or more of the following operations: deleting a string in the target data subset, updating the content of a string in the target data subset, or adding a new string to the target data subset.

[0209] In one possible design, the acquisition module 1601 is further configured to acquire the text to be processed, which includes multiple characters; the processing module 1602 is further configured to process the text to be processed through the target model to obtain a prediction result, wherein the prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character, and the prediction result also includes multiple first labels, one first label being used to indicate the semantics of a target character set, and multiple first labels being used to determine the intent of the text to be processed.

[0210] It should be noted that the information interaction and execution process between the modules / units in the text data processing device 1600 are based on the same concept as the above-mentioned method embodiments in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0211] The following describes an execution device provided in an embodiment of this application. Please refer to [link / reference]. Figure 17 , Figure 17This is a schematic diagram of an execution device provided in an embodiment of this application. The execution device 1700 can specifically be a virtual reality (VR) device, a mobile phone, a tablet, a laptop, a smart wearable device, a monitoring data processing device, or a radar data processing device, etc., and is not limited thereto. Specifically, the execution device 1700 includes: a receiver 1701, a transmitter 1702, a processor 1703, and a memory 1704 (wherein the execution device 1700 may have one or more processors 1703). Figure 17 (Taking a processor as an example), processor 1703 may include application processor 17031 and communication processor 17032. In some embodiments of this application, receiver 1701, transmitter 1702, processor 1703 and memory 1704 may be connected via a bus or other means.

[0212] Memory 1704 may include read-only memory and random access memory, and provides instructions and data to processor 1703. A portion of memory 1704 may also include non-volatile random access memory (NVRAM). Memory 1704 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0213] Processor 1703 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses are referred to as the bus system in the diagram.

[0214] The methods disclosed in the embodiments of this application can be applied to or implemented by processor 1703. Processor 1703 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 1703 or by instructions in software form. Processor 1703 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Processor 1703 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1704. Processor 1703 reads the information in memory 1704 and, in conjunction with its hardware, completes the steps of the above method.

[0215] Receiver 1701 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1702 can be used to output digital or character information through the first interface; transmitter 1702 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1702 may also include a display device such as a display screen.

[0216] In this embodiment of the application, the application processor 17031 in the processor 1703 is used to execute... Figure 10 The text data processing method executed by the execution device in the corresponding embodiment. Specifically, the application processor 17031 is used to perform the following steps:

[0217] Get the text to be processed, which includes multiple characters;

[0218] The target model processes the text to be processed to obtain prediction results. The prediction results are used to indicate that the text to be processed is split into multiple target character sets. Each target character set includes at least one character. The prediction results also include multiple first labels. One first label is used to indicate the semantics of a target character set, and multiple first labels are used to determine the intent of the text to be processed.

[0219] It should be noted that the specific manner in which the application processor 17031 executes the above steps is different from that described in this application. Figure 10 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 10 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0220] This application also provides an electronic device; please refer to [link / reference]. Figure 18 , Figure 18 This is a schematic diagram of an electronic device provided in an embodiment of this application. Specifically, the electronic device 1800 is implemented by one or more servers. The electronic device 1800 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1822 (e.g., one or more processors) and memory 1832, and one or more storage media 1830 (e.g., one or more mass storage devices) for storing application programs 1842 or data 1844. The memory 1832 and storage media 1830 can be temporary or persistent storage. The program stored in the storage media 1830 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the electronic device. Furthermore, the CPU 1822 may be configured to communicate with the storage media 1830 and execute the series of instruction operations in the storage media 1830 on the electronic device 1800.

[0221] Electronic device 1800 may also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input / output interfaces 1858, and / or one or more operating systems 1841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0222] In one embodiment of this application, the central processing unit 1822 is used to execute... Figure 3 The method executed by the training device in the corresponding embodiment. Specifically, the central processing unit 1822 can be used to execute the following steps:

[0223] The target model processes the text to be processed to obtain the prediction result. The text to be processed includes multiple characters. The prediction result is used to indicate that the text to be processed is split into multiple first character sets. Each first character set includes at least one character. The prediction result also includes multiple first labels. One first label is used to indicate the semantics of a target character set. Multiple first labels are used to determine the prediction intent of the text to be processed.

[0224] The target model is trained based on the target loss function to obtain the trained target model.

[0225] The target loss function is used to indicate the similarity between the predicted result and the expected result corresponding to the text to be processed. The expected result corresponding to the text to be processed is used to indicate that the text to be processed is split into multiple second character sets, each second character set including at least one character. The expected result corresponding to the text to be processed also includes multiple expected labels. One expected label is used to indicate the semantics of a second character set, and multiple expected labels are used to determine the correct intent of the text to be processed.

[0226] It should be noted that the specific manner in which the central processing unit 1822 executes the above steps is different from that described in this application. Figure 3 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 3 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0227] In one scenario, the central processing unit 1822 is used to execute... Figure 11 The method executed by the electronic device in the corresponding embodiment. Specifically, the central processing unit 1822 can be used to perform the following steps:

[0228] Obtain a target data subset, which includes a first subset and a second subset. The first subset includes a first string and a first expected label corresponding to the first string, and the second subset includes a second string and a second expected label corresponding to the second string.

[0229] Based on the target data subset, a second data set corresponding to the target model is determined, wherein the target model is used to perform semantic understanding on the input text to be processed, and the second data set includes multiple target data subsets. The second data set is used for the training phase and / or the inference phase of the target model.

[0230] The target data subset is used to indicate the expected result corresponding to the text to be processed. The expected result is used to indicate that the text to be processed should be split into at least two sets of strings, including a first string and a second string. The correct result also includes a first expected label corresponding to the first string and a second expected label corresponding to the second string.

[0231] It should be noted that the specific manner in which the central processing unit 1822 executes the aforementioned steps is different from that in this application. Figure 11 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 11 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0232] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figure 3 The steps performed by the training device in the method described in the illustrated embodiment, or causing the computer to perform the steps as described above. Figure 10 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figure 11 The steps performed by the electronic device in the method described in the illustrated embodiment.

[0233] This application embodiment also provides a computer-readable storage medium storing a program for performing signal processing, which, when run on a computer, causes the computer to perform the aforementioned actions. Figure 3 The steps performed by the training device in the method described in the illustrated embodiment, or causing the computer to perform the steps as described above. Figure 10 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figure 11 The steps performed by the electronic device in the method described in the illustrated embodiment.

[0234] The text data processing device, neural network training device, execution device, electronic device, and training device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to perform the aforementioned operations. Figure 10 or Figure 11 The text data processing method described in the illustrated embodiment, or, to cause the chip to perform the above... Figure 3The illustrated embodiment describes a neural network training method. Optionally, the storage unit is an in-chip storage unit, such as a register or cache. Alternatively, the storage unit can be an external storage unit located within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0235] For details, please refer to Figure 19 , Figure 19 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU190). The NPU190 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1903, which is controlled by the controller 1904 to extract matrix data from the memory and perform multiplication operations.

[0236] In some implementations, the arithmetic circuit 1903 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1903 is a two-dimensional pulsating array. The arithmetic circuit 1903 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1903 is a general-purpose matrix processor.

[0237] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1902 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1901 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1908.

[0238] Unified memory 1906 is used to store input and output data. Weight data is directly transferred to weight memory 1902 via Direct Memory Access Controller (DMAC) 1905. Input data is also transferred to unified memory 1906 via DMAC.

[0239] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1909.

[0240] The Bus Interface Unit (BIU) 1910 is used by the instruction fetch memory 1909 to fetch instructions from external memory, and also by the memory access controller 1905 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0241] The DMAC is mainly used to move input data from external memory DDR to unified memory 1906, or to weight data to weight memory 1902, or to input data to input memory 1901.

[0242] The vector computation unit 1907 includes multiple processing units that further process the output of the computation circuits as needed, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is primarily used for computations in non-convolutional / fully connected layers of neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0243] In some implementations, the vector computation unit 1907 can store the processed output vector in the unified memory 1906. For example, the vector computation unit 1907 can apply linear and / or nonlinear functions to the output of the computation circuit 1903, such as performing linear interpolation on feature planes extracted from convolutional layers, or accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1907 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as activation input to the computation circuit 1903, for example, for use in subsequent layers of the neural network.

[0244] The instruction fetch buffer 1909 connected to the controller 1904 is used to store the instructions used by the controller 1904;

[0245] Unified memory 1906, input memory 1901, weighted memory 1902, and instruction fetch memory 1909 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0246] In the above embodiments, the operations of each layer in the target model can be performed by the operation circuit 1903 or the vector calculation unit 1907.

[0247] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0248] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or electronic device, etc.) to execute the methods described in the various embodiments of this application.

[0250] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0251] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for processing text data, characterized in that, The method includes: Obtain the text to be processed, which includes multiple characters; The text to be processed is processed by a target model to obtain a prediction result. The prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes multiple first labels. One first label is used to indicate the semantics of one target character set. The multiple first labels are used to determine the intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple target character sets and the multiple first labels. The process of processing the text to be processed using the target model to obtain the prediction result includes: Each target character set is matched with multiple strings in a first data set to determine a target string that matches each target character set, and at least one second tag corresponding to the target string is obtained from the first data set, wherein a string includes at least one character; Based on each target character set and the at least one second tag, the target model is used to match each target character set with multiple tags in the first data set to obtain the first tag that matches each target character set.

2. The method according to claim 1, characterized in that, The plurality of first tags include at least two levels of tags, the at least two levels of tags include a parent tag and a child tag, and there is a hierarchical relationship between the parent tag and the child tag.

3. The method according to claim 1 or 2, characterized in that, There are N ways to split the text to be processed, where N is an integer greater than or equal to 1. The prediction result is used to indicate a target splitting method corresponding to the text to be processed, and the target splitting method belongs to the N splitting methods.

4. The method according to claim 1, characterized in that, When the at least one second tag includes at least two second tags, after obtaining at least one second tag corresponding to the target string from the first data set, the method further includes: Based on the text to be processed, the target character set, and the at least two second tags, target indication information is generated through the target model. The target indication information is used to indicate whether each second tag matches or does not match the target character set. Based on the target indication information, the at least two second tags are filtered to obtain at least one filtered tag; The step of matching each target character set with multiple tags in the first data set using the target model, based on each target character set and the at least one second tag, includes: Based on the target character set and the at least one filtered label, the target character set is matched with multiple labels in the first data set using the target model.

5. A method for training a neural network, characterized in that, The method includes: The target model processes the text to be processed to obtain a prediction result. The text to be processed includes multiple characters. The prediction result is used to indicate that the text to be processed is split into multiple first character sets. Each first character set includes at least one character. The prediction result also includes multiple first labels. Each first label is used to indicate the semantics of a first character set. The multiple first labels are used to determine the prediction intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple first character sets and the multiple first labels. The target model is trained according to the target loss function to obtain the trained target model; Wherein, the target loss function is used to indicate the similarity between the predicted result and the expected result corresponding to the text to be processed, the expected result corresponding to the text to be processed is used to indicate splitting the text to be processed into multiple second character sets, each second character set including at least one character, the expected result corresponding to the text to be processed also includes multiple expected labels, one of the expected labels is used to indicate the semantics of a second character set, and the multiple expected labels are used to determine the correct intent of the text to be processed; The process of processing the text to be processed using the target model to obtain the prediction result includes: Each of the first character sets is matched with multiple strings in the first data set to determine the target string that matches each of the first character sets, and at least one second tag corresponding to the target string is obtained from the first data set, wherein a string includes at least one character; Based on each first character set and the at least one second label, the target model is used to match each first character set with multiple labels in the first data set to obtain the first label that matches each first character set.

6. The method according to claim 5, characterized in that, The plurality of first tags include at least two levels of tags, the at least two levels of tags include a parent tag and a child tag, and there is a hierarchical relationship between the parent tag and the child tag.

7. The method according to claim 5 or 6, characterized in that, Before processing the text to be processed through the target model, the method further includes: Obtain a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, the second subset including a second string and a second expected label corresponding to the second string; Based on the target data subset, the text to be processed and the expected result corresponding to the text to be processed are determined, wherein the text to be processed includes the first string and the second string, and the expected result includes the first expected tag corresponding to the first string and the second expected tag corresponding to the second string.

8. The method according to claim 7, characterized in that, The third score corresponding to the text to be processed meets the preset conditions, and the third score is used to indicate the quality of the text to be processed.

9. A method for processing text data, characterized in that, The method includes: Obtain a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, the second subset including a second string and a second expected label corresponding to the second string; Based on the target data subset, a second data set corresponding to the target model is determined. The target model is used to perform semantic understanding on the input text to be processed to obtain a prediction result. The prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes multiple first labels, each first label indicating the semantics of a target character set. The multiple first labels are used to determine the intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple target character sets and the multiple first labels. The second data set includes multiple target data subsets and is used in the training phase and / or the inference phase of the target model. The first label matching each target character set is obtained by the target model matching each target character set with multiple labels in the first data set based on each target character set and at least one second label. The at least one second label corresponds to a target string, which is a string in the multiple strings of the first data set that matches each target character set. Each string in the first data set includes at least one character. The target data subset is used to indicate the expected result corresponding to the text to be processed. The expected result is used to indicate that the text to be processed is split into at least two sets of strings. The at least two sets of strings include the first string and the second string. The correct result also includes the first expected label corresponding to the first string and the second expected label corresponding to the second string.

10. The method according to claim 9, characterized in that, The method further includes: The modification operation corresponding to the target data subset is obtained through the display interface. The modification operation includes any one or more of the following operations: deleting a string in the target data subset, updating the content of a string in the target data subset, or adding a new string to the target data subset.

11. The method according to claim 9 or 10, characterized in that, The method further includes: Obtain the text to be processed, which includes multiple characters; The target model is used to process the text to be processed to obtain the prediction result.

12. A text data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the text to be processed, which includes multiple characters; The processing module is used to process the text to be processed through a target model to obtain a prediction result. The prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes multiple first labels, each first label being used to indicate the semantics of a target character set. The multiple first labels are used to determine the intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple target character sets and the multiple first labels. The processing module is specifically used for: Each target character set is matched with multiple strings in the first data set to determine the target string that matches each target character set, and at least one second tag corresponding to each target string is obtained from the first data set, wherein a string includes at least one character; Based on each target character set and the second tag, the target model is used to match each target character set with multiple tags in the first data set to obtain the first tag that matches each target character set.

13. The apparatus according to claim 12, characterized in that, The plurality of first tags include at least two levels of tags, the at least two levels of tags include a parent tag and a child tag, and there is a hierarchical relationship between the parent tag and the child tag.

14. The apparatus according to claim 12 or 13, characterized in that, There are N ways to split the text to be processed, where N is an integer greater than or equal to 1. The prediction result is used to indicate a target splitting method corresponding to the text to be processed, and the target splitting method belongs to the N splitting methods.

15. The apparatus according to claim 12, characterized in that, The device further includes: The generation module is used to generate target indication information based on the text to be processed, the target character set, and at least two second tags, through the target model. The target indication information is used to indicate whether each second tag matches or does not match the target character set. A filtering module is used to filter the at least two second tags according to the target indication information to obtain at least one filtered tag; The processing module is specifically used to match the target character set with multiple tags in the first data set using the target model, based on the target character set and the at least one filtered tag.

16. A training device for a neural network, characterized in that, The device includes: The processing module is used to process the text to be processed through the target model to obtain a prediction result. The text to be processed includes multiple characters. The prediction result is used to indicate that the text to be processed is split into multiple first character sets. Each first character set includes at least one character. The prediction result also includes multiple first labels. One first label is used to indicate the semantics of a first character set. The multiple first labels are used to determine the prediction intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple first character sets and the multiple first labels. The training module is used to train the target model according to the target loss function to obtain the trained target model; The processing module is specifically used for: Each of the first character sets is matched with multiple strings in the first data set to determine the target string that matches each of the first character sets, and at least one second tag corresponding to the target string is obtained from the first data set, wherein a string includes at least one character; Based on each first character set and the at least one second tag, the target model is used to match each first character set with multiple tags in the first data set to obtain the first tag that matches each first character set; Wherein, the target loss function is used to indicate the similarity between the predicted result and the expected result corresponding to the text to be processed, the expected result corresponding to the text to be processed is used to indicate splitting the text to be processed into multiple second character sets, each second character set including at least one character, the expected result corresponding to the text to be processed also includes multiple expected labels, one of the expected labels is used to indicate the semantics of a second character set, and the multiple expected labels are used to determine the correct intent of the text to be processed.

17. The apparatus according to claim 16, characterized in that, The plurality of first tags include at least two levels of tags, the at least two levels of tags include a parent tag and a child tag, and there is a hierarchical relationship between the parent tag and the child tag.

18. The apparatus according to claim 16 or 17, characterized in that, The device further includes: The acquisition module is used to acquire a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, and the second subset including a second string and a second expected label corresponding to the second string; The determining module is configured to determine the text to be processed and the expected result corresponding to the text to be processed based on the target data subset, wherein the text to be processed includes the first string and the second string, and the expected result includes the first expected tag corresponding to the first string and the second expected tag corresponding to the second string.

19. The apparatus according to claim 18, characterized in that, The third score corresponding to the text to be processed meets the preset conditions, and the third score is used to indicate the quality of the text to be processed.

20. A text data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a target data subset, the target data subset including a first subset and a second subset, the first subset including a first string and a first expected label corresponding to the first string, and the second subset including a second string and a second expected label corresponding to the second string; The processing module is configured to determine a second data set corresponding to the target model based on the target data subset. The target model is used to perform semantic understanding on the input text to be processed to obtain a prediction result. The prediction result is used to indicate that the text to be processed is split into multiple target character sets, each target character set including at least one character. The prediction result also includes multiple first labels, each first label indicating the semantics of a target character set. The multiple first labels are used to determine the intent of the text to be processed. The prediction result is obtained based on the matching degree between the multiple target character sets and the multiple first labels. The second data set includes multiple target data subsets and is used in the training phase and / or the inference phase of the target model. The first label matching each target character set is obtained by the target model matching each target character set with multiple labels in the first data set based on each target character set and at least one second label. The at least one second label corresponds to a target string, which is a string in the multiple strings of the first data set that matches each target character set. Each string in the first data set includes at least one character. The target data subset is used to indicate the expected result corresponding to the text to be processed. The expected result is used to indicate that the text to be processed is split into at least two sets of strings. The at least two sets of strings include the first string and the second string. The correct result also includes the first expected label corresponding to the first string and the second expected label corresponding to the second string.

21. The apparatus according to claim 20, characterized in that, The acquisition module is further configured to acquire the modification operation corresponding to the target data subset through the display interface. The modification operation includes any one or more of the following operations: deleting a string in the target data subset, updating the content of a string in the target data subset, or adding a new string to the target data subset.

22. The apparatus according to claim 20 or 21, characterized in that, The acquisition module is also used to acquire the text to be processed, which includes multiple characters; The processing module is also used to process the text to be processed through the target model to obtain a prediction result.

23. A computer program product, characterized in that, When the computer program is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 4, or causes the computer to perform the method as described in any one of claims 5 to 8, or causes the computer to perform the method as described in any one of claims 9 to 11.

24. A computer-readable storage medium, characterized in that, The method includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 4, or causes the computer to perform the method as described in any one of claims 5 to 8, or causes the computer to perform the method as described in any one of claims 9 to 11.

25. An execution device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the execution device to perform the method as described in any one of claims 1 to 4.

26. An electronic device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the electronic device to perform the method as described in any one of claims 5 to 8, or to cause the electronic device to perform the method as described in any one of claims 9 to 11.

Images