Text processing method and device

Abstract

The embodiment of the invention discloses a text processing method and device. The method comprises the steps of obtaining a to-be-processed text, generating to-be-processed data according to the to-be-processed text, processing the to-be-processed data through a pre-trained recognition model, obtaining a first hidden layer vector from a first predetermined encoder layer, obtaining a second hidden layer vector from a second predetermined encoder layer, and obtaining a second hidden layer vector from a second predetermined encoder layer. The first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining a first recognition result of the to-be-processed text according to the first hidden layer vector through a subject classifier, and determining a second recognition result of the to-be-processed text according to the second hidden layer vector through a black and white classifier. Therefore, streaming processing can be carried out on the text, and the real-time performance and the response speed of a large language model and the accuracy of text recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a text processing method and apparatus. Background Art

[0002] When generating text content, large language models must strictly comply with public order, good customs, ethics, and policies and regulations. The review process has become an important link to ensure the implementation of these standards. The importance of this application area lies in ensuring user safety and maintaining social values, and preventing the spread of bad information.

[0003] In the prior art, text risk assessment mainly relies on text classification models based on the Transformer architecture such as the BERT model. However, the traditional solution adopts a "generate first and then judge" mode, that is, waiting for a certain length of text to be generated before conducting a review, which leads to a lack of real-time performance. Especially in the context of the need to pursue real-time prevention and control of streaming content, the existing non-streaming solutions are inadequate. Moreover, since the BERT model requires a single category label for the entire text segment, directly using it for real-time risk assessment is not only difficult to achieve but also has poor effects. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present invention is to provide a text processing method and apparatus, which can improve the real-time performance and response speed of large language models and the accuracy of text recognition.

[0005] In a first aspect, an embodiment of the present invention provides a text processing method, the method comprising:

[0006] Obtain a text to be processed;

[0007] Generate data to be processed according to the text to be processed;

[0008] Process the data to be processed through a pre-trained recognition model, and obtain a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer. The recognition model includes a plurality of encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers;

[0009] Determine a first recognition result of the text to be processed through a topic classifier according to the first hidden layer vector;

[0010] Determine a second recognition result of the text to be processed through a black-and-white classifier according to the second hidden layer vector.

[0011] In some embodiments, the first recognition result is a topic recognition result, the second recognition result is a black-and-white classification result corresponding to each topic, and the number of black-and-white classifiers is the same as the number of topics and corresponds one by one.

[0012] In some embodiments, the text to be processed includes a query text and a response text input by a user, and the response text is generated by a large language model according to the query text.

[0013] In some embodiments, generating the data to be processed according to the text to be processed includes:

[0014] Segmenting the query text and the response text into text segments according to a predetermined length respectively;

[0015] Inserting segment markers after each of the text segments;

[0016] Inserting a sentence marker between the query text and the response text to obtain the data to be processed.

[0017] In some embodiments, processing the data to be processed by a pre-trained recognition model includes:

[0018] Generating an input vector according to the data to be processed through an input embedding layer;

[0019] Processing the input vector through multiple encoder layers of the pre-trained recognition model, wherein the input of the first encoder layer is the input vector, and the input of the second to the L-th encoder layers is the output of the previous encoder layer, L is the total number of encoder layers, and L is an integer.

[0020] In some embodiments, the pre-trained recognition model is obtained through the following steps:

[0021] Obtaining an initial recognition model and training samples, the training samples including a query text and a response text, and the response text being generated by a large language model according to the query text;

[0022] Generating first data according to the training samples;

[0023] Training the initial recognition model through the first data to obtain an intermediate recognition model;

[0024] Obtaining prediction results of each of the training samples through the intermediate recognition model;

[0025] Generating segment labels corresponding to each of the text segments according to the prediction results;

[0026] Generating second data according to the training samples;

[0027] Training the intermediate recognition model according to the second data and the segment labels to obtain the pre-trained recognition model.

[0028] In some embodiments, generating the first data according to the training sample includes:

[0029] Segmenting the query text and the reply text into text segments according to a predetermined length respectively;

[0030] Inserting a segmentation marker after each of the text segments;

[0031] Inserting a summary marker after the query text and the reply text respectively to obtain the first data.

[0032] In some embodiments, the prediction result includes a third hidden layer vector corresponding to each segmentation marker and a fourth hidden layer vector corresponding to the summary marker.

[0033] In some embodiments, generating the segmentation label corresponding to each text segment according to the prediction result includes:

[0034] Determining an attention score according to the third hidden layer vector and the fourth hidden layer vector;

[0035] Determining a sample type, where the sample type includes a positive sample and a negative sample;

[0036] For a positive sample, labeling each segmentation label as a positive label;

[0037] For a negative sample, determining each segmentation label according to the attention score.

[0038] In some embodiments, generating the second data according to the training sample includes:

[0039] Segmenting the query text and the reply text into text segments according to a predetermined length respectively;

[0040] Inserting a segmentation marker after each of the text segments;

[0041] Inserting a sentence marker between the query text and the reply text to obtain the second data.

[0042] In some embodiments, training the intermediate recognition model according to the second data and the segmentation label to obtain the pre-trained recognition model includes:

[0043] Training the intermediate recognition model in a manner of using a sliding window encoder module connected with a causal convolution to obtain the pre-trained recognition model.

[0044] In a second aspect, an embodiment of the present invention provides a text processing device, and the device includes:

[0045] A text acquisition unit, configured to acquire a text to be processed;

[0046] A data generation unit for generating data to be processed according to the text to be processed;

[0047] A vector acquisition unit for processing the data to be processed through a pre-trained recognition model, and acquiring a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer, where the recognition model includes multiple encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers;

[0048] A first recognition unit for determining a first recognition result of the text to be processed according to the first hidden layer vector through a theme classifier;

[0049] A second recognition unit for determining a second recognition result of the text to be processed according to the second hidden layer vector through a black-and-white classifier.

[0050] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory is used to store one or more computer program instructions, and where the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.

[0051] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions implement the method as described in the first aspect when executed by a processor.

[0052] The technical solution of the embodiment of the present invention obtains the text to be processed, generates the data to be processed according to the text to be processed, processes the data to be processed through a pre-trained recognition model, and acquires a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer. The first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determines a first recognition result of the text to be processed according to the first hidden layer vector through a theme classifier, and determines a second recognition result of the text to be processed according to the second hidden layer vector through a black-and-white classifier. Thus, streaming processing of the text can be realized, and the real-time performance, response speed of the large language model and the accuracy of text recognition can be improved. Description of the Drawings

[0053] Through the following description of the embodiments of the present invention with reference to the drawings, the above and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0054] Figure 1 is a schematic diagram of the text processing system according to the embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of the recognition model according to the embodiment of the present invention;

[0056] Figure 3 It is a schematic diagram of the encoder layer of an embodiment of the present invention;

[0057] Figure 4 It is a flowchart for obtaining an identification model of an embodiment of the present invention;

[0058] Figure 5 It is a flowchart for obtaining first data of an embodiment of the present invention;

[0059] Figure 6 It is a schematic diagram for obtaining shard labels of an embodiment of the present invention;

[0060] Figure 7 It is a flowchart for obtaining second data of an embodiment of the present invention;

[0061] Figure 8 It is a flowchart of a text processing method of an embodiment of the present invention;

[0062] Figure 9 It is a flowchart for obtaining data to be processed of an embodiment of the present invention;

[0063] Figure 10 It is a schematic diagram of a text processing apparatus of an embodiment of the present invention;

[0064] Figure 11 It is a schematic diagram of an electronic device of an embodiment of the present invention. Detailed implementation manners

[0065] The following describes the present application based on embodiments, but the present application is not limited to these embodiments. In the following detailed description of the present application, some specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. In order to avoid obscuring the essence of the present application, well-known methods, processes, procedures, elements and circuits are not described in detail.

[0066] In addition, those of ordinary skill in the art should understand that the drawings provided herein are all for illustrative purposes and are not necessarily drawn to scale.

[0067] Unless the context clearly requires otherwise, words such as "including" and "comprising" in the entire application document should be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, it is the meaning of "including but not limited to".

[0068] In the description of the present application, it should be understood that terms such as "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0069] In the solutions described in this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for performing a contract), and will only be processed within the specified or agreed scope. If a user refuses to process personal information other than the necessary information required for basic functions, it will not affect the user's use of the basic functions.

[0070] Today, with the rapid development of information technology, large language models (LLMs), as a core technology in the field of natural language processing (NLP), are changing the way we live and work. These models can generate coherent and relevant text content based on the prompts provided by users. However, as their application scope expands, it becomes particularly important to ensure that the generated content complies with public order, good customs, ethics, and policies and regulations. This requires effective review of the content generated by large language models to serve as a safety barrier for various tasks.

[0071] Currently, the review of the content generated by large language models mainly relies on the BERT (Bidirectional Encoder Representations from Transformers) model, which is a deep learning model based on the Transformer architecture. BERT learns language representations using a large-scale text corpus in the pre-training stage and is fine-tuned in various NLP tasks, achieving remarkable results. Although the BERT model has extensive applications in fields such as text classification and sentiment analysis, due to its characteristic that the entire text corresponds to a category label, it faces two major challenges in real-time review of the content generated by large language models: one is the lack of shard-level labels, and the other is that each shard is processed independently, making it impossible to effectively utilize historical information. This non-real-time review mode of "generate first and then judge" leads to latency problems and limits its practicality.

[0072] To solve these problems, the present invention proposes an innovative method aimed at achieving real-time content prevention and control in a streaming autoregressive paradigm. This method not only overcomes the latency problem in traditional review methods but also solves the problems of the lack of shard-level labels and insufficient utilization of historical information.

[0073] Figure 1 It is a schematic diagram of the text processing system according to an embodiment of the present invention. As Figure 1 shown, the text processing system according to an embodiment of the present invention includes at least one client 1, a network 2, and a server 3. Among them, the client 1 can be communicatively connected to the server 3 through the network 2.

[0074] In this embodiment, the client 1 is used to provide an interaction interface between the user and the server, and can be implemented through various terminal devices, such as a desktop computer, a laptop computer, a tablet computer, a mobile phone, or other data processing terminals, etc. The client 1 is responsible for receiving the query text input by the user and generating a query request to send to the server 3. In addition, the client 1 is also responsible for receiving the reply text sent by the server 3 and displaying it.

[0075] Further, the server 3 includes a large language model 31 and an identification model 32.

[0076] Among them, the large language model 31 is used to generate a reply text according to the user's query text. Specifically, large language models (LLMs) refer to those language processing models that have a large number of parameters and are built based on deep learning technologies, especially the Transformer architecture. These models learn rich language structures and patterns through training on large-scale text datasets and can generate coherent and relevant text content according to the input prompts.

[0077] The identification model 32 is used to process the query text input by the user and the reply text generated by the large language model, and determine whether there is a risk in these texts. If there is no risk, the reply text is returned to the client 1. If there is a risk, the large language model is notified to regenerate the reply text.

[0078] The network 2 can be used for the exchange of information and / or data. Among them, the network 2 can be any type of wired or wireless network, or a combination of them. In some embodiments, the network 2 can include a wired network, a wireless network, an optical fiber network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, etc., or any combination thereof. In some embodiments, the network 2 can include one or more network access points. For example, the network 2 can include wired or wireless network access points, such as base stations and / or network switching nodes, and one or more components of the text processing system can be connected to the network through the access point to exchange data and / or information.

[0079] Figure 2 It is a schematic diagram of the recognition model according to an embodiment of the present invention. Specifically, as Figure 2 shown, the recognition model includes an input embedding layer A, a plurality of encoder layers B1 - B L , a topic classifier C, and a plurality of black - and - white classifiers D1 - D3.

[0080] Among them, the input embedding layer A converts the text to be processed into an input vector. Specifically, in the embodiment of the present invention, markers are first inserted into the text to be processed to obtain the data to be processed, and then the input embedding layer generates an input vector according to the data to be processed.

[0081] Specifically, the text to be processed includes a query text input by the user and a response text, and the response text is generated by a large - language model according to the query text. The query text and the response text are respectively segmented into text segments according to a predetermined length, a segment marker is inserted after each text segment, and a sentence marker is inserted between the query text and the response text to obtain the data to be processed.

[0082] For example, assume that in a certain query, the query text input by the user is "I want to know some methods to improve work efficiency. Do you have any suggestions?". The response text generated by the large - language model is "Sure. First, use time - management skills and prioritize important and urgent tasks. Second, use task - management and collaboration software to track progress. In addition, keep the work environment clean and organized to enhance concentration. Finally, take regular breaks to restore energy. I hope these suggestions can help you improve your work efficiency!". If the predetermined length is 10 characters, the generated data to be processed is "I want to know some improve work[CLS]efficiency. Do you have any[CLS]suggestions?[CLS][SEP]Sure. First, use[CLS]time - management skills and prioritize[CLS]important and urgent tasks. Second[CLS], use task - management and collaboration[CLS]software to track progress. In addition[CLS], keep the work environment clean and organized[CLS], enhance concentration. Finally[CLS], take regular breaks to restore energy.[CLS]I hope these suggestions can help you[CLS]improve your work efficiency![CLS]". Among them, [CLS] is the segment marker, and [SEP] is the sentence marker.

[0083] It should be noted that when inserting the segment marker [CLS], if the number of texts in the last group is less than the predetermined length, [CLS] can be inserted at the end of the text. At the same time, in the above example, the response text is taken as the complete response output by the large - language model for illustration. However, in an actual scenario, during the process of the large - language model generating the response text, the response text is processed in real - time. For example, it can be processed once every predetermined period, or once after generating a predetermined number of texts.

[0084] Further, after the input embedding layer A receives the data to be processed, the data will sequentially pass through the word embedding (TokenEmbedding) and position embedding (Position Embedding) steps, and finally generate an input vector.

[0085] For word embedding (Token Embedding): First, a tokenizer is used to tokenize the data to be processed, decomposing the text into words or sub-word units. Then, each word or sub-word is converted into a corresponding sequence of Token IDs. This process utilizes a pre-trained word embedding matrix to map each Token ID to its corresponding vector representation. These vectors not only contain the basic semantic information of the words or sub-words but also lay the foundation for the subsequent processing of the model.

[0086] For position embedding (Position Embedding): The position embedding of each Token is a fixed vector pre-computed based on its specific position in the sequence. The role of position embedding is to enable the model to understand the relative order relationship between each Token.

[0087] Finally, the position embedding vector is added to the word embedding vector to jointly form the complete input vector for each Token. This combination method ensures that the model can not only capture the information at the lexical level but also recognize the sequential features in the sentence structure. Through the above steps, the input embedding layer A successfully converts the original text into a form that the model can process, thus providing a solid foundation for subsequent tasks.

[0088] In this embodiment, the L encoder layers B1 - B L are connected in sequence. The input information of the first encoder layer B1 is the input vector output by the input embedding layer A. The input information of the k-th encoder layer B k is the output information of the (k - 1)-th encoder layer B k-1 , where k is a positive integer and 2 ≤ k ≤ L.

[0089] Figure 3 is a schematic diagram of the encoder layer of the embodiment of the present invention. As Figure 3 shown, the encoder layer of the embodiment of the present invention adopts a Transformer encoder, including a multi-head attention layer E1, a normalization layer E2, a feed-forward neural network layer E3, and a normalization layer E4.

[0090] Among them, the multi-head attention layer E1 utilizes the multi-head self-attention mechanism to help the model, when processing each input position, simultaneously focus on other positions in the sequence, thereby better capturing context information. Each self-attention layer performs a linear transformation on the input sequence, then calculates the attention weights and applies them to the sequence to obtain a weighted representation.

[0091] The input end of the normalization layer E2 is connected to the output end of the multi-head attention layer E1 and the original input of the multi-head attention layer E1, and is used to implement residual connections and layer normalization. Residual connections allow information to flow directly between different layers, helping to avoid the vanishing gradient and accelerating training, while layer normalization helps to stabilize the training process and keep the input distribution of each layer within a small range.

[0092] The input end of the feedforward neural networks layer E3 is connected to the input end of the normalization layer E2 and includes two fully connected layers, which are connected by the ReLU activation function. The role of this layer is to perform a non-linear transformation and mapping on the features of each position.

[0093] The normalization layer E4 is connected to the output end of the feedforward neural networks layer E3 and the original input of the feedforward neural networks layer E3, and is used to implement residual connections and layer normalization. Residual connections allow information to flow directly between different layers, helping to avoid the vanishing gradient and accelerating training, while layer normalization helps to stabilize the training process and keep the input distribution of each layer within a small range.

[0094] Finally, the encoder converts the input sequence that has undergone multiple layers of processing into a series of high-dimensional hidden representations. These representations not only contain the semantic information of the input sequence but also the context information of each position in the sequence. Generally speaking, the role of the Transformer encoder is to convert the input sequence into context-aware hidden representations, which can contain the semantic and contextual information of the input sequence and provide the necessary information basis for the decoder to generate the output sequence. The output of the encoder layer serves as the input for subsequent layers or task modules, such as for downstream tasks like text classification, question answering systems, or machine translation. In this way, the encoder can effectively support various natural language processing tasks and significantly improve the generalization ability and prediction accuracy of the model.

[0095] In this embodiment, when the pre-trained recognition model processes the data to be processed, a first hidden layer vector is obtained from a first predetermined encoder layer (encoder layer B in the figure i ), and the theme classifier C is used to determine a first recognition result of the text to be processed according to the first hidden layer vector.

[0096] Among them, the first hidden layer vector refers to the feature representation output by a specific hidden layer (encoder layer B in the embodiment of the present invention i ) in the neural network architecture. It contains the key information and features extracted from the input data (such as text) after multiple transformations, and these features are crucial for subsequent tasks (such as theme classification).

[0097] The theme classifier C is a pre-trained classifier, usually a fully connected layer or a multi-layer perceptron (MLP), and its input is the first hidden layer vector. The goal of this classifier is to map the high-dimensional first hidden layer vector to one or more predefined theme categories. In some embodiments, first, an appropriate linear transformation (through the product of the weight matrix) is performed on the first hidden layer vector, and then an activation function (such as Softmax) is applied to convert the output into the first recognition result, and the first recognition result is the theme recognition result. In the embodiment of the present invention, three themes are taken as examples for illustration, namely the first theme X1, the second theme X2, and the third theme X3.

[0098] In a specific implementation manner, the first recognition result is in the form of a probability distribution, indicating the probability of each possible theme. For example, the first recognition result is (P X1 , P X2 , P X3 ), where P X1 , P X2 , P X3 respectively represent the probabilities that the text to be processed is the first theme X1, the second theme X2, and the third theme X3.

[0099] In this embodiment, the number of black-and-white classifiers is the same as and corresponds one-to-one with the number of themes. In the embodiment of the present invention, three themes are taken as examples for illustration, namely the first theme X1, the second theme X2, and the third theme X3. Correspondingly, the number of black-and-white classifiers is also three, namely D1-D3. Among them, the black-and-white classifiers D1-D3 correspond to the first theme X1, the second theme X2, and the third theme X3 respectively.

[0100] Among them, when the pre-trained recognition model processes the data to be processed, a second hidden layer vector is obtained from a second predetermined encoder layer (encoder layer B in the embodiment of the present invention j)Obtain the second hidden layer vector. The second hidden layer vector contains the key information and features extracted after the text to be processed undergoes multiple transformations. These features are crucial for the subsequent black-and-white classification task. Determine the second recognition result of the text to be processed based on the second hidden layer vector by the black-and-white classifier. The second recognition result is the black-and-white classification result corresponding to each theme. Among them, the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers.

[0101] In some embodiments, the black-and-white classifier includes a fully connected layer or a multi-layer perceptron (MLP), and its input is the second hidden layer vector from the encoder layer. The specific implementation steps are as follows: First, perform a linear transformation (through the product of the weight matrix) on the second hidden layer vector to map it to a low-dimensional space. Apply a non-linear activation function (such as ReLU or Tanh) to enhance the expressive power of the model. Finally, through an output layer (usually a Softmax layer), convert the result into a probability distribution form, indicating the probability of each possible black-and-white classification label. Based on the probability distribution, select the classification label with the highest probability as the final black-and-white classification result. Each black-and-white classifier is specifically designed for a certain specific theme, so they can better capture and understand the compliance requirements under that theme.

[0102] In some embodiments, the second recognition result output by each black-and-white classifier is whether the content under the corresponding theme is compliant, such as Pass (passed), Review (needs review), or Block (blocked). Pass (passed) means that the text to be processed is considered safe and compliant and can be used continuously without further manual intervention. Review (needs review) means that there may be potential problems or uncertainties in the content, and further manual review is required to determine its compliance. Block (blocked): means that the content clearly violates the relevant regulations and should be immediately blocked from publication or removed. For example, assume that the text to be processed belongs to the first theme X1 and after being processed by the black-and-white classifier D1, the second recognition result may be "Pass", indicating that the text content is compliant; if the text to be processed belongs to the second theme X2 and the result obtained after being processed by the black-and-white classifier D2 is "Block", it means that the text contains illegal content and should be blocked from publication.

[0103] In an embodiment of the present invention, by obtaining a text to be processed, generating data to be processed according to the text to be processed, processing the data to be processed through a pre-trained recognition model, obtaining a first hidden layer vector from a first predetermined encoder layer, and obtaining a second hidden layer vector from a second predetermined encoder layer, where the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining a first recognition result of the text to be processed by a topic classifier according to the first hidden layer vector, and determining a second recognition result of the text to be processed by a black-and-white classifier according to the second hidden layer vector. Thus, streaming processing of text can be achieved, improving the real-time performance, response speed of the large language model, and the accuracy of text recognition.

[0104] Figure 4 is a flowchart for obtaining a recognition model according to an embodiment of the present invention. Specifically, as Figure 4 shown, the steps for obtaining a pre-trained recognition model according to an embodiment of the present invention are as follows:

[0105] Step S110: Obtain an initial recognition model and training samples.

[0106] In this embodiment, an initial recognition model and a sample set are obtained. Among them, the architecture of the initial recognition model can refer to Figure 2 , and includes multiple encoder layers, a pre-trained topic classifier, and a pre-trained black-and-white classifier. Among them, the parameters in the multiple encoder layers are un-trained initialization parameters. The sample set includes multiple training samples, and each training sample includes a query text, a response text, label information corresponding to the query text, and label information corresponding to the response text. Among them, the response text in each training sample is generated by a large language model according to the query text. The label information includes the topic classification result and the black-and-white classification result corresponding to the text.

[0107] Step S120: Generate first data according to the training samples.

[0108] In this embodiment, the training process of the recognition model is divided into two stages. In the first stage, the query text and the response text in each training sample are respectively used as a sample pair to train the initial recognition model.

[0109] Specifically, Figure 5 is a flowchart for obtaining the first data according to an embodiment of the present invention. As Figure 5 shown, generating the first data according to the training samples includes the following steps:

[0110] Step S121: Slice the query text and the response text into text segments according to a predetermined length.

[0111] In this embodiment, the predetermined length can be set according to the actual situation. For example, when processing Chinese text, the predetermined length is the predetermined number of words, such as 10, 15, etc.

[0112] Step S122: Insert sharding markers after each of the text segments.

[0113] Step S123: Insert summary markers after the query text and the response text respectively to obtain the first data.

[0114] For example, assume that in the training sample, the query text is "I want to know some methods to improve work efficiency. Do you have any suggestions?". The response text is "Sure. First, use time management skills and prioritize important and urgent tasks. Second, use task management and collaboration software to track progress. In addition, keep the work environment clean and organized to enhance concentration. Finally, take regular breaks to restore energy. I hope these suggestions can help you improve work efficiency!".

[0115] If the predetermined length is 10 words, then after processing the query text, we get: "I want to know some methods to improve work [CLS] efficiency. Do you have any [CLS] suggestions? [CLS][CLS]". After the above processing, there are four [CLS] in total. Among them, the first three [CLS] are the sharding markers for each text segment respectively, and the last [CLS] is the summary marker.

[0116] Similarly, after processing the response text, we get: "Sure. First, use [CLS] time management skills and prioritize [CLS] important and urgent tasks. Second [CLS], use task management and collaboration [CLS] software to track progress. In addition, [CLS] keep the work environment clean and organized [CLS] to enhance concentration. Finally, [CLS] take regular breaks to restore energy. [CLS] I hope these suggestions can help you [CLS] improve work efficiency! [CLS][CLS]". After the above processing, the last [CLS] is the summary marker, and the other [CLS] are sharding markers.

[0117] Step S130: Train the initial recognition model with the first data to obtain an intermediate recognition model.

[0118] In this embodiment, the obtained first data is provided to the initial recognition model for processing, and the corresponding predicted topic classification result and predicted black-and-white classification result can be obtained. The predicted topic classification result and the predicted black-and-white classification result are compared with the topic classification result and the black-and-white classification result in the label information, and the initial recognition model is updated according to the comparison result. Thus, after training the initial recognition model with a large number of training samples, an intermediate recognition model can be obtained.

[0119] Step S140: Obtain the prediction results of each of the training samples through the intermediate recognition model.

[0120] In this embodiment, after obtaining the intermediate recognition model, the intermediate recognition model processes the training samples and can output the hidden vectors corresponding to each shard label and the summary label, that is, the prediction results. Specifically, the prediction results include the third hidden layer vectors corresponding to each shard label and the fourth hidden layer vector corresponding to the summary label. Among them, the prediction results can be extracted from any encoder layer in the intermediate recognition model, such as the first predetermined encoder layer, the second predetermined encoder layer, or other encoder layers.

[0121] Step S150: Generate shard labels corresponding to each text segment according to the prediction results.

[0122] In this embodiment, shard labels corresponding to each text segment are generated according to the prediction results and the sample types.

[0123] Specifically, Figure 6 is a schematic diagram of obtaining shard labels in an embodiment of the present invention. As Figure 6 shown, the generating shard labels corresponding to each text segment according to the prediction results includes the following steps:

[0124] Step S151: Determine attention scores according to the third hidden layer vectors and the fourth hidden layer vectors.

[0125] In natural language processing tasks, the attention mechanism is a powerful tool that allows the model to dynamically focus on different parts of the input sequence when processing sequence data. Especially in tasks such as content moderation, the attention mechanism can help identify which parts of the text may be problematic or require special attention. The attention score reflects the correlation or importance between different positions in the input sequence. Specifically, in the embodiment of the present invention, the attention score is used to measure the similarity or association degree between the summary label and each shard label. This similarity can be used to determine whether each shard contains non-compliant content.

[0126] Among them, the summary label is the [CLS] token at the sentence level, representing the comprehensive representation of the entire sentence. The shard label [CLS] token represents the overall characteristics of the shard. In the embodiment of the present invention, the prediction results can be obtained through the intermediate recognition model. The prediction results include the third hidden layer vectors corresponding to each shard label and the fourth hidden layer vector corresponding to the summary label. Thus, the attention scores are determined according to the third hidden layer vectors and the fourth hidden layer vectors.

[0127] Specifically, the similarity between each third hidden layer vector and the fourth hidden layer vector is calculated respectively to obtain the attention scores. Among them, the similarity calculation method can adopt methods such as dot product and cosine similarity.

[0128] Step S152: Determine the sample type, where the sample type includes positive samples and negative samples.

[0129] In this embodiment, the sample type is determined according to the label information in the training set, and the sample type includes positive samples and negative samples. Among them, positive samples represent compliant texts, and negative samples represent non-compliant texts.

[0130] Step S153: For positive samples, label each shard label as a positive label.

[0131] In this embodiment, for positive samples, each shard label is labeled as a positive label. That is, for positive samples, all shard labels are set to positive labels, where the positive label indicates compliance.

[0132] Step S154: For negative samples, determine each shard label according to the attention score.

[0133] In this embodiment, a probability threshold is set in advance. If the attention score of a certain shard label exceeds the set probability threshold, the corresponding shard is marked as a negative label (i.e., there may be non-compliant content). For the segments that do not reach the threshold, they are marked as positive labels (i.e., compliant content).

[0134] Thus, the shard label corresponding to each text segment can be obtained.

[0135] Step S160: Generate second data according to the training samples.

[0136] In this embodiment, after obtaining each shard label, second data is obtained based on the training samples and the shard labels.

[0137] Specifically, Figure 7 is the flowchart of obtaining the second data in the embodiment of the present invention. As Figure 7 shown, the obtaining of the second data based on the training samples and the shard labels includes the following steps:

[0138] Step S161: Shard the query text and the reply text into text segments according to a predetermined length.

[0139] In this embodiment, the predetermined length can be set according to the actual situation. For example, when processing Chinese texts, the predetermined length is a predetermined number of characters, such as 10, 15, etc.

[0140] Step S162: Insert sharding markers after each of the text segments.

[0141] Step S163: Insert sentence markers between the query text and the response text to obtain the second data.

[0142] For example, assume that in the training sample, the query text is "I want to know some methods to improve work efficiency. Do you have any suggestions?" The response text is "Sure. First, use time management techniques and prioritize important and urgent tasks. Second, use task management and collaboration software to track progress. In addition, keep the work environment clean and organized to improve concentration. Finally, take regular breaks to restore energy. I hope these suggestions can help you improve work efficiency!"

[0143] If the predetermined length is 10 words, the processed query text is: "I want to know some methods to improve work [CLS] efficiency. Do you have any [CLS] suggestions? [CLS]". Similarly, the processed response text is: "Sure. First, use [CLS] time management techniques and prioritize [CLS] important and urgent tasks. Second [CLS], use task management and collaboration [CLS] software to track progress. In addition, [CLS] keep the work environment clean and organized [CLS] to improve concentration. Finally, [CLS] take regular breaks to restore energy. [CLS] I hope these suggestions can help you [CLS] improve work efficiency! [CLS]". Finally, insert sentence markers between the query text and the response text to connect the query text and the response text to obtain the second data as: "I want to know some methods to improve work [CLS] efficiency. Do you have any [CLS] suggestions? [CLS][SEP]Sure. First, use [CLS] time management techniques and prioritize [CLS] important and urgent tasks. Second [CLS], use task management and collaboration [CLS] software to track progress. In addition, [CLS] keep the work environment clean and organized [CLS] to improve concentration. Finally, [CLS] take regular breaks to restore energy. [CLS] I hope these suggestions can help you [CLS] improve work efficiency! [CLS]". Here, [SEP] is the sentence marker and [CLS] is the sharding marker.

[0144] Step S170: Train the intermediate recognition model according to the second data and the sharding labels to obtain the pre-trained recognition model.

[0145] In this embodiment, the training process described above can be divided into two stages. In the second stage, the intermediate recognition model processes the second data to obtain a prediction result, where the prediction result includes the fifth hidden layer vectors corresponding to each shard label. Finally, the fifth hidden layer vectors are compared with the shard labels obtained above, and the intermediate recognition model is updated according to the comparison result to obtain the pre-trained recognition model.

[0146] In some embodiments, the intermediate recognition model is trained to obtain the pre-trained recognition model by using a sliding window BERT followed by causal convolution for training. After the first stage of training is completed and the shard labels are annotated, [SEP] is used to concatenate the query text and the response text into a single sample. Each shard of this sample has a label. During training, future shards are masked, and only the current segment is predicted based on the historical context. The token at the last Hidden Layer of the causal convolution is exactly the position of the inserted CLS [token].

[0147] Specifically, the second data is actually a long sequence formed by concatenating the query text and the response text. The sliding window technique is used to split it into multiple fixed-length text segments (shards). During training, for the currently processed shard, future shard information is masked, that is, the model is only allowed to see the current shard and the information before it. This simulates the concept of "historical context" in time series prediction. At the same time, a causal convolution layer is introduced to enhance the model's ability to understand the sequence. Causal convolution only focuses on past information and does not use future information, which is suitable for sequence prediction tasks. In the causal convolution layer, the output of the last hidden layer corresponds to the position of the inserted [CLS] token, providing the classification result for the entire segment. For each shard, based on its historical context (i.e., the current shard and the previous shards), the model needs to predict the category of the shard (such as topic classification and black-and-white classification). Using the annotated shard labels as the supervision signal, the model is trained by minimizing the loss function between the prediction result and the true label to obtain the pre-trained recognition model.

[0148] In the embodiment of the present invention, by obtaining the text to be processed, generating the data to be processed according to the text to be processed, processing the data to be processed by the pre-trained recognition model, obtaining the first hidden layer vector from the first predetermined encoder layer, and obtaining the second hidden layer vector from the second predetermined encoder layer, where the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining the first recognition result of the text to be processed by the topic classifier according to the first hidden layer vector, and determining the second recognition result of the text to be processed by the black-and-white classifier according to the second hidden layer vector. Thus, it is possible to achieve streaming processing of the text, improving the real-time performance, response speed of the large language model, and the accuracy of text recognition.

[0149] Figure 8 It is a flowchart of the text processing method according to an embodiment of the present invention. After obtaining the pre-trained recognition model, the pre-trained recognition model can be used to perform text processing tasks. Specifically, the text processing method according to an embodiment of the present invention includes the following steps:

[0150] Step S210, obtain the text to be processed.

[0151] In this embodiment, the text to be processed includes the query text and the reply text input by the user, and the reply text is generated by the large language model according to the query text.

[0152] It should be noted that in an actual scenario, during the process of the large language model generating the reply text, the reply text is processed in real time. For example, it can be processed every predetermined period, or after every predetermined number of texts are generated. Thus, streaming processing can be realized, and the real-time performance of text processing and the response speed of the large language model can be improved.

[0153] Step S220, generate the data to be processed according to the text to be processed.

[0154] In this embodiment, the text to be processed is preprocessed to obtain the data to be processed.

[0155] Specifically, Figure 9 It is a flowchart of obtaining the data to be processed according to an embodiment of the present invention. As Figure 9 shown, generating the data to be processed according to the text to be processed includes:

[0156] Step S221, slice the query text and the reply text respectively according to a predetermined length to obtain text segments.

[0157] Step S222, insert a slicing mark after each of the text segments;

[0158] Step S223, insert a sentence mark between the query text and the reply text to obtain the data to be processed.

[0159] For example, assume that in a certain query, the query text entered by the user is "I want to know some methods to improve work efficiency. Do you have any suggestions?". The response text generated by the large language model is "Of course. First, use time management techniques and prioritize important and urgent tasks. Second, use task management and collaboration software to track progress. In addition, keep the work environment clean and organized to enhance concentration. Finally, take regular breaks to restore energy. I hope these suggestions can help you improve your work efficiency!". If the predetermined length is 10 characters, the data to be processed generated is "I want to know some improve work [CLS] efficiency methods, you have any [CLS] suggestions? [CLS][SEP]Of course. First, use [CLS] time management techniques, prioritize [CLS] important and urgent tasks. Second [CLS], use task management and collaboration [CLS] software to track progress. In addition, [CLS] keep the work environment clean and organized [CLS], enhance concentration. Finally, [CLS] take regular breaks to restore energy. [CLS] I hope these suggestions can help you [CLS] improve your work efficiency! [CLS]". Among them, [CLS] is the shard marker and [SEP] is the sentence marker.

[0160] Step S230: Process the data to be processed through a pre-trained recognition model, and obtain a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer.

[0161] In this embodiment, the recognition model includes multiple encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers.

[0162] Specifically, after obtaining the data to be processed, the data to be processed is successively passed through the word embedding (Token Embedding) and position embedding (Position Embedding) steps through the input embedding layer, and finally an input vector is generated. Then, the input vector is provided to the encoder layer. The input information of the first encoder layer is the input vector output by the input embedding layer. The input information of other encoder layers (the second to the Lth encoder layer) is the output information of the previous encoder layer. Finally, the encoder converts the input sequence that has been processed through multiple layers into a series of high-dimensional hidden representations. These representations not only contain the semantic information of the input sequence, but also include the context information of each position in the sequence. L is the total number of encoder layers, and L is an integer.

[0163] In this embodiment, when processing the data to be processed through a pre-trained recognition model, a first hidden layer vector is obtained from a first predetermined encoder layer (the encoder layer in the figure), and a second hidden layer vector is obtained from a second predetermined encoder layer.

[0164] Among them, the first hidden layer vector and the second hidden layer vector refer to the feature representations output by a specific hidden layer in a neural network architecture. It contains the key information and features extracted after the input data (such as text) undergoes multiple transformations.

[0165] Step S240: Determine the first recognition result of the text to be processed according to the first hidden layer vector through a theme classifier.

[0166] In this embodiment, the theme classifier C is a pre-trained classifier, usually a fully connected layer or a multi-layer perceptron (MLP), and its input is the first hidden layer vector. The goal of this classifier is to map the high-dimensional first hidden layer vector to one or more predefined theme categories. In some embodiments, first perform an appropriate linear transformation on the first hidden layer vector (through the product of the weight matrix), and then apply an activation function (such as Softmax) so as to convert the output into the first recognition result, and the first recognition result is the theme recognition result.

[0167] Step S250: Determine the second recognition result of the text to be processed according to the second hidden layer vector through a black-and-white classifier.

[0168] In this embodiment, the number of black-and-white classifiers is the same as and corresponds one-to-one with the number of themes. Among them, when processing the data to be processed through a pre-trained recognition model, obtain the second hidden layer vector from the second predetermined encoder layer. Determine the second recognition result of the text to be processed according to the second hidden layer vector through a black-and-white classifier. The second recognition result is the black-and-white classification result corresponding to each theme. Among them, the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers.

[0169] In some embodiments, the black-and-white classifier includes a fully connected layer or a multi-layer perceptron (MLP), and its input is the second hidden layer vector from the encoder layer. The specific implementation steps are as follows: First, perform a linear transformation on the second hidden layer vector (through the product of the weight matrix) to map it to a low-dimensional space. Apply a non-linear activation function (such as ReLU or Tanh) to enhance the expression ability of the model. Finally, through an output layer (usually a Softmax layer), convert the result into a probability distribution form, indicating the probability of each possible black-and-white classification label. Based on the probability distribution, select the classification label with the highest probability as the final black-and-white classification result. Each black-and-white classifier is specifically targeted at a certain specific theme, so they can better capture and understand the compliance requirements under this theme.

[0170] In an embodiment of the present invention, by obtaining a text to be processed, generating data to be processed according to the text to be processed, processing the data to be processed through a pre-trained recognition model, obtaining a first hidden layer vector from a first predetermined encoder layer, and obtaining a second hidden layer vector from a second predetermined encoder layer, where the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining a first recognition result of the text to be processed by a topic classifier according to the first hidden layer vector, and determining a second recognition result of the text to be processed by a black-and-white classifier according to the second hidden layer vector. Thus, streaming processing of the text can be achieved, improving the real-time performance, response speed of the large language model, and the accuracy of text recognition.

[0171] Figure 10 is a schematic diagram of a text processing device according to an embodiment of the present invention. As Figure 10 shown, the text processing device according to an embodiment of the present invention includes a text acquisition unit 101, a data generation unit 102, a vector acquisition unit 103, a first recognition unit 104, and a second recognition unit 105. Among them, the text acquisition unit 101 is used to acquire a text to be processed. The data generation unit 102 is used to generate data to be processed according to the text to be processed. The vector acquisition unit 103 is used to process the data to be processed through a pre-trained recognition model, obtain a first hidden layer vector from a first predetermined encoder layer, and obtain a second hidden layer vector from a second predetermined encoder layer. The recognition model includes multiple encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers. The first recognition unit 104 is used to determine a first recognition result of the text to be processed by a topic classifier according to the first hidden layer vector. The second recognition unit 105 is used to determine a second recognition result of the text to be processed by a black-and-white classifier according to the second hidden layer vector.

[0172] In an embodiment of the present invention, by obtaining a text to be processed, generating data to be processed according to the text to be processed, processing the data to be processed through a pre-trained recognition model, obtaining a first hidden layer vector from a first predetermined encoder layer, and obtaining a second hidden layer vector from a second predetermined encoder layer, where the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining a first recognition result of the text to be processed by a topic classifier according to the first hidden layer vector, and determining a second recognition result of the text to be processed by a black-and-white classifier according to the second hidden layer vector. Thus, streaming processing of the text can be achieved, improving the real-time performance, response speed of the large language model, and the accuracy of text recognition.

[0173] Figure 11 is a schematic diagram of an electronic device according to an embodiment of the present invention. In this embodiment, the electronic device 11 includes a server, a terminal, etc. As Figure 11As shown, the electronic device 11: includes at least one processor 111; and a memory 112 communicatively connected to the at least one processor 111; and a communication component 113 communicatively connected to the scanning device, the communication component 113 receiving and transmitting data under the control of the processor 111; wherein, the memory 112 stores instructions executable by the at least one processor 111, and the instructions are executed by the at least one processor 111 to implement the above text processing method.

[0174] Specifically, the electronic device includes: one or more processors 111 and a memory 112. Figure 11 Taking one processor 111 as an example. The processor 111 and the memory 112 may be connected by a bus or other means. Figure 11 Taking the connection by bus as an example. The memory 112, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 111 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory 112, that is, to implement the above text processing method.

[0175] The memory 112 may include a program storage area and a data storage area, wherein, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store an option list, etc. In addition, the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 112 may optionally include a memory remotely provided with respect to the processor 111, and these remote memories can be connected to an external device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] One or more modules are stored in the memory 112, and when executed by one or more processors 111, implement the text processing method in any of the above method embodiments.

[0177] The above product can execute the method provided in the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided in the embodiments of the present application.

[0178] In an embodiment of the present invention, by obtaining a text to be processed, generating data to be processed according to the text to be processed, processing the data to be processed through a pre-trained recognition model, obtaining a first hidden layer vector from a first predetermined encoder layer, and obtaining a second hidden layer vector from a second predetermined encoder layer, where the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers, determining a first recognition result of the text to be processed by a topic classifier according to the first hidden layer vector, and determining a second recognition result of the text to be processed by a black-and-white classifier according to the second hidden layer vector. Thus, it is possible to achieve streaming processing of the text, improving the real-time performance, response speed of the large language model, and the accuracy of text recognition.

[0179] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, and the computer-readable program is used for a computer to execute the above-mentioned partial or all method embodiments.

[0180] That is, those skilled in the art can understand that all or part of the steps in implementing the above-mentioned embodiment methods can be completed by instructing relevant hardware through a program, and the program is stored in a storage medium, including several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes.

[0181] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A text processing method, characterized in that, The method includes: Obtain the text to be processed; Generate the data to be processed according to the text to be processed; Process the data to be processed through a pre-trained recognition model, and obtain a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer. The recognition model includes multiple encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers; Determine a first recognition result of the text to be processed through a topic classifier according to the first hidden layer vector; Determine a second recognition result of the text to be processed through a black-and-white classifier according to the second hidden layer vector.

2. The method according to claim 1, characterized in that The first recognition result is a topic recognition result, and the second recognition result is a black-and-white classification result corresponding to each topic. The number of black-and-white classifiers is the same as the number of topics and they correspond one by one.

3. The method according to claim 1, wherein The text to be processed includes a query text input by a user and a reply text, and the reply text is generated by a large language model according to the query text.

4. The method according to claim 3, wherein The generating the data to be processed according to the text to be processed includes: Slice the query text and the reply text respectively according to a predetermined length to obtain text segments; Insert slice markers after each of the text segments; Insert a sentence marker between the query text and the reply text to obtain the data to be processed.

5. The method according to claim 1, wherein The processing the data to be processed through a pre-trained recognition model includes: Generate an input vector according to the data to be processed through an input embedding layer; Process the input vector through multiple encoder layers of the pre-trained recognition model, where the input of the first encoder layer is the input vector, and the input of the second to the L-th encoder layers is the output of the previous encoder layer. L is the total number of encoder layers and L is an integer.

6. The method according to claim 1, characterized in that The pre-trained recognition model is obtained through the following steps: Obtain an initial recognition model and training samples, where the training samples include a query text and a reply text, and the reply text is generated by a large language model according to the query text; Generate first data according to the training samples; Train the initial recognition model through the first data to obtain an intermediate recognition model; Obtain prediction results of each of the training samples through the intermediate recognition model; Generate slice labels corresponding to each text segment according to the prediction results; Generate second data according to the training samples; Train the intermediate recognition model according to the second data and the slice labels to obtain the pre-trained recognition model.

7. The method according to claim 6, characterized in that, The generating the first data according to the training samples includes: Slice the query text and the reply text respectively according to a predetermined length to obtain text segments; Insert slice markers after each of the text segments; Insert summary markers after the query text and the reply text respectively to obtain the first data.

8. The method according to claim 7, characterized in that The prediction results include third hidden layer vectors corresponding to each slice marker and a fourth hidden layer vector corresponding to the summary marker.

9. The method according to claim 8, characterized in that The generating the slice labels corresponding to each text segment according to the prediction results includes: Determine attention scores according to the third hidden layer vector and the fourth hidden layer vector; Determine the sample type, where the sample type includes positive samples and negative samples; For positive samples, label each shard label as a positive label; For negative samples, determine each shard label according to the attention scores.

10. The method according to claim 6, wherein The generating the second data according to the training samples includes: Shard the query text and the response text into text segments according to a predetermined length; Insert shard markers after each of the text segments; Insert a sentence marker between the query text and the response text to obtain the second data.

11. The method according to claim 6, wherein The training the intermediate recognition model according to the second data and the shard labels to obtain the pre-trained recognition model includes: Train the intermediate recognition model in a way that a sliding window encoder module is connected with causal convolution to obtain the pre-trained recognition model.

12. A text processing device, characterized in that, The device includes: A text acquisition unit for acquiring a text to be processed; A data generation unit for generating data to be processed according to the text to be processed; A vector acquisition unit for processing the data to be processed through a pre-trained recognition model, and acquiring a first hidden layer vector from a first predetermined encoder layer and a second hidden layer vector from a second predetermined encoder layer, the recognition model includes a plurality of encoder layers, and the first predetermined encoder layer and the second predetermined encoder layer are different encoder layers; A first recognition unit for determining a first recognition result of the text to be processed through a topic classifier according to the first hidden layer vector; A second recognition unit for determining a second recognition result of the text to be processed through a black and white classifier according to the second hidden layer vector.

13. An electronic device, comprising a memory and a processor, characterized in that, The memory is used for storing one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, The computer program instructions, when executed by the processor, implement the method according to any one of claims 1-11.

Images