Text processing method and device

Abstract

The embodiments of the present specification provide a text processing method and device, wherein the text processing method includes: obtaining a text to be processed corresponding to a project type; constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector; constructing a fusion vector based on the adversarial vector and the encoding vector; processing the fusion vector by a processing unit corresponding to the project type in the language model to obtain a prediction result corresponding to the project type; by using a language model that integrates adversarial representation and is more robust, the prediction accuracy can be effectively improved, so that more accurate prediction results can be fed back to users in response to project types to meet the actual needs of users.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of machine learning technology, and more particularly to a text processing method and device. Background Art

[0002] With the continuous breakthroughs in theories and technologies related to artificial intelligence and deep learning, the development of natural language processing technology has been rapid in recent years. Supported by new digital infrastructure such as big data and cloud computing, large-scale pre-trained language models have emerged; pre-trained language models have been widely used in scenarios such as text classification, text summarization, machine translation, and dialogue generation. However, existing large-scale pre-trained language models focus more on improving the performance of models on natural inputs by optimizing model structures and pre-training tasks. This makes pre-trained language models vulnerable to adversarial attacks. In uncontrollable or untrusted environments, models are usually unable to accurately handle adversarial inputs containing natural noise or adversarial perturbations, that is, the robustness and security of models on adversarial inputs cannot be effectively guaranteed. In real scenarios, especially safety-critical scenarios, various types of text adversarial mutation risks are often encountered. The inherent fragility of pre-trained language models brings potential security risks to intelligent systems that are actually deployed and applied. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, the embodiments of this specification provide a text processing method. One or more embodiments of this specification also relate to a text processing device, a language model training method, a language model training device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a text processing method is provided, including:

[0005] Get the text to be processed corresponding to the project type;

[0006] Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector;

[0007] Constructing a fusion vector based on the adversarial vector and the encoding vector;

[0008] The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type.

[0009] According to a second aspect of the embodiments of this specification, there is provided a text processing device, including:

[0010] A text acquisition module is configured to acquire the to-be-processed text corresponding to the project type;

[0011] A text processing module is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0012] A vector construction module, configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0013] The vector processing module is configured to process the fusion vector through a processing unit corresponding to the project type in the language model to obtain a prediction result corresponding to the project type.

[0014] According to a third aspect of the embodiments of this specification, a second text processing method is provided, including:

[0015] Receiving a text processing request submitted by a user for a project type, wherein the text processing request carries a text to be processed;

[0016] Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector;

[0017] Constructing a fusion vector based on the adversarial vector and the encoding vector;

[0018] The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type as a response to the text processing request.

[0019] According to a fourth aspect of the embodiments of this specification, a second text processing device is provided, including:

[0020] A request receiving module is configured to receive a text processing request submitted by a user for a project type, wherein the text processing request carries a text to be processed;

[0021] A text processing module is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0022] A vector construction module, configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0023] The vector processing module is configured to process the fusion vector through a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type as a response to the text processing request.

[0024] According to a fifth aspect of the embodiments of this specification, a third text processing method is provided, including:

[0025] Displaying a text input interface based on an operation instruction submitted by a user for a project type, and receiving a text to be processed input by the user through the text input interface;

[0026] Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector;

[0027] Constructing a fusion vector based on the adversarial vector and the encoding vector;

[0028] The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type and display it to the user.

[0029] According to a sixth aspect of the embodiments of this specification, a third text processing device is provided, including:

[0030] A display interface module is configured to display a text input interface based on an operation instruction submitted by a user for a project type, and receive a text to be processed input by the user through the text input interface;

[0031] A text processing module is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0032] A vector construction module, configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0033] The vector processing module is configured to process the fusion vector through a processing unit corresponding to the item type in the language model, obtain a prediction result corresponding to the item type and display it to the user.

[0034] According to a seventh aspect of the embodiments of this specification, a method for training a language model is provided, including:

[0035] Get the initial corpus corresponding to the project type;

[0036] Processing the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus;

[0037] Constructing a corpus corresponding to the project type according to the adversarial corpus and the initial corpus;

[0038] The initial language model is trained based on the target corpus included in the corpus until a language model that meets the training stop condition is obtained.

[0039] According to an eighth aspect of the embodiments of this specification, a language model training device is provided, including:

[0040] A corpus acquisition module is configured to acquire initial corpus corresponding to a project type;

[0041] A corpus processing module is configured to process the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus;

[0042] A collection building module is configured to build a corpus set corresponding to the project type according to the adversarial corpus and the initial corpus;

[0043] The training model module is configured to train the initial language model based on the target corpus included in the corpus until a language model that meets the training stop condition is obtained.

[0044] According to a ninth aspect of the embodiments of this specification, a computing device is provided, including:

[0045] Memory and processor;

[0046] The memory is used to store computer-executable instructions, and the processor is used to implement any one of the text processing methods or the steps of training the language model when executing the computer-executable instructions.

[0047] According to the tenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of implementing any one of the text processing methods or the training of the language model are implemented.

[0048] According to an eleventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the text processing method or the training of the language model.

[0049] The text processing method provided in this specification can construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram after obtaining the text to be processed corresponding to the project type, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fused vector containing adversarial knowledge, and finally the fused vector is processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. In the process of processing text, using a more robust language model that integrates adversarial representation can effectively improve the prediction accuracy.

[0050] Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus with the initial corpus, the adversarial representation can be integrated into the language model during the training phase, effectively improving the intrinsic security of the pre-trained language model. It can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby building a more robust language model that better meets the project requirements of downstream projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a schematic diagram of a text processing method provided by an embodiment of this specification;

[0052] Figure 2 is a flowchart of a first text processing method provided by an embodiment of this specification;

[0053] Figure 3 is a processing flow chart of a text processing method provided by an embodiment of this specification;

[0054] Figure 4 is a structural schematic diagram of a first text processing device provided by an embodiment of this specification;

[0055] Figure 5 is a flow chart of a second text processing method provided by an embodiment of this specification;

[0056] Figure 6 is a structural schematic diagram of a second text processing device provided by an embodiment of this specification;

[0057] Figure 7 is a flowchart of a third text processing method provided by an embodiment of this specification;

[0058] Figure 8 is a structural schematic diagram of a third text processing device provided by an embodiment of this specification;

[0059] Fig. 9is a flow chart of a language model training method provided by an embodiment of this specification;

[0060] Fig.10 It is a structural diagram of a language model training device provided by an embodiment of this specification;

[0061] Fig.11 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0062] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0063] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0064] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0065] First, the terms involved in one or more embodiments of this specification are explained.

[0066] Natural language processing: an important direction in the fields of computer science and artificial intelligence, which mainly studies various theories and methods that can enable effective communication between humans and computers using natural language.

[0067] Pre-trained language model: The pre-trained language model is the evolution of the word vector model. Its essence is a word vector model with context. The core idea of ​​the pre-trained language model is to use unlabeled data to train the language model based on specific self-supervised tasks before using labeled data, so that it can learn some potential knowledge that is not related to labeling.

[0068] Adversarial text: Adding some small disturbances to the input of a deep learning model can easily change the model's prediction results. This kind of disturbance is called adversarial disturbance, the perturbed input is called an adversarial sample, and the process of misleading the model with the input adversarial sample is called an adversarial attack. If the sample input to the model is a perturbed text, it is called adversarial text.

[0069] Robustness: Robustness, also known as robustness or robustness, refers to the ability of a computer system to handle errors during execution, and the ability of an algorithm to continue to operate normally and ensure stable performance when encountering input, calculation and other anomalies.

[0070] In this specification, three text processing methods are provided. This specification also involves three text processing devices, a language model training method, a language model training device, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0071] In practical applications, most of the existing pre-trained natural language processing models are built based on large-scale unlabeled training corpora, relying on pre-training tasks such as masked language models and next sentence prediction. The training corpora usually do not contain adversarial corpora, so the models focus more on extracting universal semantic representations of a single modality. Since universal representations do not contain prior knowledge such as adversarial knowledge and adversarial relations, they are usually sensitive to adversarial variations. Therefore, their performance often degrades severely in uncontrollable or untrusted environments. For example, when users ask questions to intelligent question-answering robots, the intelligent question-answering system cannot work properly because the input questions contain spelling errors caused by homographs and homophones.

[0072] Although the prior art proposes that semantically rich representations can be constructed by fusing the glyph representations and the representations of the Chinese characters extracted from a single modality. However, it is difficult for the model to directly learn the association between adversarial variations from pre-training data, nor can it associate the variation content with specific semantic information. In addition, the implicit patterns learned in the pre-training stage may be catastrophically forgotten in the downstream fine-tuning stage. In other words, by processing the text through the pre-trained language model, the classification processing, text summary extraction, machine translation processing or keyword extraction of the text can be realized according to the needs of the specific application scenario. For example, in the information search scenario, when the user searches for the information corresponding to football player A through the search application, in order to facilitate the user to browse the queried information about football player A; usually, the corresponding article will be matched in the information database according to the keywords entered by the user, and then the summary will be extracted from the matched article to achieve the simplification of the content of the article, so that the user can understand the main content of the article by browsing the summary, so that the user can determine the article about football player A that needs to be read in a shorter time.

[0073] In this process, only by ensuring the accuracy of the abstract extraction of the article can the user be supported to browse the article he needs to read. Therefore, when extracting the abstract of the article, the text processing method provided in this embodiment can be used; see Figure 1 The schematic diagram shown includes a server 102 and a terminal 104. Specifically, in order to provide a more robust language model for text processing, the server 102 can obtain the initial corpus of the corresponding summary extraction project during training, and then use a preset adversarial processing algorithm to process the initial corpus to obtain an adversarial corpus with an adversarial relationship. Then, a corpus set corresponding to the summary extraction project is constructed based on the adversarial corpus and the initial corpus, and the initial language model of the corresponding summary extraction project is trained on this basis to obtain a language model that meets the training stop condition; the adversarial representation is integrated into the language model during the training phase, which effectively improves the inherent security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model that better meets the project requirements of downstream projects.

[0074] Furthermore, after the training process of the language model is completed, in the use stage, after receiving the keywords submitted by the user through the terminal 104 and extracting the articles from the database, the adversarial vector corresponding to the article can be constructed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the article into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit for extracting the summary in the language model to obtain the summary content of an article, and so on. After the summaries of all articles are extracted, a query page responsive to the keywords input by the user can be formed and feedback can be provided to facilitate the user to view the summary content of multiple articles. This process is completed by using a language model that integrates adversarial representation and is more robust, which can effectively improve the prediction accuracy, so that the user can browse the article summary with sufficiently high accuracy, so that the article to be read can be determined in a shorter time, and the article content can be guaranteed to be more in line with the actual reading needs.

[0075] That is to say, during the model training phase, adversarial vectors with adversarial knowledge are incorporated, and in the downstream fine-tuning phase, the adversarial vectors are incorporated into the encoding vectors. This allows the model to implicitly learn potential adversarial relationships such as similar sound, similar shape, and similar meaning in the text during the learning phase, effectively solving the problem of limited expression of single-modal knowledge, and achieving the purpose of enhancing the robustness of the natural language processing system against adversarial variations based on similar sound, similar shape, and similar meaning during the reasoning phase, thereby ensuring the performance of intelligent systems for practical applications in uncontrolled environments.

[0076] The following is an implementation of the first text processing method provided in this embodiment, which is specifically described as follows:

[0077] Figure 2 A flowchart of a first text processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0078] Step S202: Obtain the text to be processed corresponding to the project type.

[0079] Specifically, the project type refers to the type of project that processes the text to be processed, including but not limited to text classification, text summarization, machine translation, keyword extraction, etc. Correspondingly, the text to be processed refers to the text input by the upstream port that needs to be processed according to the processing strategy of the project type. The upstream port can be connected to the user or the service platform.

[0080] In actual applications, after receiving the processed text of the corresponding project type submitted by the upstream port, in order to improve the accuracy of subsequent text processing, the processed text can also be preprocessed, that is, data cleaning and / or word segmentation operations can be performed on the processed text to eliminate redundant data in the processed text, so that subsequent processing operations can be completed based on standard text.

[0081] Step S204, constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector.

[0082] Specifically, after obtaining the text to be processed corresponding to the project type as mentioned above, further, in order to improve the accuracy of text processing, that is, to avoid the influence of erroneous information contained in the text to be processed on the accuracy of text processing, the adversarial vector corresponding to the text to be processed can be constructed in combination with the adversarial relationship graph, and at the same time, the text to be processed is encoded by the encoding unit in the pre-trained language model to obtain the encoding vector corresponding to the text to be processed; so that the adversarial vector and the encoding vector can be combined in the subsequent process, so that when the model processes the text, the adversarial representation of adversarial knowledge such as adversarial relationship can be integrated, and the text processing process can be completed through a more robust language model.

[0083] Among them, the adversarial relationship graph specifically refers to a relationship graph constructed based on common characters of the same language as the text to be processed, and its structure is connected by characters with adversarial relationships. The characters that are connected to each other have adversarial relationships. The adversarial relationship refers to the relationship between characters in terms of shape, pronunciation, and / or meaning, etc., which is used to assist in creating the adversarial vector corresponding to the text to be processed. Correspondingly, the adversarial vector specifically refers to the adversarial representation of adversarial knowledge such as the adversarial relationship corresponding to each character in the text to be processed, which is constructed by combining the learned node representation in the adversarial relationship graph based on the text to be processed; that is, the adversarial vector can reduce the impact of the adversarial relationship when it is subsequently processed by the language model, thereby improving the accuracy of text processing prediction. Correspondingly, the language model specifically refers to a model obtained through pre-training that meets the usage requirements of the project type; when the project type is a text summary extraction project, the language model is a summary extraction model, which is used to extract the corresponding text summary in the text to be processed; when the project type is a keyword extraction project, the language model is a keyword extraction model, which is used to extract keywords from the text to be processed; when the project type is a text classification project, the language model is a text classification model, which is used to classify the text to be processed. Correspondingly, the encoding vector specifically refers to the vector expression obtained after the encoding unit in the language model encodes the text to be processed.

[0084] Based on this, in order to improve the accuracy of text processing, before processing the text to be processed through the language model, an adversarial vector corresponding to the text to be processed can be constructed according to the preset adversarial relationship graph, so as to integrate the adversarial relationship of the corresponding text to be processed into the adversarial vector, so as to reduce the impact of the adversarial relationship on the model prediction performance when processing the text to be processed later.

[0085] Furthermore, before constructing the adversarial vector corresponding to the text to be processed based on the adversarial relationship graph, the adversarial relationship graph needs to be pre-constructed so that multiple reuse can be achieved, and the adversarial vector corresponding to the text to be processed input by the upstream port can be constructed for any project type; in this embodiment, the construction of the adversarial relationship graph is specifically implemented as steps S2042 to S2048;

[0086] Step S2042, obtaining basic characters corresponding to the item type.

[0087] Step S2044, determining basic shape confrontation information and basic sound confrontation information between the basic characters by traversing the basic characters.

[0088] Specifically, the basic characters specifically refer to the common characters of the corresponding project types. This embodiment takes the Chinese language corresponding to the text to be processed as an example for explanation. Correspondingly, the basic characters are the common Chinese characters. The descriptions of other scenarios can refer to the same or corresponding descriptions of this embodiment, and will not be repeated here. Correspondingly, the basic shape confrontation information specifically refers to the information corresponding to the shape proximity relationship of the corresponding glyph dimension, that is, if the glyphs between the characters are similar, they have basic shape confrontation information with each other, and the degree of shape proximity can be determined by calculating the edit distance; correspondingly, the basic sound confrontation information specifically refers to the information corresponding to the sound proximity relationship of the corresponding pronunciation dimension, that is, if the pronunciations between the characters are similar, the project has a basic sound correspondence relationship, and the degree of sound proximity can be determined by calculating the similarity of the basic phonemes.

[0089] Based on this, when constructing an adversarial relationship graph, in order to ensure that the text to be processed input at any time in the project type scenario can construct an adversarial vector corresponding to the text to be processed, the adversarial relationship graph can be constructed based on the language corresponding to the usage scenario, that is, the basic characters of the corresponding project type are obtained, and then the basic shape adversarial information and basic sound adversarial information between the basic characters are determined by traversing the basic characters, so as to facilitate the subsequent construction of an adversarial relationship graph based on the basic shape adversarial information and the basic sound adversarial information.

[0090] Furthermore, considering that the amount of basic character data corresponding to the project type is large, and the adversarial relationship diagram constructed subsequently needs to cover a wider range of usage scenarios, when traversing the basic characters, the basic character image can be constructed through the traversal results, and then the basic shape adversarial information between the characters is calculated based on this, and the basic sound adversarial information is calculated by calculating the similarity between the phoneme sequences. In this embodiment, the specific implementation method is as follows:

[0091] By traversing the basic characters, a basic character image corresponding to the basic character is constructed; the image features of the basic character image are extracted as the character vector corresponding to the basic character; the basic shape confrontation information between the basic characters is calculated according to the character vector; and by traversing the basic characters, the phoneme sequence corresponding to the basic character is determined; and the basic sound confrontation information between the basic characters is calculated according to the phoneme sequence.

[0092] Specifically, the basic character image specifically refers to the image representation obtained by converting each basic character; correspondingly, the character vector specifically refers to the vector expression corresponding to the character constructed by using the visual features of the convolutional neural network; correspondingly, the phoneme sequence specifically refers to the sequence composed of phonemes obtained by decomposing the basic characters into phonemes.

[0093] Based on this, after obtaining the basic characters, we can traverse the basic characters to construct the basic character images corresponding to each basic character, and then use the convolutional neural network to extract the image features of the basic character images to define the character vectors corresponding to each basic character. Finally, we calculate the distance between the character vectors to determine the degree of similarity between the basic characters, and select the basic characters corresponding to the character vectors whose distance is less than the threshold to construct the basic shape adversarial information, that is, to construct the shape proximity relationship between the basic characters.

[0094] At the same time, the basic characters can be traversed to determine the phoneme sequences corresponding to each basic character, and then the phoneme sequence similarity can be calculated to determine the phonetic similarity between the basic characters corresponding to the character vectors. The basic characters corresponding to the phoneme sequences with a similarity less than a threshold are selected to construct basic phonetic adversarial information, that is, to construct the phonetic similarity relationship between the basic characters, so as to facilitate the subsequent combination of the basic shape adversarial information and the basic sound adversarial information to complete the construction of the adversarial relationship diagram.

[0095] It should be noted that when the upstream port is connected to an alphabetic language, such as English, French, etc., the similarity between the glyph vectors can be directly calculated to determine the basic shape confrontation information between the basic characters based on the similarity calculation results. Or when the upstream port is connected to a pictographic language, such as Chinese, Japanese, etc., the basic shape confrontation information between the basic characters can be comprehensively calculated by combining the glyph vectors, four-corner codes, and stroke order between the characters. In actual applications, the determination of the basic shape confrontation information between basic characters can select a suitable solution based on the actual application scenario, and this embodiment does not make any limitation here.

[0096] In addition, when calculating the basic sound adversarial information between basic characters, you can also calculate the pronunciation similarity of the characters, such as calculating the similarity between Chinese pinyin or English phonetic symbols, to quantify the basic sound adversarial information between basic characters, so as to facilitate the subsequent construction of an adversarial relationship graph containing the adversarial relationships between all basic characters from the dimensions of similar shape and similar sound.

[0097] In summary, by calculating the basic shape adversarial information based on the basic character images corresponding to the characters, and determining the basic sound adversarial information between basic characters by calculating the similarity of phoneme sequences, the accuracy of determining the relationship between similar shapes and sounds can be guaranteed, so that an adversarial relationship graph covering a wider range of adversarial relationships can be constructed in the future to support the accuracy of text processing in project type scenarios.

[0098] Step S2046, constructing a basic character node according to the basic character, and constructing basic edge information according to the basic shape confrontation information and the basic sound confrontation information.

[0099] Step S2048, establishing a connectivity relationship between the basic character nodes and the basic edge information, and constructing the adversarial relationship graph according to the connectivity relationship establishment result.

[0100] Specifically, after determining the basic sound confrontation information and the basic shape confrontation information between the basic characters as mentioned above, further, based on the basic sound confrontation information and the basic shape confrontation information, characters with similar shape and sound relationships can be determined. Then, the basic characters are used as nodes, and the basic shape confrontation information and the basic sound confrontation information are used as edges. The connectivity relationship between nodes is established using the edges, so that an adversarial relationship graph can be obtained based on the connectivity relationship establishment result, with the basic characters as nodes and the similar shape and sound relationships between characters as edges.

[0101] The basic character node specifically refers to a node constructed with a single character in the adversarial relationship graph, and the basic edge information specifically refers to an edge constructed with a single basic shape adversarial information or basic sound adversarial information in the adversarial relationship graph, and each basic character node can be connected to at least one basic edge. In other words, any basic character node is connected to its corresponding similar-looking character and / or similar-sounding character.

[0102] In addition, considering that the adversarial relationship graph includes shape-similarity relationships, sound-similarity relationships, and characters, and there are shape-similarity relationships or sound-similarity relationships between adjacent characters, there are also indirect shape-similarity relationships and sound-similarity relationships between indirect characters. For example, the shape-similar characters corresponding to character A include B, C, D, etc., and the characters similar in shape to character B include E, F, etc. Then, it can be determined that characters E, F, etc. have an indirect shape-similarity relationship with character A. Correspondingly, there is also an indirect sound-similarity relationship. Therefore, in the application stage, when constructing the adversarial vector corresponding to the text to be processed based on the adversarial relationship graph, other adversarial relationships mapped after the construction of the adversarial relationship graph can be used in combination. This embodiment does not make any limitations here.

[0103] In summary, by using the basic characters as the contact character nodes and the basic sound adversarial information and basic shape adversarial information as the basic edges to construct the adversarial relationship graph, it is possible to fully reflect the adversarial relationships between characters through the adversarial relationship graph, so as to facilitate subsequent full exploration of the adversarial relationship graph according to project requirements and construct adversarial vectors representing multiple dimensions for improving the accuracy of text processing.

[0104] Furthermore, in the process of constructing the adversarial vector corresponding to the text to be processed based on the above-mentioned constructed adversarial relationship graph, considering that there may be some redundant data in the text to be processed that will affect the processing accuracy, the text to be processed can be preprocessed first and then mapped to the adversarial relationship graph to construct the adversarial vector corresponding to the text to be processed, so as to complete the processing operation through the model subsequently. In this embodiment, the specific implementation is as shown in steps S2142 to S2148.

[0105] Step S2142: Preprocess the text to be processed to obtain the text sequence corresponding to the text to be processed.

[0106] Step S2144: Determine the character nodes corresponding to each character in the text sequence and the edge information associated with the character nodes in the preset adversarial relationship graph.

[0107] Step S2146: Determine the global adversarial information corresponding to the text to be processed according to the character nodes and the edge information.

[0108] Specifically, preprocessing specifically refers to data cleaning and / or word segmentation of the text to be processed; accordingly, the text sequence specifically refers to a set of characters contained in the text to be processed; accordingly, the character node specifically refers to the node corresponding to the basic character node in the adversarial relationship graph between the characters in the text sequence; accordingly, the edge information specifically refers to the information corresponding to the edge that has a connectivity relationship with the character node in the adversarial relationship graph, which is used to reflect the adversarial relationship of each character in the text sequence; accordingly, the global adversarial information specifically refers to the set of adversarial relationships corresponding to each character in the text sequence, which is used for the subsequent construction of the adversarial vector.

[0109] Based on this, after obtaining the text to be processed, in order to facilitate subsequent processing, the text to be processed can be cleaned and / or segmented to obtain the text sequence corresponding to the text to be processed. Then, the character nodes corresponding to each character in the text sequence and the edge information associated with each character node are determined in the preset adversarial relationship graph to determine the adversarial relationship corresponding to each character in the text sequence; finally, the global adversarial information corresponding to the text to be processed can be determined based on the character nodes and edge information, that is, the adversarial relationship corresponding to each character in the text to be processed, so as to facilitate the subsequent construction of the adversarial vector corresponding to the text to be processed based on this, so as to achieve accurate output when processed through the language model.

[0110] Furthermore, when determining the global adversarial information corresponding to the text to be processed based on the character nodes and edge information, it is actually to mine all adversarial relationships corresponding to each character in the text to be processed in the adversarial relationship graph, so that the adversarial information can be determined from multiple dimensions, so as to facilitate the subsequent construction of the adversarial vector with better adversarial performance. In this embodiment, the specific implementation method is as follows:

[0111] According to the connection relationship between the character nodes and the edge information in the adversarial relationship graph, the shape adversarial information, the sound adversarial information and the semantic adversarial information corresponding to the text sequence are determined; by integrating the shape adversarial information, the sound adversarial information and the semantic adversarial information, the global adversarial information corresponding to the text to be processed is obtained.

[0112] Specifically, the connection relationship refers to the adversarial relationship between character nodes and edge information in the adversarial relationship graph; correspondingly, the shape adversarial information refers to the shape-similar relationship between characters; the sound adversarial information refers to the sound-similar relationship between characters; correspondingly, the semantic adversarial information refers to the semantically similar relationship between characters.

[0113] Based on this, since the adversarial relationship graph is constructed by combining the phonetic and geometric relationships between basic characters, the connection relationship between character nodes and edge information in the adversarial relationship graph can be directly used to determine the geometric adversarial information and phonetic adversarial information corresponding to each character in the text sequence; at the same time, since the adversarial relationship graph covers a large number of basic characters, there are first-degree and second-degree relationships between them, so the semantic adversarial information corresponding to the characters in the text sequence can be determined through semantic analysis; by integrating the geometric adversarial information, phonetic adversarial information and semantic adversarial information, the global adversarial information corresponding to the text to be processed can be obtained, so as to facilitate the subsequent construction of an adversarial vector based on this, and realize the integration into the language model for text processing operations.

[0114] In summary, the shape adversarial information, sound adversarial information and semantic adversarial information corresponding to the characters in the text sequence are obtained through the adversarial relationship graph, so that the adversarial vector corresponding to the text to be processed can be constructed from multiple dimensions, thereby ensuring that the integrated adversarial relationship is more comprehensive, promoting the accuracy of text processing in subsequent language model processing, and avoiding interference caused by redundant data.

[0115] Step S2148, constructing the adversarial vector based on the global adversarial information and the text to be processed.

[0116] Furthermore, after obtaining the global adversarial information, since the global adversarial information is a set of adversarial relationships corresponding to the text to be processed mined from the adversarial relationship graph, the adversarial vector corresponding to the text to be processed can be obtained based on the global adversarial information, which is convenient for subsequent use.

[0117] In practical applications, when constructing adversarial vectors, graph embedding technology is actually used to learn the node representations in the adversarial relationship graph, so as to obtain adversarial vectors containing adversarial knowledge such as adversarial relationships, which fully represent the adversarial information of the text to be processed. At this stage, graph embedding technologies such as RandWalk, DeepWalk, Node2Vec, and GNN can be used. When implementing it specifically, it can be determined according to the actual application scenario, and this embodiment does not make any restrictions here. In other words, through graph embedding technology, the vector representation of each character node in the adversarial relationship graph can be learned to map the adversarial knowledge in the adversarial relationship graph to a high-dimensional vector that can be processed by the computer, so as to obtain the adversarial knowledge representation corresponding to the text to be processed, that is, the adversarial vector.

[0118] This embodiment takes the project type as keyword extraction as an example to illustrate the text processing method. The processing scenarios in other scenarios can refer to the same or corresponding description content of this embodiment, and will not be elaborated here. It should be noted that in actual applications, the text to be processed input from the upstream port may contain a large number of characters. For the convenience of description, this embodiment only takes part of the content as an example for explanation.

[0119] For example, the text to be processed obtained from the upstream port input is {... Let's go to see the sunrise...}. At this time, the text to be processed is cleaned, and the punctuation marks and headers and footers in the text to be processed are removed to obtain the standard text {Let's go to see the sunrise}. Then, the standard text is segmented to obtain the text sequence {Let's go to see the sunrise}. Further, the graph embedding technology is used to learn the node representation in the pre-constructed adversarial relationship graph to obtain the adversarial vector S1 corresponding to the text to be processed. That is, the character nodes corresponding to each character in the text sequence and the edge information associated with the character nodes are determined in the adversarial relationship graph. By traversing the connection relationship between the character nodes and the edge information in the adversarial relationship graph, the shape-similar relationship, the sound-similar relationship and the meaning-similar relationship corresponding to each character node in the text sequence are determined, that is, the adversarial relationship corresponding to each character in {Let's go to see the sunrise}; finally, based on the adversarial relationship corresponding to each character, the adversarial vector corresponding to the text to be processed can be obtained, that is, the adversarial representation of the adversarial relationship of each character is integrated, so that the keywords in the text to be processed can be accurately extracted during the subsequent language model processing.

[0120] In summary, by extracting global adversarial information based on the adversarial relationship graph, we can fully explore the adversarial relationships contained in the adversarial relationship graph, so as to construct the adversarial vector corresponding to the text to be processed, which can cover a wider range of dimensions, thereby ensuring that when the language model is processed, it can complete accurate predictions in the scenario of the project type, thereby improving the accuracy of text processing.

[0121] Step S206, constructing a fusion vector based on the adversarial vector and the encoding vector.

[0122] Specifically, after obtaining the adversarial vector and encoding vector corresponding to the text to be processed, since both the adversarial vector and the encoding vector are expressions of the text to be processed in their respective vector mapping spaces, before responding to the downstream task to process the encoding vector corresponding to the text to be processed, the adversarial vector and the encoding vector can be fused to obtain a fused vector. In other words, the adversarial relationship in the adversarial vector is integrated into the encoding vector, so that the language model can reduce the impact of adversarial information on text processing when processing it, thereby making the text processing more accurate. That is, the fused fusion vector can be understood as an enhanced robust representation.

[0123] Furthermore, since the adversarial vector is constructed from the adversarial relationship graph to construct an adversarial vector carrying an adversarial relationship, and the encoding vector is a vector obtained by encoding the text to be processed by the encoding unit in the language model, the two belong to vector expressions of different modalities, so cross-modal fusion is required. Considering that fusion at different stages will have different effects on the output results of the language model, in actual applications, fusion can be selected at different stages in combination with actual scenarios. In this embodiment, the specific implementation method is as follows:

[0124] (1) The adversarial vector and the encoding vector are fused to obtain the fused vector.

[0125] Specifically, in the early modal fusion stage, the encoding vector and the adversarial vector can be directly fused, that is, after obtaining the adversarial vector and the encoding vector, the two can be directly fused to obtain a fused vector.

[0126] In addition, during the model training phase, directly fusing the vectors can reduce the volume of the vectors, so that the model in the training phase can be given fewer parameters, making the model smaller and more accurate.

[0127] (2) Inputting the adversarial vector into an adversarial language model for processing to obtain a target adversarial vector, and inputting the encoding vector into an encoding language model for processing to obtain a target encoding vector; fusing the target adversarial vector and the target encoding vector to obtain the fused vector.

[0128] Specifically, in the mid-term modal fusion stage, the adversarial vector can be processed to obtain the target adversarial vector, and the encoding vector can be processed to obtain the encoding vector, and then the processed vectors can be fused. In other words, in this stage, the adversarial vector and the encoding vector can be processed respectively using the preset language models corresponding to the fusion requirements, so as to fuse the processing results.

[0129] Based on this, after obtaining the adversarial vector and the encoding vector, the adversarial vector can be input into its corresponding adversarial language model for processing to obtain the target adversarial vector; and the encoding vector can be input into the encoding language model for processing to obtain the target encoding vector; finally, the target adversarial vector and the target encoding vector are fused to obtain the fused vector.

[0130] That is to say, in the mid-term modal fusion stage, vector fusion processing is performed, and the fused vectors are all target vectors processed by the preset language model, which can make the fusion more refined, thereby promoting the accuracy of subsequent text processing.

[0131] In summary, by adopting fusion processing methods at different stages to obtain fusion vectors, different practical application scenarios can be achieved, thereby meeting the usage requirements in the current scenario and improving the accuracy of subsequent text processing.

[0132] Step S208: Process the fusion vector by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type.

[0133] Specifically, after the encoding vector and the adversarial vector are fused as described above, the fused vector can be further processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. The processing unit corresponding to the project type specifically refers to the unit that processes the fused vector, which corresponds to the project scenario of the project type. The processing unit may have different capabilities in different scenarios. For example, in the summary generation scenario, the processing unit is a decoding unit, which is used to decode the text summary from the fused vector; in the text classification scenario, the processing unit is a classification unit, which is used to score the fused vector and determine the classification result corresponding to the text to be processed. Correspondingly, the prediction result specifically refers to the result of the corresponding project type obtained after the text to be processed is processed by the language model; for example, if the project type is a summary extraction project, the prediction result is the summary text extracted from the text to be processed; or if the project type is a keyword extraction project, the prediction result is the keyword extracted from the text to be processed; or if the project type is a text classification project, the prediction result is the text classification result.

[0134] That is to say, after obtaining the fusion vector, the operation of processing the fusion vector depends on the downstream task. When the downstream task corresponds to a generation task (the corresponding project type is a generated text project), such as translation, summary extraction, keyword extraction, etc., the fusion vector can be processed by a trained decoding unit to obtain the target text of the corresponding generation task; when the downstream task corresponds to a non-generation task (the corresponding project type is a non-generation project), such as text classification, text scoring, etc., the fusion vector can be directly processed by a project processing unit that meets the usage scenario to obtain the target text of the corresponding non-generation task.

[0135] In actual applications, the processing unit corresponding to the project type in the language model can be set according to the actual application scenario. The setting relationship depends on the usage requirements of the project type, and this embodiment does not make any limitation here.

[0136] Using the above example, after obtaining the adversarial vector S1 corresponding to the text to be processed, the text to be processed can be input into the pre-trained language model corresponding to the keyword extraction, and the encoding unit in the language model encodes the text to be processed to obtain the encoding vector S2, and then the adversarial vector S1 and the encoding vector S2 are fused in the early modal fusion stage to obtain the fusion vector S corresponding to the text to be processed. Furthermore, the fusion vector is input into the decoding unit in the pre-trained language model for decoding, and the keyword "sunrise" corresponding to the text to be processed can be obtained according to the decoding result, and it can be output.

[0137] In summary, the fusion vector is processed by the processing unit in the language model to obtain the prediction result corresponding to the project type. In the process of text processing, the use of a more robust language model that integrates adversarial representation can effectively improve the prediction accuracy.

[0138] Furthermore, in the model training stage, in order to improve the prediction accuracy of the language model, it is also necessary to enable the model to learn adversarial capabilities, so that in the model fine-tuning stage, the potential adversarial relationships such as similar sound, similar form, and similar meaning in the text can be implicitly learned, which effectively solves the problem of limited expression of single-modal knowledge, and achieves the purpose of enhancing the robustness of the natural language processing system against adversarial variations based on similar sound, similar form, and similar meaning in the reasoning stage, and ensures the performance of the intelligent system for practical application in an uncontrolled environment; in this process, in order to enable the trained language model to be applied to the corresponding project scenario, when obtaining the initial corpus of the training model, it can be completed based on the specific downstream task. In this embodiment, the specific implementation method is as follows:

[0139] Step S2082, obtaining an initial corpus corresponding to the project type;

[0140] Step S2084, processing the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus;

[0141] Specifically, the initial corpus refers to the corpus required for training the language model, and the initial corpus is associated with the project type, that is, the determination of the initial corpus depends on the corresponding downstream project requirements. Correspondingly, the adversarial processing algorithm specifically refers to an algorithm that replaces and modifies the initial corpus by replacing characters, word units, or sentence units, and the replaced or modified characters, word units, or sentence units are similar in semantics or pronunciation to the original content, or the characters are similar in shape, so that adversarial information can be integrated into the corpus; accordingly, the adversarial corpus is the sample corpus after the adversarial information is integrated.

[0142] Furthermore, in order to improve the richness of the samples, when the initial corpus is processed based on the adversarial processing algorithm, the adversarial corpus can be created by combining the processing results and the intermediate results. In this embodiment, the specific implementation method is as follows:

[0143] According to the adversarial processing algorithm, in the adversarial relationship diagram, the associated characters and adjacent characters associated with the initial corpus are screened; the initial corpus is updated according to the associated characters and the adjacent characters respectively to obtain an intermediate corpus and a target corpus; and the intermediate corpus and the target corpus are determined as the adversarial corpus.

[0144] Specifically, the associated characters specifically refer to the characters in the adversarial relationship graph that have a first-degree relationship with the characters included in the initial corpus; correspondingly, the adjacent characters specifically refer to the characters in the adversarial relationship graph that have a second-degree relationship with the characters included in the initial corpus. Accordingly, updating the initial corpus specifically refers to the operation of replacing the characters in the initial corpus with adjacent characters or associated characters; accordingly, the intermediate corpus specifically refers to the corpus obtained after each character replacement, and the target corpus specifically refers to the corpus obtained after all replacement processing operations are completed.

[0145] Based on this, after obtaining the initial corpus, in order to improve the accuracy of model training and enhance the robustness of the model, the adversarial relationship graph can be used to determine the associated characters and adjacent characters corresponding to the initial corpus, and then the associated characters and adjacent characters are used to gradually replace the characters in the initial corpus, and the intermediate text is retained after each replacement as the intermediate corpus, and the target corpus is obtained after all replacements are completed. At this time, the intermediate corpus and the target corpus are used as adversarial corpora for subsequent construction of the corpus set for the training model in combination with the initial expectations, thereby improving the richness of the samples and training a language model with better predictive ability.

[0146] In summary, by adopting adversarial processing algorithms to expand the number and richness of the corpus, the subsequently constructed corpus can contain more corpora corresponding to different levels of difficulty. Retraining the model in this way can effectively improve the model's prediction accuracy and robustness.

[0147] Step S2086, constructing a corpus corresponding to the project type according to the adversarial corpus and the initial corpus;

[0148] Step S2088: training the initial language model based on the target corpus included in the corpus until the language model that meets the training stop condition is obtained.

[0149] Specifically, after obtaining the adversarial corpus as described above, by integrating the adversarial corpus and the initial corpus, a corpus set for training the model can be obtained. Finally, the initial language model is trained in combination with the target corpus included in the corpus set until the language model that meets the training stop condition is obtained.

[0150] Among them, the target corpus specifically refers to the samples in the corpus set that can train the initial language model; correspondingly, the training stop condition specifically refers to the condition for stopping training the model after adjusting the parameters of the model, which can be the number of model iterations or the comparison of loss values; that is, after the number of model iterations or the comparison of loss values meets the conditions, the training can be stopped, and the trained model can be used as the language model corresponding to the project type.

[0151] Furthermore, during the process of training the model, it is actually continuously adjusting the parameters of the model, that is, inputting the target corpus included in the corpus set into the initial language model for processing to obtain the predicted text corresponding to the target corpus included in the corpus set; adjusting the parameters of the initial language model according to the sample text and the predicted text corresponding to the target corpus included in the corpus set until the language model that meets the training stop condition is obtained.

[0152] For example, if a model for keyword extraction needs to be trained, in order to improve the accuracy of the model in extracting keywords, after obtaining the initial corpus {"A goes hiking"}, the initial corpus can be processed using a preset adversarial processing algorithm, that is, determining the associated characters {"by..."} corresponding to the character "A" in the adversarial relationship graph, and the adjacent characters {"home..."}, the associated characters {"area..."} corresponding to the character "goes", and the adjacent characters {"bend..."}, and successively determining the adjacent characters and associated characters corresponding to each character; then using the associated characters and associations to update the initial corpus to obtain intermediate corpora {"by goes hiking"}, {"home goes hiking"}... {"A area hiking"}, {"A bend hiking"}; and target corpora {"by area hiking"}, {"home bend hiking"}, {"by bend hiking"}, {"home area hiking"}...; then constructing a corpus set based on the intermediate corpora, target corpora, and the initial corpus.

[0153] Furthermore, input the target corpus included in the corpus set into the keyword extraction model to be trained to obtain the predicted text corresponding to each target corpus. Finally, adjust the parameters of the keyword extraction model according to the sample text and the predicted text corresponding to the target corpus included in the corpus set until the target keyword extraction model that meets the training stop condition is obtained.

[0154] In summary, by adopting the adversarial processing algorithm to actively generate adversarial text for the language model, and adding the generated adversarial text together with the intermediate form of perturbation text to the model fine-tuning corpus, the language model of the corresponding project type is adversarially fine-tuned, so that a language model with higher prediction accuracy can be trained.

[0155] The text processing method provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model, and the prediction result corresponding to the project type can be obtained. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby building a more robust language model to better meet the project requirements of downstream projects.

[0156] The following combination Figure 3 , taking the application of the text processing method provided in this specification in the text summary extraction scenario as an example, the text processing method is further explained. Figure 3 A flowchart of a text processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0157] Step S302, obtaining the to-be-processed text corresponding to the project type.

[0158] Step S304: preprocess the text to be processed to obtain a text sequence corresponding to the text to be processed.

[0159] Step S306, determining the character node corresponding to each character in the text sequence and the edge information associated with the character node in the preset adversarial relationship graph.

[0160] Step S308, determining the shape adversarial information, the sound adversarial information and the semantic adversarial information corresponding to the text sequence according to the connection relationship between the character nodes and the edge information in the adversarial relationship graph.

[0161] Step S310, obtaining global confrontation information corresponding to the text to be processed by integrating the shape confrontation information, the sound confrontation information and the semantic confrontation information.

[0162] Step S312, constructing an adversarial vector based on the global adversarial information and the text to be processed.

[0163] Step S314: Process the text to be processed by the encoding unit in the language model to obtain an encoding vector.

[0164] Step S316, fusing the adversarial vector and the encoding vector to obtain a fused vector.

[0165] Step S318, processing the fusion vector by the processing unit corresponding to the project type in the language model to obtain a prediction result corresponding to the project type.

[0166] In summary, after obtaining the text to be processed corresponding to the project type, the adversarial vector corresponding to the text to be processed can be constructed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. In the process of text processing, the use of a more robust language model that integrates adversarial representations can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby building a more robust language model that better meets the project requirements of downstream projects.

[0167] Corresponding to the above method embodiment, this specification also provides a text processing device embodiment, Figure 4 FIG. 1 shows a schematic diagram of the structure of a first text processing device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0168] The text acquisition module 402 is configured to acquire the to-be-processed text corresponding to the project type;

[0169] The text processing module 404 is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0170] A vector construction module 406, configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0171] The vector processing module 408 is configured to process the fusion vector through a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type.

[0172] In an optional embodiment, the text processing module 404 is further configured to:

[0173] Preprocess the text to be processed to obtain a text sequence corresponding to the text to be processed; determine the character node corresponding to each character in the text sequence and the edge information associated with the character node in the preset adversarial relationship graph; determine the global adversarial information corresponding to the text to be processed based on the character node and the edge information; and construct the adversarial vector based on the global adversarial information and the text to be processed.

[0174] In an optional embodiment, the text processing module 404 is further configured to:

[0175] According to the connection relationship between the character nodes and the edge information in the adversarial relationship graph, the shape adversarial information, the sound adversarial information and the semantic adversarial information corresponding to the text sequence are determined; by integrating the shape adversarial information, the sound adversarial information and the semantic adversarial information, the global adversarial information corresponding to the text to be processed is obtained.

[0176] In an optional embodiment, the construction of the adversarial relationship graph includes:

[0177] Obtain basic characters corresponding to the project type; determine basic shape confrontation information and basic sound confrontation information between the basic characters by traversing the basic characters; construct basic character nodes according to the basic characters, and construct basic side information according to the basic shape confrontation information and the basic sound confrontation information; establish a connectivity relationship between the basic character nodes and the basic side information, and construct the confrontation relationship graph according to the connectivity relationship establishment result.

[0178] In an optional embodiment, determining the basic shape confrontation information between the basic characters by traversing the basic characters includes:

[0179] By traversing the basic characters, a basic character image corresponding to the basic character is constructed; the image features of the basic character image are extracted as the character vector corresponding to the basic character; the basic shape confrontation information between the basic characters is calculated according to the character vector; correspondingly, the basic sound confrontation information between the basic characters is determined by traversing the basic characters, including: by traversing the basic characters, a phoneme sequence corresponding to the basic character is determined; and the basic sound confrontation information between the basic characters is calculated according to the phoneme sequence.

[0180] In an optional embodiment, the constructing vector module 406 is further configured to:

[0181] The adversarial vector and the encoding vector are fused to obtain the fused vector; or, the adversarial vector is input into an adversarial language model for processing to obtain a target adversarial vector, and the encoding vector is input into an encoding language model for processing to obtain a target encoding vector; the target adversarial vector and the target encoding vector are fused to obtain the fused vector.

[0182] In an optional embodiment, the training of the language model includes:

[0183] Acquire initial corpus corresponding to the project type; process the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus; construct a corpus set corresponding to the project type according to the adversarial corpus and the initial corpus; train an initial language model based on the target corpus included in the corpus set until the language model that meets the training stop condition is obtained.

[0184] In an optional embodiment, the processing the initial corpus according to a preset adversarial processing algorithm to obtain the adversarial corpus includes:

[0185] According to the adversarial processing algorithm, in the adversarial relationship diagram, the associated characters and adjacent characters associated with the initial corpus are screened; the initial corpus is updated according to the associated characters and the adjacent characters respectively to obtain an intermediate corpus and a target corpus; and the intermediate corpus and the target corpus are determined as the adversarial corpus.

[0186] In an optional embodiment, the training of the initial language model based on the target corpus included in the corpus until the language model that satisfies the training stop condition is obtained includes:

[0187] The target corpus contained in the corpus is input into the initial language model for processing to obtain the predicted text corresponding to the target corpus contained in the corpus; the initial language model is adjusted according to the sample text corresponding to the target corpus contained in the corpus and the predicted text until the language model that meets the training stop condition is obtained.

[0188] The text processing device provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model to better meet the project requirements of downstream projects.

[0189] The above is a schematic scheme of a text processing device of this embodiment. It should be noted that the technical scheme of the text processing device and the technical scheme of the above text processing method belong to the same concept, and the details not described in detail in the technical scheme of the text processing device can be referred to the description of the technical scheme of the above text processing method.

[0190] Figure 5 A flow chart of a second text processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0191] Step S502: receiving a text processing request submitted by a user for a project type, wherein the text processing request carries a text to be processed.

[0192] Step S504, constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector.

[0193] Step S506, constructing a fusion vector based on the adversarial vector and the encoding vector.

[0194] Step S508: Process the fusion vector by the processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type as a response to the text processing request.

[0195] The text processing method provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model, and the prediction result corresponding to the project type can be obtained. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby building a more robust language model to better meet the project requirements of downstream projects.

[0196] It should be noted that the second text processing method provided in this embodiment belongs to the same concept as the scheme of the first text processing method mentioned above, and the same or corresponding description contents can be referred to the above embodiment, and this embodiment will not be elaborated in detail here.

[0197] Corresponding to the above method embodiment, this specification also provides a text processing device embodiment, Figure 6 FIG. 2 shows a schematic diagram of the structure of a second text processing device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0198] The request receiving module 602 is configured to receive a text processing request submitted by a user for a project type, wherein the text processing request carries a text to be processed;

[0199] The text processing module 604 is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0200] A vector construction module 606, configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0201] The vector processing module 608 is configured to process the fusion vector through the processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type as a response to the text processing request.

[0202] The text processing device provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model to better meet the project requirements of downstream projects.

[0203] The above is a schematic scheme of the second text processing device of this embodiment. It should be noted that the technical scheme of the text processing device and the technical scheme of the second text processing method mentioned above are of the same concept, and the details not described in detail in the technical scheme of the text processing device can be found in the description of the technical scheme of the second text processing method mentioned above.

[0204] Figure 7 A flowchart of a third text processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0205] Step S702: displaying a text input interface based on the operation instruction submitted by the user for the project type, and receiving the text to be processed input by the user through the text input interface.

[0206] Step S704, constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector.

[0207] Step S706, constructing a fusion vector based on the adversarial vector and the encoding vector.

[0208] Step S708: Process the fusion vector by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type and display it to the user.

[0209] The text processing method provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model, and the prediction result corresponding to the project type can be obtained. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby building a more robust language model to better meet the project requirements of downstream projects.

[0210] It should be noted that the third text processing method provided in this embodiment belongs to the same concept as the scheme of the first text processing method mentioned above, and the same or corresponding description contents can be referred to the above embodiment, and this embodiment will not be elaborated in detail here.

[0211] Corresponding to the above method embodiment, this specification also provides a text processing device embodiment, Figure 8 FIG. 2 shows a schematic diagram of the structure of a third text processing device provided by an embodiment of this specification. Figure 8 As shown, the device comprises:

[0212] The display interface module 802 is configured to display a text input interface based on an operation instruction submitted by a user for a project type, and receive a text to be processed input by the user through the text input interface;

[0213] The text processing module 804 is configured to construct an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship diagram, and to process the text to be processed through an encoding unit in a language model to obtain an encoding vector;

[0214] A vector construction module 806 is configured to construct a fusion vector based on the adversarial vector and the encoding vector;

[0215] The vector processing module 808 is configured to process the fusion vector through a processing unit corresponding to the item type in the language model, obtain a prediction result corresponding to the item type and display it to the user.

[0216] The text processing device provided in this specification, after obtaining the text to be processed corresponding to the project type, can construct the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram, and after the encoding unit in the language model processes the text to be processed into an encoding vector, the encoding vector and the adversarial vector can be fused to obtain a fusion vector containing adversarial knowledge, and finally the fusion vector is processed by the processing unit corresponding to the project type in the language model to obtain the prediction result corresponding to the project type. In the process of processing the text, the use of a language model that integrates adversarial representation and is more robust can effectively improve the prediction accuracy. Furthermore, when training the language model used in the text processing process, by combining the adversarial corpus and the initial corpus, the adversarial representation can be integrated into the language model during the training stage, which effectively improves the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model to better meet the project requirements of downstream projects.

[0217] The above is a schematic scheme of the third text processing device of this embodiment. It should be noted that the technical scheme of the text processing device and the technical scheme of the third text processing method mentioned above are of the same concept, and the details not described in detail in the technical scheme of the text processing device can be found in the description of the technical scheme of the third text processing method mentioned above.

[0218] Fig. 9 A flow chart of a language model training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0219] Step S902: Obtain the initial corpus corresponding to the project type.

[0220] Step S904: Process the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus.

[0221] Step S906: construct a corpus corresponding to the project type according to the adversarial corpus and the initial corpus.

[0222] Step S908: training the initial language model based on the target corpus included in the corpus until a language model that meets the training stop condition is obtained.

[0223] Furthermore, in the model training stage, in order to improve the prediction accuracy of the language model, it is also necessary to enable the model to learn adversarial capabilities, so that in the model fine-tuning stage, the potential adversarial relationships such as similar pronunciation, similar form, and similar meaning in the text can be implicitly learned, which effectively solves the problem of limited expression of single-modal knowledge, and achieves the purpose of enhancing the robustness of the natural language processing system against adversarial variations based on similar pronunciation, similar form, and similar meaning in the reasoning stage, and ensures the performance of the intelligent system for practical application in an uncontrolled environment; in this embodiment, the specific implementation method is as follows:

[0224] Step S9082, obtaining an initial corpus corresponding to the project type;

[0225] Step S9084, processing the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus;

[0226] Specifically, the initial corpus refers to the corpus required for training the language model, and the initial corpus is associated with the project type, that is, the determination of the initial corpus depends on the corresponding downstream project requirements. Correspondingly, the adversarial processing algorithm specifically refers to an algorithm that replaces and modifies the initial corpus by replacing characters, word units, or sentence units, and the replaced or modified characters, word units, or sentence units are similar in semantics or pronunciation to the original content, or the characters are similar in shape, so that adversarial information can be integrated into the corpus; accordingly, the adversarial corpus is the sample corpus after the adversarial information is integrated.

[0227] Furthermore, in order to improve the richness of the samples, when the initial corpus is processed based on the adversarial processing algorithm, the adversarial corpus can be created by combining the processing results and the intermediate results. In this embodiment, the specific implementation method is as follows:

[0228] According to the adversarial processing algorithm, in the adversarial relationship diagram, the associated characters and adjacent characters associated with the initial corpus are screened; the initial corpus is updated according to the associated characters and the adjacent characters respectively to obtain an intermediate corpus and a target corpus; and the intermediate corpus and the target corpus are determined as the adversarial corpus.

[0229] Specifically, the associated characters specifically refer to the characters in the adversarial relationship graph that have a first-degree relationship with the characters included in the initial corpus; correspondingly, the adjacent characters specifically refer to the characters in the adversarial relationship graph that have a second-degree relationship with the characters included in the initial corpus. Accordingly, updating the initial corpus specifically refers to the operation of replacing the characters in the initial corpus with adjacent characters or associated characters; accordingly, the intermediate corpus specifically refers to the corpus obtained after each character replacement, and the target corpus specifically refers to the corpus obtained after all replacement processing operations are completed.

[0230] Based on this, after obtaining the initial corpus, in order to improve the accuracy of model training and enhance the robustness of the model, the adversarial relationship graph can be used to determine the associated characters and adjacent characters corresponding to the initial corpus, and then the associated characters and adjacent characters are used to gradually replace the characters in the initial corpus, and the intermediate text is retained after each replacement as the intermediate corpus, and the target corpus is obtained after all replacements are completed. At this time, the intermediate corpus and the target corpus are used as adversarial corpora for subsequent construction of the corpus set for the training model in combination with the initial expectations, thereby improving the richness of the samples and training a language model with better predictive ability.

[0231] In summary, by adopting adversarial processing algorithms to expand the number and richness of the corpus, the subsequently constructed corpus can contain more corpora corresponding to different levels of difficulty. Retraining the model in this way can effectively improve the model's prediction accuracy and robustness.

[0232] Step S9086, constructing a corpus corresponding to the project type according to the adversarial corpus and the initial corpus;

[0233] Step S9088: training the initial language model based on the target corpus contained in the corpus until the language model that meets the training stop condition is obtained.

[0234] Specifically, after obtaining the adversarial corpus as mentioned above, the adversarial corpus and the initial corpus are integrated to obtain a corpus set for training the model. Finally, the initial language model is trained in combination with the target corpus contained in the corpus until the language model that meets the training stop condition is obtained.

[0235] Among them, the target corpus specifically refers to the samples in the corpus that can be used to train the initial language model; correspondingly, the training stop condition specifically refers to the condition for stopping the training of the model after training and adjusting the model parameters, which can be the number of model iterations, or the comparison of loss values; that is, when the number of model iterations or the comparison of loss values ​​meets the conditions, the training can be stopped, and the trained model can be used as the language model corresponding to the project type.

[0236] Furthermore, in the process of training the model, the model parameters are actually continuously adjusted, that is, the target corpus contained in the corpus is input into the initial language model for processing to obtain the predicted text corresponding to the target corpus contained in the corpus; the initial language model parameters are adjusted according to the sample text corresponding to the target corpus contained in the corpus and the predicted text until the language model that meets the training stop condition is obtained.

[0237] For example, when training a model capable of keyword extraction, to improve the accuracy of keyword extraction by the model, after obtaining the initial corpus "{A goes hiking}", the preset adversarial processing algorithm can be used to process the initial corpus. That is, in the adversarial relationship graph, determine the associated characters corresponding to the character "A" {...}, and adjacent characters {...}, determine the associated characters corresponding to the character "qu" {...}, and adjacent characters {...}, and sequentially determine the adjacent characters and associated characters corresponding to each character; then use the associated characters and associations to update the initial corpus to obtain intermediate corpora "{You goes hiking}", "{Jia goes hiking}"... "{A qu goes hiking}", "{A qu goes hiking}"; and target corpora "{You qu goes hiking}", "{Jia qu goes hiking}", "{You qu goes hiking}", "{Jia qu goes hiking}"...; then construct a corpus set based on the intermediate corpora, target corpora, and initial corpus.

[0238] Furthermore, input the target corpora included in the corpus set into the keyword extraction model to be trained to obtain the predicted text corresponding to each target corpus. Finally, adjust the parameters of the keyword extraction model according to the sample text and predicted text corresponding to the target corpora included in the corpus set until a target keyword extraction model that meets the training stop condition is obtained.

[0239] In summary, by using the adversarial processing algorithm to actively generate adversarial texts for the language model and adding the generated adversarial texts together with the intermediate form of perturbation texts to the model fine-tuning corpus set, adversarial fine-tuning is performed on the language model of the corresponding project type, resulting in a language model with higher prediction accuracy being trained.

[0240] The training method of the language model provided in this specification, when training the language model used in the text processing process, can be completed by combining adversarial corpora and initial corpora, enabling the integration of adversarial representations into the language model during the training phase, effectively improving the endogenous security of the pre-trained language model, and effectively ensuring the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model that better meets the project requirements of downstream projects.

[0241] It should be noted that the training method of the language model provided in this embodiment belongs to the same concept as the training process solution of the language model in the above first text processing method. The same or corresponding description content can be referred to in the above embodiment, and this embodiment will not be elaborated here too much.

[0242] Corresponding to the above method embodiment, this specification also provides an embodiment of a training device for a language model. Fig.10 The structural schematic diagram of a training device for a language model provided by an embodiment of this specification is shown. As Fig.10 shown, the device includes:

[0243] The corpus acquisition module 1002 is configured to acquire initial corpus corresponding to the project type;

[0244] The corpus processing module 1004 is configured to process the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus;

[0245] A building set module 1006 is configured to build a corpus set corresponding to the project type according to the adversarial corpus and the initial corpus;

[0246] The training model module 1008 is configured to train the initial language model based on the target corpus included in the corpus until a language model that meets the training stop condition is obtained.

[0247] The language model training device provided in this specification, when training the language model used in the text processing process, combines the adversarial corpus with the initial corpus, so that the adversarial representation can be integrated into the language model during the training phase, effectively improving the intrinsic security of the pre-trained language model, and can effectively ensure the accuracy and robustness of the pre-trained language model in an uncontrolled and untrusted environment, thereby constructing a more robust language model to better meet the project requirements of downstream projects.

[0248] The above is a schematic scheme of a language model training device of this embodiment. It should be noted that the technical scheme of the language model training device and the technical scheme of the language model training method described above belong to the same concept, and the details of the technical scheme of the language model training device that are not described in detail can be found in the description of the technical scheme of the language model training method described above.

[0249] Fig.11 The block diagram of a computing device 1100 according to one embodiment of the present specification is shown. The components of the computing device 1100 include but are not limited to a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0250] The computing device 1100 also includes an access device 1140 that enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0251] In one embodiment of the present specification, the above components of the computing device 1100 and Fig.11 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.11 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0252] The computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 1100 may also be a mobile or stationary server.

[0253] The processor 1120 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned text processing method or language model training method.

[0254] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned text processing method or language model training method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned text processing method or language model training method.

[0255] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned text processing method or language model training method.

[0256] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned text processing method or language model training method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned text processing method or language model training method.

[0257] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned text processing method or language model training method.

[0258] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned text processing method or language model training method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned text processing method or language model training method.

[0259] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0260] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0261] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0262] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0263] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A text processing method, comprising: Get the text to be processed corresponding to the project type; Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector, wherein the adversarial relationship graph refers to a relationship graph constructed based on universal characters of the same language as the text to be processed, and the structure of the adversarial relationship graph is connected by characters having an adversarial relationship, and the characters connected to each other all have an adversarial relationship; Constructing a fusion vector based on the adversarial vector and the encoding vector; The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type.

2. According to the text processing method of claim 1, constructing the adversarial vector corresponding to the text to be processed according to the preset adversarial relationship diagram comprises: Preprocessing the text to be processed to obtain a text sequence corresponding to the text to be processed; Determine, in the preset adversarial relationship graph, a character node corresponding to each character in the text sequence, and edge information associated with the character node; Determine the global adversarial information corresponding to the text to be processed according to the character node and the edge information; The adversarial vector is constructed based on the global adversarial information and the text to be processed.

3. According to the text processing method of claim 2, the step of determining the global adversarial information corresponding to the text to be processed according to the character nodes and the edge information comprises: Determine, according to the connection relationship between the character nodes and the edge information in the adversarial relationship graph, the shape adversarial information, the sound adversarial information, and the semantic adversarial information corresponding to the text sequence; The global confrontation information corresponding to the text to be processed is obtained by integrating the shape confrontation information, the sound confrontation information and the semantic confrontation information.

4. According to the text processing method according to any one of claims 1 to 3, the construction of the adversarial relationship graph comprises: Get the basic character corresponding to the item type; By traversing the basic characters, basic shape confrontation information and basic sound confrontation information between the basic characters are determined; constructing a basic character node according to the basic character, and constructing basic edge information according to the basic shape confrontation information and the basic sound confrontation information; A connectivity relationship between the basic character nodes and the basic edge information is established, and the adversarial relationship graph is constructed according to the connectivity relationship establishment result.

5. The text processing method according to claim 4, wherein determining the basic shape confrontation information between the basic characters by traversing the basic characters comprises: By traversing the basic characters, constructing a basic character image corresponding to the basic character; Extracting image features of the basic character image as a character vector corresponding to the basic character; Calculate the basic shape confrontation information between the basic characters according to the character vector; Accordingly, determining the basic sound confrontation information between the basic characters by traversing the basic characters includes: By traversing the basic characters, determining the phoneme sequence corresponding to the basic characters; The basic sound confrontation information between the basic characters is calculated according to the phoneme sequence.

6. The text processing method according to claim 1, wherein the step of constructing a fusion vector based on the adversarial vector and the encoding vector comprises: Fusing the adversarial vector and the encoding vector to obtain the fused vector; or, The adversarial vector is input into an adversarial language model for processing to obtain a target adversarial vector, and the encoding vector is input into an encoding language model for processing to obtain a target encoding vector; the target adversarial vector and the target encoding vector are fused to obtain the fused vector.

7. The text processing method according to claim 1, wherein the training of the language model comprises: Obtaining initial corpus corresponding to the project type; Processing the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus; Constructing a corpus corresponding to the project type according to the adversarial corpus and the initial corpus; The initial language model is trained based on the target corpus included in the corpus until the language model that meets the training stop condition is obtained.

8. The text processing method according to claim 7, wherein the initial corpus is processed according to a preset adversarial processing algorithm to obtain adversarial corpus, comprising: According to the adversarial processing algorithm, the associated characters and adjacent characters associated with the initial corpus are screened in the adversarial relationship graph, wherein the associated characters refer to characters in the adversarial relationship graph that have a first-degree relationship with the characters included in the initial corpus, and the adjacent characters refer to characters in the adversarial relationship graph that have a second-degree relationship with the characters included in the initial corpus; The initial corpus is updated according to the associated characters and the adjacent characters to obtain an intermediate corpus and a target corpus; The intermediate corpus and the target corpus are determined as the adversarial corpus.

9. The text processing method according to claim 7, wherein the initial language model is trained based on the target corpus included in the corpus until the language model that satisfies the training stop condition is obtained, comprising: Inputting the target corpus contained in the corpus into the initial language model for processing to obtain a predicted text corresponding to the target corpus contained in the corpus; The initial language model is adjusted according to the sample text corresponding to the target corpus contained in the corpus and the predicted text until the language model that meets the training stop condition is obtained.

10. A text processing method, comprising: Receiving a text processing request submitted by a user for a project type, wherein the text processing request carries a text to be processed; Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector, wherein the adversarial relationship graph refers to a relationship graph constructed based on universal characters of the same language as the text to be processed, and the structure of the adversarial relationship graph is connected by characters having an adversarial relationship, and the characters connected to each other all have an adversarial relationship; Constructing a fusion vector based on the adversarial vector and the encoding vector; The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type as a response to the text processing request.

11. A text processing method, comprising: Displaying a text input interface based on an operation instruction submitted by a user for a project type, and receiving a text to be processed input by the user through the text input interface; Constructing an adversarial vector corresponding to the text to be processed according to a preset adversarial relationship graph, and processing the text to be processed by an encoding unit in a language model to obtain an encoding vector, wherein the adversarial relationship graph refers to a relationship graph constructed based on universal characters of the same language as the text to be processed, and the structure of the adversarial relationship graph is connected by characters having an adversarial relationship, and the characters connected to each other all have an adversarial relationship; Constructing a fusion vector based on the adversarial vector and the encoding vector; The fusion vector is processed by a processing unit corresponding to the item type in the language model to obtain a prediction result corresponding to the item type and display it to the user.

12. A method for training a language model, comprising: Get the initial corpus corresponding to the project type; Processing the initial corpus according to a preset adversarial processing algorithm to obtain adversarial corpus with an adversarial relationship, wherein the processing the initial corpus according to the preset adversarial processing algorithm to obtain adversarial corpus with an adversarial relationship includes: determining adversarial corpus with an adversarial relationship in a preset adversarial relationship graph according to the preset adversarial processing algorithm, wherein the adversarial relationship graph refers to a relationship graph constructed based on universal characters of the same language as the text to be processed, wherein the structure of the adversarial relationship graph is connected through characters with an adversarial relationship, and characters that are connected to each other all have an adversarial relationship; Constructing a corpus corresponding to the project type according to the adversarial corpus and the initial corpus; The initial language model is trained based on the target corpus included in the corpus until a language model that meets the training stop condition is obtained.

13. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 11 or 12 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11 or 12.

Images