Text processing method and device and related equipment

Abstract

The invention discloses a text processing method and device and related equipment. The method comprises the steps of obtaining a to-be-processed text; determining a language type of the to-be-processed text; configuring an adaptive component for a preset language model based on the language type; wherein the adaptive component is obtained by pre-training based on a training sample of the language type; obtaining a target unique language feature of the to-be-processed text based on the configured preset language model; and performing subsequent processing on the to-be-processed text based on the target unique language feature of the to-be-processed text to obtain a processing result of the to-be-processed text. According to the text processing method provided by the embodiment of the invention, the adaptive component is configured in the language model, and the unique language features of different languages can be obtained by utilizing the adaptive component, so that the expression ability of the unique language features of different languages can be enhanced when to-be-processed texts of different language types are processed.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a text processing method, apparatus and related equipment. Background Art

[0002] In the field of artificial intelligence, cross-language understanding is an important research direction. Currently existing artificial intelligence models have excellent capabilities in processing single language tasks, but in multilingual environments, especially when dealing with natural language processing (NLP) tasks in multiple languages, these artificial intelligence models still face many challenges.

[0003] For example, the commonly used Bloom (Big Science Large Open-science Open-access Multilingual Language Model) model can well represent the common features between different languages, has a certain generalization ability, and can be used across languages ​​to a certain extent. However, for the unique expressions, meanings, grammatical structures and other features of different languages, the performance of the Bloom model is still relatively poor. Summary of the invention

[0004] The main purpose of this application is to propose a text processing method, apparatus and related equipment, aiming to solve the problem that the Bloom model in the prior art has poor ability to express unique language features between different languages.

[0005] To achieve the above purpose, this application proposes a text processing method, including: Get the text to be processed; Determining the language type of the text to be processed; Configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; Acquire target unique language features of the text to be processed based on the configured preset language model; The text to be processed is subsequently processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

[0006] In an embodiment of the present application, the adaptation component includes a first unique feature acquisition module, an activation module, and a second unique feature acquisition module that are sequentially communicatively connected, the preset model is a Bloom model, and the configuration of the adaptation component for the preset language model based on the language type includes: Configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model; The adaptation component is configured with adaptation parameters corresponding to the language type of the text to be processed.

[0007] In the embodiment of the present application, the adaptation component is pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different language types to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

[0008] In an embodiment of the present application, the step of obtaining target unique language features of the text to be processed based on the configured preset language model includes: Receiving the original text features output by the multi-layer perceptron based on the first unique feature acquisition module, and acquiring the first unique language features in the original text features; Performing a nonlinear transformation on the first unique language feature based on the activation module to obtain a transformation result; Acquire a second unique language feature in the transformation result based on the second unique feature acquisition module; The second unique language feature is used as a target unique language feature of the text to be processed.

[0009] In an embodiment of the present application, the subsequent processing of the text to be processed based on the target unique language features of the text to be processed to obtain the processing result of the text to be processed includes: Based on the second unique feature acquisition module, the target unique language feature is superimposed on the original text feature to obtain an enhanced feature; The text to be processed is subsequently processed based on the enhanced features to obtain a processing result of the text to be processed.

[0010] The present application also proposes a text processing device, the text processing device comprising: A text acquisition module is used to acquire the text to be processed; A processing module, used for determining the language type of the text to be processed; Configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; Acquire target unique language features of the text to be processed based on the configured preset language model; The text to be processed is subsequently processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

[0011] In an embodiment of the present application, the adaptation component includes a first unique feature acquisition module, an activation module, and a second unique feature acquisition module that are communicatively connected in sequence, the preset model is a Bloom model, and the processing module is configured to configure the adaptation component for the preset language model based on the language type in the following manner: Configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model; The adaptation component is configured with adaptation parameters corresponding to the language type of the text to be processed.

[0012] In the embodiment of the present application, the adaptation component is pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different language types to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

[0013] In the embodiment of the present application, the processing module is further configured as follows: Receiving the original text features output by the multi-layer perceptron based on the first unique feature acquisition module, and acquiring the first unique language features in the original text features; Performing a nonlinear transformation on the first unique language feature based on the activation module to obtain a transformation result; Acquire a second unique language feature in the transformation result based on the second unique feature acquisition module; The second unique language feature is used as a target unique language feature of the text to be processed.

[0014] In the embodiment of the present application, the processing module is further configured as follows: Based on the second unique feature acquisition module, the target unique language feature is superimposed on the original text feature to obtain an enhanced feature; The text to be processed is subsequently processed based on the enhanced features to obtain a processing result of the text to be processed.

[0015] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, any of the above-mentioned text processing methods is implemented.

[0016] An embodiment of the present application further provides a computing device, including a processor, on which a computer program is stored, and when the computer program is executed, the text processing method described in any of the above embodiments is implemented.

[0017] The text processing method in the embodiment of the present application configures an adaptation component in the language model, and utilizes the adaptation component to obtain the unique language features of different languages, thereby enhancing the ability to express the unique language features of different languages ​​when processing texts to be processed in different language types. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0019] Figure 1 This is an architectural diagram of the Bloom model in the prior art; Figure 2 It is an architecture diagram of a decoding module in a Bloom model in the prior art; Figure 3 A step diagram of a text processing method in an embodiment of the present application; Figure 4 This is an architecture diagram of a decoding module configured with an adaptation component in one embodiment of the present application; Figure 5 A module diagram of an adapter component in an embodiment of the present application; Figure 6 is a module diagram of a text processing device in an embodiment of the present application; Figure 7 A module diagram of a computer-readable storage medium in an embodiment of the present application; Figure 8 A module diagram of a computing device in one embodiment of the present application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] Those skilled in the art know that the embodiments of the present application can be implemented as a system, device, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0023] According to the implementation manner of the present application, a text processing method, apparatus and related equipment are proposed.

[0024] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0025] The principles and spirit of the present application are explained in detail below with reference to several representative implementations of the present application.

[0026] Exemplary Methods like Figure 1 , Figure 2 As shown in FIG. 1 , a schematic diagram of a Bloom model in the prior art is shown. Figure 2 The figure is an architecture diagram of a decoding module in the prior art, where the Bloom model mainly includes an embedding layer and multiple decoding modules, each of which includes multiple parallel multilayer perceptrons (MLP) and multi-head attention modules.

[0027] Among them, the Embedding layer is mainly used to map the input text to be processed into word vectors; the multi-layer perceptron is used to extract and transform the word vectors output by the embedding layer; the multi-head attention module adds position information (ALiBi) to the standard multi-head attention (HeadFusion), so that the model can better understand the positional relationship of the text to be processed.

[0028] In addition, a normalization layer (LN) can be added to the output of each layer. LN is a regularization method commonly used in deep learning. It is mainly used to normalize the output of each layer to ensure that the data is within the range of mean 0 and variance 1, so that the gradient calculation result will not become too large or too small due to chain calculation. When processing sequence data, it can effectively prevent gradient explosion or gradient disappearance, thereby improving the expression ability and training efficiency of the model.

[0029] After training, the Bloom model can have a good representation effect on the common features between different languages, has a certain generalization ability, and can be used across languages ​​to a certain extent. However, for the unique language features of different languages, the Bloom model's expression ability is relatively poor. Therefore, the present application proposes a text processing method to solve the problem that the natural language processing solution in the prior art has a poor expression ability for the unique language features of different languages.

[0030] like Figure 3 As shown, the embodiment of the present application proposes a text processing method, comprising the following steps S100-S500: Step S100: Obtain the text to be processed.

[0031] In an embodiment of the present application, the text to be processed may be text of different language types, the text to be processed may be text that needs to be translated, or may be text that needs sentiment analysis, or may be text that needs to be classified, and so on.

[0032] In an embodiment of the present application, the text to be processed can be input into the Bloom model based on the input port of a Bloom model that has been trained in the prior art and has a certain generalization ability, and the input port of the Bloom model is the port of the Bloom model in the prior art for the user to input the text to be processed.

[0033] Step S200: Determine the language type of the text to be processed.

[0034] In the embodiment of the present application, when inputting the text to be processed, a language identifier can also be input into the Bloom model, and the language identifier is used to indicate the language type of the text to be processed. For example, when the text to be processed is in English, its language identifier can be en, and when the text to be processed is in Chinese, its language identifier can be cn, etc. Then, when the user inputs the text to be processed and its corresponding language identifier, the language type of the text to be processed can be determined according to the language identifier.

[0035] Step S300: configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on training samples of the language type.

[0036] In step S100, the text to be processed is input into a trained Bloom model. That is, in the embodiment of the present application, the preset language model is the trained Bloom model with a certain generalization ability. The overall architecture of the Bloom model proposed in the embodiment of the present application can be referred to Figure 1 shown.

[0037] After determining that the preset language model is a Bloom model, in the embodiment of the present application, an adaptation component may be configured for the Bloom model based on the following steps S310-S320: Step S310: configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model.

[0038] like Figure 1 , Figure 2 , Figure 4 As shown, the Bloom model includes multiple decoding modules, each of which includes multiple multi-layer perceptrons. In the embodiment of the present application, when configuring the adaptation component, the adaptation component can be set after each multi-layer perceptron in at least one decoding module of the Bloom model, that is, for each decoding module of the Bloom model, it is not necessary to set the above-mentioned adaptation component in all decoding modules, and the number of configured adaptation components can be determined based on the expression ability of the existing Bloom model for different languages. In addition, the adaptation component can be configured between the decoding module and the normalization layer after the decoding module.

[0039] For example, for a common language, the Bloom model can already express its common features and unique language features well. In this case, there is no need to set an adapter component in the Bloom model, or the adapter component can be set only in a few decoding modules. For uncommon languages, when the Bloom model has poor expression ability for its unique language features, adapter components can be set in most or all decoding modules to improve the expression ability for its unique language features. That is, the stronger the Bloom model's expression ability for a certain language, the fewer the number of adapter components that need to be set when processing the text to be processed of that language type, and the weaker the expression ability for a certain language, the more the number of adapter components that need to be set when processing the text to be processed of that language type.

[0040] In addition, in an embodiment of the present application, when the adaptation component needs to be set in only some of the decoding modules, it can be set from front to back in the order of the decoding modules, that is, the adaptation component is preferentially set in the decoding modules that are positioned at the front in the Bloom model. Moreover, when the adaptation component is set in a certain decoding module, it can be set in all the multi-layer perceptrons of the decoding module.

[0041] Step S320: configuring the adaptation component with adaptation parameters corresponding to the language type of the text to be processed.

[0042] In an embodiment of the present application, the adaptation component is configured to be pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different languages ​​to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

[0043] In an embodiment of the present application, after the adaptation component is configured in the Bloom model, the backbone model is the other modules in the Bloom model except the adaptation component, and the backbone model is mainly used to obtain the original text features of different training samples, and the adaptation component is used to enhance the unique language features in the original text features. Therefore, during training, the parameters of the backbone model can be frozen and only the adaptation component is trained, thereby greatly reducing the amount of calculation during training.

[0044] In addition, during training, the adaptation components can be trained separately based on training samples of different languages. For example, if the adaptation component is trained only based on Chinese samples, the obtained adaptation parameters can better express the unique language features of Chinese. For another example, if the adaptation component is trained only based on English training samples, the obtained adaptation parameters can better express the unique language features of English.

[0045] Thus, the adaptation components are trained respectively using training samples of multiple language types to obtain adaptation parameters corresponding to different languages. When faced with texts to be processed in different languages, the adaptation components are configured based on the adaptation parameters corresponding to the language, that is, the unique language features of the text to be processed in the language can be well expressed. That is, in the embodiment of the present application, the Bloom model is also configured to: based on the language type of the text to be processed, configure the adaptation parameters corresponding to the language type of the text to be processed for the adaptation component to process the text to be processed.

[0046] Step S400: Acquire target unique language features of the text to be processed based on the configured preset language model.

[0047] In the embodiment of the present application, the unique language features of the text to be processed can be obtained through the following steps S410-S430: Step S410: receiving the original text features output by the multilayer perceptron based on the first unique feature acquisition module, and acquiring the first unique language features in the original text features.

[0048] like Figure 5As shown, in an embodiment of the present application, the adaptation component includes a first unique feature acquisition module 100, an activation module 200, and a second unique feature acquisition module 300 that are communicatively connected in sequence. When configuring the adaptation component, the first unique feature acquisition module 100, the activation module 200, and the second unique feature acquisition module 300 can be sequentially arranged after the multi-layer perceptron of the decoding module, and the first unique feature acquisition module 100 can receive the output result of the multi-layer perceptron that is communicatively connected thereto.

[0049] In the embodiment of the present application, after the text to be processed passes through the multi-layer perceptron of the Bloom model, the multi-layer perceptron can extract the original text features in the text to be processed, some of which are common features between different languages, and some of which are unique language features unique to each of the different languages. An adaptation component is set after the multi-layer perceptron, and the unique language features of different languages ​​are added to the original text features output by the multi-layer perceptron through the adaptation component, thereby enhancing the Bloom model's ability to express the unique language features of different languages.

[0050] In the embodiment of the present application, the first unique feature acquisition module 100 is used to receive the original text features output by the multi-layer perceptron, reduce the dimension of the original text features, and obtain the first unique language features in the original text features after the dimension reduction. For example, in the embodiment of the present application, the first unique feature acquisition module 100 is provided with a dimension reduction matrix W1 and a first bias matrix b1. After the first unique feature acquisition module 100 receives the original text features output by the multi-layer perceptron, assuming that the original text features are h, the original text features are first reduced in dimension by the dimension reduction matrix, and after the dimension reduction matrix is ​​reduced in dimension, it can be obtained Then it is biased by the first bias matrix b1 to get , which is the first unique language feature.

[0051] Among them, the dimensionality reduction matrix can reduce the dimension of the original text features, which can reduce the computational complexity of the training process. After the original text features are reduced in dimension, the first bias matrix is ​​introduced. Through the first bias matrix, the first unique language features after the dimensionality reduction of the original text features under different language types can be captured and enhanced, thereby improving the expression ability of the adaptation component for the unique language features of different language types.

[0052] Step S420: performing a nonlinear transformation on the first unique language feature based on the activation module to obtain a transformation result.

[0053] like Figure 5As shown, in the embodiment of the present application, the activation module 200 is used to receive the output result of the first unique feature acquisition module 100, and perform a nonlinear transformation on the output result of the first unique feature acquisition module 100 to obtain a transformation result. For example, in the embodiment of the present application, the activation module 200 is provided with a nonlinear activation function, such as GeLu. Assuming that represents the activation function, the output result of the first unique feature acquisition module 100 After being input into the activation module 200, the transformation result can be obtained after the nonlinear activation function Among them, the first unique feature acquisition module 100 reduces the dimension of the original text features, and enhances the unique language features of different languages ​​through the first bias matrix, and then inputs it into the activation module 200 for nonlinear transformation, which can further increase the expression ability of the adaptation component for the unique language features of different languages.

[0054] Step S430: acquiring a second unique language feature in the transformation result based on the second unique feature acquisition module.

[0055] like Figure 5 As shown, in the embodiment of the present application, the second unique feature acquisition module 300 is used to receive the transformation result output by the activation module 200, perform dimension upgrade on the transformation result, and obtain the second unique language feature in the transformed result after dimension upgrade. For example, in the embodiment of the present application, the second unique feature acquisition module 300 is provided with a dimension upgrade matrix W2 and a second bias matrix b2. After the activation module 200 outputs the transformation result to the second unique feature acquisition module 300, the second unique feature acquisition module 300 first performs dimension upgrade on the transformation result to obtain Then it is biased by the second bias matrix to obtain , which is the second unique language feature.

[0056] Among them, the second unique feature acquisition module 300 upgrades the transformation result to the original dimension of the original text feature, which can facilitate subsequent operations in the Bloom model, and the second bias matrix can capture the unique language features of the original text features under different language types after dimensionality upgrade and enhance them, thereby further improving the expression ability of the adaptation component for the unique language features of different language types.

[0057] Step S430: using the second unique language feature as the target unique language feature of the text to be processed.

[0058] In the embodiment of the present application, in step S420, the second unique feature acquisition module 300 can output , that is, the second unique language feature. In step S430, the second unique language feature output by the second unique feature acquisition module 300 can be used as the target unique language feature of the text to be processed.

[0059] Step S500: performing subsequent processing on the text to be processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

[0060] In the embodiment of the present application, the target unique language feature can be superimposed on the original text feature based on the second unique feature acquisition module 300 to obtain an enhanced feature, and the to-be-processed text can be subsequently processed based on the enhanced feature.

[0061] Among them, the second unique language feature is the target unique language feature, specifically , the original text is h, and the unique language features in the original text features can be enhanced by superimposing the two using the second unique feature acquisition module 300. Specifically, the superposition can be based on the following formula: +h Among them, h represents the original text features of the multi-layer perceptron, represents the dimension reduction matrix, represents the first bias matrix, represents the activation function, represents a dimension-raising matrix, represents the second bias matrix, Represents the enhanced features after superposition.

[0062] In the embodiment of the present application, after the adaptation component obtains the superimposed enhanced features, it outputs them to other modules of the Bloom model, which can be used for subsequent processing by other modules. Since the enhanced features have enhanced the unique language features of the text to be processed, the processing results obtained in the subsequent processing process will inevitably have a better expression effect on the unique language features of the text to be processed. The text processing method in the embodiment of the present application configures an adaptation component in the language model, and utilizes the adaptation component to obtain the unique language features of different languages, thereby enhancing the ability to express the unique language features of different languages ​​when processing texts to be processed in different language types.

[0063] Exemplary Devices like Figure 6 As shown, this exemplary embodiment provides a text processing device 400, and the text processing device 400 includes: The text acquisition module 410 is used to acquire the text to be processed; A processing module 420, used to determine the language type of the text to be processed; Configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; Acquire target unique language features of the text to be processed based on the configured preset language model; The text to be processed is subsequently processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

[0064] In the embodiment of the present application, the adaptation component includes a first unique feature acquisition module 100, an activation module 200, and a second unique feature acquisition module 300 that are communicatively connected in sequence, the preset model is a Bloom model, and the processing module 420 is configured to configure the adaptation component for the preset language model based on the language type in the following manner: Configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model; The adaptation component is configured with adaptation parameters corresponding to the language type of the text to be processed.

[0065] In the embodiment of the present application, the adaptation component is pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different language types to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

[0066] In the embodiment of the present application, the processing module 420 is further configured to: Based on the first unique feature acquisition module 100, the original text features output by the multi-layer perceptron are received, and the first unique language features in the original text features are acquired; Performing a nonlinear transformation on the first unique language feature based on the activation module 200 to obtain a transformation result; Acquire a second unique language feature in the transformation result based on the second unique feature acquisition module 300; The second unique language feature is used as a target unique language feature of the text to be processed.

[0067] In the embodiment of the present application, the processing module 420 is further configured to: Based on the second unique feature acquisition module 300, the target unique language feature is superimposed on the original text feature to obtain an enhanced feature; The text to be processed is subsequently processed based on the enhanced features to obtain a processing result of the text to be processed.

[0068] The specific processing methods of each module of the text processing device 400 in the embodiment of the present application can refer to the various embodiments of the text processing method in the above exemplary method, and will not be described one by one here.

[0069] The text processing device 400 in the embodiment of the present application configures an adaptation component in the language model through the processing module 420, and can obtain unique language features of different languages ​​by using the adaptation component, so that when processing texts to be processed of different language types, the expression ability of unique language features of different languages ​​can be enhanced.

[0070] Exemplary Media After introducing the method, medium and system of the exemplary embodiments of the present application, next, reference is made to Figure 7 For a description of the computer-readable storage medium of the exemplary embodiment of the present application, please refer to Figure 7 , the computer-readable storage medium shown is a CD 70, on which a computer program (i.e., a program product) is stored. When the computer program is executed by the processor, each step recorded in the above method implementation will be implemented, for example, obtaining a text to be processed; determining the language type of the text to be processed; configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; obtaining the target unique language features of the text to be processed based on the configured preset language model; performing subsequent processing on the text to be processed based on the target unique language features of the text to be processed to obtain the processing result of the text to be processed. The specific implementation methods of each step will not be repeated here. It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which will not be repeated here.

[0071] Exemplary Computing Devices After introducing the methods, systems, and media of the exemplary embodiments of the present application, reference is now made to Figure 8 A computing device according to an exemplary embodiment of the present application.

[0072] Figure 8 A block diagram of an exemplary computing device 80 suitable for implementing embodiments of the present application is shown. The computing device 80 may be a computer system or a server. Figure 8The computing device 80 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0073] like Figure 8 As shown, the components of the computing device 80 may include, but are not limited to: one or more processors or processing units 801 , a system memory 802 , and a bus 803 connecting various system components (including the system memory 802 and the processing unit 801 ).

[0074] The computing device 80 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 80, including volatile and non-volatile media, removable and non-removable media.

[0075] The system storage 802 may include computer system readable media in the form of volatile storage, such as random access memory (RAM) 8021 and / or cache memory 8022. The computing device 80 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the ROM 8023 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 is not shown in the Figure 8 As shown in FIG. 8 , a disk drive for reading and writing a removable non-volatile disk (e.g., a “floppy disk”) and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to the bus 803 via one or more data medium interfaces. The system storage 802 may include at least one program product, which has a set (e.g., at least one) of program modules, which are configured to perform the functions of each embodiment of the present application.

[0076] A program / utility 8025 having a set (at least one) of program modules 8024 may be stored, for example, in system memory 802, and such program modules 8024 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 8024 generally perform the functions and / or methods of the embodiments described herein.

[0077] The computing device 80 may also communicate with one or more external devices 804 (e.g., a keyboard, a pointing device, a display, etc.). Such communication may be performed via an input / output (I / O) interface. In addition, the computing device 80 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 806. Figure 8 As shown, the network adapter 806 communicates with other modules (such as the processing unit 801, etc.) of the computing device 80 via the bus 803. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with computing device 80 .

[0078] The processing unit 801 executes various functional applications and data processing by running the program stored in the system memory 802, for example, obtaining the text to be processed; determining the language type of the text to be processed; configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on the training sample of the language type; obtaining the target unique language features of the text to be processed based on the configured preset language model; performing subsequent processing on the text to be processed based on the target unique language features of the text to be processed to obtain the processing result of the text to be processed. The specific implementation of each step is not repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the computing device are mentioned in the above detailed description, this division is only exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be concretized in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be concretized.

[0079] In the description of the present application, it should be noted that the terms "first", "second" and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] In the several embodiments provided in the present application, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. The system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0083] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0084] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computing device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0085] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0086] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. A text processing method, the text processing method comprising: Get the text to be processed; Determining the language type of the text to be processed; Configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; Acquire target unique language features of the text to be processed based on the configured preset language model; The text to be processed is subsequently processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

2. The text processing method according to claim 1, wherein: The adaptation component includes a first unique feature acquisition module, an activation module, and a second unique feature acquisition module that are sequentially communicatively connected, the preset model is a Bloom model, and configuring the adaptation component for the preset language model based on the language type includes: Configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model; The adaptation component is configured with adaptation parameters corresponding to the language type of the text to be processed.

3. The text processing method according to claim 2, wherein: The adaptation component is pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different language types to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

4. The text processing method according to claim 2, wherein: The step of obtaining target unique language features of the text to be processed based on the configured preset language model includes: Receiving the original text features output by the multi-layer perceptron based on the first unique feature acquisition module, and acquiring the first unique language features in the original text features; Performing a nonlinear transformation on the first unique language feature based on the activation module to obtain a transformation result; Acquire a second unique language feature in the transformation result based on the second unique feature acquisition module; The second unique language feature is used as a target unique language feature of the text to be processed.

5. The text processing method according to claim 4, wherein: The subsequent processing of the text to be processed based on the target unique language features of the text to be processed to obtain the processing result of the text to be processed includes: Based on the second unique feature acquisition module, the target unique language feature is superimposed on the original text feature to obtain an enhanced feature; The text to be processed is subsequently processed based on the enhanced features to obtain a processing result of the text to be processed.

6. A text processing device, comprising: A text acquisition module is used to acquire the text to be processed; A processing module, used for determining the language type of the text to be processed; Configuring an adaptation component for a preset language model based on the language type; wherein the adaptation component is pre-trained based on a training sample of the language type; Acquire target unique language features of the text to be processed based on the configured preset language model; The text to be processed is subsequently processed based on the target unique language features of the text to be processed to obtain a processing result of the text to be processed.

7. The text processing device according to claim 6, wherein: The adaptation component includes a first unique feature acquisition module, an activation module, and a second unique feature acquisition module that are communicatively connected in sequence, the preset model is a Bloom model, and the processing module is configured to configure the adaptation component for the preset language model based on the language type in the following manner: Configuring the adaptation component after each multi-layer perceptron of at least one decoding module of the Bloom model; The adaptation component is configured with adaptation parameters corresponding to the language type of the text to be processed.

8. The text processing device according to claim 7, wherein: The adaptation component is pre-trained based on the following method: After configuring the adaptation component in the Bloom model, the main model parameters of the Bloom model are frozen, and the adaptation component is trained based on training samples of different language types to obtain multiple groups of adaptation parameters; wherein each group of adaptation parameters corresponds to a language type.

9. The text processing device according to claim 7, wherein: The processing module is further configured to: Receiving the original text features output by the multi-layer perceptron based on the first unique feature acquisition module, and acquiring the first unique language features in the original text features; Performing a nonlinear transformation on the first unique language feature based on the activation module to obtain a transformation result; Acquire a second unique language feature in the transformation result based on the second unique feature acquisition module; The second unique language feature is used as a target unique language feature of the text to be processed.

10. The text processing device according to claim 9, wherein: The processing module is further configured to: Based on the second unique feature acquisition module, the target unique language feature is superimposed on the original text feature to obtain an enhanced feature; The text to be processed is subsequently processed based on the enhanced features to obtain a processing result of the text to be processed.

11. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to execute the method according to claims 1-5.

12. A computing device, comprising a processor, wherein a computer program is stored on the processor, and when the computer program is executed, the processing method according to any one of claims 1 to 5 is implemented.

Images