Natural language based text processing method and apparatus, electronic device, and medium

Abstract

This application provides a text processing method, apparatus, electronic device, and medium based on natural language, belonging to the field of artificial intelligence technology. The method includes: performing word embedding processing on the lexical indexes corresponding to word segmentation in the target text to obtain initial lexical vectors; extracting contextual features from the initial lexical vectors to obtain lexical context vectors; performing deep semantic extraction on the lexical context vectors to obtain deep semantic vectors; obtaining lexical modulation vectors corresponding to the lexical indexes according to the task complexity of the target text; and performing text tasks on the target text based on the lexical modulation vectors and deep semantic vectors. This application, by performing contextual feature extraction, deep semantic extraction, and vector modulation processing on the lexical indexes corresponding to word segmentation, can solve the problem of homogenization of all lexical elements while preserving their identity features, thus improving the accuracy of text processing.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a text processing method and apparatus, electronic device and medium based on natural language. Background Technology

[0002] Traditional text processing methods typically rely on multiple parallel expert subnetworks (such as multiple FFNs) deployed in the feedforward network layer of a Mixture of Experts (MoE) model to process the input text. These expert subnetworks are dynamically activated based on the context vector of the input text through a gating network. For example, in machine translation tasks, the text to be translated is input into a MoE for multi-layer Transformer encoding, outputting deep semantic vectors of terms. A gating network is then used to calculate the matching scores between these deep semantic vectors and each expert subnetwork. Finally, the two expert subnetworks with the highest scores are selected to process the deep semantic vectors of terms, and the weighted result of the two expert subnetworks is used to determine the text translation content. However, this method relies on a gating network to route the deep semantic vectors of words output by the deep network to the relevant expert subnetwork, homogenizing all words. This not only wastes computational resources but also reduces the retention rate of key word information. Furthermore, traditional methods do not directly link the original identity of words (such as part-of-speech tagging or domain labeling). Since the original identity of words in deep networks is easily blurred due to repeated context fusion, the original identity information of words is often lost in the routing decisions of the gating network during text processing, leading to a decrease in text processing accuracy. Therefore, improving text processing accuracy has become an urgent problem to be solved. Summary of the Invention

[0003] The main objective of this application is to provide a text processing method, apparatus, electronic device, and medium based on natural language, which aims to improve the accuracy of text processing.

[0004] To achieve the above objectives, a first aspect of this application proposes a text processing method based on natural language, the method comprising:

[0005] In response to a text task processing request for the target text, the target text is segmented into words to obtain text words;

[0006] Obtain the word index corresponding to the text segmentation from the preset vocabulary, and perform word embedding processing on the word index to obtain the initial word vector;

[0007] Context features are extracted from the initial word vector to obtain the word context vector;

[0008] Deep semantic extraction is performed on the word context vector to obtain the word deep semantic vector;

[0009] The task complexity of the text task is obtained, and the word index is modulated according to the task complexity to obtain the word modulation vector.

[0010] The target text is processed using the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result.

[0011] In some embodiments, the task complexity includes low-level task complexity, medium-level task complexity, or high-level task complexity.

[0012] The lexical modulation vector includes a single-dimensional modulation vector, a four-dimensional modulation vector, or a four-sample-dimensional modulation vector.

[0013] The step of modulating the lexical index according to the task complexity to obtain the lexical modulation vector includes:

[0014] If the task complexity is the same as the low-level task complexity, then the lexical index is subjected to single-dimension modulation processing to obtain the single-dimension modulation vector corresponding to the lexical index; wherein, the vector dimension of the single-dimension modulation vector is the same as the vector dimension of the initial lexical vector.

[0015] If the task complexity is the same as the intermediate task complexity, then the single-dimensional modulation vector corresponding to the lexical index is scaled by four times to obtain the four-dimensional modulation vector.

[0016] If the task complexity is the same as the high-level task complexity, then the single-dimensional modulation vector corresponding to the lexical index is sampled and scaled by four times to obtain the four-dimensional modulation vector.

[0017] In some embodiments, scaling the unit-dimensional modulation vector by four times to obtain the four-times-sampled-dimensional modulation vector includes:

[0018] The four-dimensional modulation vector corresponding to the single-dimensional modulation vector is downsampled to obtain the dimension-reduced modulation vector.

[0019] The reduced-dimensional modulation vector is grouped and reshaped to obtain a reshaped modulation vector;

[0020] The reshaped modulation vector is upsampled to obtain the four-times-sampled-dimension modulation vector.

[0021] In some embodiments, performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result includes:

[0022] If the lexical modulation vector is the single-dimensional modulation vector, then the deep semantic vector of the lexical and the single-dimensional modulation vector are fused element by element to obtain the target calibrated lexical vector.

[0023] The target text is processed using the target calibration lexical vector to obtain the text task processing result.

[0024] In some embodiments, the step of element-wise fusing the deep semantic vector of the lexical unit with the single-dimensional modulation vector to obtain the target calibrated lexical unit vector includes:

[0025] Obtain the modulation category of the word modulation vector; wherein the modulation category includes at least one of multiplicative modulation category, additive modulation category and affine transformation modulation category;

[0026] If the modulation category includes the multiplicative modulation category, then the deep semantic vector of the lexical unit is multiplied element-wise with the single-dimensional modulation vector to obtain the target calibrated lexical unit vector;

[0027] If the modulation category includes the additive modulation category, then the deep semantic vector of the lexical unit is added element-wise to the single-dimensional modulation vector to obtain the target calibrated lexical unit vector;

[0028] If the modulation category includes the affine transformation modulation category, then the single-dimensional modulation vector is divided into a single-dimensional multiplicative modulation vector and a single-dimensional additive modulation vector. The deep semantic vector of the lexical unit is multiplied element-wise with the single-dimensional multiplicative modulation vector to obtain a multiplicative modulated lexical vector. The deep semantic vector of the lexical unit is added element-wise with the single-dimensional additive modulation vector to obtain an additive modulated lexical vector. The target calibration lexical vector is determined based on the multiplicative modulated lexical vector and the additive modulated lexical vector.

[0029] In some embodiments, the step of performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result further includes:

[0030] If the word modulation vector is the four-dimensional modulation vector, then the word deep semantic vector and the four-dimensional modulation vector are fused element by element to obtain the target enhanced word vector.

[0031] The text task is performed on the target text based on the target enhanced lexical vector to obtain the text task processing result.

[0032] In some embodiments, the step of performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result further includes:

[0033] If the word modulation vector is the four-times sampling dimension modulation vector, then the word deep semantic vector and the four-times sampling dimension modulation vector are fused element by element to obtain the target optimized word vector;

[0034] Based on the target optimized lexical vector, a text task is performed on the target text to obtain the text task processing result.

[0035] To achieve the above objectives, a second aspect of this application provides a text processing apparatus based on natural language, the apparatus comprising:

[0036] The word segmentation module is used to respond to the text task processing request of the target text, perform word segmentation on the target text, and obtain text words;

[0037] The word embedding processing module is used to obtain the word index corresponding to the text segmentation from the preset vocabulary, and to perform word embedding processing on the word index to obtain the initial word vector;

[0038] The context feature extraction module is used to extract context features from the initial word vector to obtain a word context vector;

[0039] The deep semantic extraction module is used to perform deep semantic extraction on the word context vector to obtain the word deep semantic vector;

[0040] The lexical modulation processing module is used to obtain the task complexity of the text task and modulate the lexical index according to the task complexity to obtain the lexical modulation vector.

[0041] The text task execution module is used to perform text tasks on the target text based on the word modulation vector and the word deep semantic vector, and obtain the text task processing result.

[0042] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0043] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first aspect described above.

[0044] The natural language-based text processing method, apparatus, electronic device, and medium proposed in this application first obtain the lexical index corresponding to the text segmentation of the target text from a pre-defined vocabulary, and then perform word embedding processing on the lexical index to obtain an initial lexical vector. By processing the lexical index, the original features of the lexical are preserved, and contextual features are extracted from the initial lexical vector, effectively capturing the contextual information of the lexical vector. Deep semantic extraction is then performed on the lexical context vector to further capture the deep semantic information of the lexical, providing contextual and deep semantic support for subsequent text task processing. Second, the lexical index is modulated according to the task complexity to obtain a lexical modulation vector, further preserving the original features of the lexical and effectively avoiding the loss of original lexical information in deep networks. Finally, the text task is performed on the target text based on the lexical modulation vector and the lexical deep semantic vector. This not only avoids the problem of homogenizing all lexicals in traditional methods, but also dynamically adjusts the lexical modulation vector according to the complexity of the task, effectively improving the retention rate of key lexical information and significantly improving the efficiency and accuracy of text processing. Attached Figure Description

[0045] Figure 1 This is a flowchart of a natural language-based text processing method provided in an embodiment of this application;

[0046] Figure 2 yes Figure 1 The flowchart of step S105 in the process;

[0047] Figure 3 yes Figure 2 The flowchart of step S203 in the process;

[0048] Figure 4 yes Figure 1 The flowchart of step S106 in the process;

[0049] Figure 5 yes Figure 4 The flowchart of step S401 in the process;

[0050] Figure 6 yes Figure 1 Another flowchart of step S106 in the process;

[0051] Figure 7 yes Figure 1 Another flowchart of step S106 in the process;

[0052] Figure 8 This is a schematic diagram of the structure of a text processing device based on natural language provided in an embodiment of this application;

[0053] Figure 9This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0057] First, let's analyze some of the terms used in this application:

[0058] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0059] The RWKV (Receptance Weighted Key Value) architecture is a novel and efficient large language model architecture. It has linear computational complexity and combines the advantages of recurrent neural networks (RNN) and Transformers. In the text processing process, it consists of multiple residual blocks, each of which includes two sub-blocks: time-mixing and channel-mixing. The two sub-blocks are combined to capture the semantic features of text vectors, thereby achieving efficient processing of text tasks.

[0060] This application provides a natural language-based text processing method, apparatus, electronic device, and medium, aiming to improve text processing accuracy.

[0061] The text processing method, apparatus, electronic device, and medium based on natural language provided in this application are specifically described through the following embodiments. First, the text processing method based on natural language in the embodiments of this application are described.

[0062] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0063] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0064] The natural language-based text processing method provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the natural language-based text processing method, but is not limited to the above forms.

[0065] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0066] Figure 1 This is an optional flowchart of a natural language-based text processing method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0067] Step S101: In response to the text task processing request of the target text, the target text is segmented to obtain text segmentation.

[0068] Step S102: Obtain the word index corresponding to the text segmentation from the preset vocabulary, and perform word embedding processing on the word index to obtain the initial word vector.

[0069] Step S103: Extract context features from the initial word vector to obtain word context vector.

[0070] Step S104: Perform deep semantic extraction on the word context vector to obtain the word deep semantic vector.

[0071] Step S105: Obtain the task complexity of the text task, and modulate the word index according to the task complexity to obtain the word modulation vector.

[0072] Step S106: Perform a text task on the target text based on the word modulation vector and the word deep semantic vector to obtain the text task processing result.

[0073] Steps S101 to S106 of this embodiment first obtain the word index corresponding to the text segmentation of the target text from a preset vocabulary, and perform word embedding processing on the word index to obtain an initial word vector. By processing the word index, the original features of the words can be preserved, and the contextual features of the initial word vector can be extracted, effectively capturing the contextual information of the word vector. Deep semantic extraction is performed on the word context vector to further capture the deep semantic information of the words, providing contextual and deep semantic support for subsequent text task processing. Second, the word index is modulated according to the complexity of the task to obtain a word modulation vector, which further preserves the original features of the words and effectively avoids the loss of original word information in deep networks. Finally, the text task is performed on the target text according to the word modulation vector and the word deep semantic vector. This not only avoids the problem of homogenizing all words in traditional methods, but also dynamically adjusts the word modulation vector according to the complexity of the task, effectively improving the retention rate of key word information and significantly improving the efficiency and accuracy of text processing.

[0074] In step S101 of some embodiments, specifically, the target text refers to the original text sequence to be processed.

[0075] For example, an English sentence that needs to be translated, a news article that needs to be categorized, or a sentence that needs to be replied to.

[0076] For example, the text task processing request can be determined based on the actual application scenario, and there are no restrictions here.

[0077] For example, text task processing requests can be text classification task requests, text translation task requests, intelligent customer service response tasks, or text sentiment analysis requests, etc.

[0078] Specifically, text segmentation refers to dividing continuous natural language text into discrete word units.

[0079] For example, in response to a user's question text "Why hasn't my order been shipped yet?", the question text can be segmented using WordPiece to obtain word units of ["I", "of", "order", "why", "not yet", "shipped", "?"].

[0080] In this embodiment, by responding to the text task processing request of the target text, the target text is segmented into words to obtain text words, which can provide structured input for subsequent word embedding processing, ensure the independence of each word, and at the same time preserve the semantic information of each word.

[0081] In step S102 of some embodiments, specifically, the preset vocabulary refers to a database that stores the mapping relationship between each word element in text segmentation and its corresponding word element index.

[0082] Specifically, a lexical index refers to the unique numerical identifier corresponding to each lexical unit in the text segmentation within the vocabulary.

[0083] For example, by looking up the vocabulary, you can find that the lexicon index for the word "language" is 123, and the lexicon index for the word "artificial intelligence" is 456, etc.

[0084] Specifically, the initial lexical vector refers to a continuous numerical lexical vector representation with a fixed dimension, which is used to characterize the spatial semantic features of each lexical index.

[0085] Specifically, the RWKV model also includes a word embedding layer, which can perform WordEmbedding on each word index to obtain an initial word vector, thereby converting the word index into a continuous vector space representation.

[0086] In this embodiment, by obtaining the lexical index corresponding to the text segmentation from the preset vocabulary and performing word embedding processing on the lexical index to obtain the initial lexical vector, the initial lexical vector corresponding to the lexical index can be determined, so that the original attributes of the lexical (such as part of speech and domain features) can continue to participate in subsequent decisions, effectively preventing the original lexical identity information from being diluted or lost in the word embedding process, and improving the accuracy of text task processing.

[0087] In step S103 of some embodiments, specifically, the lexical context vector refers to a vector that takes into account the contextual information surrounding the lexical index.

[0088] Specifically, the initial lexical vectors can be input into the RWKV model in the order of the target text. The initial lexical vectors are then processed by the channel-mixing sub-block within the residual block of the model through token shift processing. This means that the current lexical vector is semantically associated with its predecessor lexical vector. During this process, the channel-mixing sub-block works in collaboration with the time-mixing sub-block at the same layer. The time-mixing sub-block performs temporal decay and aggregation of the historical information of the current initial lexical vector, while the channel-mixing sub-block uses the token shift mechanism to fuse the filtered effective preceding information in the channel dimension.

[0089] For example, if the current initial lexical vector is "why", then the previous initial lexical vector "order" is reviewed through the temporal mixing sub-block to capture the semantic association between "why" and "order", and the lexical of the previous "order" is introduced by combining TokenShift to output the lexical context vector of the current initial lexical vector "why".

[0090] In this embodiment, contextual features are extracted from the initial lexical vector to obtain the lexical context vector. This allows for contextual feature extraction of the vector corresponding to the lexical index, avoiding the situation where the original identity of lexical units in deep networks is easily blurred due to repeated context fusion. This often results in the loss of the original identity information of lexical units in the routing decision of the gating network during text processing. It also avoids the problem of long sequence dependencies and effectively captures the contextual information of the vector corresponding to the lexical index, which helps to improve the accuracy of subsequent text processing.

[0091] In step S104 of some embodiments, specifically, the deep semantic vector of a word refers to removing redundant and interfering information from the semantic vector of intermediate words and retaining only the core and effective semantic feature vector.

[0092] Specifically, deep semantic extraction of word context vectors to obtain deep semantic vectors of words can include: performing non-linear processing on word context vectors to obtain intermediate word semantic vectors; and performing activation processing on intermediate word semantic vectors to obtain deep semantic vectors of words.

[0093] Among them, the intermediate word semantic vector refers to the transition vector after the word context vector has undergone nonlinear transformation.

[0094] Specifically, the RWKV model also includes a feedforward network, which can input the word context vector into the feedforward neural network for nonlinear transformation processing. Since the FFN includes two linear layers and a nonlinear activation function (the activation function here is only used for nonlinear transformation, unlike the subsequent separate activation processing step), the first linear layer can map the word context vector to a high-dimensional vector (e.g., from 128 dimensions to 1024 dimensions) to expand the expression space of word semantic features. The high-dimensional vector is then nonlinearly activated by a nonlinear activation function (e.g., the GeLU function) to obtain activated word vectors, which are used to mine semantic associations between words. The second linear layer then maps the activated word feature vectors back to the same dimension as the input dimension of the word context vector (e.g., from 1024 dimensions back to 128 dimensions) to obtain intermediate word semantic vectors.

[0095] For example, when a 128-dimensional word context vector is input into the FFN, the first linear layer of the FFN maps the 128-dimensional word context vector to a 1024-dimensional high-dimensional vector. The GELU function then performs a nonlinear transformation on the 1024-dimensional high-dimensional vector to obtain a 1024-dimensional activated word vector. The second linear layer then maps the 1024-dimensional activated word vector back to 128 dimensions to obtain a 128-dimensional intermediate word semantic vector.

[0096] In this embodiment, by performing nonlinear processing on the word context vector, the limitation that linear transformation cannot capture complex semantic relationships is broken. Deeper semantic features can be extracted from the word context vector. Furthermore, since the feedforward network has a simple structure and high computational efficiency, it matches the linear complexity advantage of the RWKV model and will not increase too many computational resources due to nonlinear processing, thus ensuring the processing efficiency of the target text.

[0097] In this embodiment, by performing deep semantic extraction on the context vector of word units, not only is the deep semantic association of word units effectively captured, but also invalid and redundant information in the deep semantics of word units is effectively removed, thereby improving the accuracy of deep semantic extraction of word units.

[0098] Please see Figure 2 In some embodiments, the task complexity includes low-level task complexity, medium-level task complexity, or high-level task complexity; the word modulation vector includes a single-dimensional modulation vector, a four-dimensional modulation vector, or a four-sample-dimensional modulation vector; step S105 includes, but is not limited to, steps S201 to S203:

[0099] Step S201: If the task complexity is low-level, then the word index is subjected to single-dimensional modulation processing to obtain the single-dimensional modulation vector corresponding to the word index; wherein, the vector dimension of the single-dimensional modulation vector is the same as the vector dimension of the initial word vector.

[0100] Step S202: If the task complexity is medium, then the single-dimensional modulation vector corresponding to the word index is scaled four times to obtain a four-dimensional modulation vector.

[0101] Step S203: If the task complexity is high-level, then the single-dimensional modulation vector corresponding to the word index is sampled and scaled four times to obtain a four-dimensional modulation vector.

[0102] In step S201 of some embodiments, specifically, task complexity refers to the task processing level divided according to the task type, task data processing volume and computing resources required for the text task. The task complexity includes low-level task complexity, medium-level task complexity or high-level task complexity.

[0103] Specifically, low-level task complexity refers to low-difficulty tasks that involve simple semantic understanding, small amounts of data processing, and low computational resources. For example, in a sentiment classification task that processes a short text (such as "This news article is very well written"), if the average number of tokens is 30, which meets the low-level average token processing threshold (e.g., less than 200), and the required video memory is 512MB and the computing power is 0.5 TFLOPS (Floating Point Operations per Second), which meets the low-level video memory threshold (e.g., less than 4GB) and low-level computing power threshold (e.g., less than 3 TFLOPS), then the sentiment classification task is determined to be of low-level task complexity.

[0104] Specifically, intermediate task complexity refers to a task of medium difficulty determined by the professional domain classification task based on the required professional domain knowledge, the moderate amount of task data processing, and the moderate amount of computing resources required. For example, when processing a task of classifying multiple themes from 100,000 words of news articles, if the average number of tokens is 256, which meets the intermediate average token processing threshold (e.g., greater than or equal to 200 and less than 1000), the required video memory is 8GB, which meets the intermediate video memory threshold (e.g., greater than or equal to 4GB and less than 25GB) and the intermediate computing power threshold (e.g., greater than or equal to 3TFLOPS and less than 15TFLOPS), and the computing power is 5TFLOPS, then the task is determined to be of intermediate task complexity.

[0105] Specifically, advanced task complexity refers to complex task types that require deep semantic reasoning, involving large amounts of data processing and highly defined computational resources. For example, for the translation of a 1 million-word book, if the average number of tokens is 1024, which meets the advanced average token processing threshold (e.g., greater than 1000), the required video memory is 32GB, and the computing power is 20TFLOPS, which meets the advanced video memory threshold (e.g., greater than 25GB) and the advanced computing power threshold (e.g., greater than 15TFLOPS), then the task is classified as advanced task complexity.

[0106] Specifically, a single-dimensional modulation vector refers to a DeepEmb vector with the same dimension as the initial lexical vector.

[0107] Specifically, the RWKV model also includes a lexical modulation embedding layer. If the task complexity is low, the lexical index can be scaled by one dimension through the lexical modulation embedding layer to further improve the spatial semantic representation of the lexical index. The lexical modulation embedding layer is related to the size of the vocabulary to ensure that each lexical index in the vocabulary has a corresponding modulation vector, avoiding index out-of-bounds situations. The lexical modulation embedding layer includes modulation functions of single modulation embedding, four-fold modulation embedding, and four-fold sampling modulation embedding. The use of different modulation functions is determined based on the task complexity.

[0108] For example, if the vocabulary stores 50,000 lexical indices, the lexical modulation embedding layer can be used to represent the modulation vectors corresponding to the 50,000 lexical indices respectively.

[0109] For example, if the lexical index of "language" in the vocabulary is 123, then the lexical modulation embedding layer performs single modulation embedding on 123, and the single-dimensional modulation vector corresponding to 123 can be 4-dimensional [0.15, 0.2, 0.4, 0.45].

[0110] In step S202 of some embodiments, specifically, the four-dimensional modulation vector refers to the DeepEmb vector, which expands the dimension of the single-dimensional modulation vector by four times. It is used to expand the dimension of the lexical index of intermediate task complexity to provide richer semantic information, thereby further improving the spatial semantic representation of lexical units.

[0111] For example, if the word index of "language" in the vocabulary is 123, then 123 is single-modulation embedded through the word modulation embedding layer, and the single-dimensional modulation vector corresponding to 123 can be 4-dimensional [0.15,0.2,0.4,0.45]. This single-dimensional modulation vector is then scaled by 4 times to obtain 16-dimensional [0.1,0.2,0.3,0.4,0.15,0.25,0.35,0.45,0.2,0.3,0.4,0.5,0.25,0.35,0.45,0.55].

[0112] Please see Figure 3 In some embodiments, step S203 includes, but is not limited to, steps S301 to S303:

[0113] Step S301: Downsample the four-dimensional modulation vector corresponding to the single-dimensional modulation vector to obtain the dimension-reduced modulation vector.

[0114] Step S302: The reduced-dimensional modulation vector is grouped and reshaped to obtain the reshaped modulation vector.

[0115] Step S303: Upsample the reshaped modulation vector to obtain a modulation vector with four times the sampling dimension.

[0116] In step S301 of some embodiments, specifically, the dimension reduction modulation vector refers to the DeepEmb vector after downsampling the four-dimensional modulation vector corresponding to the single-dimensional modulation vector.

[0117] Specifically, a reduced-dimensional modulation vector can be obtained by applying the Lora (Low-Rank Adaptation) technique to the four-dimensional modulation vector.

[0118] For example, in the task of translating a 1 million-word book, if the dimension of the single-dimensional modulation vector is 128, and the dimension after quadrupling is 512, downsampling can be achieved by introducing a low-rank matrix of Lora. The low-rank matrix of Lora can be represented as the product of two matrices, where the dimension of the first matrix A can be 256×2, and the dimension of the second matrix B can be 2×512. By multiplying the low-rank matrices A and B, the 512-dimensional quadrupling modulation vector can be mapped to a 256-dimensional reduced modulation vector, which can effectively capture the key features in the quadrupling modulation vector.

[0119] In this embodiment, by downsampling the four-dimensional modulation vector, not only is the computational resource overhead reduced, but the key semantic information of the lexical units is also preserved, thereby improving computational efficiency while ensuring the semantic accuracy of the lexical units.

[0120] In step S302 of some embodiments, specifically, reshaping the modulation vector refers to the vector after semantic space adjustment of the dimension-reduced modulation vector, and the dimension of the reshaping modulation vector is the same as the dimension of the dimension-reduced modulation vector.

[0121] For example, the 256-dimensional reduced modulation vector can be divided into 4 groups, each with 64-dimensional grouped reduced modulation vectors. Then, each grouped reduced modulation vector is processed nonlinearly by an independent lightweight MLP (Multilayer Perceptron) to obtain 64-dimensional reshaped grouped reduced modulation vectors. Finally, the 64-dimensional reshaped grouped reduced modulation vectors are concatenated to obtain the 256-dimensional reshaped modulation vector.

[0122] In step S303 of some embodiments, specifically, the four-times-sampled-dimension modulation vector refers to the DeepEmb vector obtained by expanding the dimension of the single-dimension modulation vector by four times and sampling it.

[0123] Specifically, the reshaped modulation vector can be upsampled to a four-times-sampled-dimensional modulation vector by performing transpose convolution on the reshaped modulation vector.

[0124] For example, a 256-dimensional reshaped modulation vector can be transposed and convolved with a kernel size of 2, a stride of 2, and padding of 0 to expand the dimension of the reshaped modulation vector from 256 dimensions to a 512-dimensional modulation vector with four times the sampling dimension.

[0125] Through steps S301 to S303, by downsampling, grouping and reshaping, and upsampling, it is possible to reduce the overhead of computing resources and ensure the accuracy of text task processing when dealing with tasks of high complexity.

[0126] Through steps S201 to S203, by dynamically adjusting the modulation vector dimension of the lexical index according to the task complexity, precise adaptation to different task complexity requirements is achieved. That is, the differentiated modulation mechanism not only enhances the model's adaptability and semantic expression ability, but also optimizes the computational efficiency. Moreover, the modulation vector is obtained by modulating and embedding the lexical index in different dimensions, which not only effectively preserves the original identity information of the lexical, but also further captures the deep embedded semantics of the lexical, which helps to improve the efficiency and accuracy of subsequent text processing.

[0127] In one optional embodiment of this application, before performing the text task, the deep semantic vector of the word unit can be residually processed with the initial word unit vector to obtain the enhanced word unit semantic vector. That is, the deep semantic vector of the word unit is added element-wise with the initial word unit vector using the residual connection method, and the fused vector after addition is determined as the enhanced word unit semantic vector. The enhanced word unit semantic vector is then determined as the final deep semantic vector of the word unit. This further preserves the original features of the word unit while incorporating the deep semantic information of the word unit, and further effectively avoids the loss of the original information of the word unit in the deep network.

[0128] For example, if the deep semantic vector of a word is [0.4, 0.6, 0.3, 0.5] and the initial word vector is [0.2, 0.3, 0.4, 0.2], then the enhanced semantic vector of the word can be obtained as [0.6, 0.9, 0.7, 0.7] through residual connection.

[0129] Please see Figure 4 In some embodiments, step S106 includes, but is not limited to, steps S401 to S402:

[0130] Step S401: If the word modulation vector is a single-dimensional modulation vector, then the word deep semantic vector and the single-dimensional modulation vector are fused element by element to obtain the target calibrated word vector.

[0131] Step S402: Perform a text task on the target text based on the target calibration lexical vector to obtain the text task processing result.

[0132] Please see Figure 5 , in some embodiments, step S401 further includes but is not limited to steps S501 to S504:

[0133] Step S501, obtaining the modulation category of the token modulation vector; wherein, the modulation category includes at least one of multiplicative modulation category, additive modulation category, and affine transformation modulation category.

[0134] Step S502, if the modulation category includes the multiplicative modulation category, then element-wise multiply the token deep semantic vector by the single-dimensional modulation vector to obtain the target calibrated token vector.

[0135] Step S503, if the modulation category includes the additive modulation category, then element-wise add the token deep semantic vector to the single-dimensional modulation vector to obtain the target calibrated token vector.

[0136] Step S504, if the modulation category includes the affine transformation modulation category, then divide the single-dimensional modulation vector into a single-dimensional multiplicative modulation vector and a single-dimensional additive modulation vector, element-wise multiply the token deep semantic vector by the single-dimensional multiplicative modulation vector to obtain the multiplicative adjusted token vector, element-wise add the token deep semantic vector to the single-dimensional additive modulation vector to obtain the additive adjusted token vector, and determine the target calibrated token vector according to the multiplicative adjusted token vector and the additive adjusted token vector.

[0137] In step S501 of some embodiments, specifically, the modulation category refers to the classification of transformation methods for semantic adjustment of the token deep semantic vector, and the modulation category includes at least one of multiplicative modulation category, additive modulation category, and affine transformation modulation category.

[0138] Specifically, the multiplicative modulation category is a method for token vector modulation achieved by element-wise multiplication, which is used to enhance or weaken the semantic feature intensity of the token vector.

[0139] For example, for the sentiment classification task of short texts, the token "wonderful" is a positive sentiment word in sentiment classification. The semantic feature intensity of the token "wonderful" can be enhanced to improve the influence of the "wonderful" semantics on sentiment classification.

[0140] Specifically, the additive modulation category is a method for vector modulation achieved by element-wise addition, which is used to correct the overall semantic deviation of the token vector.

[0141] For example, for the sentiment classification task of short texts, the token "fair" is inclined to be positive but the original semantics is neutral (overall semantic deviation) in sentiment classification. The overall semantic deviation can be corrected by bias modulation to improve the accuracy of neutral to positive sentiment classification.

[0142] Specifically, affine transformation modulation is a modulation method that combines multiplicative and additive transformations, used to adjust the semantic feature strength and overall semantic bias of lexical vectors.

[0143] For example, in the sentiment classification task of short text, the word "good" tends to be positive in sentiment classification, but its original semantics are neutral (overall semantic bias). The positive feature strength of "good" can be enhanced by combining affine transformation with multiplicative modulation, and the overall neutral bias can be corrected by additive modulation, so that the semantics of "good" are more in line with the actual sentiment tendency and the classification accuracy can be improved.

[0144] In step S502 of some embodiments, specifically, the target calibration lexical vector refers to the lexical vector after being modulated by a single-dimensional modulation vector for deep semantic vectors of lexical units in low-level task complexity scenarios.

[0145] For example, if the word modulation vector is a single-dimensional modulation vector and the modulation category is a multiplicative modulation category, for a low-level task complexity text sentiment classification task (such as "This news article is very well written"), whether it is positive or negative, if the deep semantic vector of the word "wonderful" is [0.6,0.9,0.7,0.7], and the single-dimensional modulation vector corresponding to the word index of "wonderful" is [1.2,0.8,1.0,0.6], then the target calibration word vector after single multiplicative modulation can be: [0.6x1.2,0.9x0.8,0.7x1.0,0.7x0.6]=[0.72,0.72,0.7,0.42].

[0146] In step S503 of some embodiments, for example, if the word modulation vector is a single-dimensional modulation vector and the modulation category is an additive modulation category, for a low-level task complexity text sentiment classification task (such as "This article is written fairly well"), if the word deep semantic vector of "fairly well" is [0.5, 0.6, 0.7, 0.8], and the single-dimensional modulation vector corresponding to the word index of "fairly well" is [0.1, 0.2, 0.3, 0.4], then the target calibration word vector after single additive modulation can be: [0.5+0.1, 0.6+0.2, 0.7+0.3, 0.8+0.4]=[0.6, 0.8, 1.0, 1.2].

[0147] In step S504 of some embodiments, for example, if the word modulation vector is a single-dimensional modulation vector and the modulation category is an affine transformation modulation category, for a low-level task complexity text sentiment classification task (such as "This paper is well written"), if the deep semantic vector of the word "good" is [0.7, 0.8, 0.9, 1.0], and the single-dimensional modulation vector corresponding to the word index of "good" is divided into a single-dimensional multiplicative modulation vector [1.1, 1.0, 0.9, 0.8] and a single-dimensional additive modulation vector [0.2, 0.3, 0.4, 0.5], then the multiplicative modulated word vector after single-dimensional multiplicative modulation can be: [0 [0.77,0.8,0.81,0.8] = [0.77,0.8,0.81,0.8]. After single-fold additive modulation, the additive adjustment lexical vector can be: [0.7+0.2,0.8+0.3,0.9+0.4,1.0+0.5] = [0.9,1.1,1.3,1.5]. Finally, the target calibration lexical vector determined based on the multiplicative adjustment lexical vector and the additive adjustment lexical vector can be: [0.77+0.2,0.8+0.3,0.81+0.4,0.8+0.5] = [0.97,1.1,1.21,1.3].

[0148] Through steps S501 to S504, the modulation vectors corresponding to different modulation categories are fused element-by-element with the deep semantic vectors of the word units. This can adapt to the needs of different text tasks and effectively solve the problems of inaccurate semantic expression of word units and poor adaptability to different text tasks. It can accurately adjust the semantic features of word units and effectively solve the problems of wasted computing resources and loss of original identity and key information of word units caused by the homogenization of all word units in traditional methods.

[0149] In step S402 of some embodiments, specifically, the text task processing result refers to the task execution result of the target text under low-level task complexity.

[0150] For example, in a sentiment classification task for short texts (such as "This article is well written"), the corresponding text task processing result is that the short text can be classified as a positive sentiment category.

[0151] Specifically, all target calibration word vectors can be concatenated in the order of the target text to obtain a text fusion vector. Then, the text fusion vector can be activated using an activation function (such as the Softmax function) to obtain the text task processing result.

[0152] Through steps S401 to S402, it is possible to combine single-word modulation vectors and word deep semantic vectors to perform text tasks with low task complexity. While preserving the original identity of words, the semantic distribution of word space is adjusted, improving the accuracy of text processing. Furthermore, the low-level task complexity is not expanded in a targeted manner to ensure the efficiency of text processing tasks.

[0153] Please see Figure 6 In some embodiments, step S106 may include, but is not limited to, steps S601 to S602:

[0154] Step S601: If the word modulation vector is a four-dimensional modulation vector, then the word deep semantic vector and the four-dimensional modulation vector are fused element by element to obtain the target enhanced word vector.

[0155] Step S602: Perform a text task on the target text based on the target enhanced lexical vector to obtain the text task processing result.

[0156] In step S601 of some embodiments, specifically, the target-enhanced lexical vector refers to the lexical vector after being modulated by a four-fold dimensional modulation vector for deep semantic vectors of lexical vectors in medium-level task complexity scenarios.

[0157] Specifically, if the task complexity is medium and the word modulation vector is a four-dimensional modulation vector, a modulation category can be determined from multiplicative modulation category, additive modulation category, and affine transformation modulation category for the medium-level task complexity scenario, so as to determine the element-wise fusion of word deep semantic vector and four-dimensional modulation vector based on the modulation category.

[0158] Furthermore, since the method of fusing the deep semantic vector of a word element with the four-dimensional modulation vector is the same as the method of fusing the deep semantic vector of a word element with the single-dimensional modulation vector, it will not be repeated here.

[0159] For example, in multi-topic text classification tasks with intermediate task complexity, for lexical units that express the core concept of a topic, such as "innovation," the weight of the "innovation" lexical unit can be strengthened through multiplicative modulation to improve the ability to identify the "innovation" topic.

[0160] For example, in multi-topic text classification tasks with intermediate task complexity, for a word motif such as "development" that is slightly biased towards the topic of "innovation" but whose semantics are not clear enough, additive modulation can add an appropriate bias to "development" to make the word motif of "development" perform more accurately in multi-topic classification.

[0161] For example, in a multi-topic text classification task with intermediate task complexity, the term "robust" is represented positively in the risk control topic and negatively in the innovation topic. Furthermore, the original semantics of the term "robust" is neutral. For the risk control topic classification, the spatial semantics of "robust" can be enhanced by combining affine transformation with multiplicative modulation. For the innovation topic classification, the spatial semantics of "robust" can be weakened, and additive modulation can be used to correct the overall neutrality bias, thereby improving the classification accuracy of the term "robust" in multi-topic text classification tasks.

[0162] In step S602 of some embodiments, specifically, the text task processing result refers to the task execution result of the target text under medium task complexity.

[0163] For example, when performing a multi-category thematic classification task on a 100,000-word news article, the corresponding text task processing results can be "technological innovation" and "environmental development" themes.

[0164] Specifically, the method for performing text tasks on the target text based on the target enhanced lexical vector and obtaining the text task processing result is the same as the method for performing text tasks on the target text based on the target calibrated lexical vector and obtaining the text task processing result, and will not be repeated here.

[0165] Through steps S601 to S602, text tasks of medium complexity can be performed by combining four-fold word modulation vectors and deep word semantic vectors. While preserving the original word identity features, four-fold dimension modulation is used to capture deeper word semantics, thereby improving the retention rate of key word information and thus improving the accuracy of text processing.

[0166] Please see Figure 7 In some embodiments, step S106 may include, but is not limited to, steps S701 to S702:

[0167] Step S701: If the word modulation vector is a four-times sampling dimension modulation vector, then the word deep semantic vector and the four-times sampling dimension modulation vector are fused element by element to obtain the target optimized word vector.

[0168] Step S702: Perform a text task on the target text based on the target optimized word vector to obtain the text task processing result.

[0169] In step S701 of some embodiments, specifically, the target optimized lexical vector refers to the lexical vector after being modulated by a four-fold sampling dimension modulation vector for deep semantic vectors of lexical units in medium-to-high-level task complexity scenarios.

[0170] Specifically, if the task complexity is high-level and the word modulation vector is a four-times-sampled-dimension modulation vector, a modulation category can still be determined from the multiplicative modulation category, additive modulation category, and affine transformation modulation category for the high-level task complexity scenario. This allows for the element-wise fusion of the word deep semantic vector and the four-times-sampled-dimension modulation vector based on the modulation category.

[0171] Furthermore, since the method of fusing the deep semantic vector of the lexical unit with the modulation vector of four times the sampling dimension element-wise to obtain the target optimized lexical unit vector and fusing the modulation vector of one times the sampling dimension element-wise to obtain the target calibrated lexical unit vector is the same, it will not be repeated here.

[0172] For example, in a translation task involving a 1 million-word book with high task complexity, the weight of the lexical unit "technology" can be increased through multiplicative modulation to improve the translation accuracy of related concepts.

[0173] For example, in a translation task involving a 1 million-word book with high task complexity, if the meaning of the word "application" needs to be emphasized in a technological context during translation, additive modulation can adjust the semantic representation of the word "application" in a technological context to better fit the context.

[0174] For example, in a high-complexity task like translating a 1 million-word book, if the word "efficiency" represents positive and negative meanings in "work efficiency" and "energy efficiency" respectively, affine transformation modulation combined with multiplicative modulation can enhance the positive semantics of "efficiency" in the context of "work efficiency," while additive modulation can correct the negative bias of "efficiency" in the context of "energy efficiency," thereby improving the classification accuracy of the word "efficiency" in the translation task.

[0175] In step S702 of some embodiments, specifically, the text task processing result refers to the task execution result of the target text under high task complexity.

[0176] For example, for a Chinese-to-English translation task of a 1 million-word book, the corresponding text task processing result can be the English translation of the book's content.

[0177] Specifically, the method of performing text tasks on the target text based on the optimized lexical vectors of the target and obtaining the text task processing results is the same as the method of performing text tasks on the target text based on the calibrated lexical vectors of the target and obtaining the text task processing results, and will not be elaborated here.

[0178] Through steps S701 to S702, it is possible to combine the four-fold sampled lexical modulation vector with the deep semantic vector of the lexical to perform text tasks with high task complexity. While preserving the original lexical identity features, it not only captures deeper lexical semantics through four-fold dimension modulation, thus improving the accuracy of text processing, but also avoids the computational redundancy caused by simple dimension expansion by combining the sampling mechanism, further improving the efficiency of text processing.

[0179] In one optional embodiment of this application, the single-dimensional modulation vector, the four-dimensional modulation vector, and the four-sampled-dimensional modulation vector are stored in system memory (such as RAM, Random Access Memory) or solid-state drive (SSD), and the relevant model parameters for determining the deep semantic vector of the word are stored in the video processor memory (VRAM, Video Random Access Memory). During text processing, the modulation vector corresponding to a single word can be loaded from RAM or SSD on demand and efficiently through a pre-reading mechanism, thereby improving the efficiency of the RWVK model in performing text task processing with almost no increase in video memory usage.

[0180] This embodiment first obtains the lexical index corresponding to the text segmentation of the target text from a preset vocabulary, and performs word embedding processing on the lexical index to obtain an initial lexical vector. By processing the lexical index, the original features of the lexical are preserved, and contextual features are extracted from the initial lexical vector, effectively capturing the contextual information of the lexical vector. Deep semantic extraction is then performed on the lexical context vector to further capture the deep semantic information of the lexical, providing contextual and deep semantic support for subsequent text task processing. Second, the lexical index is modulated according to the task complexity to obtain a lexical modulation vector, further preserving the original features of the lexical and effectively avoiding the loss of original lexical information in deep networks. Finally, the text task is performed on the target text based on the lexical modulation vector and the deep semantic vector of the lexical. This not only avoids the problem of homogenizing all lexicals in traditional methods, but also dynamically adjusts the lexical modulation vector according to the complexity of the task, effectively improving the retention rate of key lexical information and significantly improving the efficiency and accuracy of text processing.

[0181] Please see Figure 8 This application also provides a natural language-based text processing apparatus that can implement the above-described natural language-based text processing method. The apparatus includes:

[0182] The word segmentation module is used to respond to the text task processing request of the target text, perform word segmentation on the target text, and obtain the text words;

[0183] The word embedding processing module is used to perform word embedding processing on the text segmentation to obtain the initial word vector;

[0184] The context feature extraction module is used to extract context features from the initial word vector to obtain the word context vector;

[0185] The deep semantic extraction module is used to extract deep semantics from the context vector of word units to obtain deep semantic vectors of word units;

[0186] The lexical modulation processing module is used to obtain the task complexity of the text task and modulate the lexical index according to the task complexity to obtain the lexical modulation vector.

[0187] The text task execution module is used to perform text tasks on the target text based on the word modulation vector and the word deep semantic vector, and obtain the text task processing results.

[0188] The specific implementation of this natural language-based text processing device is basically the same as the specific implementation of the natural language-based text processing method described above, and will not be repeated here.

[0189] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described natural language-based text processing method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0190] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0191] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0192] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the processing system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the natural language-based text processing method of the embodiments of this application.

[0193] The input / output interface 903 is used to implement information input and output;

[0194] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0195] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0196] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0197] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described natural language-based text processing method.

[0198] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0200] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0201] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0202] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0203] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0204] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0206] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0208] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A natural language based text processing method, characterized by, The method includes: In response to a text task processing request for the target text, the target text is segmented into words to obtain text words; Obtain the word index corresponding to the text segmentation from the preset vocabulary, and perform word embedding processing on the word index to obtain the initial word vector; Context features are extracted from the initial word vector to obtain the word context vector; Deep semantic extraction is performed on the word context vector to obtain the word deep semantic vector; The task complexity of the text task is obtained, and the word index is modulated according to the task complexity to obtain a word modulation vector; wherein, the task complexity includes low-level task complexity, medium-level task complexity, or high-level task complexity; the word modulation vector includes a single-dimensional modulation vector, a four-dimensional modulation vector, or a four-sample-dimensional modulation vector. The target text is processed using the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result. The step of modulating the lexical index according to the task complexity to obtain the lexical modulation vector includes: If the task complexity is the same as the low-level task complexity, then the lexical index is subjected to single-dimension modulation processing to obtain the single-dimension modulation vector corresponding to the lexical index; wherein, the vector dimension of the single-dimension modulation vector is the same as the vector dimension of the initial lexical vector. If the task complexity is the same as the intermediate task complexity, then the single-dimensional modulation vector corresponding to the lexical index is scaled by four times to obtain the four-dimensional modulation vector. If the task complexity is the same as the high-level task complexity, then the single-dimensional modulation vector corresponding to the lexical index is sampled and scaled by four times to obtain the four-dimensional modulation vector.

2. The method of claim 1, wherein, The step of scaling the single-dimensional modulation vector corresponding to the lexical index by four times to obtain the four-dimensional modulation vector includes: The four-dimensional modulation vector corresponding to the single-dimensional modulation vector is downsampled to obtain the dimension-reduced modulation vector. The reduced-dimensional modulation vector is grouped and reshaped to obtain a reshaped modulation vector; The reshaped modulation vector is upsampled to obtain the four-times-sampled-dimension modulation vector.

3. The method of claim 1, wherein, The step of performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result includes: If the lexical modulation vector is the single-dimensional modulation vector, then the deep semantic vector of the lexical and the single-dimensional modulation vector are fused element by element to obtain the target calibrated lexical vector. The target text is processed using the target calibration lexical vector to obtain the text task processing result.

4. The method of claim 3, wherein, The step of fusing the deep semantic vector of the lexical unit with the single-dimensional modulation vector element-wise to obtain the target calibrated lexical unit vector includes: Obtain the modulation category of the word modulation vector; wherein the modulation category includes at least one of multiplicative modulation category, additive modulation category and affine transformation modulation category; If the modulation category includes the multiplicative modulation category, then the deep semantic vector of the lexical unit is multiplied element-wise with the single-dimensional modulation vector to obtain the target calibrated lexical unit vector; If the modulation category includes the additive modulation category, then the deep semantic vector of the lexical unit is added element-wise to the single-dimensional modulation vector to obtain the target calibrated lexical unit vector; If the modulation category includes the affine transformation modulation category, then the single-dimensional modulation vector is divided into a single-dimensional multiplicative modulation vector and a single-dimensional additive modulation vector. The deep semantic vector of the lexical unit is multiplied element-wise with the single-dimensional multiplicative modulation vector to obtain a multiplicative modulated lexical vector. The deep semantic vector of the lexical unit is added element-wise with the single-dimensional additive modulation vector to obtain an additive modulated lexical vector. The target calibration lexical vector is determined based on the multiplicative modulated lexical vector and the additive modulated lexical vector.

5. The method of claim 1, wherein, The step of performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result further includes: If the word modulation vector is the four-dimensional modulation vector, then the word deep semantic vector and the four-dimensional modulation vector are fused element by element to obtain the target enhanced word vector. The text task is performed on the target text based on the target enhanced lexical vector to obtain the text task processing result.

6. The method of claim 1, wherein, The step of performing a text task on the target text based on the lexical modulation vector and the lexical deep semantic vector to obtain the text task processing result further includes: If the word modulation vector is the four-times sampling dimension modulation vector, then the word deep semantic vector and the four-times sampling dimension modulation vector are fused element by element to obtain the target optimized word vector; Based on the target optimized lexical vector, a text task is performed on the target text to obtain the text task processing result.

7. A natural language based text processing apparatus, characterized by, The device includes: The word segmentation module is used to respond to the text task processing request of the target text, perform word segmentation on the target text, and obtain text words; The word embedding processing module is used to obtain the word index corresponding to the text segmentation from the preset vocabulary, and to perform word embedding processing on the word index to obtain the initial word vector; The context feature extraction module is used to extract context features from the initial word vector to obtain a word context vector; The deep semantic extraction module is used to perform deep semantic extraction on the word context vector to obtain the word deep semantic vector; The lexical modulation processing module is used to obtain the task complexity of the text task and modulate the lexical index according to the task complexity to obtain the lexical modulation vector; wherein, the task complexity includes low-level task complexity, medium-level task complexity or high-level task complexity; the lexical modulation vector includes single-dimensional modulation vector, four-dimensional modulation vector or four-sample-dimensional modulation vector. The text task execution module is used to perform text tasks on the target text based on the word modulation vector and the word deep semantic vector, and obtain the text task processing result. The word modulation processing module is specifically used for: If the task complexity is the same as the low-level task complexity, then the lexical index is subjected to single-dimension modulation processing to obtain the single-dimension modulation vector corresponding to the lexical index; wherein, the vector dimension of the single-dimension modulation vector is the same as the vector dimension of the initial lexical vector. If the task complexity is the same as the intermediate task complexity, then the single-dimensional modulation vector corresponding to the lexical index is scaled by four times to obtain the four-dimensional modulation vector. If the task complexity is the same as the high-level task complexity, then the single-dimensional modulation vector corresponding to the lexical index is sampled and scaled by four times to obtain the four-dimensional modulation vector.

8. An electronic device, comprising: The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the natural language-based text processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by a processor, it implements the natural language-based text processing method according to any one of claims 1 to 6.

Images