Text processing method, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention discloses a text processing method, electronic equipment, a computer readable storage medium and a computer program product, and relates to the field of data processing in a large model technology. The method comprises the steps of obtaining a to-be-processed text; performing pre-filling processing on a first text in the to-be-processed text by using a first model parameter of a text processing model stored on the client to obtain a first pre-filling result of the first text; processing the to-be-processed text based on the first pre-filling result by using the first model parameter and a second model parameter of a text processing model stored on the cloud to obtain a processing result of the to-be-processed text, the first model parameter being used for representing a part of model parameters in the text processing model, and the second model parameter being used for representing a part of model parameters in the text processing model; the second model parameters at least comprise other parameters except the first model parameters in the text processing model. According to the method and the device, the technical problems of low user experience, waste of client resources and high model operation cost caused by delay of the first token when the user input is replied in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of large model technology data processing, and in particular, to a text processing method, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rise of ChatGPT, the demand for large-scale model inference is exploding on the user side. To meet these inference needs, one of the main methods currently used by large-scale model service providers is to rent a model deployed in the cloud to be responsible for inference and complete all calculations. The client application (Application, APP) is only responsible for uploading the prompt word and displaying the results. However, the cloud service undertakes all calculations, which means that the user's input data needs to be transmitted to the cloud via the Internet for processing before the results are returned to the user. Due to the limitations of Internet transmission speeds and the time consumed by cloud computing, even if the cloud service itself has a fast computing speed, the time required for data transmission and processing will affect the overall latency. This results in the model being unable to generate the corresponding answer in real time when responding to user input, resulting in a high first character (token) delay, which in turn leads to a poor user experience.

[0003] In addition, since the cloud undertakes all calculations, the client's resources are not fully utilized, and the cost of cloud computing is higher than that of client computing, which leads to waste of client resources and high model operation costs.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] Embodiments of the present application provide a text processing method, an electronic device, a computer-readable storage medium, and a computer program product to at least solve the technical problems in the related art in which a first token delay is generated when responding to user input, resulting in a low user experience, waste of client resources, and high model operation costs.

[0006] According to one aspect of an embodiment of the present application, a text processing method is provided, including: obtaining a text to be processed; using a first model parameter of a text processing model stored on a client to pre-fill a first text in the text to be processed, and obtaining a first pre-fill result of the first text; using the first model parameter and a second model parameter of the text processing model stored on the cloud to process the text to be processed based on the first pre-fill result, and obtaining a processing result of the text to be processed, wherein the first model parameter is used to characterize part of the model parameters in the text processing model, and the second model parameter at least includes other parameters in the text processing model except the first model parameter.

[0007] According to another aspect of an embodiment of the present application, a text processing method is also provided, which is applied to a client, and the method includes: responding to receiving an input text to be processed; using a first model parameter of a text processing model stored on the client to pre-fill a first text in the text to be processed to obtain a first pre-filled result of the first text, wherein the first model parameter is used to characterize part of the model parameters in the text processing model; sending the text to be processed to the cloud, and receiving an intermediate result sent by the cloud, wherein the intermediate result is a result obtained by processing the text to be processed using a second model parameter of the text processing model stored on the cloud, and the second model parameter includes at least other parameters in the text processing model except the first model parameter; using the first model parameter, processing the intermediate result based on the first pre-filled result to obtain a processing result of the text to be processed.

[0008] According to another aspect of an embodiment of the present application, a text processing method is also provided, which is applied to the cloud, and the method includes: receiving a text to be processed sent by a client, wherein the text to be processed is a text of external input received by the client, and a first model parameter of a text processing model is stored on the client, and the first model parameter is used to characterize part of the model parameters in the text processing model, and the first model parameter is used to pre-fill the first text in the text to be processed to obtain a first pre-filled result of the first text; using the second model parameter of the text processing model stored on the cloud to process the text to be processed to obtain an intermediate result of the text to be processed, wherein the second model parameter at least includes other parameters in the text processing model except the first model parameter; sending the intermediate result to the client, wherein the intermediate result is processed by the client based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

[0009] According to another aspect of an embodiment of the present application, a text processing device is also provided, including: an acquisition module for acquiring a text to be processed; a first processing module for pre-filling a first text in the text to be processed using a first model parameter of a text processing model stored on a client, and obtaining a first pre-filled result of the first text; a second processing module for processing the text to be processed based on the first pre-filled result using the first model parameter and a second model parameter of the text processing model stored on the cloud, and obtaining a processing result of the text to be processed, wherein the first model parameter is used to characterize part of the model parameters in the text processing model, and the second model parameter at least includes other parameters in the text processing model except the first model parameter.

[0010] According to another aspect of an embodiment of the present application, a text processing device is also provided, which is applied to a client, and the device includes: a receiving module for responding to receiving an input text to be processed; a first processing module for pre-filling a first text in the text to be processed using a first model parameter of a text processing model stored on the client, to obtain a first pre-filled result of the first text, wherein the first model parameter is used to characterize part of the model parameters in the text processing model; a sending module for sending the text to be processed to the cloud and receiving an intermediate result sent by the cloud, wherein the intermediate result is a result obtained by processing the text to be processed using a second model parameter of the text processing model stored on the cloud, and the second model parameter includes at least other parameters in the text processing model except the first model parameter; a second processing module for processing the intermediate result based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

[0011] According to another aspect of an embodiment of the present application, a text processing device is also provided, which is applied to the cloud. The device includes: a receiving module, which is used to receive a text to be processed sent by a client, wherein the text to be processed is a text of external input received by the client, and a first model parameter of a text processing model is stored on the client, and the first model parameter is used to characterize part of the model parameters in the text processing model. The first model parameter is used to pre-fill the first text in the text to be processed to obtain a first pre-filled result of the first text; a processing module, which is used to process the text to be processed using a second model parameter of the text processing model stored on the cloud to obtain an intermediate result of the text to be processed, wherein the second model parameter at least includes other parameters in the text processing model except the first model parameter; a sending module, which is used to send the intermediate result to the client, wherein the intermediate result is processed by the client based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

[0012] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein any one of the above methods is executed when the program is running.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above methods.

[0014] According to another aspect of an embodiment of the present application, a computer program product is further provided, including a computer program, which implements any of the above methods when executed by a processor.

[0015] In an embodiment of the present application, a method is adopted in which a text to be processed is obtained; a first model parameter of a text processing model stored on a client is used to pre-fill a first text in the text to be processed, and a first pre-fill result of the first text is obtained; and the text to be processed is processed based on the first pre-fill result using the first model parameter and a second model parameter of a text processing model stored on a cloud, and a processing result of the text to be processed is obtained, wherein the first model parameter is used to characterize part of the model parameters in the text processing model, and the second model parameter at least includes other parameters in the text processing model except the first model parameter. It is easy to notice that the first text in the text to be processed is directly processed by the first model parameter of the text processing model on the client, without uploading the first text to the cloud and then having the cloud process the first text, and the model parameters of the text processing model on the client are not compressed, which not only saves the processing time of the text processing model, but also ensures the processing accuracy of the text processing model, thereby solving the problem of the first token delay. In addition, the text processing models on the client and the cloud can jointly process the second text in the text to be processed. Through the combination of end and cloud, the computing power of the client is used to process the first text in the text to be processed. This can not only make full use of the computing resources of the client and reduce the workload on the cloud, but also reduce the operating cost of the model, thereby improving the overall throughput of the text processing model and realizing cloud cost optimization, achieving the purpose of improving user experience, making full use of client resources, and reducing the operating cost of large models, thereby achieving the technical effect of not generating the first token delay when responding to user input, and thus solving the technical problem in the related technology that the first token delay will be generated when responding to user input, resulting in low user experience, waste of client resources and high model operating cost.

[0016] It is easy to notice that the above general description and the following detailed description are merely for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of a text processing method according to Example 1 of the present application;

[0020] Figure 3is a flowchart of an optional text processing according to Example 1 of the present application;

[0021] Figure 4 is a flowchart of a text processing method according to Example 2 of the present application;

[0022] Figure 5 is a flowchart of a text processing method according to Example 3 of the present application;

[0023] Figure 6 is a schematic diagram of a text processing device according to Example 4 of the present application;

[0024] Figure 7 is a schematic diagram of a text processing device according to Example 5 of the present application;

[0025] Figure 8 is a schematic diagram of a text processing device according to Example 6 of the present application;

[0026] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The technical solution provided in this application is mainly implemented using large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. The large model can also be called a foundation model / foundation model. It is pre-trained by using large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large-scale language models (LLMs) and multimodal pre-training models.

[0030] It should be noted that when the large model is actually used, the pre-trained model can be fine-tuned through a small number of samples, so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, data processing is performed by the text processing model in the large model in the question and answer scenario as an example for explanation.

[0031] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0032] Token: refers to the abstract character of a single word in a large language model. Longer English words may be composed of multiple tokens.

[0033] Prefill: Prefill refers to the stage when the Transformer model first calculates the attention mechanism (Attent ion).

[0034] Decode: Decoding refers to the stage where the Transformer model gradually generates tokens.

[0035] Example 1

[0036] According to an embodiment of the present application, a text processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Considering the huge number of model parameters of large models and the limited computing resources of mobile terminals, the above text processing method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present application. Figure 1 In the illustrated application scenario, the large model is deployed in a cloud 10, which can be connected to one or more client devices 20 via a local area network, a wide area network, the Internet, or other types of data networks. The client devices 20 herein may include, but are not limited to, smartphones, tablet computers, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. The client devices 20 can interact with users via a graphical user interface to access the large model and thereby implement the methods provided in the embodiments of the present application.

[0038] In an embodiment of the present application, the system composed of the client device and the cloud can perform the following steps: the client device executes to obtain the text to be processed, uses the first model parameters of the text processing model stored on the client to pre-fill the first text in the text to be processed, obtains a first pre-filled result of the first text, uses the first model parameters to process the text to be processed based on the first pre-filled result, and obtains a processing result of the text to be processed; the cloud executes to process the text to be processed based on the first pre-filled result using the second model parameters of the text processing model stored on the cloud, and obtains a processing result of the text to be processed. It should be noted that if the operating resources of the client device can meet the deployment and operation conditions of the large model, the embodiment of the present application can be performed on the client device.

[0039] Under the above operating environment, this application provides Figure 2 The text processing method shown. Figure 2 This is a flowchart of a text processing method according to Example 1 of the present application. Figure 2 As shown, the method may include the following steps:

[0040] Step S202: Obtain the text to be processed.

[0041] The above-mentioned text to be processed can be a text obtained after receiving user input in real time, or can be character (token) data. The user input can be voice input or text input, but is not limited thereto.

[0042] In an optional embodiment, when a user inputs into a dialog box of a user terminal, the user terminal can first receive the user input in real time, and then convert the received user input in real time to obtain a text to be processed. For example, when a user inputs "What is XXX" in a dialog box by voice, the client can first receive the "XXX" in real time and convert the "XXX" into a first text to be processed. Then, the client can receive the "XXX" in real time and convert the "XXX" into a second text to be processed. Similarly, the voice input can be received and converted sequentially and in real time, and finally the text to be processed can be obtained as the character data corresponding to "What is XXX".

[0043] Step S204 , using the first model parameter of the text processing model stored on the client, pre-filling the first text in the text to be processed to obtain a first pre-filling result for the first text.

[0044] In the prior art, after receiving the entire text to be processed, the client uploads it to the cloud via the network. The cloud then pre-populates the text before returning it to the client, leading to a delay in the first token. To reduce the delay in the first token, the embodiments of the present application can, after acquiring the text to be processed, directly pre-populate the text in real time using a text processing model stored on the client. Therefore, the aforementioned text processing model may include, but is not limited to, a pre-population module. The client stores all network parameters for the pre-population module, meaning that the first model parameters may include, but are not limited to, all network parameters for the pre-population module. Based on all network parameters for the pre-population module, the client can pre-populate the first text in the text to be processed in real time upon acquiring the text to be processed, without having to upload the text to the cloud. This effectively reduces the delay in the first token. It should be noted that, according to statistics, the rate of voice or keyboard typing input rarely exceeds 4 tokens per second. Therefore, the embodiments of the present application offer the opportunity to utilize streamlined computing to process user input. That is, in this embodiment, the client can pre-fill the text to be processed in real time after dividing it into chunks, one by one. That is to say, the client pre-fills the text to be processed simultaneously with the user's input. The text to be processed can be divided into chunks according to time periods, but is not limited thereto. It can also be divided into chunks according to the length of characters. In this embodiment, taking time periods as an example, the specific time period is not limited in this embodiment and can be 10s, 8s, 12s, etc. The above-mentioned first text can be other texts in the text to be processed except for the last input chunk.

[0045] It's important to note that while client-side computing power isn't comparable to the cloud, due to the limited speed of user input, client-side prefill can still keep pace with input. That is, the client can complete prefill calculations for the current input segment before the user completes the next. Therefore, as user input progresses, prefill calculations for the previous segments are gradually completed on the client, and the resulting KV Cache is continuously sent to the cloud.

[0046] In an optional embodiment, in order to effectively reduce the delay of the first token, when the client obtains the text to be processed, it can firstly segment the text to be processed obtained in real time to obtain multiple input segments, and then directly pre-fill the first text after segmentation in the text to be processed in real time through all the network parameters of the pre-filling module in the first model parameters of the text processing model stored on the client, and obtain the first pre-filling result of the first text and the cache data corresponding to the first text. In which, since the pre-filling of the last segment has not been completed when the user stops inputting, the client can pre-fill part of the text first. It should be noted that in the process of pre-filling the text to be processed, cache data (KV cache) will be generated, wherein KV cache is a key-value pair cache, which stores data in the form of key-value pairs, and can quickly access and obtain the corresponding value through the key. KV cache is usually used to improve the reading speed and performance of data, especially when frequent access and retrieval of data is required, it can effectively reduce the access pressure on the database or other data sources.

[0047] It should be noted that, since the prefill of all fragments except the last one is performed simultaneously with the user input, the delay of returning the first token is greatly reduced.

[0048] Step S206: Using the first model parameters and the second model parameters of the text processing model stored on the cloud, the text to be processed is processed based on the first pre-filled result to obtain a processing result of the text to be processed, wherein the first model parameters are used to characterize part of the model parameters in the text processing model, and the second model parameters at least include other parameters in the text processing model except the first model parameters.

[0049] In order to realize the end-cloud combined processing flow of the text to be processed, the above-mentioned text processing model may also include but is not limited to: a decoding module. The client and the cloud can respectively store some parameters of the decoding module. For example, when the client stores the network parameters of the latter part of the decoding module, the cloud can store the network parameters of the front part of the decoding module. And when the client stores the network parameters of the front part of the decoding module, the cloud can store the network parameters of the latter part of the decoding module. It should be noted that after the network parameters of the latter part and the network parameters of the front part are merged, the complete network parameters of the decoding module are obtained.

[0050] In order to enable the cloud to pre-fill the second text in the text to be processed, the cloud may also store the network parameters of the front part of the pre-filling module for pre-filling the second text.

[0051] Therefore, the first model parameters mentioned above may include, but are not limited to: all network parameters of the pre-filled module stored on the client, the network parameters of the front part of the decoding module, or the network parameters of the back part of the decoding module. The second model parameters mentioned above may include, but are not limited to: the network parameters of the front part of the pre-filled module stored in the cloud, the network parameters of the back part of the decoding module, or the network parameters of the front part of the decoding module.

[0052] In an optional embodiment, when the first model parameters include all network parameters of the pre-filling module stored in the client and the network parameters of the rear part of the decoding module, and the second model parameters include the network parameters of the front part of the pre-filling module stored in the cloud and the network parameters of the front part of the decoding module, after obtaining the first pre-filling result, the client can first send the text to be processed to the cloud, and the cloud can pre-fill the second text in the text to be processed based on the network parameters of the front part of the pre-filling module in the second model parameters to obtain an intermediate result of the second text (for example, it can be a pre-filled hidden state), and return the intermediate result to the client. The client can first process the intermediate result based on other network parameters of the pre-filling module in the first model parameters to obtain the pre-filling result of the second text and the cached data corresponding to the second text, wherein the other network parameters are parameters other than the network parameters of the front part. Then, the first pre-filling result and the pre-filling result of the second text can be summarized to obtain the pre-filling result of the text to be processed, and the cached data corresponding to the first text and the cached data corresponding to the second text can be summarized to obtain the cached data. Finally, based on the pre-filling result of the text to be processed, the first character of the processing result of the text to be processed can be generated, and the first character and the cached data can be sent to the cloud. This allows the cloud to decode the text to be processed and the first character based on the cached data to obtain other characters in the processing result of the text to be processed.

[0053] Optionally, after the cloud receives the text to be processed, the first character and the cached data, it can first decode the text to be processed and the first character based on the network parameters and cached data of the front part of the decoding module in the second model parameters, and obtain the intermediate result of any character except the first character in the processing result of the text to be processed (for example, it can be a decoded hidden state). It should be noted that data will be generated in the process of decoding the pre-filled result, so the cloud can send the intermediate result and the data generated in the decoding process to the client. The client can first update the cached data based on the generated data to obtain new cached data, and secondly, it can decode the intermediate result based on the network parameters and new cached data of the rear part of the decoding module in the first model parameters, that is, the result to be processed can be obtained. Any character except the first character, then any character can be used as the first character, the new cache data can be used as the cache data, and the network parameters and cache data of the front part of the decoding module in the first model parameters are repeatedly used to decode the text to be processed and the first character to obtain the decoded hidden state of any character, based on the data generated in the process of decoding the pre-filled result of the text to be processed, the cache data is updated to obtain the updated cache data, and the network parameters of the rear part of the decoding module in the first model parameters are used to decode the decoded hidden state based on the updated cache data to obtain the steps of any character, until the number of generated characters reaches the preset number, and finally the preset number of characters can be combined, that is, the processing result of the text to be processed can be obtained.

[0054] In another optional embodiment, where the first model parameters include all network parameters of the pre-fill module stored on the client and the network parameters of the front portion of the decoding module, and the second model parameters include the network parameters of the front portion of the pre-fill module stored in the cloud and the network parameters of the back portion of the decoding module, the steps for obtaining the first character are the same as those described above and are not further described here. After obtaining the first character, the cached data, and the text to be processed, the client may send the first character, the cached data, and the text to be processed to the cloud. During the decoding process, the client can first decode the text to be processed and the first character based on the cached data and the network parameters of the front part of the decoding module in the first model parameters, and obtain the intermediate result of any character except the first character in the processing result of the text to be processed (for example, it can be a decoded hidden state). It should be noted that data will be generated in the process of decoding the pre-filled result, so the client can also send the intermediate result and the data generated in the decoding process to the cloud. After receiving the first character, cached data, text to be processed, intermediate result and generated data, the cloud will first update the cached data based on the generated data to obtain new cached data. Secondly, it can decode the intermediate result based on the new cached data and the network parameters of the rear part of the decoding module in the second model parameters, that is, it can Obtain any character except the first character in the result to be processed, then any character can be used as the first character, and the new cache data can be used as the cache data, and repeatedly use the network parameters and cache data of the front part of the decoding module in the first model parameters to decode the text to be processed and the first character to obtain the decoded hidden state of any character, based on the data generated in the process of decoding the pre-filled result of the text to be processed, update the cache data to obtain the updated cache data, and use the network parameters of the rear part of the decoding module in the second model parameters to decode the decoded hidden state based on the updated cache data to obtain the step of any character, until the number of generated characters reaches the preset number, and finally the preset number of characters can be combined, that is, the processing result of the text to be processed can be obtained.

[0055] In an embodiment of the present application, a method is adopted in which a text to be processed is obtained; a first model parameter of a text processing model stored on a client is used to pre-fill a first text in the text to be processed, and a first pre-fill result of the first text is obtained; and the text to be processed is processed based on the first pre-fill result using the first model parameter and a second model parameter of a text processing model stored on a cloud, and a processing result of the text to be processed is obtained, wherein the first model parameter is used to characterize part of the model parameters in the text processing model, and the second model parameter at least includes other parameters in the text processing model except the first model parameter. It is easy to notice that the first text in the text to be processed is directly processed by the first model parameter of the text processing model on the client, without uploading the first text to the cloud and then having the cloud process the first text, and the model parameters of the text processing model on the client are not compressed, which not only saves the processing time of the text processing model, but also ensures the processing accuracy of the text processing model, thereby solving the problem of the first token delay. In addition, the text processing models on the client and the cloud can jointly process the second text in the text to be processed. Through the combination of end and cloud, the computing power of the client is used to process the first text in the text to be processed. This can not only make full use of the computing resources of the client and reduce the workload on the cloud, but also reduce the operating cost of the model, thereby improving the overall throughput of the text processing model and realizing cloud cost optimization, achieving the purpose of improving user experience, making full use of client resources, and reducing the operating cost of large models, thereby achieving the technical effect of not generating the first token delay when responding to user input, and thus solving the technical problem in the related technology that the first token delay will be generated when responding to user input, resulting in low user experience, waste of client resources and high model operating cost.

[0056] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the client stores at least the network parameters of the pre-filling module and the network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers located at the back in the decoding module; the cloud stores at least the network parameters of the target pre-filling layer in the pre-filling module and the network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers located at the front in the pre-filling module, and the second decoding layer is used to represent the decoding layers in the decoding module other than the first decoding layer.

[0057] The above-mentioned first decoding layer can be located on the client side, and the network parameters of the first decoding layer are the network parameters of the latter part of the above-mentioned decoding module. The above-mentioned first preset number can be set in advance by the user and can be expressed as (K+1) to N, but is not limited to this. The above-mentioned target pre-filled layer can be a filling layer in the pre-filled module located in the cloud, which is used to pre-fill the second text. The above-mentioned second preset number can be set in advance by the user and can be expressed as 1 to K, but is not limited to this. The above-mentioned second decoding layer can be located in the cloud.

[0058] In an optional embodiment, the text processing model includes a pre-filling module and a decoding module. The network parameters of the pre-filling module are stored on the client, and the network parameters of the target pre-filling layer in the pre-filling module are stored in the cloud. The target pre-filling layer represents the second predetermined number of pre-filling layers located in front of the pre-filling module.

[0059] In another optional embodiment, the network parameters of the first decoding layer in the decoding module are stored in the client, and the network parameters of the second decoding layer in the decoding module are stored in the cloud, wherein the first decoding layer is the first preset number of decoding layers located at the back of the decoding module, and the second decoding layer is the decoding layer in the decoding module other than the first decoding layer.

[0060] In the above embodiment of the present application, the first model parameters of the text processing model stored on the client are used to pre-fill the first text in the text to be processed to obtain a first pre-fill result of the first text, including: reading the network parameters of the pre-fill module stored in the client layer by layer; and using the read network parameters to pre-fill the first text to obtain the first pre-fill result.

[0061] In an optional embodiment, before pre-filling the first text in the text to be processed by the first model parameters of the text processing model to obtain the first pre-filling result of the first text, first, based on the hierarchical structure of the pre-filling module, the first layer network parameters of the pre-filling module can be read, and the first text can be pre-filled by the read first layer network parameters to obtain the first sub-pre-filling result. Secondly, the second layer network parameters of the pre-filling module can be read, and the first sub-pre-filling result can be processed by the read second layer network parameters to obtain the second sub-pre-filling result. And so on, until the last layer network parameters of the pre-filling module are read, and the first pre-filling result of the previous layer is processed by the last layer network parameters, that is, the first pre-filling result can be obtained. In this step, by reading the network parameters of the pre-filling module layer by layer to process the text to be processed, the amount of data read by the text processing model at one time can be reduced, and further, the processing efficiency and accuracy of the text processing model can be improved.

[0062] In the above embodiment of the present application, the first model parameters and the second model parameters of the text processing model stored on the cloud are used to process the text to be processed based on the first pre-filled result to obtain the processing result of the text to be processed, including: using the network parameters of other pre-filled layers stored in the client and the network parameters of the target pre-filled layer to pre-fill the second text in the text to be processed to obtain a second pre-filled result of the second text, wherein the other pre-filled layers are used to represent the pre-filled layers other than the target pre-filled layer in the pre-filled module; generating the first character in the processing result through the client based on the first pre-filled result and the second pre-filled result, wherein the second text is used to represent the text other than the first text in the text to be processed; using the network parameters of the first decoding layer and the network parameters of the second decoding layer, decoding the text to be processed and the first character based on the cached data to obtain other characters in the processing result, wherein the cached data is used to represent the data generated by pre-filling the text to be processed, and the other characters are used to represent at least one character located after the first character in the processing result.

[0063] In an optional embodiment, the second text can first be pre-filled using the network parameters of other pre-filled layers of the client and the network parameters of the target filling layer to obtain a second pre-filled result of the second text and cached data, wherein the second text is the text other than the first text in the text to be processed. For example, first the client can send the second text and cached data to the cloud, and the cloud can obtain an intermediate result (for example, a pre-filled hidden state) after pre-filling the second text using the network parameters of the target filling layer. Secondly, the cloud can send the pre-filled Hidden States to the client, and the client can pre-fill the pre-filled Hidden States based on the network parameters of other pre-filled layers, that is, it can obtain the second pre-filled result of the second text and the cached data corresponding to the second text. At the same time, the client can also aggregate the cached data corresponding to the second text and the cached data corresponding to the first text to obtain the cached data. Among them, other pre-filled layers are pre-filled layers other than the target pre-filled layer in the pre-filled module. Then, the client can generate the first character in the processing result based on the first pre-filled result and the second pre-filled result. Finally, the network parameters of the first decoding layer and the network parameters of the second decoding layer can be used to decode the text to be processed and the first character based on the cached data to obtain other characters in the processing result. Among them, the cached data is the data generated after the text to be processed is pre-filled with data, and the other characters are at least one character located after the first character in the processing result. For example, the cloud can first decode the text to be processed and the first character through the network parameters and cached data of the second decoding layer to obtain decoded Hidden States and generate data. Then, the cloud can send the decoded HiddenStates and the generated data to the client. The client can first update the cached data based on the generated data to obtain new cached data, and then can decode the decoded HiddenStates through the network parameters of the first decoding layer and the new cache parameters to obtain other characters in the processing result. In this step, the second text in the text to be processed is pre-filled through end-cloud collaboration. Since the computing power of the client is utilized, the workload on the cloud is reduced, thereby improving the overall throughput of the text processing model and achieving cloud cost optimization.

[0064] In the above embodiment of the present application, the network parameters of other pre-filled layers stored in the client and the network parameters of the target pre-filled layer are used to pre-fill the second text in the text to be processed to obtain a second pre-filled result of the second text, including: using the network parameters of the target pre-filled layer to pre-fill the second text to obtain a pre-filled hidden state of the second text; using the network parameters of other pre-filled layers to pre-fill the pre-filled hidden state to obtain a second pre-filled result of the second text.

[0065] In an optional embodiment, the client can first send the text to be processed to the cloud, and the cloud can pre-fill the second text in the text to be processed through the network parameters of the target pre-fill layer to obtain the pre-filled hidden state of the second text. Secondly, the cloud can send the pre-filled hidden state of the second text to the client, and the client can obtain the second pre-filled result and cached data of the second text after pre-filling the pre-filled hidden state through the network parameters of other pre-filled layers.

[0066] In the above embodiment of the present application, the network parameters of the first decoding layer and the network parameters of the second decoding layer are used to decode the text to be processed and the first character based on the cached data to obtain other characters in the processing result, including: using the network parameters of the first decoding layer and the network parameters of the second decoding layer, decoding the text to be processed and the first character based on the cached data, and sequentially generating at least one character in the processing result; combining the first character and at least one character to generate the processing result.

[0067] In an optional embodiment, the network parameters and cache data of the second decoding layer can be used to first decode the text to be processed and the first character to obtain the decoded hidden state of any one of the at least one character, and generate data. Secondly, the decoded hidden state of any one of the characters and the generated data can be sent to the client. The client can first update the cache data based on the generated data to obtain new cache data. Secondly, the decoded hidden state of any one of the characters can be decoded based on the network parameters of the first decoding layer and the new cache data to obtain any one of the characters. Then, the new cache data can be used as cache data, and any one of the characters can be used as the first character. The network parameters and cache data of the second decoding layer are used to repeatedly decode the text to be processed and the first character to obtain the decoded hidden state of any one of the at least one character, and generate data. Secondly, the decoded hidden state of any one of the characters and the generated data can be sent to the client. The client can first update the cache data based on the generated data to obtain new cache data. Secondly, the decoded hidden state of any one of the characters can be decoded based on the network parameters of the first decoding layer and the new cache data to obtain any one of the characters, until a complete processing result of the text to be processed is generated. Finally, the first character and the at least one character can be combined to obtain the processing result of the text to be processed.

[0068] In the above embodiment of the present application, the network parameters of the first decoding layer and the network parameters of the second decoding layer are used to decode the processing text and the first character based on the cached data, and at least one character in the processing result is sequentially generated, including: using the network parameters of the second decoding layer to decode the processing text and the first character based on the cached data to obtain a decoded hidden state of any one of the at least one character; updating the cached data based on data generated in the process of decoding the pre-filled result of the processing text to obtain updated cached data; decoding the decoded hidden state based on the updated cached data using the network parameters of the first decoding layer to obtain any one character; taking the any one character as the first character and the updated cached data as the cached data, and repeatedly performing the steps of decoding the processing text and the first character based on the cached data using the network parameters of the second decoding layer to obtain a decoded hidden state of any one character, updating the cached data based on data generated in the process of decoding the pre-filled result of the processing text to obtain updated cached data, and decoding the decoded hidden state based on the updated cached data using the network parameters of the first decoding layer to obtain any one character, until the number of generated characters reaches a preset number; and determining that the generated character is at least one character.

[0069] The above-mentioned preset number is determined by the text to be processed. Since the processing result is a reply to the text to be processed, the number of characters in the processing result is associated with the text to be processed. Based on the specific content of the text to be processed, the preset number of characters can be obtained. Since the same processed text may have different processing results, the value of the preset number will also be different. Therefore, the specific value of the preset number is not limited in this embodiment.

[0070] In an optional embodiment, the network parameters and cached data of the second decoding layer of the cloud can be used to first decode the processed text and the first character to obtain the decoded hidden state of any one of the at least one character, and the data generated by the decoding can also be obtained. Secondly, the cloud can send the decoded hidden state and the data generated by the decoding to the client. The client can first update the cached data through the data generated by the decoding to obtain new cached data. Secondly, the client can decode the decoded hidden state based on the network parameters of the first decoding layer and the updated cached data to obtain any character. Then, the arbitrary character can be used as the first character, and the updated cached data can be used as the cached data. The steps of decoding the processed text and the first character based on the cached data using the network parameters of the second decoding layer to obtain the decoded hidden state of any character, updating the cached data based on the data generated in the process of decoding the pre-filled result of the processed text to obtain the updated cached data, and decoding the decoded hidden state based on the updated cached data using the network parameters of the first decoding layer to obtain any character are repeated until the number of generated characters reaches a preset number. Finally, it can be determined that the generated character is at least one character.

[0071] Figure 3 is a flowchart of an optional text processing according to Example 1 of the present application, such as Figure 3 As shown, the user input can be "What do you think of the future of AI?", wherein the client (Device) user can input by voice (or keyboard typing), the first input segment can be "What do you", the second input segment can be "think of the", and the third input segment can be "future of AI". When the user inputs, the network parameters of the client's pre-fill module can pre-fill the first input segment and the second input segment in real time. Figure 3 As shown, when the pre-filling module has N layers, the first input segment and the second input segment can be pre-filled in real time through the 1-K layers and the (K+1)-N layers, as shown in FIG. Figure 3In the above example, 1 ScanWeights Layer by Layer and 2 Prefill Pref ll can generate cache data (KV Cache). Since the Pref ll of the third input segment is not completed when the user stops inputting, the generated cache data can be sent to the cloud (Server) layer by layer through the network (Network), as shown in the following example: Figure 3 Upload KVCache Layer by Layer and send the third input segment to the cloud (Server). Figure 3 The 8 upload characters in Upload Tokens.

[0072] Until the user input is finished, because the Prefill of the last input segment has not been completed ( Figure 3 6+7), these tokens will be sent to the cloud ( Figure 3 8), the cloud and client collaborate to complete the remaining Prefill work ( Figure 3 9-10). Assuming the model has N layers, after the cloud receives the cached data and the third input segment, it is responsible for processing layers 1 to K of the text processing model ( Figure 3 9 prefills the server in the cloud and sends the intermediate result (i.e. prefilled hidden state) to the client ( Figure 3 In 10, the pre-filled hidden state is sent. The client is responsible for continuing to process the (K+1) to N layers ( Figure 3 In step 11, the client prefills the Device Prefill. After completing the prefill of the third input segment, the client can display the first token and send it to the cloud ( Figure 3 12 upload characters (Uploadtokens). Because the cloud only participates in prefiling the last segment, rather than prefiling the entire input, the time the cloud is occupied by prefiling is effectively reduced, saving cloud resources. In addition, because prefiling of all segments except the last segment is performed simultaneously with user input, the delay in returning the first token is effectively reduced.

[0073] After the pre-filling process is completed, the decoding phase can be carried out. The cloud first uses speculative decoding to generate draft tokens (that is, the data generated in the above decoding process) and completes the verification of the first 1-K layers of the model ( Figure 313, 16, and 18 perform speculative decoding on the cloud (Server Speculative Decoding) and send the intermediate results (i.e., decoded hidden states) to the client or other client (Other Device) ( Figure 3 14, 19 send decoded hidden states Send Hidden States), the client continues to verify the (K+1) to N layers ( Figure 3 In 15 and 20, the client performs speculative decoding and returns the result to the cloud ( Figure 3 17 upload characters Upload tokens) are used to continue decoding, and the final processing result can be obtained as "the future of AI holds great potent ial". Since the cloud is only responsible for processing the 1-K layers instead of all N layers during each decoding process, under the same circumstances, the cloud's graphics processing unit (GPU) memory occupancy and computing resource consumption are significantly reduced, and the freed-up memory and computing resources can be used to process more requests, thereby improving throughput. Experiments show that the embodiments of the present application can increase the cloud service volume by up to 50%, which means a significant reduction in service provider costs. In the case of voice output, the output rate of the client of the embodiment of the present application can still reach 4 tokens per second, ensuring that the user experience will not be reduced.

[0074] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0075] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0077] Example 2

[0078] According to an embodiment of the present application, a text processing method is also provided, which is applied to a client. Figure 4 This is a flowchart of a text processing method according to Example 2 of this application. Figure 4 As shown, the method may include the following steps:

[0079] Step S402, in response to receiving input text to be processed;

[0080] Step S404: Pre-filling a first text in the text to be processed using a first model parameter of the text processing model stored on the client to obtain a first pre-filled result of the first text, wherein the first model parameter is used to represent a portion of model parameters in the text processing model;

[0081] Step S406: Send the text to be processed to the cloud, and receive an intermediate result sent by the cloud, wherein the intermediate result is a result obtained by processing the text to be processed using the second model parameters of the text processing model stored on the cloud, and the second model parameters at least include other parameters of the text processing model except the first model parameters;

[0082] Step S408 : Processing the intermediate result based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

[0083] In an optional embodiment, when a user inputs in a dialog box of a client, the client can first receive the input text to be processed, and secondly, can pre-fill the first text in the text to be processed using the first model parameter of the text processing model stored on the client to obtain a first pre-filled result of the first text. The client can then send the text to be processed to the cloud. After the cloud receives the text to be processed, it can process the text to be processed to obtain an intermediate result. The cloud can then return the intermediate result to the client. After the client receives the intermediate result, it can process the text to be processed based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed. Wherein, the cloud stores the second model parameter of the text processing model, the first model parameter is used to characterize some model parameters in the text processing model, and the second model parameter at least includes other parameters in the text processing model except the first model parameter.

[0084] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers with a connection relationship at the end of the decoding module; the second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a connection relationship at the front of the pre-filling module, and the second decoding layer is used to represent the pre-filling layers in the decoding module other than the first decoding layer.

[0085] In an optional embodiment, the text processing model includes a pre-filling module and a decoding module. The text processing model also includes: first model parameters and second model parameters. The first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module. The first model parameters are stored on the client, and the first decoding layer is the first preset number of decoding layers in the decoding module that are connected later.

[0086] In another optional embodiment, the second model parameters include network parameters of a target pre-population layer in the pre-population module and network parameters of a second decoding layer in the decoding module, and the second model parameters are stored in the cloud. The target pre-population layer is a second predetermined number of pre-population layers that are connected closer to the front in the pre-population module. The second decoding layer is a decoding layer in the decoding module other than the first decoding layer.

[0087] In the above embodiment of the present application, the first model parameters of the text processing model stored on the client are used to pre-fill the first text in the text to be processed to obtain a first pre-fill result of the first text, including: reading the network parameters of the pre-fill module stored in the client layer by layer; and using the read network parameters to pre-fill the first text to obtain the first pre-fill result.

[0088] In an optional embodiment, before pre-filling the first text in the text to be processed through the first model parameters of the text processing model to obtain the first pre-filling result of the first text, the first layer network parameters of the pre-filling module can be read based on the hierarchical structure of the pre-filling module, and the first text can be pre-filled through the read first layer network parameters to obtain a first sub-pre-filling result. Secondly, the second layer network parameters of the pre-filling module can be read, and the first sub-pre-filling result can be processed through the read second layer network parameters to obtain a second sub-pre-filling result. And so on, until the last layer network parameters of the pre-filling module are read, and the first pre-filling result of the previous layer is processed through the last layer network parameters, so that the first pre-filling result can be obtained.

[0089] In the above embodiment of the present application, the first model parameter is used to process the text to be processed based on the first pre-filled result to obtain the processing result of the text to be processed, including: sending cached data to the cloud, and receiving the pre-filled intermediate result sent by the cloud, wherein the cached data is used to represent the data generated by pre-filling the text to be processed, and the pre-filled intermediate result is the result obtained by the cloud using the network parameters of the target pre-filled layer to pre-fill the second text in the text to be processed, and the second text is used to represent the text other than the first text in the text to be processed; using the network parameters of other pre-filled layers stored in the client, the pre-filled intermediate result is pre-filled to obtain a second pre-filled result of the second text, wherein the other pre-filled layers are used to represent the pre-filled layers other than the target pre-filled layers in the pre-filled module. layer; based on the first pre-filled result and the second pre-filled result, generate the first character in the processing result; send the first character to the cloud, and receive the decoding intermediate results and updated cache data of other characters sent by the cloud, wherein the decoding intermediate result is the result obtained by the cloud side decoding the first character and the text to be processed based on the cached data using the network parameters of the second decoding layer, the other characters are at least one character located after the first character in the processing result, and the updated cached data is obtained by updating the cached data based on the data generated in the process of decoding the first character and the text to be processed; use the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cached data to obtain other characters; combine the first character and the other characters to generate the processing result.

[0090] In an optional embodiment, the client can first pre-fill the second text using the network parameters of other pre-fill layers stored in the client to obtain a second pre-fill result of the second text and cached data, wherein the cached data is the data generated after the text to be processed is pre-filled, and the second text is the text other than the first text in the text to be processed. Secondly, the client can send the cached data to the cloud, and the cloud can obtain a pre-filled intermediate result (for example, it can be a pre-filled hidden state) after pre-filling the second text using the network parameters of the target filling layer. Next, the cloud can send the pre-filled Hidden States to the client, and the client can pre-fill the pre-filled Hidden States based on the network parameters of other pre-fill layers, that is, it can obtain the second pre-fill result of the second text and the cached data corresponding to the second text. At the same time, the client can also summarize the cached data corresponding to the second text and the cached data corresponding to the first text to obtain the cached data. The "other pre-filled layers" are pre-filled layers in the pre-filled module other than the target pre-filled layer. The client can then generate the first character in the processing result based on the first and second pre-filled results and send the first character to the cloud. The cloud can then decode the processed text and the first character based on cached data using the network parameters of the first and second decoding layers, obtaining the other characters in the processing result and the cached data generated by the decoding. The "other characters" are at least one character in the processing result that follows the first character. For example, the cloud can first decode the processed text and the first character using the network parameters and cached data of the second decoding layer to obtain decoded Hidden States and the cached data generated by the decoding. The cloud can then send the decoded Hidden States and the cached data generated by the decoding to the client. The client can first update the cached data based on the decoded cached data to obtain updated cached data. The client can then decode the decoded Hidden States using the network parameters of the first decoding layer and the updated cached parameters to obtain the other characters in the processing result. Finally, the client can combine the first character with the other characters to obtain the processing result.

[0091] Example 3

[0092] According to an embodiment of the present application, a text processing method is also provided, which is applied in the cloud. Figure 5 This is a flowchart of a text processing method according to Example 3 of this application. Figure 5 As shown, the method may include the following steps:

[0093] Step S502: receiving a text to be processed sent by a client, wherein the text to be processed is text received by the client as external input, and the client stores first model parameters of a text processing model, the first model parameters being used to represent a portion of model parameters in the text processing model, and the first model parameters being used to perform pre-filling processing on a first text in the text to be processed, thereby obtaining a first pre-filling result for the first text;

[0094] Step S504: Process the text to be processed using the second model parameters of the text processing model stored on the cloud to obtain an intermediate result of the text to be processed, wherein the second model parameters at least include other parameters of the text processing model except the first model parameters;

[0095] Step S506 : sending the intermediate result to the client, wherein the client processes the intermediate result based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

[0096] In an optional embodiment, when the user inputs in the dialog box of the client, the client will receive the text to be processed input by the user. Then, the client can send the text to be processed to the cloud. The cloud can receive the text to be processed sent by the client and process the text to be processed through the second model parameter of the text processing model stored on the cloud to obtain an intermediate result of the text to be processed. Then, the cloud can return the intermediate result to the client. After the client receives the intermediate result, it can process the intermediate result based on the first model parameter stored on the client and the first pre-filled result to obtain the processing result of the text to be processed. Wherein, the text to be processed is the text of external input received by the client, and the first model parameter of the text processing model is stored on the client. The first model parameter is used to represent some model parameters in the text processing model. The first model parameter is used to pre-fill the first text in the text to be processed to obtain the first pre-filled result of the first text. The second model parameter at least includes other parameters in the text processing model except the first model parameter.

[0097] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers with a connection relationship at the end of the decoding module; the second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a connection relationship at the front of the pre-filling module, and the second decoding layer is used to represent the decoding layers in the decoding module other than the first decoding layer.

[0098] In an optional embodiment, the text processing model includes a pre-filling module and a decoding module. The text processing model also includes: first model parameters and second model parameters. The first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module. The first model parameters are stored on the client, and the first decoding layer is the first preset number of decoding layers in the decoding module that are connected later.

[0099] In another optional embodiment, the second model parameters include network parameters of a target pre-population layer in the pre-population module and network parameters of a second decoding layer in the decoding module, and the second model parameters are stored in the cloud. The target pre-population layer is a second predetermined number of pre-population layers that are connected closer to the front in the pre-population module. The second decoding layer is a decoding layer in the decoding module other than the first decoding layer.

[0100] In the above embodiment of the present application, the second model parameters of the text processing model stored on the cloud are used to process the text to be processed to obtain an intermediate result of the text to be processed, including: receiving cached data sent by the client, wherein the cached data is used to represent the data generated by pre-filling the text to be processed; based on the network parameters of the target pre-filling layer, pre-filling the second text in the text to be processed to obtain a pre-filled intermediate result, wherein the second text is used to represent the text other than the first text in the text to be processed; receiving the first character in the processing result sent by the client, wherein the first character is generated by the client based on the first pre-filling result and the second pre-filling result of the second text, and the second pre-filling result is the result obtained by pre-filling the pre-filled intermediate result using the network parameters of other pre-filling layers stored in the client, wherein the other pre-filling layers are used to represent the pre-filling layers other than the target pre-filling layer in the pre-filling module; using the network parameters of the second decoding layer, based on the cached data, decode the text to be processed and the first character to obtain a decoding intermediate result of other characters in the processing result, wherein the other characters are used to represent at least one character located after the first character in the processing result.

[0101] In an optional embodiment, the second text can first be pre-filled using the network parameters of other pre-filled layers of the client and the network parameters of the target filling layer to obtain a second pre-filled result of the second text and cached data, wherein the second text is the text to be processed other than the first text. For example, first the client can send the cached data to the cloud, the cloud can first accept the cached data, and then the second text can be pre-filled using the network parameters and cached data of the target filling layer to obtain a pre-filled intermediate result (for example, a pre-filled hidden state). Then the cloud can send the pre-filled Hidden States to the client, and the client can pre-fill the pre-filled Hidden States based on the network parameters of other pre-filled layers, that is, the second pre-filled result of the second text and the cached data corresponding to the second text can be obtained. At the same time, the client can also aggregate the cached data corresponding to the second text and the cached data corresponding to the first text to obtain the cached data. Among them, other pre-filled layers are pre-filled layers other than the target pre-filled layer in the pre-filled module. Then, the client can generate the first character in the processing result based on the first pre-filled result and the second pre-filled result, and send the first character to the cloud. Finally, the cloud can decode the text to be processed and the first character based on the cached data through the network parameters of the second decoding layer to obtain other characters in the processing result. Among them, the cached data is the data generated after the text to be processed is pre-filled with data, and the other characters are at least one character located after the first character in the processing result. For example, the cloud can first decode the text to be processed and the first character through the network parameters and cached data of the second decoding layer to obtain decoded HiddenStates and generate data. Then, the cloud can send the decoded Hidden States and the generated data to the client. The client can first update the cached data based on the generated data to obtain new cached data, and then decode the decoded Hidden States through the network parameters of the first decoding layer and the new cache parameters to obtain other characters in the processing result.

[0102] In the above embodiment of the present application, the method also includes: updating the cache data based on the data generated in the process of decoding the first character and the text to be processed to obtain updated cache data; sending the intermediate result and the updated cache data to the client, wherein the processing result is that the client uses the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cache data to obtain other characters, and the processing result is obtained by the client combining the first character and other characters, and the other characters are obtained by the client using the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cache data.

[0103] In an optional embodiment, data is generated during the cloud decoding of the first character and the text to be processed. At this time, the cloud can update the cached data based on the generated data to obtain updated cached data. Secondly, the cloud can send the intermediate result and the updated cached data to the client. The client can decode the decoding intermediate result based on the updated cached data through the network parameters of the first decoding layer to obtain other characters. Finally, the client can combine the first character and the other characters to obtain the processing result.

[0104] Example 4

[0105] According to an embodiment of the present application, a text processing device for implementing the above-mentioned text processing method is also provided. Figure 6 is a schematic diagram of a text processing device according to Example 4 of the present application, such as Figure 6 As shown, the device includes: an acquisition module 62 , a first processing module 64 and a second processing module 66 .

[0106] Among them, the acquisition module is used to obtain the text to be processed; the first processing module is used to use the first model parameters of the text processing model stored on the client to pre-fill the first text in the text to be processed, and obtain a first pre-filled result of the first text; the second processing module is used to use the first model parameters and the second model parameters of the text processing model stored on the cloud to process the text to be processed based on the first pre-filled result, and obtain the processing result of the text to be processed, wherein the first model parameters are used to characterize some model parameters in the text processing model, and the second model parameters at least include other parameters in the text processing model except the first model parameters.

[0107] It should be noted that the acquisition module 62, the first processing module 64, and the second processing module 66 correspond to steps S202 to S206 in Example 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be run as part of the device in the server 10 provided in Example 1.

[0108] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the client stores at least the network parameters of the pre-filling module and the network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers located at the back in the decoding module; the cloud stores at least the network parameters of the target pre-filling layer in the pre-filling module and the network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers located at the front in the pre-filling module, and the second decoding layer is used to represent the decoding layers in the decoding module other than the first decoding layer.

[0109] In the above embodiment of the present application, the first processing module includes: a reading unit and a first processing unit.

[0110] The reading unit is used to read the network parameters of the pre-filling module stored in the client layer by layer; the first processing unit is used to perform pre-filling processing on the first text using the read network parameters to obtain a first pre-filling result.

[0111] In the above embodiment of the present application, the second processing module includes: a second processing unit, a generating unit and a decoding unit.

[0112] Among them, the second processing unit is used to use the network parameters of other pre-filled layers stored in the client and the network parameters of the target pre-filled layer to pre-fill the second text in the text to be processed, and obtain a second pre-filled result of the second text, wherein the other pre-filled layers are used to represent the pre-filled layers other than the target pre-filled layer in the pre-filled module; the generation unit is used to generate the first character in the processing result based on the first pre-filled result and the second pre-filled result through the client, wherein the second text is used to represent the text other than the first text in the text to be processed; the decoding unit is used to use the network parameters of the first decoding layer and the network parameters of the second decoding layer to decode the text to be processed and the first character based on the cached data to obtain other characters in the processing result, wherein the cached data is used to represent the data generated by pre-filling the text to be processed, and the other characters are used to represent at least one character located after the first character in the processing result.

[0113] In the above embodiment of the present application, the second processing unit includes: a first processing sub-unit and a second processing sub-unit.

[0114] Among them, the first processing sub-unit is used to use the network parameters of the target pre-filling layer to pre-fill the second text to obtain the pre-filled hidden state of the second text; the second processing sub-unit is used to use the network parameters of other pre-filling layers to pre-fill the pre-filled hidden state to obtain the second pre-filling result of the second text.

[0115] In the above embodiments of the present application, the decoding unit includes: a decoding subunit and a combining subunit.

[0116] Among them, the decoding subunit is used to use the network parameters of the first decoding layer and the network parameters of the second decoding layer to decode the text to be processed and the first character based on the cached data, and generate at least one character in the processing result in sequence; the combining subunit is used to combine the first character and at least one character to generate the processing result.

[0117] In the above embodiment of the present application, the decoding sub-unit is also used to: use the network parameters of the second decoding layer to decode the processed text and the first character based on the cache data to obtain the decoded hidden state of any one character in at least one character; update the cache data based on the data generated in the process of decoding the pre-filled result of the processed text to obtain the updated cache data; use the network parameters of the first decoding layer to decode the decoded hidden state based on the updated cache data to obtain any character; use any character as the first character, use the updated cache data as the cache data, and repeatedly perform the steps of using the network parameters of the second decoding layer to decode the processed text and the first character based on the cache data to obtain the decoded hidden state of any character, update the cache data based on the data generated in the process of decoding the pre-filled result of the processed text to obtain the updated cache data, and use the network parameters of the first decoding layer to decode the decoded hidden state based on the updated cache data to obtain any character, until the number of generated characters reaches a preset number; determine that the generated character is at least one character.

[0118] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0119] Example 5

[0120] According to an embodiment of the present application, a text processing device for implementing the above-mentioned text processing method is also provided, which is applied to a client. Figure 7 is a schematic diagram of a text processing device according to Example 5 of the present application, such as Figure 7 As shown, the device includes: a receiving module 72 , a first processing module 74 , a sending module 76 and a second processing module 78 .

[0121] Among them, the receiving module is used to respond to the received input text to be processed; the first processing module is used to use the first model parameters of the text processing model stored on the client to pre-fill the first text in the text to be processed to obtain a first pre-filled result of the first text, wherein the first model parameters are used to characterize part of the model parameters in the text processing model; the sending module is used to send the text to be processed to the cloud and receive the intermediate result sent by the cloud, wherein the intermediate result is the result obtained by processing the text to be processed using the second model parameters of the text processing model stored on the cloud, and the second model parameters at least include other parameters in the text processing model except the first model parameters; the second processing module is used to use the first model parameters to process the intermediate result based on the first pre-filled result to obtain the processing result of the text to be processed.

[0122] It should be noted that the receiving module 72, the first processing module 74, the sending module 76, and the second processing module 78 correspond to steps S402 to S408 in Example 2. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 2. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be part of the device and can be run in the server 10 provided in Example 1.

[0123] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers with a connection relationship at the end of the decoding module; the second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a connection relationship at the front of the pre-filling module, and the second decoding layer is used to represent the pre-filling layers in the decoding module other than the first decoding layer.

[0124] In the above embodiment of the present application, the first processing module includes: a reading unit and a first processing unit.

[0125] The reading unit is used to read the network parameters of the pre-filling module stored in the client layer by layer; the first processing unit is used to perform pre-filling processing on the first text using the read network parameters to obtain a first pre-filling result.

[0126] In the above embodiment of the present application, the second processing module includes: a first sending unit, a second processing unit, a generating unit, a second sending unit, a decoding unit and a combining unit.

[0127] Among them, the first sending unit is used to send cached data to the cloud and receive the pre-filled intermediate result sent by the cloud, wherein the cached data is used to represent the data generated by pre-filling data for the text to be processed, and the pre-filled intermediate result is the result obtained by the cloud using the network parameters of the target pre-filled layer to pre-fill the second text in the text to be processed, and the second text is used to represent the text other than the first text in the text to be processed; the second processing unit is used to use the network parameters of other pre-filled layers stored in the client to pre-fill the pre-filled intermediate result to obtain a second pre-filled result of the second text, wherein the other pre-filled layers are used to represent the pre-filled layers in the pre-filled module other than the target pre-filled layer; the generating unit is used to generate a processing result based on the first pre-filled result and the second pre-filled result. The first character in the result; the second sending unit is used to send the first character to the cloud, and receive the decoding intermediate results and updated cache data of other characters sent by the cloud, wherein the decoding intermediate result is the result obtained by the cloud side decoding the first character and the text to be processed based on the cache data using the network parameters of the second decoding layer, the other characters are at least one character located after the first character in the processing result, and the updated cache data is obtained by updating the cache data based on the data generated in the process of decoding the first character and the text to be processed; the decoding unit is used to decode the decoding intermediate result based on the updated cache data using the network parameters of the first decoding layer to obtain other characters; the combining unit is used to combine the first character and the other characters to generate a processing result.

[0128] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0129] Example 6

[0130] According to an embodiment of the present application, a text processing device for implementing the above-mentioned text processing method is also provided, which is applied in the cloud. Figure 8 is a schematic diagram of a text processing device according to Example 6 of the present application, such as Figure 8 As shown, the device includes: a receiving module 82 , a processing module 84 and a sending module 86 .

[0131] Among them, the receiving module is used to receive the text to be processed sent by the client, wherein the text to be processed is the text of external input received by the client, and the first model parameter of the text processing model is stored on the client, and the first model parameter is used to characterize some model parameters in the text processing model. The first model parameter is used to pre-fill the first text in the text to be processed to obtain a first pre-filled result of the first text; the processing module is used to use the second model parameter of the text processing model stored on the cloud to process the text to be processed to obtain an intermediate result of the text to be processed, wherein the second model parameter at least includes other parameters in the text processing model except the first model parameter; the sending module is used to send the intermediate result to the client, wherein the intermediate result is processed by the client based on the first pre-filled result using the first model parameter to obtain the processing result of the text to be processed.

[0132] It should be noted that the receiving module 82, processing module 84, and sending module 86 correspond to steps S502 to S506 in Example 3. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 3. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be part of the device and can run in the server 10 provided in Example 1.

[0133] In the above embodiment of the present application, the text processing model includes a pre-filling module and a decoding module; the first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent the first preset number of decoding layers with a connection relationship at the end of the decoding module; the second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a connection relationship at the front of the pre-filling module, and the second decoding layer is used to represent the pre-filling layers in the decoding module other than the first decoding layer.

[0134] In the above embodiments of the present application, the processing module includes: a first receiving unit, a processing unit, a second receiving unit and a decoding unit.

[0135] Among them, the first receiving unit is used to receive cached data sent by the client, wherein the cached data is used to represent the data generated by pre-filling data for the text to be processed; the processing unit is used to pre-fill the second text in the text to be processed based on the network parameters of the target pre-filling layer to obtain a pre-filled intermediate result, wherein the second text is used to represent the text other than the first text in the text to be processed; the second receiving unit is used to receive the first character in the processing result sent by the client, wherein the first character is generated by the client based on the first pre-filling result and the second pre-filling result of the second text, and the second pre-filling result is the result obtained by pre-filling the pre-filled intermediate result using the network parameters of other pre-filling layers stored in the client, wherein the other pre-filling layers are used to represent the pre-filling layers other than the target pre-filling layer in the pre-filling module; the decoding unit is used to decode the text to be processed and the first character based on the cached data using the network parameters of the second decoding layer to obtain the decoding intermediate results of the other characters in the processing result, wherein the other characters are used to represent at least one character located after the first character in the processing result.

[0136] In the above embodiments of the present application, the device includes: an update module and a data sending module.

[0137] Among them, the update module is used to update the cache data based on the data generated in the process of decoding the first character and the text to be processed to obtain updated cache data; the data sending module is used to send the intermediate results and the updated cache data to the client, wherein the processing result is that the client uses the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cache data to obtain other characters, and the processing result is obtained by the client combining the first character and other characters, and the other characters are obtained by the client using the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cache data.

[0138] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0139] Example 7

[0140] The embodiment of the present application may provide an electronic device, which may be any electronic device in a computer terminal group. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0141] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0142] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the text processing method: obtaining the text to be processed; using the first model parameters of the text processing model stored on the client to pre-fill the first text in the text to be processed, and obtain a first pre-fill result of the first text; using the first model parameters and the second model parameters of the text processing model stored on the cloud, processing the text to be processed based on the first pre-fill result to obtain the processing result of the text to be processed, wherein the first model parameters are used to characterize part of the model parameters in the text processing model, and the second model parameters at least include other parameters in the text processing model except the first model parameters.

[0143] Optionally, Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 9 As shown, the electronic device A may include: one or more (only one is shown in the figure) processors 902, a memory 904, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module and a display.

[0144] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the text processing method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the above-mentioned text processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories can be connected to the electronic device A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0145] It can be understood by those skilled in the art that Figure 9 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 9 It does not limit the structure of the above electronic device. For example, electronic device A may also include Figure 9 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 9 Different configurations shown.

[0146] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0147] Example 8

[0148] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the text processing method provided in the first embodiment.

[0149] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0150] Example 9

[0151] An embodiment of the present application further provides a computer program product, comprising a computer program, which implements any of the above methods when executed by a processor.

[0152] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0153] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0155] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0156] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0158] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A text processing method, characterized in that: include: Get the text to be processed; Using a first model parameter of a text processing model stored on the client, pre-filling a first text in the text to be processed to obtain a first pre-filling result for the first text; The text to be processed is processed based on the first pre-filled result using the first model parameters and the second model parameters of the text processing model stored on the cloud to obtain a processing result of the text to be processed, wherein the first model parameters are used to characterize part of the model parameters in the text processing model, and the second model parameters at least include other parameters in the text processing model except the first model parameters.

2. The method according to claim 1, characterized in that The text processing model includes a pre-filling module and a decoding module; The first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent a first preset number of decoding layers with a connection relationship later in the decoding module; The second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a forward connection relationship in the pre-filling module, and the second decoding layer is used to represent the decoding layers in the decoding module other than the first decoding layer.

3. The method according to claim 2, characterized in that Using a first model parameter of a text processing model stored on the client, pre-filling a first text in the text to be processed to obtain a first pre-filled result of the first text, including: Reading the network parameters of the pre-filled module stored in the client layer by layer; The first text is pre-filled using the read network parameters to obtain the first pre-filled result.

4. The method according to claim 2, characterized in that The first model parameter and the second model parameter of the text processing model stored on the cloud are used to process the text to be processed based on the first pre-filled result to obtain a processing result of the text to be processed, including: performing pre-filling processing on a second text in the to-be-processed text using the network parameters of the other pre-filled layers stored in the client and the network parameters of the target pre-filled layer to obtain a second pre-filling result for the second text, wherein the other pre-filled layers are used to represent the pre-filled layers in the pre-filling module other than the target pre-filled layer; Generate, by the client, a first character in the processing result based on the first pre-filled result and the second pre-filled result, wherein the second text is used to represent text other than the first text in the text to be processed; Using the network parameters of the first decoding layer and the network parameters of the second decoding layer, the text to be processed and the first character are decoded based on the cached data to obtain other characters in the processing result, wherein the cached data is used to represent the data generated by pre-filling the text to be processed with data, and the other characters are used to represent at least one character in the processing result that is located after the first character.

5. The method according to claim 4, characterized in that Using the network parameters of the other pre-filled layers stored in the client and the network parameters of the target pre-filled layer, pre-filling the second text in the to-be-processed text to obtain a second pre-filled result for the second text, including: Performing a pre-filling process on the second text using the network parameters of the target pre-filling layer to obtain a pre-filled hidden state of the second text; The pre-filled hidden state is pre-filled using the network parameters of the other pre-filled layers to obtain a second pre-filled result of the second text.

6. The method according to claim 4, characterized in that Decoding the text to be processed and the first character based on cached data using the network parameters of the first decoding layer and the network parameters of the second decoding layer to obtain other characters in the processing result includes: Decoding the to-be-processed text and the first character based on the cached data using the network parameters of the first decoding layer and the network parameters of the second decoding layer, and sequentially generating at least one character in the processing result; The first character and the at least one character are combined to generate the processing result.

7. A text processing method, characterized in that: Applied to a client, the method includes: In response to receiving input text to be processed; Using first model parameters of the text processing model stored on the client, pre-filling a first text in the text to be processed to obtain a first pre-filled result for the first text, wherein the first model parameters are used to represent part of the model parameters in the text processing model; Sending the text to be processed to a cloud, and receiving an intermediate result sent by the cloud, wherein the intermediate result is a result obtained by processing the text to be processed using second model parameters of the text processing model stored on the cloud, and the second model parameters at least include other parameters of the text processing model except the first model parameters; The intermediate result is processed based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

8. The method according to claim 7, characterized in that The text processing model includes a pre-filling module and a decoding module; The first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent a first preset number of decoding layers with a connection relationship later in the decoding module; The second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a forward connection relationship in the pre-filling module, and the second decoding layer is used to represent the pre-filling layers in the decoding module other than the first decoding layer.

9. The method according to claim 8, characterized in that Using a first model parameter of a text processing model stored on the client, pre-filling a first text in the text to be processed to obtain a first pre-filled result of the first text, including: Reading the network parameters of the pre-filled module stored in the client layer by layer; The first text is pre-filled using the read network parameters to obtain the first pre-filled result.

10. The method according to claim 8, characterized in that The intermediate result is processed based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed, including: Sending cached data to the cloud, and receiving a pre-filled intermediate result sent by the cloud, wherein the cached data is used to represent data generated by pre-filling the text to be processed, and the pre-filled intermediate result is a result obtained by the cloud using the network parameters of the target pre-filled layer to pre-fill a second text in the text to be processed, and the second text is used to represent text in the text to be processed other than the first text; performing pre-filling processing on the pre-filled intermediate result using network parameters of other pre-filled layers stored in the client to obtain a second pre-filled result of the second text, wherein the other pre-filled layers are used to represent pre-filled layers in the pre-filled module other than the target pre-filled layer; generating a first character in the processing result based on the first pre-filled result and the second pre-filled result; Sending the first character to the cloud, and receiving intermediate decoding results and updated cache data of other characters sent by the cloud, wherein the intermediate decoding result is a result obtained by the cloud side decoding the first character and the text to be processed based on the cache data using the network parameters of the second decoding layer, the other characters are at least one character located after the first character in the processing result, and the updated cache data is obtained by updating the cache data based on data generated during the decoding of the first character and the text to be processed; Decoding the intermediate decoding result based on the updated buffer data using the network parameters of the first decoding layer to obtain the other characters; The first character and the other characters are combined to generate the processing result.

11. A text processing method, characterized in that: Applied to the cloud, the method includes: Receiving a text to be processed sent by a client, wherein the text to be processed is text received by the client as external input, the client storing first model parameters of a text processing model, the first model parameters being used to represent part of the model parameters in the text processing model, and the first model parameters being used to perform pre-filling processing on a first text in the text to be processed to obtain a first pre-filling result for the first text; Processing the text to be processed using second model parameters of the text processing model stored on the cloud to obtain an intermediate result of the text to be processed, wherein the second model parameters at least include other parameters in the text processing model except the first model parameters; The intermediate result is sent to the client, wherein the client processes the intermediate result based on the first pre-filled result using the first model parameter to obtain a processing result of the text to be processed.

12. The method according to claim 11, characterized in that The text processing model includes a pre-filling module and a decoding module; The first model parameters include network parameters of the pre-filling module and network parameters of the first decoding layer in the decoding module, wherein the first decoding layer is used to represent a first preset number of decoding layers with a connection relationship later in the decoding module; The second model parameters include network parameters of the target pre-filling layer in the pre-filling module and network parameters of the second decoding layer in the decoding module, wherein the target pre-filling layer is used to represent the second preset number of pre-filling layers with a forward connection relationship in the pre-filling module, and the second decoding layer is used to represent the pre-filling layers in the decoding module other than the first decoding layer.

13. The method according to claim 12, characterized in that Processing the text to be processed using the second model parameter of the text processing model stored on the cloud to obtain an intermediate result of the text to be processed, including: Receiving cache data sent by the client, wherein the cache data is used to represent data generated by pre-filling data for the text to be processed; Based on the network parameters of the target pre-filling layer, pre-filling a second text in the text to be processed to obtain the pre-filling intermediate result, wherein the second text is used to represent text other than the first text in the text to be processed; Receiving a first character in the processing result sent by the client, wherein the first character is generated by the client based on the first pre-filled result and a second pre-filled result of the second text, and the second pre-filled result is a result of pre-filling the pre-filled intermediate result using network parameters of other pre-filled layers stored in the client, wherein the other pre-filled layers are used to represent pre-filled layers in the pre-filled module other than the target pre-filled layer; Using the network parameters of the second decoding layer, the text to be processed and the first character are decoded based on the cached data to obtain intermediate decoding results of other characters in the processing result, wherein the other characters are used to represent at least one character in the processing result that is located after the first character.

14. The method according to claim 11, characterized in that The method further comprises: Based on data generated during the decoding of the first character and the text to be processed, updating the cache data to obtain updated cache data; Sending the intermediate result and the updated cached data to the client, wherein the processing result is that the client uses the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cached data to obtain the other characters, and the processing result is obtained by the client combining the first character and the other characters, and the other characters are obtained by the client using the network parameters of the first decoding layer to decode the decoding intermediate result based on the updated cached data.

15. An electronic device, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 14 when running.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 14.

17. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 14.

Images