Data processing method and apparatus using same

Abstract

A data processing method, which is applied to a first device, the method comprising: receiving a first prompt sent by a second device, the first prompt being obtained on the basis of an input request; on the basis of the first prompt, obtaining a second prompt, the length of the second prompt being less than that of the first prompt, and the second prompt comprising a meaning indicated in the first prompt; and sending the second prompt to the second device, wherein the data processing capability of the first device is greater than that of the second device. While ensuring result accuracy, the present application can reduce the computing overheads of end-side models processing prompts, thereby reducing inference delay.

Description

Data processing method and device thereof

[0001] The present application claims priority from the Chinese patent application No. 202411392865.7 filed on September 30, 2024, and entitled "A data processing method and device thereof", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method and device thereof. BACKGROUND

[0003] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, enabling machines to have perception, reasoning and decision-making functions.

[0004] In the prior art, a large model is often deployed on the end side, and the built-in large model on the end side is preferred during inference. However, due to the low computing power of the end side, the model on the cloud side needs to be used to assist in inference, for example, the model is split, the part of the model that does not need a large computing power overhead is placed on the end side for inference, and the part of the model that needs a large computing power overhead is placed on the cloud side for inference. Specifically, the end side performs inference related to the shallow part of the model, and the cloud side performs inference related to the deep part of the model.

[0005] However, this will cause frequent data transmission between the end and the cloud, resulting in large inference delay and high cost. SUMMARY

[0006] In a first aspect, the present application provides a data processing method applied to a first device, the method comprising: receiving a first prompt sent by a second device, the first prompt being obtained according to an input request; obtaining a second prompt according to the first prompt, the length of the second prompt being less than that of the first prompt, and the second prompt containing the semantics indicated in the first prompt; sending the second prompt to the second device; wherein the data processing capability of the first device is greater than that of the second device.

[0007] The second prompt contains the semantics indicated in the first prompt, which can be understood as the semantic similarity between the second prompt and the first prompt being greater, and both describing the same semantics, but the length of the former is less than that of the latter. The semantics can be directly indicated (for example, by a natural language description) or indirectly indicated (for example, by a vector representation). The semantics indicated by the vector representation can be determined by a certain task network. Of course, the second prompt can not contain the semantics of the information irrelevant to the generation of the reply to the input request in the first prompt, that is, the second prompt contains the key semantics (that is, the semantics of the information related to the generation of the reply to the input request) indicated in the first prompt.

[0008] The present application converts the first prompt sent by the device with low computing power into a second prompt with smaller length by processing the first prompt. In order to ensure the calculation accuracy of the subsequent model, the second prompt still contains the semantics of the first prompt. In addition, the end-side model only needs to process the shorter second prompt. Moreover, processing the prompt with shorter length can also reduce the size of the output result, so as to reduce the calculation overhead of the end-side model in processing the prompt and reduce the inference delay while ensuring the accuracy of the result.

[0009] The second device performs semantic-preserving compression on the prompt of the first device. Since the first device processes the compressed prompt, the data processing capability of the first device is relatively low, which greatly reduces the computing power and time overhead required by the first device to process the prompt by using the machine learning model, and ensures the processing accuracy (the compression is semantic-preserving compression). In addition, compared with the second device directly processing the prompt sent by the first device, the computing resources on the second device are usually more expensive than the computing resources on the first device in terms of the price paid by the user. Compared with the task of only compressing the prompt, the process of processing the prompt to obtain the reply to the input request requires a large number of iteration steps and requires more computing resources. Therefore, in the embodiment of the present application, the second device is used for compression of the prompt, and the first device is used for obtaining the reply to the input request based on the prompt, which can reduce the cost caused by the computing power overhead on the second device.

[0010] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a second natural language description. The second natural language description is a summarized description of the first natural language description.

[0011] Here, the so-called summary can be understood as using shorter language to summarize the content of the natural language, which can also be called an abstract.

[0012] The first natural language description can be all or part of the first prompt.

[0013] For example, the first natural language description can be summarized by constructing a prompt to utilize a language model, where the second prompt can also be referred to as a hard prompt.

[0014] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description. For example, the first natural language description can be feature-extracted to obtain the first vector representation.

[0015] In a possible implementation, the obtaining the second prompt according to the first prompt includes: compressing the first natural language description in the first prompt to obtain a second vector representation; and adjusting a dimension of the second vector representation to obtain the first vector representation, and the second prompt includes the first vector representation.

[0016] Since the data processing capability of the first device is less than that of the second device, the scale of the model (the first machine learning model) deployed on the first device is usually less than that of the model (the second machine learning model) deployed on the second device. For models of different scales, the dimension (for example, the dimension size of an embedding vector of an input) of the vector representation of the supported input is usually different. Therefore, the dimension of the vector representation obtained by the second device through compression of the second machine learning model can be different from the dimension size of the vector representation of the input supported by the first machine learning model on the first device. In an embodiment of the present application, a dimension adjustment module (which can also be referred to as a dimension adapter) can be deployed on the second device or the first device. The dimension adjustment module can adjust the dimension of the vector representation obtained by the second machine learning model to the dimension size of the vector representation of the input supported by the first machine learning model. For example, when the dimension adjustment module is deployed on the second device, the second device can compress the first prompt through the second machine learning model to obtain a second vector representation. Then, the second device can adjust the dimension of the second vector representation through the dimension adjustment module, so as to obtain a second prompt including a dimension-adjusted vector representation (a first vector representation). Then, the second device can send the second prompt including the dimension-adjusted vector representation (the first vector representation) to the first device, and the first device can process the second prompt through the first machine learning model.

[0017] In a possible implementation, the first prompt includes a plurality of prompt segments obtained by division, and the second prompt includes compression results (for example, feature representations or summary descriptions) obtained by compressing each prompt segment in the plurality of prompt segments.

[0018] The compression can be summarizing the natural language description in the first prompt to obtain a summarized description, or can be feature extraction on the natural language description in the first prompt to obtain an abstract vector representation, and the length of the vector representation is often much smaller than the natural language.

[0019] For example, the language model can be used to perform feature extraction on the first natural language description by constructing a prompt, and the second prompt can also be referred to as a soft prompt.

[0020] In a possible implementation, the obtaining the second prompt according to the first prompt comprises: constructing a third prompt according to the first prompt, the third prompt indicating processing each prompt segment obtained by dividing the first prompt; obtaining a compression result of each prompt segment by a language model according to the third prompt; and obtaining the second prompt by fusing a plurality of compression results.

[0021] In a possible implementation, the data processing capability is computing power.

[0022] In a possible implementation, the first prompt includes a plurality of segments of natural language descriptions, and the second prompt includes a summarized description or a vector representation obtained by processing each segment.

[0023] That is, each segment in the first prompt can be processed to obtain a summarized description or a vector representation of each segment, and the summarized description or the vector representation of each segment can be fused (for example, spliced) to obtain the second prompt, which can retain the semantics of each segment. Compared with summarizing or feature extraction on the whole, summarizing or feature extraction on the segments and fusing the segments can retain more complete semantics.

[0024] In a possible implementation, the first prompt includes the input request, a requirement for a reply to the output, and context information to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

[0025] The prompt can include multiple parts, of which exemplary parts include an input request, which can be input information of a user, such as a question. The prompt can also include context information, such as historical input information and output when the question is multi-turn, or environmental information of the generated result. The prompt can also include a requirement to be met by the output, which can also be referred to as a constraint on the output. Of course, the prompt can also include other types of information, which are related to a prompt template used to generate the prompt. The above three examples can also overlap with each other, for example, the input request can also contain a part of the requirement for the output.

[0026] In the embodiments of the present application, when the second device compresses the first prompt, one or more of the above information can be compressed, for example, only all or part of the input request is compressed, and other information and the first information remain unchanged, for example, only the input request and the context information are compressed, and other information and the first prompt remain unchanged.

[0027] In a possible implementation,

[0028] The first device is an end-side device, and the second device is a cloud-side server; or

[0029] The first device is an end-side device, and the second device is an edge-side server; or

[0030] The first device is an edge-side device, and the second device is a cloud-side server.

[0031] In a second aspect, the present application provides a system comprising a first device and a second device, wherein the first device and the second device are in communication, and the data processing capability of the first device is greater than that of the second device.

[0032] The first device is configured to send a first prompt to the second device, wherein the first prompt is obtained according to an input request.

[0033] The second device is configured to obtain a second prompt according to the first prompt, wherein the length of the second prompt is less than that of the first prompt, and the second prompt contains the semantics indicated in the first prompt.

[0034] The second device is configured to send the second prompt to the second device.

[0035] The first device is configured to obtain a reply to the input request by using a first machine learning model according to the second prompt.

[0036] In a possible implementation, the second device is specifically configured to obtain a second prompt by using a second machine learning model according to the first prompt.

[0037] The first machine learning model is obtained by compressing or knowledge distilling the second machine learning model.

[0038] In training the second machine learning model, the set compression rate can be used as one of the training targets, so that the trained second machine learning model has the compression capability of the set compression rate. Multiple second machine learning models can be trained, different second machine learning models have different compression capabilities of different compression rates, and one of the second machine learning models can be selected to perform the compression process of the first prompt sent by the first device.

[0039] In a possible implementation, the first device can also carry information indicating the compression rate requirement when sending the first prompt to the second device, and the second device can select a second machine learning model from multiple second machine learning models that can meet the compression rate requirement to process the first prompt.

[0040] In a possible implementation, for different first devices, the second device can select second machine learning models with different compression capabilities of different compression rates to perform the compression process, for example, the second device can select model A to process the prompt sent by device A, and select model B to process the prompt sent by device B, device A and device B are different, and the compression rates of model A and model B are different.

[0041] In a possible implementation, the training tasks of the first machine learning model and the second machine learning model are the same during training.

[0042] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description.

[0043] The first device is further configured to perform dimension adjustment on the first vector representation in the second prompt to obtain an adjusted second prompt.

[0044] The first device is specifically configured to obtain a reply to the input request through a first machine learning model according to the adjusted second prompt.

[0045] In a third aspect, the present application provides a data processing method applied to a second device, the method comprising:

[0046] sending a first prompt prompt to a first device, the first prompt being obtained according to an input request;

[0047] receiving a second prompt sent by the first device according to the first prompt, the length of the second prompt being less than that of the first prompt, and the second prompt containing the semantics indicated by the first prompt;

[0048] obtaining a reply to the input request through a first machine learning model according to the second prompt; wherein the data processing capability of the first device is greater than that of the second device.

[0049] In a possible implementation, the input request contains privacy information; the first prompt does not contain the privacy information, and the first device is further configured to fuse the privacy information into the second prompt.

[0050] The first device can obtain an initial prompt according to the input request, and the initial prompt can contain privacy information, which can not be contained in the first prompt. In order to ensure the accuracy of the generated result, the first device can fuse the privacy information into the second prompt after receiving the second prompt sent by the second device, and generate the reply to the input request based on the second prompt fused with the privacy information through the language model. Through the above manner, in the case of ensuring that the privacy information cannot be perceived by the second device and also improving the processing speed of the first device for the prompt, the processing accuracy of the first device for the input request can be ensured.

[0051] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description; and the method further includes:

[0052] dimensionally adjusting the first vector representation in the second prompt to obtain an adjusted second prompt;

[0053] obtaining, according to the second prompt, a reply to the input request through a first machine learning model, includes:

[0054] obtaining, according to the adjusted second prompt, a reply to the input request through the first machine learning model.

[0055] When the dimension adjustment module is deployed on the first device, the first device can adjust the dimension of the vector representation included in the second prompt after receiving the second prompt sent by the second device, so as to obtain the second prompt containing the vector representation after dimension adjustment, and the second device can process the second prompt containing the vector representation after dimension adjustment through the first machine learning model.

[0056] In a fourth aspect, the present application provides a data processing apparatus applied to a first device, the apparatus comprising:

[0057] a transceiving module configured to receive a first prompt prompt sent by a second device, the first prompt being obtained according to an input request;

[0058] a prompt compression module configured to obtain a second prompt according to the first prompt, the length of the second prompt being less than that of the first prompt, and the second prompt containing semantics indicated in the first prompt;

[0059] The transceiving module is further configured to send the second prompt to the second device; and the data processing capability of the first device is greater than the data processing capability of the second device.

[0060] In a possible implementation, the first prompt comprises a plurality of prompt segments obtained by division, and the second prompt comprises compression results of the plurality of prompt segments.

[0061] In a possible implementation, the prompt compression module is specifically configured to:

[0062] construct a third prompt according to the first prompt, the third prompt indicating processing of each prompt segment obtained by division of the first prompt;

[0063] obtain, according to the third prompt, compression results of each prompt segment by using a language model;

[0064] obtain the second prompt by fusing the compression results.

[0065] In a possible implementation, the first prompt comprises a first natural language description, and the second prompt comprises a second natural language description, the second natural language description being a summarized description of the first natural language description.

[0066] In a possible implementation, the first prompt comprises a first natural language description, and the second prompt comprises a first vector representation of the first natural language description.

[0067] In a possible implementation, the prompt compression module is specifically configured to:

[0068] compress the first natural language description in the first prompt to obtain a second vector representation;

[0069] The apparatus further comprises:

[0070] a dimension adjustment module configured to perform dimension adjustment on the second vector representation to obtain the first vector representation, and the second prompt comprises the first vector representation.

[0071] In a possible implementation, the processing module is specifically configured to:

[0072] The first device is an end-side device, and the second device is a cloud-side server; or

[0073] The first device is an end-side device, and the second device is an edge-side server; or

[0074] The first device is an edge-side device, and the second device is a cloud-side server.

[0075] In a possible implementation, the data processing capability is computing power.

[0076] In a possible implementation,

[0077] The first prompt includes the input request, a requirement for a reply to the output, and context information that needs to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

[0078] In a fifth aspect, an embodiment of the present application provides a data processing apparatus applied to a second device, and the apparatus includes:

[0079] A transceiving module is configured to send a first prompt prompt to the second device, the first prompt being obtained according to an input request;

[0080] A prompt compression module is configured to receive a second prompt sent by the second device according to the first prompt, the second prompt having a length smaller than that of the first prompt, and the second prompt containing semantics indicated by the first prompt;

[0081] A model inference module is configured to obtain a reply to the input request by using a first machine learning model according to the second prompt.

[0082] In a possible implementation, the input request contains privacy information; the first prompt does not contain the privacy information, and the first device is further configured to fuse the privacy information into the second prompt.

[0083] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description; and the apparatus further includes:

[0084] A dimension adjustment module is configured to perform dimension adjustment on the first vector representation in the second prompt to obtain an adjusted second prompt.

[0085] The model inference module is specifically configured to:

[0086] Obtain a reply to the input request by using a first machine learning model according to the adjusted second prompt.

[0087] In a sixth aspect, an embodiment of the present application provides a data processing apparatus, which can include a memory, a processor and a bus system, wherein the memory is configured to store a program, and the processor is configured to execute the program in the memory to perform the method of the first aspect and any optional method of the first aspect, and the method of the third aspect and any optional method of the third aspect.

[0088] In a seventh aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is run on a computer, the computer is enabled to perform the method of the first aspect and any optional method of the first aspect, and the method of the third aspect and any optional method of the third aspect.

[0089] In an eighth aspect, an embodiment of the present application provides a computer program, and when the computer program is run on a computer, the computer is enabled to perform the method of the first aspect and any optional method of the first aspect, and the method of the third aspect and any optional method of the third aspect.

[0090] In a ninth aspect, an embodiment of the present application provides a chip system, which includes a processor configured to support the execution of the data processing apparatus to perform the functions involved in the above aspects, such as sending or processing the data involved in the above methods; or, information. In a possible design, the chip system further includes a memory, and the memory is configured to store the necessary program instructions and data for the execution device or the training device. The chip system can be composed of a chip, or can include a chip and other discrete devices. BRIEF DESCRIPTION OF DRAWINGS

[0091] FIG. 1A is a structural schematic diagram of an artificial intelligence main body framework;

[0092] FIG. 1B and FIG. 1C are schematic diagrams of an application system framework of the present application;

[0093] FIG. 1D is a schematic diagram of an optional hardware structure of a terminal;

[0094] FIG. 2 is a structural schematic diagram of a server;

[0095] FIG. 3 to FIG. 5 are schematic diagrams of a system architecture of the present application;

[0096] FIG. 6 is a flow of a cloud service;

[0097] FIG. 7 is a flow schematic diagram of a data processing method provided by an embodiment of the present application;

[0098] FIG. 8 is a flow schematic diagram of a data processing method provided by an embodiment of the present application;

[0099] FIG. 9 and FIG. 10 are flow schematic diagrams of a data processing method provided by an embodiment of the present application;

[0100] Figure 11 is a schematic diagram of a data processing device provided in an embodiment of this application;

[0101] Figure 12 is a schematic diagram of a terminal device provided in an embodiment of this application;

[0102] Figure 13 is a schematic diagram of a server structure provided in an embodiment of this application;

[0103] Figure 14 is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0104] The embodiments of the present invention will now be described with reference to the accompanying drawings. The terminology used in the embodiments section is for illustrative purposes only and is not intended to limit the scope of the invention.

[0105] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0106] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0107] The terms “substantially,” “about,” and similar terms used herein are used as approximations rather than as terms of degree, and are intended to take into account the inherent biases of measurements or calculations known to those skilled in the art. Furthermore, the use of “may” in describing embodiments of the invention refers to “one or more possible embodiments.” The terms “use,” “using,” and “used” used herein are to be considered synonymous with the terms “utilize,” “utilizing,” and “utilized,” respectively. Additionally, the term “exemplary” is intended to refer to an instance or illustration.

[0108] First, the overall workflow of the artificial intelligence system is described, please refer to FIG. 1A, which shows a structural diagram of an artificial intelligence subject framework, and the following describes the artificial intelligence subject framework from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0109] (1) Infrastructure

[0110] The infrastructure provides computing power support for the artificial intelligence system, realizes communication with the outside world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips (CPU, NPU, GPU, ASIC, FPGA, etc. Hardware acceleration chips); the underlying platform includes distributed computing framework and network-related platform support and support, which can include cloud storage and computing, interconnection network, etc. For example, sensors and external communication acquire data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for calculation.

[0111] (2) Data

[0112] The data on the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. Data involves graphics, images, speech, text, and also involves Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, and humidity.

[0113] (3) Data processing

[0114] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0115] Among them, machine learning and deep learning can symbolize and formalize intelligent information modeling, extraction, preprocessing, training, etc.

[0116] Reasoning refers to simulating human intelligent reasoning methods in computers or intelligent systems, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0117] Decision-making refers to the process of decision-making after intelligent information reasoning, which usually provides functions such as classification, sorting, and prediction.​​​​​​​​​

[0118] (4) General capabilities

[0119] After the data is processed as mentioned above, further based on the results of data processing, some general capabilities can be formed, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0120] (5) Intelligent products and industry applications

[0121] Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields, which are the packaging of overall solutions of artificial intelligence, and realize the application of intelligent information decision products. Its application fields mainly include: intelligent terminal, intelligent transportation, intelligent medical treatment, automatic driving, smart city, etc.

[0122] First, the application scenario of the present application is introduced. The present application can be but is not limited to the application of a generative artificial intelligence (artificial intelligence generated content, AIGC) function (hereinafter referred to as a synthetic application program) or a cloud service provided by a cloud-side server, etc., which will be introduced next:

[0123] I. Synthetic application program

[0124] The product form of the embodiments of the present application can be a synthetic application program. The synthetic application program can run on a terminal device or a server on the cloud side.

[0125] In one possible implementation, the synthetic application program can implement a data generation task based on input data (such as images, text, audio, video, etc.). The synthetic application program can execute the data generation task in response to the input input data (such as images, text, audio, video, etc.) to obtain a synthesis result.

[0126] For example, the above-mentioned data generation task can be but is not limited to:

[0127] Text generation task: It can generate various types of text content, including news reports, blog articles, product descriptions, social media posts, etc. It can generate logically coherent and coherent text according to the given theme and requirements.

[0128] Image generation task: It can generate images, including illustrations, artwork, design drafts, etc. It can generate image content related to the given description or keywords.

[0129] Audio generation task: Can generate speech content, including reading text, voice assistant answers, etc. It can simulate human voice characteristics and intonation, making the generated speech sound more natural.

[0130] Content summary and summary task: Can read a large amount of text content and generate a summary or summary. It can extract key information from the text and present it to the user in a concise manner.

[0131] Language translation task: Can perform language translation, translating text in one language into another language. It can handle multiple language pairs and provide accurate translation results.

[0132] Automatic reply and customer service: Can be used to automatically reply to user questions and provide customer service. It can understand the user's intent and give accurate answers or suggestions.

[0133] In one possible implementation, the user can open the synthetic application installed on the terminal device and input the input data (such as image, text, audio, video, etc.), and the synthetic application can generate data for the input data through the method provided by the present application. Embodiments, and the synthesis result is presented to the user (the presentation method can be but is not limited to display, save, upload to the cloud side, etc.).

[0134] In one possible implementation, the user can open the synthetic application installed on the terminal device and input the input data, and the synthetic application can send the input data to the server on the cloud side, and the server on the cloud side can generate data for the input data through the method provided by the present application. Embodiments, and the synthesis result is returned to the terminal device, and the terminal device can present the synthesis result to the user (the presentation method can be but is not limited to display, save, upload to the cloud side, etc.).

[0135] Next, the synthetic application in the present application embodiment is introduced from the functional architecture and the product architecture for realizing the function.

[0136] Referring to FIG. 1B, FIG. 1B is a functional architecture of the synthetic application in the present application embodiment:

[0137] In one possible implementation, as shown in FIG. 1B, the synthetic application 102 can receive the input parameter 101 (for example, containing input data) and generate the synthesis result 103. The synthetic application 102 can be executed on (for example) at least one computer system, and includes computer code that, when executed by one or more computers, causes the computers to perform a natural language model trained by the method provided by the present application.

[0138] Referring to FIG. 1C, FIG. 1C is a schematic diagram of an entity architecture for running a synthetic application according to an embodiment of the present application.

[0139] Referring to FIG. 1C, FIG. 1C is a schematic diagram of an entity architecture for running a synthetic application according to an embodiment of the present application.

[0140] The terminal 100 can install a synthetic application or open a webpage related to the synthetic function. The application and the webpage can provide an interface. The terminal 100 can receive parameters input by a user on the interface of the synthetic function and send the parameters to the server 200. The server 200 can obtain a processing result based on the received parameters and return the processing result to the terminal 100.

[0141] It should be understood that, in some optional implementations, the terminal 100 can also complete the action of obtaining a processing result based on received parameters by itself without the cooperation of the server. The embodiments of the present application are not limited in this regard.

[0142] Next, the product form of the terminal 100 in FIG. 1C is described.

[0143] The terminal 100 in the embodiments of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like, and the embodiments of the present application are not limited in this regard.

[0144] FIG. 1D shows a schematic diagram of an optional hardware structure of the terminal 100.

[0145] Referring to FIG. 1D, the terminal 100 can include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a processor 170, an external interface 180, a power supply 190, and the like. Those skilled in the art can understand that FIG. 1D is merely an example of a terminal or a multifunctional device, and does not constitute a limitation on the terminal or the multifunctional device, which can include more or fewer components than those shown, or combine certain components, or different components.

[0146] The input unit 130 can be configured to receive inputted digital or character information, and to generate key signal inputs related to user settings of the portable multifunctional device and control of functions. Specifically, the input unit 130 can include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect touch operations of a user thereon or therearound (such as operations of the user using a finger, a knuckle, a stylus, or any suitable object on or near the touch screen), and drive corresponding connected devices according to a pre-set program. The touch screen can detect touch actions of the user on the touch screen, convert the touch actions into touch signals and send the touch signals to the processor 170, and can receive commands from the processor 170 and execute the commands; the touch signals at least include touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, etc.

[0147] The input device 132 can receive inputted input data, etc.

[0148] The display unit 140 can be configured to display information inputted by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playing of any kind of multimedia files. In the embodiments of the present application, the display unit 140 can be configured to display interfaces of the composition application, generated composition results, etc.

[0149] The storage 120 can be configured to store instructions and data. The storage 120 can mainly include a storage instruction area and a storage data area. The storage data area can store various data such as multimedia files, texts, etc.; the storage instruction area can store software units such as operating systems, applications, instructions required by at least one function, etc., or their subsets, expanded sets. It can also include a non-volatile random access memory; provide the processor 170 with software and applications that include management of hardware, software, and data resources in the computing processing device, support control. It is also used for storage of multimedia files, and storage of running programs and applications.

[0150] The processor 170 is the control center of the terminal 100, connects each part of the whole terminal 100 by various interfaces and lines, executes various functions of the terminal 100 and processes data by running or executing the instructions stored in the memory 120 and calling the data stored in the memory 120, thereby overall controlling the terminal device. Optionally, the processor 170 can include one or more processing units; preferably, the processor 170 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 170. In some embodiments, the processor, the memory, can be implemented on a single chip, and in some embodiments, they can also be implemented on separate chips respectively. The processor 170 can also be used to generate corresponding operation control signals to the corresponding components of the computing processing device, read and process the data in the software, especially read and process the data and programs in the memory 120, so that each functional module therein executes corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0151] The memory 120 can be used to store software codes related to the data processing method, and the processor 170 can execute the steps of the data processing method of the chip, or can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to realize corresponding functions.

[0152] The RF unit 110 (optional) can be used to receive and send signals in information or communication processes. For example, after receiving the downlink information from the base station, the processor 170 processes the information. In addition, the uplink data is sent to the base station. Generally, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0153] In the embodiments of the present application, the RF unit 110 can send input data to the server 200 and receive the synthesis result sent by the server 200.

[0154] It should be understood that the RF unit 110 is optional, which can be replaced by other communication interfaces, for example, a network interface.

[0155] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so as to realize the functions of power management, discharge management, and power consumption management through the power management system.

[0156] The terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector. The external interface 180 can be used to connect the terminal 100 to other devices for communication, and can also be used to connect a charger to charge the terminal 100.

[0157] Although not shown, the terminal 100 can also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, different function sensors, etc., which will not be described here. Some or all of the methods described below can be applied in the terminal 100 as shown in FIG. 1D.

[0158] Next, the product form of the server 200 in FIG. 1C is described;

[0159] FIG. 2 provides a structural schematic diagram of a server 200. As shown in FIG. 2, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate through the bus 201.

[0160] The bus 201 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in FIG. 2, but it does not mean that there is only one bus or only one type of bus.

[0161] The processor 202 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0162] The memory 204 can include a volatile memory, such as a random access memory (RAM). The memory 204 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard drive (HDD), or a solid state drive (SSD).

[0163] The memory 204 can be used to store software codes related to the data processing method, and the processor 202 can execute the steps of the chip data processing method or schedule other units to realize the corresponding functions.

[0164] It should be understood that the terminal 100 and the server 200 described above can be centralized or distributed devices, and the processors (for example, the processor 170 and the processor 202) in the terminal 100 and the server 200 can be hardware circuits (for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor, a microcontroller, or the like) or a combination of the hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, a DSP, or the like, or a hardware system without an instruction execution function, such as an ASIC, an FPGA, or the like, or a combination of the hardware system without an instruction execution function and the hardware system with an instruction execution function.

[0165] It should be understood that the steps related to the model inference process in the embodiments of the present application involve AI-related operations. When performing AI operations, the instruction execution architecture of the terminal device and the server is not limited to the architecture of the processor combined with the memory described above. The system architecture provided by the embodiments of the present application will be described in detail below with reference to FIG. 5.

[0166] FIG. 5 is a schematic diagram of a system architecture provided by an embodiment of the present application. As shown in FIG. 5, the system architecture 500 includes an execution device 510, a training device 520, a database 530, a client device 540, a data storage system 550, and a data collection device 560.

[0167] The execution device 510 includes a computing module 511, an I / O interface 512, a preprocessing module 513, and a preprocessing module 514. The target model / rule 501 can be included in the computing module 511, and the preprocessing module 513 and the preprocessing module 514 are optional.

[0168] The execution device 510 can be a terminal device or a server running a synthetic application program as described above.

[0169] The data collection device 560 is configured to collect training samples. The training samples can be program files (including program codes and program input data) and the like. After the training samples are collected, the data collection device 560 stores the training samples in the database 530.

[0170] The training device 520 can train a neural network to be trained based on the training samples maintained in the database 530 to obtain the target model / rule 501.

[0171] It should be noted that, in actual application, the training samples maintained in the database 530 do not necessarily all come from the collection of the data collection device 560, but can also be received from other devices. In addition, it should be noted that the training device 520 does not necessarily train the target model / rule 501 based on the training samples maintained in the database 530, but can also obtain training samples from the cloud or other places for model training. The above description should not be regarded as a limitation of the embodiments of the present application.

[0172] The target model / rule 501 trained by the training device 520 can be applied to different systems or devices, such as the execution device 510 shown in FIG. 5. The execution device 510 can be a terminal such as a mobile phone terminal, a tablet computer, a notebook computer, an augmented reality (AR) / virtual reality (VR) device, a vehicle-mounted terminal, etc., and can also be a server, etc.

[0173] Specifically, the training device 520 can deliver the trained model to the execution device 510.

[0174] In FIG. 5, the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with external devices. The user can input data (for example, input data in the embodiments of the present application) to the I / O interface 512 through the client device 540.

[0175] The pre-processing modules 513 and 514 are used for pre-processing the input data received by the I / O interface 512. It should be understood that there can be no pre-processing modules 513 and 514 or only one pre-processing module. When there is no pre-processing module 513 and 514, the input data can be directly processed by the calculation module 511.

[0176] During the pre-processing of the input data by the execution device 510 or during the calculation process performed by the calculation module 511 of the execution device 510, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, or store the data, instructions, etc. obtained by the corresponding processing in the data storage system 550.

[0177] Finally, the I / O interface 512 provides the processing result (for example, the synthesis result, etc.) to the client device 540, thereby providing the user.

[0178] In the case shown in FIG. 5, the user can manually give input data, which can be operated through the interface provided by the I / O interface 512. In another case, the client device 540 can automatically send input data to the I / O interface 512, and if the user's authorization is required for the client device 540 to automatically send input data, the user can set the corresponding permission in the client device 540. The user can view the results output by the execution device 510 in the client device 540, and the specific presentation form can be a specific way such as display, sound, action, etc. The client device 540 can also act as a data collection end, collect the input data input into the I / O interface 512 and the output results output from the I / O interface 512 as new sample data, and store them into the database 530. Of course, the collection can also be performed without the client device 540, and the I / O interface 512 can directly store the input data input into the I / O interface 512 and the output results output from the I / O interface 512 as new sample data into the database 530.

[0179] It is worth noting that FIG. 5 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules and the like shown in the figure does not constitute any limitation. For example, in FIG. 5, the data storage system 550 is an external memory relative to the execution device 510, and in other cases, the data storage system 550 can also be placed in the execution device 510. It should be understood that the above execution device 510 can be deployed in the client device 540.

[0180] From the inference side of the model:

[0181] In the embodiment of the present application, the computing module 511 of the execution device 510 can obtain the code stored in the data storage system 550 to implement the steps related to the model inference process in the embodiment of the present application.

[0182] In the embodiment of the present application, the computing module 511 of the execution device 510 can include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 can be a hardware system with an execution instruction function, such as a CPU, a DSP, etc., or a hardware system without an execution instruction function, such as an ASIC, an FPGA, etc., or a combination of the hardware system without an execution instruction function and the hardware system with an execution instruction function.

[0183] Specifically, the computing module 511 of the execution device 510 can be a hardware system with an execution instruction function, and the steps related to the model inference process provided in the embodiments of the present application can be software codes stored in the memory. The computing module 511 of the execution device 510 can obtain the software codes from the memory and execute the obtained software codes to implement the steps related to the model inference process provided in the embodiments of the present application.

[0184] It should be understood that the computing module 511 of the execution device 510 can be a combination of a hardware system without an execution instruction function and a hardware system with an execution instruction function, and part of the steps related to the model inference process provided in the embodiments of the present application can also be implemented by the hardware system without an execution instruction function in the computing module 511 of the execution device 510, which is not limited here.

[0185] From the training side of the model:

[0186] In the embodiments of the present application, the training device 520 can obtain the codes stored in the memory (not shown in FIG. 5, which can be integrated in the training device 520 or deployed separately from the training device 520) to implement the steps related to the model training in the embodiments of the present application.

[0187] In the embodiments of the present application, the training device 520 can include hardware circuits (such as application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the training device 520 can be a hardware system with an execution instruction function, such as a CPU, a DSP, etc., or a hardware system without an execution instruction function, such as an ASIC, an FPGA, etc., or a combination of the hardware system without an execution instruction function and the hardware system with an execution instruction function.

[0188] It should be understood that the training device 520 can be a combination of a hardware system without an execution instruction function and a hardware system with an execution instruction function, and part of the steps related to the model training provided in the embodiments of the present application can also be implemented by the hardware system without an execution instruction function in the training device 520, which is not limited here.

[0189] II. Server-provided integration function cloud service:

[0190] In a possible implementation, the server can provide a service of the synthesis function for the end side through an application programming interface (API).

[0191] The terminal device can send relevant parameters (for example, containing input data) to the server through the API provided by the cloud, and the server can obtain a processing result (for example, a synthesis result, etc.) based on the received parameters and return the processing result to the terminal.

[0192] The description of the terminal and the server can be described in the above embodiments, which will not be described here.

[0193] As shown in FIG. 6, a process of using a cloud service of a synthesis function provided by a cloud platform is shown.

[0194] 1. Open and purchase a synthesis function service.

[0195] 2. The user can download a software development kit (SDK) corresponding to the synthesis function service. The cloud platform usually provides multiple development versions of the SDK for the user to select according to the needs of the development environment, such as a JAVA version of the SDK, a python version of the SDK, a PHP version of the SDK, an Android version of the SDK, etc.

[0196] 3. The user downloads the corresponding version of the SDK to the local according to the needs, imports the SDK project to the local development environment, and configures and debugs in the local development environment. The local development environment can also develop other functions, so as to form an application that integrates the synthesis function type capability.

[0197] 4. During the use of the synthesis function type application, when the synthesis function is needed, the API call of the synthesis function can be triggered. When the application triggers the synthesis function, an API request is initiated to the running instance of the synthesis function type service in the cloud environment. The API request carries input data, which is processed by the running instance in the cloud environment to obtain a processing result (for example, a synthesis result, etc.).

[0198] 5. The cloud environment returns the processing result to the application, thereby completing a synthesis function service call.

[0199] In addition to the application program and the cloud service, the implementation form of the present application can also be a large model inference acceleration library and a large model application SDK.

[0200] In order to better understand the scheme of the embodiments of the present application, the following takes text generation as an example to briefly introduce the possible application scenarios of the embodiments of the present application in combination with FIGS. 3-4.

[0201] Figure 3 shows a natural language processing system, which includes a user device and a data processing device. The user device includes a smart terminal such as a mobile phone, a personal computer, or an information processing center. The user device is the initiator of natural language data processing, and is the initiator of language query or inquiry. Usually, the user initiates the request through the user device.

[0202] The data processing device can be a cloud server, a network server, an application server, a management server, or other devices or servers with data processing functions. The data processing device receives query statements, voices, texts, etc. from the smart terminal through an interactive interface, and performs language data processing in the form of machine learning, deep learning, searching, reasoning, decision-making, etc. through the storage memory and the data processing processor, and feeds back the processing results to the user device. The storage memory in the data processing device can be a general term, including local storage and database for storing historical data. The database can be on the data processing device or on other network servers.

[0203] In the natural language processing system shown in Figure 3, the user device can receive the user's instructions, for example, the user device can receive a piece of text input by the user, and then initiate a request to the data processing device, so that the data processing device performs natural language processing applications (such as natural language generation, text classification, text reasoning, named entity recognition, translation, etc.) on the piece of text obtained by the user device, thereby obtaining the processing results (such as predicted word results, classification results, reasoning results, named entity recognition results, translation results, etc.) of the corresponding natural language processing applications on the piece of text.

[0204] In the embodiments of the present application, the user device can receive the user's instructions, for example, the user device can receive a piece of text (such as input data) input by the user, and then initiate a request to the data processing device, so that the data processing device performs natural language processing applications (such as text synthesis, etc.) on the piece of text obtained by the user device, thereby obtaining the processing results (such as synthesis results, etc.) of the corresponding natural language processing applications on the piece of text.

[0205] In Figure 3, the data processing device can process the above-mentioned text data by the method provided in the embodiments of the present application.

[0206] Figure 4 shows another natural language processing system, in which the user device directly serves as the data processing device. The user device can directly receive input from the user and directly process it by the hardware of the user device itself. The specific process is similar to that of Figure 3, and reference can be made to the above description, which will not be repeated here.

[0207] FIG. 4 is a schematic diagram of a device for natural language processing according to an embodiment of the present application.

[0208] The processor in FIG. 3 and FIG. 4 can train data / machine learn / deep learn through a neural network model or other model, and use the model (e.g., a natural language model in an embodiment of the present application, etc.) trained or learned finally by using data to perform natural language processing application (e.g., program synthesis, etc.) on text data (e.g., input data text described in an embodiment of the present application), so as to obtain corresponding processing results.

[0209] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the related terms and concepts related to neural networks involved in the embodiments of the present application will be introduced first.

[0210] (1) Neural network

[0211] The neural network can be composed of neural units, and the neural unit can be an operation unit with xs (i.e., input data) and intercept 1 as input. The output of the operation unit can be:

[0212] where s = 1, 2, … n, n is a natural number greater than 1, Ws is the weight of xs, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. The neural network is a network formed by connecting a plurality of single neural units, i.e., the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of a plurality of neural units.

[0213] (2) Natural language processing (NLP)

[0214] Natural language is human language, and natural language processing (NLP) is the processing of human language. Natural language processing is a process of systematic analysis, understanding and information extraction of text data in an intelligent and efficient manner. By using NLP and its components, we can manage very large blocks of text data, or perform a large number of automated tasks, and solve a variety of problems, such as automatic summarization, machine translation (MT), named entity recognition (NER), relation extraction (RE), information extraction (IE), sentiment analysis, speech recognition, question answering and topic segmentation, etc.

[0215] (3) Pre-trained language model

[0216] A pre-trained language model is a natural language sequence encoder that encodes each word in a natural language sequence into a vector representation to perform a prediction task. Its training includes two stages. In the pre-training stage, the model is trained on large-scale unsupervised text for language model tasks, thereby learning a word representation. In the fine-tuning stage, the model uses the parameters learned in the pre-training stage for initialization, and performs a few-step training on text classification, sequence labeling and other downstream tasks, so that the semantic information obtained by pre-training can be successfully transferred to the downstream tasks.

[0217] (4) Autoregressive language model

[0218] An autoregressive language model is a model that can predict the next word that may follow (such as "not bad") based on the given context (such as "mobile phone is very"). This model usually predicts the word in the right context given the left context, but it can also predict a word in the middle given the left and right contexts.

[0219] (5) Large Language Model (LLM): A large language model refers to a natural language processing model trained on a large scale of data, usually with tens of billions or hundreds of billions of parameters. These models capture the general features of language through learning a large amount of text data in the pre-training stage, and can then be fine-tuned on downstream tasks to adapt to the needs of specific tasks.

[0220] (6) Token: Refers to the smallest unit in text. Usually, a token can be a word, a number, a punctuation mark, a single letter, or any single element that can be analyzed as text.

[0221] (7) Back Propagation Algorithm

[0222] The convolutional neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, the forward propagation of the input signal until the output produces an error loss, and the error loss information is updated through back propagation to update the parameters in the initial super-resolution model, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the super-resolution model, such as the weight matrix.

[0223] (8) Loss Function

[0224] In the process of training a deep neural network, because it is desired that the output of the deep neural network is as close as possible to the value that is truly intended to be predicted, the weight vector of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the truly intended target value (of course, there is usually an initialization process before the first update, i.e., pre-configuring parameters for each layer in the deep neural network), for example, if the predicted value of the network is too high, adjust the weight vector to make it predict lower, and constantly adjust until the deep neural network can predict the truly intended target value or a value very close to the truly intended target value. Therefore, it is necessary to define "how to compare the difference between the predicted value and the target value" in advance, which is the loss function or objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the deep neural network becomes a process of trying to minimize this loss.

[0225] Nowadays, a large model is often deployed on the terminal side, and a large model built-in on the terminal side is preferred during inference. However, due to the low computing power of the terminal side, the model on the cloud side needs to assist in inference. For example, the model is split, and the part of the model that does not require a large computing power overhead is placed on the terminal side for inference, and the part of the model that requires a large computing power overhead is placed on the cloud side for inference. Specifically, the terminal side performs an inference process related to the shallow part of the model, and the cloud side performs an inference process related to the deep part of the model.

[0226] However, this causes frequent data transmission between the terminal and the cloud, resulting in large inference delay and high cost.

[0227] To solve the above problems, an embodiment of the present application provides a data processing method. The data processing method of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0228] Referring to FIG. 7, FIG. 7 is a flowchart of a data processing method provided by an embodiment of the present application. As shown in FIG. 7, the data processing method provided by an embodiment of the present application can include steps 701 to 703, which will be described in detail below.

[0229] 701. Receive a first prompt prompt sent by a second device, wherein the first prompt is obtained according to an input request.

[0230] The first device can receive an input request, for example, the input request can be processing a long text. After receiving the input request, it can be converted into a prompt that can be processed by a large model. The essence of prompt is to unify downstream tasks into pre-training tasks, and to convert the data of downstream tasks into natural language form in a specific template.

[0231] In one possible implementation, the first device is a terminal device, and the second device is a server on the cloud side.

[0232] In one possible implementation, the first device is a terminal device, and the second device is a server on the edge side.

[0233] In one possible implementation, the first device is an edge device, and the second device is a server on the cloud side.

[0234] 702. Obtain a second prompt according to the first prompt, wherein the length of the second prompt is less than that of the first prompt, and the second prompt contains the semantics indicated in the first prompt.

[0235] The second prompt contains the semantics indicated in the first prompt, which can be understood as the semantic similarity between the second prompt and the first prompt is greater, and both are information describing the same semantics, but the length of the former is less than that of the latter. The semantics can be directly indicated (for example, indicated by a natural language description) or indirectly indicated (for example, indicated by a vector representation). The semantics indicated by the vector representation can be determined by a certain task network. Of course, the second prompt can not contain the semantics of the information irrelevant to the generation of the reply to the input request in the first prompt, that is, the second prompt contains the key semantics (that is, the semantics of the information related to the generation of the reply to the input request) indicated in the first prompt.

[0236] The idea of the embodiment of the application is to process the first prompt sent from the device with low computing power to convert it into a second prompt with smaller length. To ensure the calculation accuracy of the subsequent model, the second prompt still contains the semantics of the first prompt. Therefore, the end-side model only needs to process the shorter second prompt. In addition, processing the prompt with shorter length can also reduce the size of the output result, thereby reducing the calculation overhead of the end-side model in processing the prompt and reducing the inference delay while ensuring the accuracy of the result.

[0237] The second device performs semantic-preserving compression on the prompt of the first device. Since the first device processes the compressed prompt, the data processing capability of the first device is relatively low, which greatly reduces the computing power and time overhead required by the first device to process the prompt through the machine learning model, and ensures the processing accuracy (the compression is semantic-preserving compression). In addition, compared with the second device directly processing the prompt sent by the first device, the computing resources on the second device are usually more expensive than the computing resources on the first device in terms of the price paid by the user. Compared with the task of only compressing the prompt, the process of processing the prompt to obtain the reply to the input request requires a large number of iteration steps and requires more computing resources. Therefore, in the embodiment of the application, the second device is used for compression of the prompt, and the reply to the input request based on the prompt is obtained on the first device, which can reduce the cost caused by the computing power overhead on the second device.

[0238] In a possible implementation, the first prompt includes the input request, a requirement for the reply to the output, and context information to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

[0239] The prompt can include multiple parts, of which exemplary ones include an input request, which can be input information of the user, such as a question, the prompt can also include context information, for example, when the question is multi-turn, the context can be historical input information and output, or environmental information of the generated result, the prompt can also include conditions that need to be met for output, which can also be referred to as constraints on output, of course, the prompt can also include other types of information, which are related to the prompt template used to generate the prompt, and the above three examples can also overlap with each other, for example, the input request can also contain part of the requirements for the output.

[0240] In an embodiment of the present application, the second device can compress one or more of the above information when compressing the first prompt, for example, only compressing all or part of the input request, and keeping other information and the first information consistent, for example, only compressing the input request and the context information, and keeping other information and the first prompt consistent.

[0241] In a possible implementation, the terminal side can receive a user request, and the terminal side can determine whether the cloud side needs to optimize the prompt, for example, when the length is greater than a threshold, it is determined that the cloud side needs to optimize the prompt, and then the prompt to be optimized is sent to the cloud side.

[0242] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a second natural language description, which is a summarized description of the first natural language description.

[0243] Here, the so-called summary can be understood as summarizing the content of the natural language using shorter language, which can also be referred to as an abstract.

[0244] The first natural language description can be all or part of the first prompt.

[0245] For example, the first natural language description can be summarized by constructing a prompt using a language model, and the second prompt can also be referred to as a hard prompt.

[0246] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a vector representation of the first natural language description.

[0247] For example, the first natural language description can be summarized by constructing a prompt using a language model, and the second prompt can also be referred to as a hard prompt.

[0248] In a possible implementation, the first prompt includes a plurality of pieces of natural language description, and the second prompt includes a summarized description or a vector representation obtained by processing each of the pieces.

[0249] That is, each of the plurality of pieces in the first prompt can be processed to obtain a summarized description or a vector representation of each piece, and the summarized descriptions or vector representations are fused (for example, concatenated) to obtain the second prompt. That is, the semantics of each piece can be retained, and compared with summarizing or feature extraction on the whole, summarizing or feature extraction on the pieces and fusion can retain more complete semantics.

[0250] In a possible implementation, a third prompt can be constructed according to the first prompt, the third prompt indicating processing of each of a plurality of pieces of natural language description obtained by dividing the first prompt, a summarized description or a vector representation of each of the pieces can be obtained by a machine learning model according to the third prompt, and the second prompt can be obtained by concatenating the summarized descriptions or vector representations.

[0251] In a possible implementation, the first prompt can be split to obtain a plurality of pieces, for example, the first prompt can be split according to a template and structure of the prompt, or split according to an index of a token, for example, each 1k tokens are taken as a piece.

[0252] Referring to FIG. 8, FIG. 8 is a schematic of processing pieces of a prompt. The plurality of soft prompts generated can be directly concatenated, or concatenated according to a certain template format, and additional privacy data can be spliced on an end side.

[0253] Next, an optimization schematic of a prompt is given:

[0254] The first prompt can be optimized by a machine learning model, which can be a causal language model based on autoregressive generation or a conditional generation model. The causal language modeling (CLM) cloud side performs multiple inferences, and CLM focuses on generating a text sequence based on a given context. In this modeling method, the model attempts to predict the next word in a given context, which usually includes all words before the current word. This modeling method follows the causal principle, that is, the current word is only affected by the words before it, not by the words after it. Conditional generation (CG) only performs one inference, and CG focuses on generating a text sequence that meets certain conditions or constraints. In this modeling method, the model generates an output that meets the conditions based on the given input (which can be text, pictures, etc.). Unlike CLM, CG not only focuses on the coherence of the given context, but also focuses on meeting the predetermined task requirements.

[0255] In a possible implementation, the first device is configured to obtain, according to the second prompt, a reply to the input request by a first machine learning model, and the second device is configured to obtain, according to the first prompt, a second prompt by a second machine learning model.

[0256] The first machine learning model is obtained by compressing or knowledge distilling the second machine learning model.

[0257] Through compression or knowledge distillation, the first machine learning model can have similar data processing capabilities while being smaller in size than the second machine learning model.

[0258] In a possible implementation, the first machine learning model and the second machine learning model have the same training task when training.

[0259] For example, the training task is a fitting task Output, or a fitting compressed prompt (that is, the compressed prompt is used as the task itself).

[0260] In a possible implementation, obtaining the second prompt according to the first prompt includes compressing a first natural language description in the first prompt to obtain a second vector representation, adjusting the dimension of the second vector representation to obtain the first vector representation, and the second prompt includes the first vector representation.

[0261] Since the data processing capability of the first device is less than that of the second device, the scale of the model (the first machine learning model) deployed by the first device is usually smaller than the scale of the model (the second machine learning model) deployed by the second device. For models of different scales, the dimension size of the vector representation of the supported input (for example, the dimension size of an embedding vector of the input) is usually different. Therefore, the dimension of the vector representation obtained by the second device through the second machine learning model compression may be different from the dimension size of the vector representation of the input supported by the first machine learning model on the first device. In an embodiment of the present application, a dimension adjustment module (also referred to as a dimension adapter) can be deployed on the second device or the first device. The dimension adjustment module can adjust the dimension of the vector representation obtained by the second machine learning model to the dimension size of the vector representation of the input supported by the first machine learning model. For example, when the dimension adjustment module is deployed on the second device, the second device can compress the first prompt through the second machine learning model to obtain a second vector representation. Then, the second device can adjust the dimension of the second vector representation through the dimension adjustment module to obtain a second prompt containing a dimension-adjusted vector representation (a first vector representation). After that, the second device can send the second prompt containing the dimension-adjusted vector representation (the first vector representation) to the first device, and the first device can process the second prompt through the first machine learning model.

[0262] 703、sending the second prompt to the second device.

[0263] For example, referring to FIG. 9 and FIG. 10, FIG. 9 is an exemplary process of online and offline of a device, and FIG. 10 is an exemplary process of online of a device.

[0264] Wherein, online includes:

[0265] 1. The end side receives a user request;

[0266] 2. The predictor determines whether cloud assistance is needed;

[0267] 3. Synchronous end-side prefix matching, multiplexing historical shared prefixes;

[0268] 4. The prompt sent to the cloud side is optimized and compressed by KV Cache;

[0269] 5. End-side delivery;

[0270] 6. End-side inference based on optimized refine prompt;

[0271] Wherein, the main steps of offline include:

[0272] End side: timing trigger, send shared prefix and system prompt to the cloud, optimize and update, cloud side returns, end side updates KV Cache according to the prompt of the cloud side.

[0273] Referring to FIG. 11, FIG. 11 is a structural diagram of a data processing apparatus provided by an embodiment of the present application. As shown in FIG. 11, the data processing apparatus 1100 provided by an embodiment of the present application comprises:

[0274] The transceiver module 1101 is configured to receive a first prompt sent by a second device, wherein the first prompt is obtained according to an input request.

[0275] The prompt compression module 1102 is configured to obtain a second prompt according to the first prompt, wherein the length of the second prompt is less than that of the first prompt, and the second prompt contains the semantics indicated in the first prompt.

[0276] The transceiver module 1101 is further configured to send the second prompt to the second device, wherein the data processing capability of the first device is greater than that of the second device.

[0277] In a possible implementation, the first prompt comprises a plurality of prompt segments obtained by division, and the second prompt comprises compression results obtained by compressing each prompt segment in the plurality of prompt segments.

[0278] In a possible implementation, the prompt compression module is specifically configured to:

[0279] construct a third prompt according to the first prompt, wherein the third prompt indicates processing of each prompt segment obtained by dividing the first prompt;

[0280] obtain a compression result of each prompt segment through a language model according to the third prompt;

[0281] obtain the second prompt by fusing a plurality of compression results.

[0282] In a possible implementation, the first prompt comprises a first natural language description, and the second prompt comprises a second natural language description, wherein the second natural language description is a summarized description of the first natural language description.

[0283] In a possible implementation, the first prompt comprises a first natural language description, and the second prompt comprises a first vector representation of the first natural language description.

[0284] In a possible implementation, the prompt compression module is specifically configured to:

[0285] compressing the first natural language description in the first prompt to obtain a second vector representation;

[0286] The apparatus further includes:

[0287] a dimension adjustment module configured to adjust a dimension of the second vector representation to obtain the first vector representation, the second prompt including the first vector representation.

[0288] In a possible implementation, the processing module is specifically configured to:

[0289] The first device is an end-side device, and the second device is a cloud-side server; or

[0290] The first device is an end-side device, and the second device is an edge-side server; or

[0291] The first device is an edge-side device, and the second device is a cloud-side server.

[0292] In a possible implementation, the data processing capability is computing power.

[0293] In a possible implementation,

[0294] The first prompt includes the input request, a requirement for a reply to an output, and context information that needs to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

[0295] Embodiments of the present application also provide a data processing apparatus applied to a second device, the apparatus including:

[0296] a transceiving module configured to send a first prompt prompt to the second device, the first prompt being obtained according to an input request;

[0297] receive a second prompt sent by the second device according to the first prompt, the second prompt being shorter than the first prompt, and the second prompt containing semantics indicated by the first prompt;

[0298] a model inference module configured to obtain a reply to the input request by a first machine learning model according to the second prompt.

[0299] In a possible implementation, the input request contains privacy information; the first prompt does not contain the privacy information, and the first device is further configured to fuse the privacy information into the second prompt.

[0300] In a possible implementation, the first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description; and the apparatus further includes:

[0301] a dimension adjustment module configured to perform dimension adjustment on the first vector representation in the second prompt to obtain an adjusted second prompt;

[0302] The model inference module is specifically configured to:

[0303] obtain, according to the adjusted second prompt, a reply to the input request through a first machine learning model.

[0304] Next, an execution device provided in an embodiment of the present application is introduced. Referring to FIG. 12, FIG. 12 is a structural schematic diagram of an execution device provided in an embodiment of the present application. The execution device 1200 can be specifically a virtual reality (VR) device, a mobile phone, a tablet computer, a notebook computer, a smart wearable device, a monitoring data processing device, a server, or the like, which is not limited herein. Specifically, the execution device 1200 includes a receiver 1201, a transmitter 1202, a processor 1203, and a memory 1204 (wherein the number of processors 1203 in the execution device 1200 can be one or more, and one processor is taken as an example in FIG. 12). The processor 1203 can include an application processor 12031 and a communication processor 12032. In some embodiments of the present application, the receiver 1201, the transmitter 1202, the processor 1203, and the memory 1204 can be connected through a bus or other means.

[0305] The memory 1204 can include a read-only memory and a random access memory, and provide the processor 1203 with instructions and data. A part of the memory 1204 can also include a non-volatile random access memory (NVRAM). The memory 1204 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or expanded sets thereof, wherein the operation instructions can include various operation instructions for implementing various operations.

[0306] The processor 1203 controls the operation of the execution device. In specific applications, various components of the execution device are coupled together through a bus system, which can include a data bus, a power supply bus, a control bus, and a state signal bus, etc. However, for the sake of clarity, all kinds of buses are referred to as a bus system in the figure.

[0307] The method disclosed in the embodiments of the present application can be applied to the processor 1203 or implemented by the processor 1203. The processor 1203 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor 1203 or by instructions in the form of software. The processor 1203 described above can be a general processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The processor 1203 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or the like mature storage medium in the art. The storage medium is located in the memory 1204, and the processor 1203 reads the information in the memory 1204 and combines the hardware to complete the steps related to the model inference process in the above method.

[0308] The receiver 1201 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1202 can be used to output digital or character information through the first interface; the transmitter 1202 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1202 can also include a display device such as a display screen.

[0309] The embodiment of the present application further provides a server device. Please refer to FIG. 13, which is a structural schematic diagram of a server according to the embodiment of the present application. Specifically, the server 1300 is implemented by one or more servers. The server 1300 can be quite different due to different configurations or performances. The server 1300 can include one or more central processing units (CPUs) 1313 (for example, one or more processors) and a memory 1332, and one or more storage media 1330 (for example, one or more mass storage devices) for storing application programs 1342 or data 1344. The memory 1332 and the storage media 1330 can be temporary storage or persistent storage. The programs stored in the storage media 1330 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Furthermore, the central processing unit 1313 can be configured to communicate with the storage media 1330 and execute the series of instruction operations in the storage media 1330 on the server 1300.

[0310] The server 1300 can further include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358, or one or more operating systems 1341, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0311] In the embodiment of the present application, the central processing unit 1313 is used to execute the data processing method in the above embodiment.

[0312] The embodiment of the present application further provides a computer program product, which, when running on a computer, causes the computer to execute the steps performed by the above execution device, or causes the computer to execute the steps performed by the above training device.

[0313] The embodiment of the present application further provides a computer readable storage medium, which stores a program for signal processing. When the program runs on a computer, the computer is caused to execute the steps performed by the above execution device, or the computer is caused to execute the steps performed by the above training device.

[0314] The execution device, the training device or the terminal device provided in the embodiments of the present application can specifically be a chip, which comprises a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the execution device executes the data processing method described in the above embodiments, or so that the chip in the training device executes the data processing method described in the above embodiments. Alternatively, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0315] Specifically, please refer to FIG. 14, which is a structural schematic diagram of a chip provided in the embodiments of the present application. The chip can be represented as a neural network processor NPU 1400, which is mounted on a host CPU (Host CPU) as a coprocessor and is assigned tasks by the Host CPU. The core part of the NPU is an operation circuit 1403, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1404.

[0316] In some implementations, the operation circuit 1403 internally comprises a plurality of processing units (PEs). In some implementations, the operation circuit 1403 is a two-dimensional systolic array. The operation circuit 1403 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1403 is a general-purpose matrix processor.

[0317] For example, it is assumed that there are an input matrix A, a weight matrix B and an output matrix C. The operation circuit takes corresponding data of the matrix B from the weight memory 1402 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1401 and performs matrix operation with the matrix B, and the partial result or final result of the obtained matrix is saved in an accumulator 1408.

[0318] The unified memory 1406 is used to store input data and output data. The weight data is transferred to the weight memory 1402 through a direct memory access controller (DMAC) 1405. The input data is also transferred to the unified memory 1406 through the DMAC.

[0319] BIU is Bus Interface Unit, which is used for the interaction between AXI bus and DMAC and instruction fetch buffer (IFB) 1409.

[0320] The bus interface unit 1410 (Bus Interface Unit, referred to as BIU) is used for the instruction fetch buffer 1409 to obtain instructions from the external memory, and is also used for the storage unit access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0321] The DMAC is mainly used to carry the input data in the external memory DDR to the unified memory 1406, or carry the weight data to the weight memory 1402, or carry the input data to the input memory 1401.

[0322] The vector calculation unit 1407 includes a plurality of operation processing units, which further process the output of the operation circuit 1403 as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolution / full connection layer network calculation in neural network, such as batch normalization, pixel-level summation, upsampling of feature plane, etc.

[0323] In some implementations, the vector calculation unit 1407 can store the processed output vector to the unified memory 1406. For example, the vector calculation unit 1407 can apply a linear function; or, a nonlinear function to the output of the operation circuit 1403, such as linear interpolation on the feature plane extracted by the convolution layer, and for example, a vector of accumulated values to generate activation values. In some implementations, the vector calculation unit 1407 generates normalized values, pixel-level summation values, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1403, for example, for use in subsequent layers in the neural network.

[0324] The controller 1404 is connected to the instruction fetch buffer 1409, which is used to store instructions used by the controller 1404;

[0325] The unified memory 1406, the input memory 1401, the weight memory 1402, and the instruction fetch buffer 1409 are all on-chip memories. The external memory is private to the NPU hardware architecture.

[0326] The processor mentioned in any of the above can be a general central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling execution of the above program.

[0327] It should be noted that the above-described apparatus embodiments are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiments provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0328] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware, and of course it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including a plurality of instructions for making a computer device (which can be a personal computer, training device, or network device, etc.) execute the methods described in various embodiments of the present application.

[0329] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part.

[0330] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

Claims

1. A data processing method, characterized by, Applied to a first device, the method comprises: receiving a first prompt sent by a second device, the first prompt being obtained according to an input request; obtaining a second prompt according to the first prompt, the second prompt being shorter than the first prompt and containing semantics indicated by the first prompt; sending the second prompt to the second device, wherein the data processing capability of the first device is greater than that of the second device.

2. The method of claim 1, wherein, The first prompt comprises a plurality of prompt segments obtained by division, and the second prompt comprises compression results obtained by compressing each of the plurality of prompt segments.

3. The method of claim 1, wherein, The second prompt is obtained according to the first prompt, comprising: constructing a third prompt according to the first prompt, the third prompt indicating processing of each of a plurality of prompt segments obtained by dividing the first prompt; obtaining a compression result of each of the prompt segments through a language model according to the third prompt; obtaining the second prompt by fusing a plurality of compression results.

4. The method according to any one of claims 1 to 3, characterized in that, The first prompt comprises a first natural language description, and the second prompt comprises a second natural language description which is a summarized description of the first natural language description.

5. The method according to any one of claims 1 to 4, characterized in that, The first prompt comprises a first natural language description, and the second prompt comprises a first vector representation of the first natural language description.

6. The method of claim 5, wherein, The second prompt is obtained according to the first prompt, comprising compressing the first natural language description in the first prompt to obtain a second vector representation; dimensionally adjusting the second vector representation to obtain the first vector representation, and the second prompt comprises the first vector representation.

7. The method of any one of claims 1-5, wherein: the first device is an end-side device, and the second device is a cloud-side server; or the first device is an end-side device, and the second device is an edge-side server; or the first device is an edge-side device, and the second device is a cloud-side server.

8. The method according to any one of claims 1 to 7, characterized in that, The data processing capability is computing power.

9. The method according to any one of claims 1 to 8, characterized in that, The first prompt comprises the input request, a requirement for a reply to an output, and context information to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

10. A system, characterized by The first device and the second device are in communication, and the data processing capability of the first device is greater than that of the second device; The first device is configured to send a first prompt to the second device, the first prompt being obtained according to an input request; The second device is configured to obtain a second prompt according to the first prompt; The second prompt is shorter than the first prompt and contains semantics indicated by the first prompt; The second device is configured to send the second prompt to the second device; The first device is configured to obtain a reply to the input request through a first machine learning model according to the second prompt.

11. The system of claim 10, wherein, The second device is specifically configured to obtain a second prompt by a second machine learning model according to the first prompt. The first machine learning model is obtained by compressing or knowledge distilling the second machine learning model.

12. The system of claim 10 or 11, wherein, The first machine learning model and the second machine learning model have the same training task during training.

13. The system of any of claims 10 to 12, wherein, The first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description. The first device is further configured to adjust the dimension of the first vector representation in the second prompt to obtain an adjusted second prompt. The first device is specifically configured to obtain a reply to the input request by a first machine learning model according to the adjusted second prompt.

14. A data processing method, characterized by, The method applied to the second device includes: sending a first prompt prompt to a first device, the first prompt being obtained according to an input request; receiving a second prompt sent by the first device according to the first prompt, the length of the second prompt being less than that of the first prompt, and the second prompt containing the semantics indicated by the first prompt; obtaining a reply to the input request by a first machine learning model according to the second prompt; wherein the data processing capacity of the first device is greater than that of the second device.

15. The method of claim 14, wherein, The input request contains privacy information; the first prompt does not contain the privacy information, and the first device is further configured to fuse the privacy information into the second prompt.

16. The method according to claim 14 or 15, characterized in that The first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description; the method further includes: adjusting the dimension of the first vector representation in the second prompt to obtain an adjusted second prompt; obtaining a reply to the input request by a first machine learning model according to the second prompt, including: obtaining a reply to the input request by a first machine learning model according to the adjusted second prompt.

17. A data processing apparatus, characterized by The device applied to the first device includes: a transceiver module for receiving a first prompt prompt sent by a second device, the first prompt being obtained according to an input request; a prompt compression module for obtaining a second prompt according to the first prompt; the length of the second prompt is less than that of the first prompt, and the second prompt contains the semantics indicated in the first prompt; the transceiver module is further configured to send the second prompt to the second device; wherein the data processing capacity of the first device is greater than that of the second device.

18. The apparatus of claim 17, wherein, The first prompt includes a plurality of prompt segments obtained by division, and the second prompt includes compression results obtained by compressing each prompt segment in the plurality of prompt segments.

19. The apparatus of claim 17, wherein, The prompt compression module is specifically configured to: construct a third prompt according to the first prompt, the third prompt indicating processing of each prompt segment obtained by dividing the first prompt; obtain a compression result of each prompt segment by a language model according to the third prompt; The second prompt is obtained by fusing a plurality of the compression results.

20. The apparatus of any one of claims 17 to 19, wherein, The first prompt includes a first natural language description, and the second prompt includes a second natural language description which is a summarized description of the first natural language description.

21. The apparatus of any one of claims 17 to 20, wherein, The first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description.

22. The apparatus of any one of claims 17 to 21, wherein, The prompt compression module is specifically configured to: compress the first natural language description in the first prompt to obtain a second vector representation; The device further includes: a dimension adjustment module configured to adjust the dimension of the second vector representation to obtain the first vector representation, and the second prompt includes the first vector representation.

23. The apparatus of any one of claims 17 to 22, wherein, The processing module is specifically configured to: The first device is an end-side device, and the second device is a cloud-side server; or The first device is an end-side device, and the second device is an edge-side server; or The first device is an edge-side device, and the second device is a cloud-side server.

24. The apparatus of any one of claims 17 to 23, wherein, The data processing capability is computing power.

25. The apparatus of any one of claims 17-24, wherein The first prompt includes the input request, a requirement for a reply to the output, and context information that needs to be referred to; and the second prompt is obtained by compressing one or more of the input request, the requirement for the output, and the context information.

26. A data processing apparatus, characterized by The method is applied to a second device, and the method includes: a transceiver configured to send a first prompt prompt to a second device, the first prompt being obtained according to an input request; receive a second prompt sent by the second device according to the first prompt, the length of the second prompt being less than that of the first prompt, and the second prompt containing the semantics indicated by the first prompt; a model inference module configured to obtain a reply to the input request by a first machine learning model according to the second prompt.

27. The apparatus of claim 26, wherein, The input request contains privacy information; the first prompt does not contain the privacy information, and the first device is further configured to fuse the privacy information into the second prompt.

28. The apparatus of claim 26 or 27, wherein, The first prompt includes a first natural language description, and the second prompt includes a first vector representation of the first natural language description; and the method further includes: a dimension adjustment module configured to adjust the dimension of the first vector representation in the second prompt to obtain an adjusted second prompt; The model inference module is specifically configured to: obtain a reply to the input request by a first machine learning model according to the adjusted second prompt.

29. A computer storage medium, comprising, The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1-9, 14-16.

30. A computer program product, characterised in that, The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1-9, 14-16. The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1-9, 14-16.

31. A system, comprising: comprising at least one processor, at least one memory; the processor, the memory are connected through a communication bus and complete mutual communication; the at least one memory is used for storing code; the at least one processor is used for executing the code to execute the method as claimed in any one of claims 1-9, 14-16.

32. A chip, comprising: comprising at least one processing unit and interface circuit, the interface circuit is used for providing program instruction or data for the at least one processing unit, the at least one processing unit is used for executing the program instruction to realize the method as claimed in any one of claims 1-9, 14-16.

Images