Data processing method and device, chip and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and device, a chip and electronic equipment. The method is applied to terminal equipment and comprises the following steps: converting a natural language input by a user into an embedded vector; outputting the embedded vector; receiving an information vector, wherein the information vector is a vector generated by reasoning a model trunk of a large language model (LLM) according to the embedded vector; and reasoning according to the information vector to obtain a character reply or an instruction request corresponding to the information vector. According to the method provided by the embodiment of the invention, the terminal equipment does not output the character information input by the user and the user information, so that the privacy of the user is effectively guaranteed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, chip and electronic device. Background Art

[0002] Large Language Model (LLM) refers to a neural network language model with a very large number of parameters (generally greater than 1 billion). With the advancement of model training methods, the parameter scale of language models continues to increase, and the application ability of LLM in real-world scenarios has been confirmed. LLM can be used to write / repair code, text summaries, answer basic questions, copywriting, etc. Its interactive method using natural language has also greatly reduced the threshold for use.

[0003] Although LLM has attracted a lot of attention due to its convenient interactive mode and wide range of usage scenarios, due to its large number of parameters, running such models is still mainly done on cloud servers.

[0004] Since LLM runs on cloud servers, relevant user information must be uploaded to the cloud when using LLM, which is not conducive to information security and privacy protection.

[0005] Therefore, a new data processing method is needed to improve information security and privacy protection in the process of using LLM. Summary of the invention

[0006] In order to improve information security and privacy protection in the process of using LLM, the present application provides a data processing method and an electronic device, and the present application also provides a computer-readable storage medium.

[0007] The present application embodiment adopts the following technical solutions:

[0008] In a first aspect, the present application provides a data processing method, which is applied to a terminal device, and the method includes:

[0009] Convert the natural language input by the user into an embedding vector;

[0010] Outputting the embedding vector;

[0011] receiving an information vector, wherein the information vector is a vector generated by a model backbone of a large language model (LLM) performing reasoning based on the embedding vector;

[0012] Inference is performed based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

[0013] According to the method of the first aspect, the terminal device will not output the text information and user information input by the user, so that the user's privacy is effectively protected.

[0014] In an implementation of the first aspect:

[0015] The natural language is converted into the embedding vector using an input module, wherein the input module is deployed in the terminal device and is a module cut out from an input end of the LLM.

[0016] In an implementation of the first aspect: the input module includes a word segmenter of the LLM.

[0017] In an implementation of the first aspect:

[0018] The information vector is converted into a service or answer provided to the user by using an output module, wherein the output module is deployed in the terminal device and is a module cut out from the output end of the LLM.

[0019] In an implementation of the first aspect: the output module includes a decoder of the LLM.

[0020] In an implementation of the first aspect, the method further includes:

[0021] The output module is trained according to the user's usage habits for the terminal device.

[0022] According to the method of the above implementation mode, the output module deployed on the terminal device is trained according to the user's usage habits of the terminal device, so that the service output of the output module is more in line with the user's usage habits, thereby improving the user experience of the LLM service.

[0023] In an implementation of the first aspect, the method further includes:

[0024] Output the text reply.

[0025] In an implementation of the first aspect, the method further includes:

[0026] According to the instruction request, information data and / or application are called to generate a service or answer provided to the user, wherein the information data is stored on the terminal device and the application is installed on the terminal device.

[0027] According to the above-mentioned implementation method, when implementing the LLM service, the output module deployed on the terminal device calls the information data stored on the terminal device, which reduces the difficulty of accessing the information data, avoids the leakage of the information data on the terminal device, and ensures data security.

[0028] According to the method of the above implementation mode, when implementing the LLM service, the output module deployed on the terminal device calls the application installed on the terminal device. The output module can directly obtain the application installation status on the terminal device, so that according to the application installation status on the terminal device, an application that meets the functional requirements can be selected from multiple applications installed on the terminal device. This not only reduces the difficulty of application calling, but also makes the final called application more in line with the application installation status of the terminal device, thereby improving the user experience.

[0029] In a second aspect, the present application further proposes a data processing method, which is applied to a cloud server, and the cloud server is loaded with a model backbone of a large language model (LLM), and the method comprises:

[0030] Receiving an embedding vector, where the embedding vector is a vector generated by converting a natural language input by a user;

[0031] Using the model backbone of the LLM, reasoning based on the embedding vector to generate an information vector;

[0032] The information vector is output.

[0033] According to the method of the second aspect, the cloud server will not directly receive the text information and user information input by the user, and the cloud server will not obtain the text information and user information input by the user through decryption or data parsing, so that the user's privacy is effectively protected.

[0034] In a third aspect, the present application further proposes a data processing device, which is applied to a terminal device, and comprises:

[0035] An input module, which is used to convert the natural language input by the user into an embedding vector;

[0036] A sending module, configured to output the embedding vector;

[0037] A receiving module, configured to receive an information vector, wherein the information vector is a vector generated by a model backbone of a large language model (LLM) performing reasoning based on the embedding vector;

[0038] The output module is used to perform reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

[0039] In an implementation of the third aspect:

[0040] The input module is a module separated from the input end of the LLM.

[0041] In an implementation of the third aspect: the input module includes a word segmenter of the LLM.

[0042] In an implementation of the third aspect:

[0043] The output module is a module separated from the output end of the LLM.

[0044] In an implementation of the third aspect: the output module includes a decoder of the LLM.

[0045] In an implementation of the third aspect, the device further includes:

[0046] A training module is used to train the output module according to the user's usage habits for the terminal device.

[0047] In an implementation of the third aspect, the output module is further used to:

[0048] Output the text reply.

[0049] In an implementation of the third aspect, the output module is further used to:

[0050] According to the instruction request, information data and / or application are called to generate a service or answer provided to the user, wherein the information data is stored on the terminal device and the application is installed on the terminal device.

[0051] In a fourth aspect, the present application further proposes a data processing device, which is applied to a cloud server and includes:

[0052] A receiving module, configured to receive an embedding vector, wherein the embedding vector is a vector generated by converting a natural language input by a user;

[0053] An inference module, the inference module comprising a model backbone of a large language model (LLM), configured to generate an information vector by performing inference based on the embedding vector using the model backbone of the LLM;

[0054] A sending module is used to output the information vector.

[0055] In a fifth aspect, the present application also proposes a large language model (LLM), wherein the LLM comprises:

[0056] An input module, which is used to convert the natural language input by the user into an embedding vector, wherein the input module is deployed on a terminal device;

[0057] A model backbone, which is used to perform reasoning based on the embedding vector to generate an information vector, wherein the model backbone is deployed on a cloud server;

[0058] An output module is used to perform reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector, wherein the output module is deployed on a terminal device.

[0059] In a sixth aspect, the present application further proposes an electronic chip, comprising a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic chip is triggered to execute the method steps described in the first aspect or the second aspect.

[0060] In the seventh aspect, the present application also proposes an electronic device, comprising a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps described in the first aspect or the second aspect.

[0061] In an eighth aspect, the present application further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The figure is a schematic diagram of an application scenario according to an embodiment of the present application;

[0063] Figure 2 The figure is a schematic diagram of an LLM application scenario according to an embodiment of the present application;

[0064] Figure 3 FIG. 1 is a schematic diagram of the LLM model structure according to an embodiment of the present application;

[0065] Figure 4 The figure is a schematic diagram of the LLM service process according to an embodiment of the present application;

[0066] Figure 5 The figure is a schematic diagram of an LLM application scenario according to an embodiment of the present application;

[0067] Figure 6 The figure is a schematic diagram of an LLM application scenario according to an embodiment of the present application;

[0068] Figure 7 The figure is a schematic diagram of an LLM application scenario according to an embodiment of the present application;

[0069] Figure 8 is a schematic diagram of a device structure according to an embodiment of the present application;

[0070] Fig. 9is a schematic diagram of a device structure according to an embodiment of the present application;

[0071] Fig.10 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0073] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0074] Figure 1 Shown is a schematic diagram of an application scenario according to an embodiment of the present application.

[0075] like Figure 1 As shown, the terminal device 100 is connected to the cloud server 110 .

[0076] LLM111 is deployed on the cloud server 110 .

[0077] If the terminal device 100 needs to use the service of LLM111, the terminal device 100 needs to send text information (for example, customer input) in the form of a string to the cloud server 110. LLM111 processes the string sent by the terminal device 100 and generates a processing result, which is fed back to the terminal device 100.

[0078] In the above service process, the terminal device 100 transmits direct text messages to the cloud server 110, and in some cases even transmits user information on the terminal device 100 to the cloud server 110. These actions will increase the risk of user information exposure and do not provide adequate protection for user privacy.

[0079] Furthermore, the application framework allows LLM to search for external information. When LLM111 provides services, it may be necessary to involve information data stored on the terminal device 100 (for example, photos, itinerary records, etc.). For example, if a command such as "find photos of me attending a meeting at venue A" is issued on the terminal device 100, LLM111 needs to search for photos on the terminal device 100; for another example, if a command such as "call up my itinerary for tomorrow afternoon at 2 o'clock" is issued on the terminal device 100, LLM111 needs to retrieve the user's itinerary on the terminal device 100.

[0080] When LLM111 needs to involve information stored on the terminal device 100 when providing services, the terminal device 100 needs to open data access rights to LLM111, or the terminal device 100 needs to upload all information involved in data processing to the cloud server 110, otherwise the service will not be able to proceed smoothly. This will not only occupy a large amount of communication resources and increase the difficulty of LLM111 in performing services, but also may cause the information on the terminal device 100 to be leaked, affecting data security.

[0081] Furthermore, the application framework allows LLM to call external software interfaces. When LLM111 provides services, it may be necessary to call applications installed on the terminal device 100. Since different terminal devices have different applications installed on them, LLM111 deployed on the cloud server 110 cannot know the application installation status of the terminal device 100, so it is difficult for LLM111 to select the best application to call from the terminal device 100. For example, if an instruction is issued on the terminal device 100 such as: "Order me a bowl of braised chicken with rice, and deliver it at 12 o'clock". LLM111 needs to call a food delivery software on the terminal device 100. Since there are multiple different food delivery software on the market, LLM111 cannot know which food delivery software is installed on the terminal device 100, and thus cannot call the food delivery software.

[0082] Furthermore, LLM111 deployed on the cloud server 110 provides services for multiple terminal devices, that is, LLM111 provides templated, common services. It is difficult for LLM111 to optimize for a certain terminal device and provide restrained services. For example, multiple food delivery apps may be installed on the terminal device 100 at the same time, and LLM111 may not be able to accurately know which food delivery app installed on the terminal device 100 the user of the terminal device 100 wants to use (involving membership, discounts, etc.), resulting in difficulty in application calling.

[0083] In view of the problems existing in the above-mentioned LLM service, an embodiment of the present application proposes a method for data processing using LLM. In the method of the embodiment of the present application, the LLM is not deployed as a whole on the cloud server, but is split and deployed on the terminal device and the cloud server respectively, so that there is no direct interaction of text information and user information between the terminal device and the cloud server, reducing the risk of exposure of text information and user information, and providing protection for user privacy.

[0084] In the embodiment of the present application, the terminal device is a device that can perform wired or wireless communication with the cloud server to achieve data interaction. The terminal device is used to obtain user input and provide services to the user.

[0085] Terminal equipment can be user equipment (UE), mobile station (MS), mobile terminal (MT), customer premise equipment (CPE), etc. Terminal equipment can be widely used in various scenarios, such as device-to-device (D2D), vehicle to everything (V2X) communication, machine-type communication (MTC), Internet of things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wear, smart transportation, smart city, etc.

[0086] For example, the terminal device may be a handheld device, a vehicle-mounted device, a machine type communication (MTC) terminal, etc. The terminal device may be a personal computer, a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, or a wireless terminal in smart home, etc. For example, the wireless terminal in self-driving may be a drone, a helicopter, or an airplane, etc. For example, the wireless terminal in the Internet of Vehicles may be a vehicle-mounted device, a vehicle-mounted device, a vehicle-mounted module, a vehicle, or a ship, etc. The wireless terminal in industrial control may be a camera, a robot, or a robotic arm, etc. The wireless terminals in a smart home can be TVs, air conditioners, vacuum cleaners, speakers, or set-top boxes.

[0087] It should be noted that in the embodiments of the present application, the terminal device may be a device or apparatus with a chip, or a device or apparatus with integrated circuits, or a chip, module or control unit in the device or apparatus shown above, and the present application does not specifically limit this.

[0088] In the embodiment of the present application, the cloud server is a device that can perform wired or wireless communication with the terminal device to achieve data interaction. The cloud server can load the LLM model to provide remote LLM services for the terminal device.

[0089] The cloud server can be a computer device directly connected to the terminal device via wireless or wired connection; the cloud server can also be a computer device connected to the terminal device via a network. The cloud server can be a single computer device, multiple interconnected computer devices, or a virtual device created based on a single computer device or multiple interconnected computer devices.

[0090] It should be noted that in the embodiments of the present application, the cloud server may be a device or apparatus with a chip, or a device or apparatus with integrated circuits, or a chip, module or control unit in the device or apparatus shown above, and the present application does not make any specific limitation.

[0091] Figure 2 Shown is a schematic diagram of an LLM application scenario according to an embodiment of the present application.

[0092] like Figure 2 As shown, in one embodiment, LLM200 is divided into three parts: an input module 210, a LLM backbone 220 and an output module 230.

[0093] The input module 210 is used to convert the input natural language into an embedding vector.

[0094] For example, in one embodiment, the input module 210 converts the input natural language into token embedding.

[0095] In the embodiments of this specification, a token is a basic unit of model input. Specifically, a "token" refers to a basic unit in a text, which can usually be a word, a phrase, a punctuation mark, a character, etc., depending on the needs and methods of text processing. For example, in the Chinese Bidirectional Encoder Representation from Transformers (BERT), a token can be a word or an identifier.

[0096] Embedding is a method of mapping words or other text units in a text into a continuous vector space. This representation method can convert words or other text units in a text into real number vectors so that computers can process them. The goal of embedding is to represent the words in a text as a set of continuous vectors, so that words with similar semantics are closer in the vector space, while words with dissimilar semantics are farther away in the vector space. This continuous vector representation can capture the semantic and grammatical relationships between words, so that it can better represent the similarities and differences between words in text processing tasks.

[0097] The token itself is not computable. In machine learning and deep learning, text usually needs to be converted into digital representation before it can be passed to the algorithm as input for processing. That is, the token needs to be mapped to a continuous vector space before subsequent operations can be performed. The result of this mapping is the embedding corresponding to the token, that is, Token Embedding. TokenEmbedding is a dense vector used to represent a token.

[0098] In other embodiments, the input module 210 may also convert the input natural language into other types of embedding vectors.

[0099] The present application does not specifically limit the structure of the input module 210. Those skilled in the art can configure the LLM 200 according to actual needs and separate the input module 210 from the LLM 200. For example, in one embodiment, the input module 210 includes a tokenizer.

[0100] The LLM backbone network 220 is the model backbone of the LLM 200, which is used to process complex inputs and output processed information vectors. Specifically, the LLM backbone network 220 performs reasoning based on the input embedding vectors to generate reasoning results, which are information vectors.

[0101] The present application does not specifically limit the structure of the LLM backbone network 220. Those skilled in the art can configure the LLM 200 according to actual needs and divide the LLM backbone network 220 from the LLM 200.

[0102] The output module 230 is used to convert the information vector output by the LLM backbone network 220 into a service or answer provided to the user.

[0103] The present application does not specifically limit the structure of the output module 230. Those skilled in the art can configure the LLM 200 according to actual needs and separate the output module 230 from the LLM 200. For example, in one embodiment, the output module 230 includes a classifier and a decoder.

[0104] For example, Figure 3 Shown is a schematic diagram of the LLM model structure according to an embodiment of the present application.

[0105] Taking the decoder-only transformer used by models such as LLaMA, Vicuna, and GPT-3 as an example, the LLM model structure is as follows Figure 3 shown.

[0106] The input module 310 of the LLM 300 is a tokenizer. When the LLM 300 performs services, the input module 310 obtains natural language information input by the user, and the natural language information input by the user is processed by the input module 310 and converted into a string of token embeddings.

[0107] The Token Embedding output by the input module 310 is input to the LLM backbone network 320 of the LLM 300. The LLM backbone network 320 includes a plurality of transformation blocks (Transformer Block 1-N). The Token Embedding input to the LLM backbone network 320 is processed by Transformer Block 1-N to generate a string of information vectors (Information Vectors).

[0108] The Information Vectors generated by the LLM backbone network 320 are input to the output module 330. The output module 330 is composed of a token classifier and a token decoder. The Information Vectors are inferred by the output module 330 to obtain the corresponding text reply or instruction request.

[0109] For example, in an application scenario, the user inputs "I want to listen to A1 A2 A3" (A1A2A3 refers to the name of a singer (the name is three characters)). "I want to listen to A1 A2 A3" is processed by the input module 310 and converted into the corresponding TokenEmbedding (X1~X6). The LLM backbone network 320 performs reasoning based on the Token Embedding and generates the corresponding Information Vectors (Y1~Y6). The output module 330 performs reasoning based on the Information Vectors and generates an instruction request "Play B1B2 now" (B1B2 refers to the name of a song by singer A1 A2A3 (two characters)).

[0110] Further, such as Figure 2 As shown, the terminal device 201 is connected to the cloud server 202 via a network. The input module 210 and the output module 230 are deployed on the terminal device 201, and the LLM backbone network 220 is deployed on the cloud server 202.

[0111] Figure 4 The figure is a schematic diagram of the LLM service process according to an embodiment of the present application.

[0112] exist Figure 2 When the LLM200 is serviced, execute Figure 4 The following process is shown.

[0113] S400, the terminal device 201 obtains natural language information (eg, text information) input by a user.

[0114] S410, the input module 210 processes the natural language information input by the user and generates a corresponding embedding vector.

[0115] S420, the terminal device 201 sends the embedding vector output by the input module 210 to the cloud server 202 through the network.

[0116] S421 , the cloud server 202 receives the embedding vector and inputs the embedding vector into the LLM backbone network 220 .

[0117] S430, the LLM backbone network 220 performs reasoning according to the embedded vector to generate a reasoning result, which is an information vector (Information Vectors).

[0118] S440, the cloud server 202 sends the information vector generated by the LLM backbone network 220 to the terminal device 201 through the network.

[0119] S441 , the terminal device 201 receives the information vector and inputs the information vector to the output module 230 .

[0120] S450, the output module 230 performs reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

[0121] In the above service process, the terminal device 201 sends the embedding vector to the cloud server 202 through the network; the cloud server 202 sends the information vector to the terminal device 201 through the network. The embedding vector and information vector transmitted on the network do not contain direct text information and user information, and the cloud server side will not obtain direct text information and user information, so the user's privacy is effectively protected.

[0122] According to the method of the embodiment of the present application, the terminal device will not output the text information and user information input by the user, the cloud server will not directly receive the text information and user information input by the user, and the cloud server will not obtain the text information and user information input by the user through decryption or data parsing, so that the user's privacy is effectively protected.

[0123] Furthermore, in one embodiment, for different tasks, different functional modules are added to the output module 230 to achieve the purpose of processing different requirements.

[0124] For example, if the output module 230 infers a text reply in S450, then after S450, the output module 230 directly outputs the text reply to display to the user. If the output module 230 infers an instruction request in S450, then after S450, the output module 230 calls the relevant resources on the terminal device 201 to execute the instruction request.

[0125] Figure 5 Shown is a schematic diagram of an LLM application scenario according to an embodiment of the present application.

[0126] like Figure 5 As shown, in one embodiment, LLM500 is divided into three parts: input module 510, LLM backbone network 520, and output module 530. Terminal device 501 is connected to cloud server 502 via a network. Input module 510 and output module 530 are deployed on terminal device 501, and LLM backbone network 520 is deployed on cloud server 502.

[0127] When LLM500 is providing services, the terminal device 501 obtains the natural language information input by the user. The input module 510 processes the natural language information input by the user and generates a string of token embeddings. The terminal device 501 sends the Token Embedding output by the input module 510 to the cloud server 502 through the network. The cloud server 502 inputs the Token Embedding into the LLM backbone network 520. The LLM backbone network 520 performs reasoning based on the Token Embedding and generates a reasoning result, which is an information vector. The cloud server 502 sends the Information Vectors generated by the LLM backbone network 520 to the terminal device 501 through the network. The terminal device 501 inputs the Information Vectors into the output module 530. The output module 530 performs reasoning based on the Information Vectors and obtains the reasoning result of the corresponding Information Vectors.

[0128] When the inference result generated by the output module 530 is a text reply, the output module 530 outputs the text reply to the display module 503 of the terminal device 501 for text display.

[0129] For example, the natural language information input by the user is "the largest mammal in the world". The inference result generated by the output module 530 is a text reply "the largest mammal in the world is the blue whale". The output module 530 outputs the text reply "the largest mammal in the world is the blue whale" to the display module 503, and the display module 503 displays "the largest mammal in the world is the blue whale".

[0130] When the inference result generated by the output module 530 is an instruction request, the output module 530 calls the relevant resources (eg, application programs and / or information data) on the terminal device 501 to execute the instruction request according to the instruction request.

[0131] In one embodiment, the output module 530 accesses the information data 504 stored on the terminal device 501 according to the instruction request, and generates a service output according to the information data 504.

[0132] For example, the natural language information input by the user is "a meeting scheduled at 3:30", and the inference result generated by the output module 530 is an instruction request "retrieve the travel record data of time (3:30) and type (meeting) from the user's travel schedule, and display the search results". According to the instruction request, the output module 530 accesses the user's travel schedule stored on the terminal device 501, searches with time (3:30) and type (meeting) as search conditions, and outputs the retrieved travel record data to the display module 503 for display.

[0133] Since the output module 530 is deployed on the terminal device 501, when the output module 530 accesses the information data 504 stored on the terminal device 501, the terminal device 501 does not need to open the remote access rights to the information data 504, and the terminal device 501 does not need to remotely transmit the information data 504. This reduces the difficulty for the output module 530 to access the information data 504, avoids the leakage of the information data 504 on the terminal device 100, and ensures data security.

[0134] According to the method of the embodiment of the present application, when implementing the LLM service, the output module deployed on the terminal device calls the information data stored on the terminal device, which reduces the difficulty of accessing the information data, avoids the leakage of the information data on the terminal device, and ensures data security.

[0135] In another embodiment, the output module 530 calls the application 505 that implements the instruction request from multiple applications installed on the terminal device 501 according to the instruction request, and runs the application 505 to generate service output.

[0136] For example, the natural language information input by the user is "order a coffee takeaway", and the inference result generated by the output module 530 is the instruction request "start the takeaway software and search for coffee takeaway in the takeaway software". According to the instruction request, the output module 530 determines the takeaway application 505 from multiple applications installed on the terminal device 501, starts the takeaway application 505, and enters "coffee" in the order interface of the takeaway application 505 for search. Finally, the terminal device 501 displays the order interface of the takeaway application 505 to the user, and the order interface displays the coffee takeaway order, waiting for the user to confirm the order.

[0137] Since the output module 530 is deployed on the terminal device 501, when the output module 530 needs to call an application for implementing a certain function, the output module 530 can directly obtain the application installation status on the terminal device 501, and thus can select the application 505 that meets the functional requirements from the multiple applications installed on the terminal device 501 based on the application installation status on the terminal device 501, thereby reducing the difficulty of application calling.

[0138] According to the method of the embodiment of the present application, when implementing the LLM service, the output module deployed on the terminal device calls the application installed on the terminal device. The output module can directly obtain the application installation status on the terminal device, so that according to the application installation status on the terminal device, an application that meets the functional requirements can be selected from multiple applications installed on the terminal device. This not only reduces the difficulty of application calling, but also makes the final called application more in line with the application installation status of the terminal device, thereby improving the user experience.

[0139] Furthermore, in another embodiment, the output module 520 is trained according to the user's usage habits for the terminal device 501, so that the service output of the output module 520 is more in line with the user's usage habits. Specifically, the output module 520 is trained with the user's usage records (e.g., application usage records) on the terminal device 501 as training samples.

[0140] Since the input module 510 and the output module 530 are deployed on the terminal device 501, the training of the output module 520 does not involve the cloud server 502. This is equivalent to the LLM backbone network 520 on the cloud server 502 providing templated, public reasoning services for multiple terminal devices (multiple users); the input module 510 and the output module 530 on the terminal device 501 provide customized services for the terminal device 501 (the user of the terminal device 501). In this way, LLM can support user customized services and improve the service experience of LLM.

[0141] For example, multiple food delivery applications are installed on the terminal device 501. The output module 520 is trained according to the user's order history records on the terminal device 501 (including the order application opened, the coupon used, the type of coffee ordered, the store ordered, the settlement price, etc.).

[0142] The natural language information input by the user is "order a coffee takeaway", and the inference result generated by the output module 530 is the instruction request "start the takeaway software, and search for coffee takeaway in the takeaway software". The output module 530 determines the takeaway application from multiple applications installed on the terminal device 501 according to the instruction request and based on the previous model training results. The takeaway application that the user often uses is selected from multiple takeaway applications. The takeaway application is called to select a coffee shop and coffee type according to the user's ordering habits and the user's coupon holdings, so that the final ordered coffee type and settlement price meet the user's ordering habits.

[0143] According to the method of the embodiment of the present application, the output module deployed on the terminal device is trained according to the user's usage habits of the terminal device, so that the service output of the output module is more in line with the user's usage habits, thereby improving the user experience of the LLM service.

[0144] Figure 6 Shown is a schematic diagram of an LLM application scenario according to an embodiment of the present application.

[0145] like Figure 6 As shown, in one embodiment, LLM is divided into three parts: input module 610, LLM backbone network 620 and output module 630. Terminal device 601 is connected to cloud server 602 via a network. Input module 610 and output module 630 are deployed on terminal device 601, and LLM backbone network 620 is deployed on cloud server 602.

[0146] The input module 610 includes a prompt generation program 611 , a tokenizer 612 , and a multilayer perceptron (MLP) network 613 .

[0147] The prompt generation program 611 is used to generate a prompt corresponding to the model according to the natural language information input by the user. The function of the prompt is to provide the model with input text and guide the model to generate a suitable answer.

[0148] The Tokenizer 612 is used to convert the prompt text into a token vector.

[0149] The MLP network 613 is used to project the token vector to generate a Token Embedding.

[0150] The LLM backbone network 620 includes multiple transformer decoder blocks. The LLM backbone network 620 performs reasoning based on Token Embedding and generates reasoning results, which are information vectors composed of multiple token vectors. Specifically, the LLM backbone network 620 uses a non-autoregressive method during reasoning to calculate all token vectors of the reasoning results at one time.

[0151] The output module 630 includes a recurrent neural network (RNN) module 631 , an MLP network 632 and a decoder 633 .

[0152] The RNN module 631 is composed of a layer of long short-term memory (LSTM) network. The RNN module 631 is used to process the information vector composed of multiple token vectors in sequence to generate a sentence composed of the information vector.

[0153] The MLP network 632 is used to perform latent space conversion on the information vector output by the RNN module 631 to generate a latent space vector.

[0154] Decoder 633 is used to convert the latent space vector output by the MLP network 632 into text.

[0155] like Figure 6 As shown, the user enters text information on the terminal device 601: "Is there more high-speed train service to place A at station Z1 or Z2?" (A refers to the place name of a high-speed train destination. Z1 and Z2 refer to two different high-speed train stations in a city respectively).

[0156] The text information "Which high-speed train station to A has more trains, Z1 or Z2?" is processed by the Prompt generation program 611, Tokenizer 612 and MLP network 613 of the input module 610 to generate the corresponding Token Embedding 603.

[0157] The terminal device 601 sends the Token Embedding 603 to the cloud server 602 .

[0158] The cloud server 602 inputs the Token Embedding 603 into the LLM backbone network 620 .

[0159] The LLM backbone network 620 performs reasoning based on the Token Embedding 603 and generates Information Vectors 604 .

[0160] The cloud server 602 sends Information Vectors 604 to the terminal device 601 .

[0161] The terminal device 601 inputs the Information Vectors 604 to the output module 530 .

[0162] The information vectors 604 are processed by the RNN module 631, the MLP network 632 and the decoder 633 of the output module 630 to generate text information "If the destination is A, there are more trains at Z2 station."

[0163] The display module of the terminal device 601 displays the text message "If the destination is A, there are more trains at Z2 station" to the user.

[0164] Figure 7 Shown is a schematic diagram of an LLM application scenario according to an embodiment of the present application.

[0165] like Figure 7 As shown, in one embodiment, LLM is divided into three parts: input module 710, LLM backbone network 720 and output module 730. Terminal device 701 is connected to cloud server 702 via a network. Input module 710 and output module 730 are deployed on terminal device 701, and LLM backbone network 720 is deployed on cloud server 702.

[0166] The input module 710 includes a prompt instruction template 711 , a tokenizer 712 , and an MLP network 713 .

[0167] The prompt instruction template 711 is used to generate a prompt of the corresponding model according to the instruction input of the user. The function of the prompt is to provide the model with input instructions and guide the model to generate instruction requests corresponding to the input instructions.

[0168] Tokenizer 712 is used to convert the prompt text into a token vector.

[0169] The MLP network 713 is used to project the token vector to generate a Token Embedding.

[0170] The LLM backbone network 720 performs reasoning based on Token Embedding to generate reasoning results, which are information vectors composed of multiple token vectors.

[0171] The output module 730 includes an MLP network 731 , a decoder 732 , and an instruction template 733 .

[0172] The MLP network 731 is used to perform latent space conversion on the information vector output by the LLM backbone network 720 to generate a latent space vector.

[0173] Decoder 732 is used to convert the latent space vector output by the MLP network 632 into text.

[0174] The instruction template 733 is used to convert the text output by the Decoder 732 into an instruction request.

[0175] like Figure 7As shown, the user enters a command-style text message on the terminal device 701: "Remove the reflection of the glass in this picture and brighten the face."

[0176] The text information "remove the reflection of the glass in this picture and brighten the face" is processed by the Prompt instruction template 711, Tokenizer 712 and MLP network 713 of the input module 710 to generate the corresponding TokenEmbedding 703.

[0177] The terminal device 701 sends the Token Embedding 703 to the cloud server 702 .

[0178] The cloud server 702 inputs the Token Embedding 703 to the LLM backbone network 720 .

[0179] The LLM backbone network 720 performs reasoning based on the Token Embedding 703 and generates Information Vectors 704 .

[0180] The cloud server 702 sends the Information Vectors 704 to the terminal device 701 .

[0181] The terminal device 701 inputs the Information Vectors 704 to the output module 730 .

[0182] The information vectors 704 are processed by the MLP network 731, the decoder 732 and the instruction template 733 of the output module 730 to generate an instruction request. The operation corresponding to the instruction request is: calling the image processing application, using the image processing application to load the currently displayed image, removing the reflection of the glass in the image, and increasing the brightness of the face in the image.

[0183] The output module 730 calls the image processing application 705 on the terminal device 701 according to the instruction request, uses the image processing application 705 to access the image library 706 on the terminal device 701, and loads the image A in the image library 706 (the image A is the image currently being displayed by the terminal device 701). The image processing application 705 is used to remove the reflection of the glass in the image A, improve the brightness of the face in the image A, and generate the image processing result 707. The image processing result 707 is displayed to the user.

[0184] According to the data processing method provided in the embodiment of the present application, the embodiment of the present application also provides a data processing device, which is applied to a terminal device.

[0185] Figure 8Schematic diagram of a device structure according to an embodiment of the present application.

[0186] like Figure 8 As shown, the data processing device 800 is a component that can be configured in a terminal device. The data processing device 800 includes:

[0187] An input module 810, which is used to convert the natural language input by the user into an embedding vector;

[0188] A sending module 820, configured to output an embedding vector;

[0189] A receiving module 830 is used to receive an information vector, where the information vector is a vector generated by a model backbone of a large language model (LLM) through reasoning based on the embedding vector;

[0190] The output module 840 is used to perform reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

[0191] In one embodiment, the input module 810 is a module separated from the input end of the LLM. Optionally, the input module 810 includes a word segmenter of the LLM.

[0192] In one embodiment, the output module 840 is a module separated from the output end of the LLM. Optionally, the output module 840 includes a decoder of the LLM.

[0193] Optionally, in one embodiment, the data processing device 800 further includes:

[0194] The training module 850 is used to train the output module 840 according to the user's usage habits for the terminal device.

[0195] Optionally, in one embodiment, the output module 840 is further used to: output a text reply.

[0196] Optionally, in one embodiment, the output module 840 is further configured to:

[0197] According to the instruction request, the information data and / or application are called to generate services or answers provided to the user, wherein the information data is stored on the terminal device 801 and the application is installed on the terminal device 801.

[0198] According to the data processing method provided in the embodiment of the present application, the embodiment of the present application also provides a data processing device, which is applied to a cloud server.

[0199] Fig. 9 Schematic diagram of a device structure according to an embodiment of the present application.

[0200] like Fig. 9As shown, the data processing device 900 is a component that can be configured in a cloud server. The data processing device 900 includes:

[0201] A receiving module 910 is used to receive an embedding vector, where the embedding vector is a vector generated by converting a natural language input by a user;

[0202] An inference module 920, which includes a model backbone of a large language model (LLM), and is used to generate an information vector by inferring based on the embedding vector using the model backbone of the LLM;

[0203] The sending module 930 is used to output the information vector. In the description of the embodiment of the present application, for the convenience of description, the device is described by being divided into various modules according to their functions. The division of each module is only a division of logical functions. When implementing the embodiment of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0204] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in a processor element or instructions in the form of software.

[0205] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0206] An embodiment of the present application further provides an electronic device, which may be the terminal device or cloud server described in the above embodiment.

[0207] Fig.10Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0208] like Fig.10 As shown, the electronic device 1000 includes a memory 1002 for storing computer program instructions and a processor 1001 for executing the program instructions, wherein, when the computer program instructions are executed by the processor 1001, the electronic device 1000 is triggered to execute the method steps executed by the terminal device or the cloud server in the method described in the embodiment of the present application.

[0209] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory 1002, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned electronic device 1000, the above-mentioned electronic device 1000 executes the method steps described in the embodiment of the present application.

[0210] It is to be understood that the structural description of the electronic device 1000 in the embodiment of the present application does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include other components besides the processor 1001 and the memory 1002.

[0211] The processor 1001 may be a device on a chip (SOC), and the processor 1001 may include a central processing unit (CPU), and may further include other types of processors.

[0212] The processor involved in the processor 1001 may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units, NPU) and an image signal processor (Image Signal Processing, ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0213] The processor 1001 may include one or more processing units. For example, the processor may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent components or integrated into one or more processors. In some embodiments, the electronic device 1000 may also include one or more processors 1001. Among them, the controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0214] In some embodiments, the processor 1001 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. Among them, the USB interface is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge an electronic device, and can also be used to transmit data between an electronic device and a peripheral device.

[0215] The electronic device 1000 may also include an external memory interface, which is used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 1001 through the external memory interface to implement a data storage function. For example, files such as music and videos are stored in the external memory card.

[0216] The memory 1002 may include a code storage area and a data storage area. The code storage area may store an operating system. The data storage area may store data created during the use of the electronic device 1000, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc.

[0217] The memory 1002 may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or may be any computer-readable medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0218] The processor 1001 and the memory 1002 may be integrated into one processing device, or more commonly, they are independent components.

[0219] For ease of explanation, Fig.10 Only one memory, processor and transceiver are shown. In an actual electronic device product, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or a storage device. The memory may be set independently of the processor or may be integrated with the processor, and the embodiments of the present application do not limit this.

[0220] Optionally, in one embodiment, the electronic device 1000 further includes a receiving module 1003 and a sending module 1004. The sending module 1004 is used to perform the sending operation in the above method embodiment. The receiving module 1003 is used to perform the receiving operation in the above method embodiment.

[0221] For example, in one embodiment, the electronic device is a terminal device, and when the computer program instructions on the memory 1002 are executed by the processor 1001, the execution is triggered:

[0222] Processor 1001 converts the natural language input by the user into an embedding vector;

[0223] The sending module 1004 outputs the embedding vector;

[0224] The receiving module 1003 receives an information vector, which is a vector generated by the model backbone of the large language model (LLM) through reasoning based on the embedding vector;

[0225] Processor 1001 performs inference based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

[0226] For another example, in one embodiment, the electronic device is a cloud server, and when the computer program instructions on the memory 1002 are executed by the processor 1001, the execution is triggered:

[0227] The receiving module 1003 receives an embedding vector, where the embedding vector is a vector generated by converting a natural language input by a user;

[0228] Processor 1001 uses the model backbone of LLM to perform reasoning based on the embedded vector and generate an information vector;

[0229] The sending module 1004 outputs an information vector.

[0230] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0231] An embodiment of the present application further proposes an electronic chip. In one embodiment, the electronic chip may be a chip installed on the terminal device in the above embodiment. When the electronic chip is a chip installed on the terminal device, the sending operation of the terminal device in the above method embodiment may be understood as the output of the chip, and the receiving operation of the terminal device in the above method embodiment may be understood as the input of the chip.

[0232] In another embodiment, the electronic chip may be a chip installed on the cloud server in the above embodiment. When the electronic chip is a chip installed on the cloud server, the sending operation of the cloud server in the above method embodiment may be understood as the output of the chip, and the receiving operation of the cloud server in the above method embodiment may be understood as the input of the chip.

[0233] The electronic chip includes a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the electronic chip is triggered to execute the method steps executed by the terminal device or the cloud server in the method described in the embodiment of the present application. The processor of the electronic chip can refer to the processor of the above-mentioned electronic device.

[0234] Optionally, in one embodiment, the electronic chip further comprises a memory for storing computer program instructions.

[0235] Optionally, in one embodiment, the electronic chip further includes a transceiver, wherein the transceiver may be an input-output circuit or a communication interface.

[0236] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0237] In the several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0238] Specifically, in one embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0239] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is run on a computer, the computer executes the method provided by the embodiment of the present application.

[0240] The embodiments described in this application are described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0241] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0242] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0243] It should also be noted that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0244] In the embodiments of the present application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0245] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0246] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0247] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0248] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0249] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method is applied to a terminal device, and the method comprises: Convert the natural language input by the user into an embedding vector; Outputting the embedding vector; receiving an information vector, wherein the information vector is a vector generated by a model backbone of a large language model (LLM) performing reasoning based on the embedding vector; Inference is performed based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

2. The method according to claim 1, characterized in that: The natural language is converted into the embedding vector using an input module, wherein the input module is deployed in the terminal device and is a module cut out from an input end of the LLM.

3. The method according to claim 2, characterized in that: The input module includes a word segmenter of the LLM.

4. The method according to claim 1, characterized in that: The information vector is converted into a service or answer provided to the user by using an output module, wherein the output module is deployed in the terminal device and is a module cut out from the output end of the LLM.

5. The method according to claim 4, characterized in that: The output module includes a decoder of the LLM.

6. The method according to claim 4 or 5, characterized in that: The method further comprises: The output module is trained according to the user's usage habits for the terminal device.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Output the text reply.

8. The method according to any one of claims 1 to 6, characterized in that The method further comprises: According to the instruction request, information data and / or application are called to generate a service or answer provided to the user, wherein the information data is stored on the terminal device and the application is installed on the terminal device.

9. A data processing method, characterized in that: The method is applied to a cloud server, the cloud server is loaded with a model backbone of a large language model (LLM), and the method comprises: Receiving an embedding vector, where the embedding vector is a vector generated by converting a natural language input by a user; Using the model backbone of the LLM, reasoning based on the embedding vector to generate an information vector; The information vector is output.

10. A data processing device, characterized in that: The device is applied to a terminal device, and the device includes: An input module, which is used to convert the natural language input by the user into an embedding vector; A sending module, configured to output the embedding vector; A receiving module, configured to receive an information vector, wherein the information vector is a vector generated by a model backbone of a large language model (LLM) performing reasoning based on the embedding vector; The output module is used to perform reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector.

11. The device according to claim 10, characterized in that: The input module is a module separated from the input end of the LLM.

12. The device according to claim 11, characterized in that: The input module includes a word segmenter of the LLM.

13. The device according to claim 10, characterized in that: The output module is a module separated from the output end of the LLM.

14. The device according to claim 13, characterized in that: The output module includes a decoder of the LLM.

15. The device according to claim 13 or 14, characterized in that The device also includes: A training module is used to train the output module according to the user's usage habits for the terminal device.

16. The device according to any one of claims 10 to 15, characterized in that The output module is also used for: Output the text reply.

17. The device according to any one of claims 10 to 15, characterized in that The output module is also used for: According to the instruction request, information data and / or application are called to generate a service or answer provided to the user, wherein the information data is stored on the terminal device and the application is installed on the terminal device.

18. A data processing device, characterized in that: The device is applied to a cloud server, and the device includes: A receiving module, configured to receive an embedding vector, wherein the embedding vector is a vector generated by converting a natural language input by a user; An inference module, the inference module comprising a model backbone of a large language model (LLM), configured to generate an information vector by performing inference based on the embedding vector using the model backbone of the LLM; A sending module is used to output the information vector.

19. A large language model (LLM), characterized in that The LLM includes: An input module, which is used to convert the natural language input by the user into an embedding vector, wherein the input module is deployed on a terminal device; A model backbone, which is used to perform reasoning based on the embedding vector to generate an information vector, wherein the model backbone is deployed on a cloud server; An output module is used to perform reasoning based on the information vector to obtain a text reply or instruction request corresponding to the information vector, wherein the output module is deployed on a terminal device.

20. An electronic chip, characterized in that: The electronic chip comprises a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic chip is triggered to execute the method steps of any one of claims 1 to 8 or claim 9.

21. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps as described in any one of claims 1 to 8 or claim 9.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 8 or claim 9.

Images