Distributed transmission method and system for large model reasoning and training

Abstract

The invention relates to a distributed transmission method and system for large model reasoning and training, and relates to the technical field of large model distributed data transmission. The method comprises the steps that a client receives model input data and a model target instruction corresponding to the model input data; inputting the model input data and the model target instruction into the first model module, outputting to obtain a model intermediate processing result, and sending the intermediate processing result and the model target instruction to a server; and the server inputs the intermediate processing result and the target model instruction into the second model module, and outputs to obtain a model processing result. Under the condition that the corresponding model module in the server is processed, distributed deployment and common application of the large model between the server and the client are realized, so that the data processing amount of the server is reduced, and meanwhile, the privacy security of the client is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of large model distributed data transmission, and in particular to a distributed transmission method and system for large model reasoning and training. Background Art

[0002] Reasoning and training of large models are important links in the field of artificial intelligence, which involve complex algorithms and a large amount of computing resources.

[0003] In the related technology, in the process involving the local terminal and the server where the model is located, data processing is usually performed on the local terminal side to generate sample data for server processing. After that, the local terminal sends the sample data to the server, and the large model pre-stored in the server will process the sample data, or train based on the sample data, so as to achieve the purpose of result generation or model training.

[0004] However, the above method has poor confidentiality for local terminal data and is prone to cause user privacy leakage. Summary of the invention

[0005] The present invention relates to a distributed transmission method and system for large model reasoning and training, which can reduce the data processing volume on the server side and improve the privacy security of the client side. The technical solution is as follows: On the one hand, a distributed transmission method for large model reasoning and training is provided, the method is applied to a distributed transmission system for large model reasoning and training, the system including a client and a server; The client communicates with the server; The client is configured with a first model module, and the server is configured with a second model module. The first model module and the second model module are combined to obtain a target large model; The method includes: The client receives the model input data and the model target instruction corresponding to the model input data; inputs the model input data and the model target instruction into the first model module, outputs the model intermediate processing result, and sends the intermediate processing result and the model target instruction to the server; The server receives the intermediate processing results and the model target instructions; inputs the intermediate processing results and the target model instructions into the second model module, outputs the model processing results, and the model processing results correspond to the result output type indicated in the model target instructions; and feeds back the model processing results to the client.

[0006] In an optional embodiment, the method further comprises: The server obtains the target large model; splits the target large model based on the vertical federated learning technology to obtain a first model module and a second model module; and sends the first model module to the client; The client receives and configures the first model module.

[0007] In an optional embodiment, the client sends the intermediate processing result and the model target instruction to the server, including: Based on the intermediate results and model target instructions, the client generates at least one set of distributed transmission data in combination with the remote procedure call framework gPRC distributed communication technology; and sends the distributed transmission data to the server.

[0008] In an optional embodiment, the format of the distributed transmission data includes a message header portion and a data portion; The message header part includes a message type word part, a node identification word part and a message code word part.

[0009] In an optional embodiment, after the client generates at least one set of distributed transmission data in combination with the remote procedure call framework gPRC distributed communication technology, the following steps are included: The client compresses and splits the distributed transmission data to obtain data fragments.

[0010] In an optional embodiment, the server receives the intermediate processing result and the model target instruction, including: The server receives the data fragment; Decompress and merge the data slices to obtain intermediate processing results and model target instructions.

[0011] In an optional embodiment, the method further comprises: The client sends a handshake request to the server; The server receives the handshake request; sends a verification certificate based on the handshake request; The client and the server exchange keys based on the verification certificate; in response to the key exchange being completed, a communication connection is established.

[0012] In an optional embodiment, the model target instructions include training instructions, result output instructions, model segmentation instructions, model loading instructions, and model reasoning instructions.

[0013] In another aspect, a distributed transmission system for large model reasoning and training is provided, the system comprising a client and a communication connection between the client and a server; The client is configured with a first model module, and the server is configured with a second model module. The first model module and the second model module are combined to obtain a target large model; The system is used to perform any of the above large model reasoning and training distributed transmission methods.

[0014] In an optional embodiment, the client is configured with a first network card and a first graphics processing unit (GPU) module, and the server is configured with a second network card and a second GPU module; The first network card is connected to the first GPU module for communication, and the second network card is connected to the second GPU module for communication.

[0015] The beneficial effects brought about by the technical solution provided by the present invention include at least: By configuring distributed server model modules in the server and client, and having the client send intermediate data to the server after preliminary data processing, and the corresponding model module in the server processes it, distributed deployment and joint application of large models between the server and the client are achieved, thereby reducing the data processing volume on the server side and improving the privacy security of the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of the structure of a distributed transmission system for large model reasoning and training provided by an exemplary embodiment of the present invention is shown.

[0018] Figure 2 A detailed layout diagram of a distributed transmission system for large model reasoning and training provided by an exemplary embodiment of the present invention is shown.

[0019] Figure 3 A schematic diagram of model segmentation provided by an exemplary embodiment of the present invention is shown.

[0020] Figure 4 A flow chart of a distributed transmission method for large model reasoning and training provided by an exemplary embodiment of the present invention is shown.

[0021] Figure 5 A schematic diagram of a process of establishing communication between a client and a server provided by an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0023] Figure 1 A schematic diagram of the structure of a distributed transmission system for large model reasoning and training provided by an exemplary embodiment of the present invention is shown. Figure 2 For a detailed layout diagram, please refer to Figure 1 as well as Figure 2 In an embodiment of the present invention, a distributed transmission system for large model reasoning and training includes a client 110 and a server 120, wherein the client 110 is in communication connection with the server 120; wherein the client is configured with a first model module 311, and the server is configured with a second model module 312, and the first model module 311 and the second model module 312 are combined to obtain a target large model.

[0024] In the embodiment of the present invention, the target large model is implemented as a large language model (Large Language Model). It should be noted that the LLM can be applied to text processing or semantic recognition scenarios, and the present invention does not limit the specific application scope of the LLM. Please refer to Figure 3 The LLM310 is divided into a first model module 311 configured on the client and a second model module 312 configured on the server, wherein the first model module 311 includes an embedding layer 3111 and at least two first encoding and decoding layers 3112, and the second model module 312 includes a head language module 3121 and at least two second encoding and decoding layers 3122.

[0025] In the embodiment of the present invention, please refer to Figure 2 To ensure the transmission efficiency during communication between the client and the server, the client 110 is configured with a first network card 111 and a first GPU module 112, and the server 120 is configured with a second network card 121 and a second GPU module 122. The server side directly transmits the data in the network card to the GPU for processing through the GPU to network card architecture, thereby improving the processing speed.

[0026] Combined with the above description, Figure 4 A schematic diagram of a distributed transmission method for large model reasoning and training provided by an exemplary embodiment of the present invention is shown, and the method is applied to Figure 1 or Figure 2 Taking the system shown in FIG. 1 as an example, the method includes: Step 401: The client receives model input data and a model target instruction corresponding to the model input data.

[0027] In the embodiment of the present invention, the LLM can be a model that has been trained and can be put into use, or it can be a model that is still in the training and learning process. Therefore, in the embodiment of the present invention, the model target instruction is used to configure the model with corresponding parameters and perform the corresponding training or application process. In this case, the model target instruction optionally includes a training instruction and a result output instruction.

[0028] Step 402: The client inputs the model input data and the model target instruction into the first model module, and outputs the model intermediate processing result.

[0029] In the embodiment of the present invention, combined with Figure 3 As shown, the client inputs the model input data and the model target instruction into the first model module, and the model input data and the model target instruction are received by the embedding layer of the first model module, and after being processed by the encoding and decoding layer, the model intermediate processing result is obtained.

[0030] Optionally, in the embodiment of the present invention, the intermediate processing result is the data received by the server.

[0031] Step 403: The client sends the intermediate processing results and the model target instructions to the server.

[0032] In an embodiment of the present invention, the client generates at least one set of distributed transmission data based on the intermediate results and the model target instructions in combination with the remote procedure call framework gPRC distributed communication technology.

[0033] In one example, the client uses gRPC distributed communication technology to transmit the intermediate calculation results of the model under the splitNN structure through a custom protocol, transferring data from the client to the server to achieve reasoning and training after the large model is split in a distributed manner.

[0034] The specific method in this process is to customize the transmission protocol based on the PROTOBUFF protocol of gRPC, that is, to define the message type and data format of the client and server. In one example, the format of distributed transmission data includes a message header part and a data part, wherein the message header part includes a message type word part, a node identification word part and a message code word part.

[0035] In an embodiment of the present invention, the client compresses and splits the distributed transmission data, obtains data fragments, and transmits the data fragments. Optionally, the data fragments correspond to consistent data fragment sizes, and transmit data to the server via gRPC streaming data transmission.

[0036] Step 404: The server receives the intermediate processing results and the model target instructions.

[0037] In an embodiment of the present invention, when the intermediate processing result is implemented as data fragments, the server will merge the data fragments, that is, decompress and merge the data fragments to obtain the intermediate data processing result and the model target instruction. In one example, the compression algorithm used in the data compression process is GZIP, and the DEFLATE algorithm is applied.

[0038] Step 405: The server inputs the intermediate processing result and the target model instruction into the second model module, and outputs the model processing result.

[0039] This process is the output process of the model processing results.

[0040] Step 406: The server feeds back the model processing result to the client.

[0041] In some embodiments of the present invention, since the model processing result has corresponding visual output information, the server needs to feed back data to the client to support the corresponding functions of the client.

[0042] Optionally, when the server sends a portion of data in a single processing flow to the client, the client can continue to process the data according to the result and perform the next round of iteration until the task is completed.

[0043] To summarize, the method and system provided by the embodiments of the present invention configure distributed server model modules in the server and the client, and the client sends the intermediate data to the server after preliminary data processing, and the corresponding model module in the server processes it, thereby realizing the distributed deployment and common application of large models between the server and the client, thereby reducing the data processing volume on the server side and improving the privacy security of the client.

[0044] In an optional embodiment, the splitting process of the target large model is a splitting based on the vertical federated learning technology. In one example, after the server configures the target large model, it is based on the vertical federated learning technology. In one example, the specific segmentation method of the technology is the splitNN method.

[0045] In an alternative embodiment, Figure 5 A schematic diagram of a process of establishing communication between a client and a server provided by an exemplary embodiment of the present invention is shown. The process includes: Step 501: The client sends a handshake request to the server.

[0046] Step 502: The server receives a handshake request.

[0047] Step 503: The server sends a verification certificate based on the handshake request.

[0048] Step 504: The client and the server exchange keys based on the verification certificate.

[0049] Step 505: In response to the key exchange being completed, the client and the server establish a communication connection.

[0050] The above are only optional embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed transmission method for large model reasoning and training, characterized in that: The method is applied to a distributed transmission system for large model reasoning and training, wherein the system includes a client and a server; The client is connected to the server for communication; The client is configured with a first model module, and the server is configured with a second model module, and the first model module and the second model module are combined to obtain a target large model; The method comprises: The client receives model input data and a model target instruction corresponding to the model input data; inputs the model input data and the model target instruction into the first model module, outputs a model intermediate processing result, and sends the intermediate processing result and the model target instruction to the server; The server receives the intermediate processing result and the model target instruction; inputs the intermediate processing result and the target model instruction into the second model module, and outputs a model processing result, wherein the model processing result corresponds to the result output type indicated in the model target instruction; and feeds back the model processing result to the client.

2. The distributed transmission method for large model reasoning and training according to claim 1 is characterized in that: The method further comprises: The server obtains the target large model; splits the target large model based on the vertical federated learning technology to obtain the first model module and the second model module; and sends the first model module to the client; The client receives and configures the first model module.

3. The distributed transmission method for large model reasoning and training according to claim 2 is characterized in that: The client sends the intermediate processing result and the model target instruction to the server, including: The client generates at least one set of distributed transmission data based on the intermediate results and the model target instructions in combination with the remote procedure call framework gPRC distributed communication technology; and sends the distributed transmission data to the server.

4. The distributed transmission method for large model reasoning and training according to claim 3 is characterized in that: The format of the distributed transmission data includes a message header part and a data part; The message header part includes a message type word part, a node identification word part and a message code word part.

5. The distributed transmission method for large model reasoning and training according to claim 3 is characterized in that: After the client generates at least one set of distributed transmission data by combining the remote procedure call framework gPRC distributed communication technology, it includes: The client compresses and splits the distributed transmission data to obtain data fragments.

6. The distributed transmission method for large model reasoning and training according to claim 5, characterized in that: The server receives the intermediate processing result and the model target instruction, including: The server receives the data slice; The data slices are decompressed and merged to obtain the intermediate processing results and the model target instructions.

7. The distributed transmission method for large model reasoning and training according to claim 3, characterized in that: The method further comprises: The client sends a handshake request to the server; The server receives the handshake request; sends a verification certificate based on the handshake request; The client and the server exchange keys based on the verification certificate; in response to the key exchange being completed, a communication connection is established.

8. The distributed transmission method for large model reasoning and training according to claim 1, characterized in that: The model target instructions include training instructions, result output instructions, model segmentation instructions, model loading instructions and model reasoning instructions.

9. A distributed transmission system for large model reasoning and training, characterized in that: The system includes a client and a server; The client is connected to the server for communication; The client is configured with a first model module, and the server is configured with a second model module, and the first model module and the second model module are combined to obtain a target large model; The system is used to execute the distributed transmission method for large model reasoning and training as described in any one of claims 1 to 8.

10. The distributed transmission system for large model reasoning and training according to claim 9, characterized in that: The client is configured with a first network card and a first graphics processor GPU module, and the server is configured with a second network card and a second GPU module; The first network card is communicatively connected to the first GPU module, and the second network card is communicatively connected to the second GPU module.

Images