Cloud-assisted transparent compression distributed model distribution method and distribution network system

Abstract

The invention relates to a cloud-assisted transparent compression distributed model distribution method and distribution network system. The method comprises the following steps: a cloud side prepares an AI model, generates a file of the AI model, and releases the file of the AI model; the cloud collaboratively calculates the difference part of the AI model needing to be updated; the cloud side performs transparent compression on the difference part of the AI model; the cloud end collaboratively transmits the difference part of the compressed AI model to the end side in a P2P multipath transmission mode; and the end side decompresses the difference part of the compressed AI model after receiving the difference part, and then the difference part and the local same part are assembled into a complete update model. According to the method, rapid updating and upgrading of the end side model can be realized, constraints of network and energy consumption of the end side are adapted, and the transmission process is optimized cooperatively through transparent compression and multipath transmission of the cloud.

Description

Technical Field

[0001] The present application relates to the field of model distribution technology, and in particular to a distributed model distribution method and distribution network system with cloud-assisted transparent compression. Background Art

[0002] Intelligent reasoning of nodes on the end side has become the next "blue ocean" of intelligent computing. Using the limited resources of nodes on the end side for local intelligent reasoning can reduce response latency. After the cloud side model is upgraded or fine-tuned, it needs to be quickly distributed to the node devices on the end side.

[0003] At present, model distribution is mainly achieved through direct push on the cloud side, which has the advantages of simplicity and convenience. However, as the scale of model parameters continues to grow, the waiting time and bandwidth overhead of directly transmitting model files continue to grow. Every time the model parameters are upgraded or updated, there is redundancy between the cloud-side and the end-side models, and there is also redundancy between the ends. Therefore, the transmission of redundant parts wastes the bandwidth resources of the cloud side, and does not fully utilize the redundant data between the ends to accelerate transmission. At the same time, due to the unstable network or low power consumption constraints on the end side, the model distribution efficiency of the nodes on the end side is restricted, and it is necessary to accelerate the distribution efficiency of the node models on the end side. Summary of the invention

[0004] Based on this, it is necessary to provide a distributed model distribution method and distribution network system with cloud-assisted transparent compression to address the above technical problems.

[0005] A distributed model distribution method with cloud-assisted transparent compression, the method comprising: The nodes on the cloud side prepare the AI ​​model, generate the AI ​​model file, and the nodes on the cloud side publish the AI ​​model file.

[0006] The nodes on the cloud side and the nodes on the end side calculate the different parts of the AI ​​model that need to be updated.

[0007] The nodes on the cloud side transparently compress the different parts of the AI ​​model.

[0008] The cloud-side nodes transmit the compressed difference parts of the AI ​​model to the end-side nodes through P2P multipath transmission.

[0009] The node on the end side decompresses the difference part of the received compressed AI model, and then assembles it with the same part locally into a complete updated model.

[0010] In one embodiment, the node on the cloud side prepares the AI ​​model and generates a file of the AI ​​model. The cloud side places the file of the AI ​​model on the cloud. The steps also include a system initialization step, which specifically includes: The cloud-side server and the end-side are connected to the network, prepare the local computing, storage, and network environment, and start the corresponding model publisher, model compressor, model forwarder, and model subscriber processes; among which: The model publisher is used to prepare and publish AI models and is in the sending state.

[0011] The model compressor is used to calculate model differences and compress them. It is in a forwarding state and decouples the model publisher and the model receiver through a proxy.

[0012] The model forwarder is used to optimize the transmission delay through multi-path transmission and is in the forwarding state.

[0013] The model subscriber is used to receive, decompress and assemble models and is in the receiving, sleeping and forwarding states.

[0014] In one embodiment, the cloud-side node and the terminal-side node calculate the difference between the AI ​​models that need to be updated, including: The publisher of the model and the subscribers of each model calculate the difference between the cloud and local AI models by comparing the model's tensor differences. If there is a difference between the remote and local bucket arrays, the tensor needs to be transmitted, otherwise it is skipped.

[0015] In one embodiment, the difference portion of the AI ​​model includes a number of tensors to be transmitted.

[0016] The cloud-side nodes transparently compress the differences in the AI ​​model, including: The model compressor on the cloud side uses transparent compression to compress each tensor to be transmitted. The transparent compression method refers to using a set of bucket vectors to splice the tensor into the bucket vectors, and then using sparse encoding for the bucket vectors, including only the top-k elements. At the same time, quantization is used to represent each non-zero value through low-bit quantization; the compressed model is serialized and stored in the file system through pickle.

[0017] In one embodiment, the cloud-side node transmits the compressed difference part of the AI ​​model to the terminal-side node through a P2P multipath transmission method, including: The cloud side starts the BitTorrent transmission protocol, and the nodes on the end side join the BitTorrent protocol, and the difference parts of the compressed AI model are transmitted to the nodes on the end side through the collaboration of the nodes on the end side.

[0018] In one embodiment, the terminal node decompresses the difference part of the received compressed AI model, and then assembles it with the local same part into a complete updated model, including: The model subscriber of the node on the end side reversely decompresses the sparse and quantized bucket vectors, restores each tensor, and combines them with the local unupdated parts to form a dictionary of the AI ​​model, which is serialized and stored in a local file through pickle.

[0019] In one embodiment, the AI ​​model is organized as a dictionary, including metadata at each level of the model and corresponding parameter tensors.

[0020] A cloud-assisted transparent compression distributed model distribution network system, the system comprising: a model publisher, a model compressor, a model forwarder and a model subscriber.

[0021] The model publisher is a node on the cloud side. The node is in the sending state and is used to publish the model.

[0022] The model compressor consists of dedicated sites on the cloud side, and the nodes are in a forwarding state for model compression.

[0023] The model forwarder is composed of nodes on the end side, providing multipath overlay network routing. The nodes are in the forwarding state and are used to forward the model.

[0024] The model subscriber consists of nodes on the end side. The node states include receiving, sleeping, and forwarding, which are used to upgrade and update the model.

[0025] When distributing the model, the model publisher prepares the AI ​​model, generates the AI ​​model file, and publishes the AI ​​model file; the model publisher and each model subscriber collaboratively calculate the difference part of the AI ​​model that needs to be updated, and the model compressor transparently compresses the difference part of the AI ​​model; the cloud-side node transmits the compressed difference part of the AI ​​model to the end-side node through P2P multipath transmission; the end-side node decompresses the received compressed difference part of the AI ​​model, and then assembles it with the same part locally into a complete updated model.

[0026] In one embodiment, the sleep state is only responsible for the communication of metadata and not for the distribution of the model, and the forwarding state can be used in conjunction with sending and receiving.

[0027] In one embodiment, the cloud-side node transmits the compressed difference part of the AI ​​model to the terminal-side node through a P2P multipath transmission method, including: The cloud side starts the BitTorrent transmission protocol, and the nodes on the end side join the BitTorrent protocol, and the difference parts of the compressed AI model are transmitted to the nodes on the end side through the collaboration of the nodes on the end side.

[0028] The above-mentioned cloud-assisted transparent compression distributed model distribution method and distribution network system, the method includes: the cloud side prepares the AI ​​model, generates the AI ​​model file, and the cloud side publishes the AI ​​model file; the cloud side collaboratively calculates the difference part of the AI ​​model that needs to be updated; the cloud side transparently compresses the difference part of the AI ​​model; the cloud side collaboratively transmits the compressed difference part of the AI ​​model to the end side through P2P multipath transmission; the end side decompresses the difference part of the compressed AI model after receiving it, and then assembles it with the same part locally into a complete updated model. This method can realize the rapid update and upgrade of the end-side model, adapt to the network and energy consumption constraints of the end-side, and collaboratively optimize the transmission process through transparent compression and multipath transmission in the cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a distributed model distribution method with cloud-assisted transparent compression in one embodiment.

[0030] Figure 2 A flow chart of a distributed model of cloud-assisted transparent compression in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In one embodiment, Figure 2 As shown, a distributed model distribution method with cloud-assisted transparent compression is provided, and the method comprises the following steps: Step 100: The node on the cloud side prepares the AI ​​model, generates the file of the AI ​​model, and the node on the cloud side publishes the file of the AI ​​model.

[0033] Specifically, AI models are organized as dictionaries, including metadata at each level of the model and the corresponding parameter tensors.

[0034] Node: It consists of participating processes or nodes, responsible for upgrading and updating the model, and participating in the distribution of the model.

[0035] Node state: including sending, receiving, sleeping, forwarding and other states. The sleeping state is only responsible for the communication of metadata, not the distribution of the model. The forwarding state can be used in conjunction with sending and receiving.

[0036] Step 102: The cloud-side node and the terminal-side node calculate the difference in the AI ​​model that needs to be updated.

[0037] Step 104: The cloud-side node transparently compresses the difference part of the AI ​​model.

[0038] Step 106: The cloud-side node transmits the compressed difference part of the AI ​​model to the terminal-side node via P2P multipath transmission.

[0039] Step 108: The node on the terminal side decompresses the difference part of the received compressed AI model, and then assembles it with the same part locally into a complete updated model.

[0040] In the above-mentioned distributed model distribution method with cloud-assisted transparent compression, the method includes: the cloud side prepares the AI ​​model, generates the file of the AI ​​model, and publishes the file of the AI ​​model; the cloud side collaboratively calculates the difference part of the AI ​​model that needs to be updated; the cloud side transparently compresses the difference part of the AI ​​model; the cloud side collaboratively transmits the compressed difference part of the AI ​​model to the end side through P2P multipath transmission; the end side decompresses the difference part of the compressed AI model after receiving it, and then assembles it with the same part locally into a complete updated model. This method can realize the rapid update and upgrade of the end-side model, adapt to the network and energy consumption constraints of the end-side, and collaboratively optimize the transmission process through transparent compression and multipath transmission in the cloud.

[0041] In one embodiment, a system initialization step is also included before step 100, specifically including: The cloud-side server and the end-side are connected to the network, prepare the local computing, storage, and network environment, and start the corresponding model publisher, model compressor, model forwarder, and model subscriber processes; among them: the model publisher is used to prepare and publish the AI ​​model and is in the sending state; the model compressor is used to calculate the model difference and compress it, and is in the forwarding state, decoupling the model publisher and the model receiver through the proxy method; the model forwarder is used to optimize the transmission delay through multi-path transmission, and is in the forwarding state; the model subscriber is used to receive, decompress and assemble the model, and is in the receiving, sleeping and forwarding states.

[0042] In one embodiment, step 102 includes: the publisher of the model and the subscriber of each model calculate the difference between the cloud and local AI models by comparing the tensor differences of the models (for the two tensors to be compared, each initializes a bucket array of length m (hyperparameter, set to 1024 by default), and maps each element of the tensor to a bucket of the array using the same hash function (the key is the position index of the element, ensuring that elements at the same position are mapped to the same bucket), and the bucket aggregates the elements inserted into the bucket using the sum (SUM) operator, and then the two bucket arrays are exchanged). If there is a difference between the remote and local bucket arrays, the tensor needs to be transmitted, otherwise the tensor is skipped.

[0043] In one embodiment, the difference part of the AI ​​model includes several tensors to be transmitted; step 104 includes: the model compressor on the cloud side compresses each tensor to be transmitted using a transparent compression method, and the transparent compression method refers to using a set of bucket vectors to splice the tensors into bucket vectors, and then using sparse coding on the bucket vectors, including only top-k elements (top-k is a hyperparameter, and the first 5% of elements are retained by default). At the same time, quantization is used to represent each non-zero value through low-bit quantization; the compressed model is serialized and stored in the file system through pickle.

[0044] In one embodiment, step 106 includes: the cloud side starts the BitTorrent transmission protocol, the node on the end side joins the BitTorrent protocol, and the difference part of the compressed AI model is transmitted to the node on the end side through the collaboration of the nodes on the end side.

[0045] In one embodiment, step 108 includes: the model subscriber of the node on the end side reversely decompresses the sparse and quantized bucket vectors, restores each tensor, and combines them with the local unupdated part to form a dictionary of the AI ​​model, which is serialized and stored in a local file through pickle.

[0046] In one embodiment, the AI ​​model is organized as a dictionary, including metadata at each level of the model and corresponding parameter tensors.

[0047] In a specific embodiment, the steps of the distributed model distribution method with cloud-assisted transparent compression include: a) System initialization: Cloud-side servers and end-side nodes connect to the network, prepare local computing, storage, and network environments, and start the necessary model publisher, model compressor, model forwarder, and model subscriber processes; b) Model preparation: The cloud side prepares the AI ​​model, generates the AI ​​model file, and publishes the AI ​​model on the cloud side; c) Model differences: The cloud-based collaborative computing needs to update the differences in the model, and only transmits the differences; d) Model compression: The cloud side transparently compresses the difference parts of the AI ​​model to be transmitted, and the publisher does not need to worry about compression; e) Model transmission: Transmission of cloud-coordinated models, optimizing transmission delay through P2P multipath transmission; f) Decompression and assembly: The client receives the model differences, decompresses them, and then assembles them with the local identical parts into a complete updated model.

[0048] In another specific embodiment, taking the distribution of the llama3 model as an example, the steps of the model distribution method are described in detail.

[0049] Step S1, model preparation: prepare the llama3 model, generate the model file, and place the model file in the cloud.

[0050] 1) The model publisher prepares and publishes the AI ​​model, which is located on the cloud side and is in the sending state; 2) The model compressor calculates the model difference and compresses it. It is located on the cloud side and is in a forwarding state. It decouples the model publisher and the model receiver through a proxy and transmits the compressed model difference data to the model forwarder. 3) Model forwarder, located on the cloud side or the client side, optimizes transmission delay through multi-path transmission, is in forwarding state, uses BitTorrent protocol by default, and can also use other P2P model file transfer protocols; 4) Model subscriber, located at the end-side node, is responsible for receiving, decompressing and assembling the model. It is in the receiving, sleeping and forwarding states and uses the BitTorrent protocol to download model files by default.

[0051] Step S2, model difference: The publisher of the model and each model subscriber calculate the difference between the cloud and local models. The calculation adopts the method of comparing the tensor differences of the models. For the two tensors to be compared, each initializes a bucket array of length m (hyperparameter, set to 1024 by default), and maps each element of the tensor to a bucket of the array using the same hash function (the key is the position index of the element, ensuring that elements at the same position are mapped to the same bucket). The bucket aggregates the elements inserted into the bucket using the sum (SUM) operator, and then exchanges the two bucket arrays. If the average relative difference of the buckets with the same index in the remote and local bucket arrays (average relative difference = average (|bucket (A,i) - bucket (B,i)|max(bucket (A,i), bucket (B,i)), for i ∈ [1,m])) exceeds 10% (hyperparameter, which can be adjusted according to application needs. The closer it is to zero, the more sensitive it is to the change in the difference), a significant difference occurs, and the tensor needs to be transmitted, otherwise the tensor is skipped to reduce data communication volume and improve distribution efficiency.

[0052] The steps to calculate the difference of each tensor between two nodes include: Step S2.1: Input the length m of the bucket array, where m is an integer and the default value is 1024; set the difference determination threshold threshold, where threshold is a floating point number and the default value is 0.1, i.e. 10%.

[0053] Step S2.2: Each node calculates the local bucket array. Specifically: initialize the local bucket array to 0; traverse the local tensor elements and map them to buckets: for each element index index and the corresponding value value, calculate the hash value and accumulate the element value to the corresponding bucket. The formula for calculating the hash value is: bucket_index = hash(index) % m Among them, hash() is a hash function that maps elements in the same position to the same bucket; Step S2.3: Nodes exchange bucket arrays. Specifically: Send local bucket arrays to each other: Each node sends its local_buckets to the other node; Receive the bucket array from the other node, recorded as remote_buckets.

[0054] Step S2.4: Each node calculates the average relative difference. Specifically: calculate the relative difference of each bucket, initialize the relative difference array to an array of 0; for each bucket index i (i ∈ [0, m-1]), perform the following operations: obtain the values ​​of the local and remote buckets, calculate the maximum value based on the values ​​of the local and remote buckets, and when the maximum value is greater than 0, calculate the relative difference, otherwise, set the corresponding value in the relative difference array to 0; calculate the average relative difference based on the relative difference array.

[0055] Step S2.5: Determine whether the difference exceeds the threshold. Specifically, compare the average relative difference with the preset threshold. If the average relative difference is greater than the preset threshold, the difference is considered significant and True is returned; otherwise, False is returned.

[0056] Step S2.6: Return the difference judgment result and output a Boolean value indicating whether the difference between the local and remote tensors significantly exceeds the threshold.

[0057] Step S3, model compression: For each tensor to be transmitted, a transparent compression method is used. Specifically, a set of bucket vectors is used to splice the tensor into bucket vectors, and then the bucket vectors are sparsely encoded to include only the top-k elements. At the same time, quantization is used to represent each non-zero value with low-bit quantization to further reduce storage overhead. The compressed model is serialized and stored in the file system through pickle.

[0058] The specific steps of model compression include: Step S3.1: Get the dictionary representation of model differences diff (dict); initialize the compressed difference dictionary compressed_diff to be empty; Step S3.2: Traverse each tensor in the dictionary representation of the model difference, flatten the tensor into a one-dimensional vector, then perform sparse coding, retain the top-k elements, quantize the sparse coding results, and store the quantized data in the compressed difference dictionary. Specifically: Step S3.2.1: Flatten the tensor into a one-dimensional vector: bucket_vector = tensor.flatten(tensor) Where bucket_vector is a one-dimensional vector, and tensor.flatten() is a function that expands tensor into a one-dimensional vector in row-first order. Step S3.2.2: Sparse coding, retaining the top-k elements. Specifically, obtain the indexes of the first k elements with the largest absolute values, obtain the values ​​of the elements corresponding to these indexes, and construct sparse coding data based on the indexes of the first k elements with the largest absolute values ​​and the values ​​of the corresponding elements; indices = np.argpartition(np.abs(bucket_vector), -k)[-k:] values ​​= bucket_vector[indices] sparse_encoded = {'indices': indices, 'values': values} Among them, sparse_encoded is sparsely encoded data, values ​​is the index of the first k elements with the largest absolute value and the value of the corresponding element, np.abs() is a function that calculates the absolute value of each element bit by bit and returns the result, np.argpartition() is a function that sorts the input array and returns the largest k sorted indices, and indices is the index of the first k elements with the largest absolute value.

[0059] Step S3.2.3: quantize the sparse coding result. Specifically, it includes: Step S3.2.3.1: Get the maximum absolute value of the median values ​​of the sparsely coded data: max_val = np.max(np.abs(sparse_encoded['values'])) Among them, max_val is the maximum absolute value, np.max() is the function that selects the maximum value in the array, and sparse_encoded['values'] is the median value of the sparsely encoded data.

[0060] Step S3.2.3.2: Calculate the scaling ratio based on the maximum absolute value max_val and the number of quantization bits bits: scale = (2bits - 1) / max_val Wherein, scale is the scaling ratio, bits is the number of quantization bits, and preferably, when bits=8, the median values ​​of the sparsely coded data are scaled to the range of 0-255.

[0061] Step S3.2.3.3: Determine the quantization value according to the scaling ratio: quantized_values ​​= np.round(sparse_encoded['values'] * scale).astype(np.int8) Among them, quantized_values ​​is the quantized value, np.round() is the rounding function, astype() is the function that returns the encoded integer array according to the input integer encoding style, and np.int8 is the 8-bit signed integer encoding style.

[0062] Step S3.2.3.4: According to the index, quantized value quantized_values ​​and scaling ratio in the sparsely coded data, construct the quantized data quantized as: quantized = {'indices': sparse_encoded['indices'], 'values': quantized_values, 'scale': scale} Among them, quantized is the quantitative data, and the quantized data contains 'indices', 'values' and 'scale'.

[0063] Step S3.2.4: Store the quantized data in a compressed difference dictionary compressed_diff[layer] = quantized Among them, compressed_diff is the compressed difference dictionary data to be serialized; layer is the hierarchical index of the model.

[0064] Step S3.3: Serialize the compressed difference dictionary: bytes =pickle_serialize(compressed_diff) Among them, bytes is the serialized data, and pickle_serialize() is a function that serializes the compressed difference dictionary data to be serialized according to the pickle style.

[0065] Step S4, model transmission: The cloud side starts the BitTorrent transmission protocol, and the terminal side joins the BitTorrent protocol to accelerate the transmission process of the compressed model to be transmitted through the collaboration of the terminal side.

[0066] The specific steps of using the BitTorrent protocol to transfer the compressed model differences are: Step S4.1: Get the compressed model difference data compressed_diff (bytes), and set the save path list paths (list) of the end-side device (model_diff_path (string): the path to save the model difference file, for example ". / model_diff.bin", torrent_file_path (string): the path to save the torrent file, for example ". / model_diff.torrent"); Step S4.2: Save the compressed model difference data compressed_diff to a file; Step S4.3: Create a seed file, encode the seed file and save it to a file. Specifically, it includes: creating a file storage object, then adding the model difference file to the file storage object, creating a seed object based on the file storage object, adding a tracker (called Tracker in BitTorrent) to the seed object, calculating the hash value of the seed object, generating a seed file, and finally encoding the seed file and saving it to the seed file; Step S4.4: Start the seed. Specifically, create a session object, set the listening port range, obtain the seed file information, add the seed file information to the created session, and specify the save path.

[0067] Step S4.5: The cloud waits for the client device to connect and download. Keep the cloud program running, monitor the upload status, and ensure that the model difference file is continuously available as a seed for download.

[0068] Step S4.6: The client side joins the BitTorrent protocol. Specifically, the client side device save path list is set, a session object is created and a listening port is set, the seed file is read and added to the session, and then the download is waited for completion.

[0069] Step S5, decompression and assembly: The client reversely decompresses the sparse and quantized bucket vectors, restores each tensor, and combines them with the local unupdated parts to form the dictionary of the AI ​​model, which is serialized and stored in a local file through pickle. The specific steps of decompressing and assembling the model include: Step S5.1: Obtain the compressed model difference data and the dictionary representation of the local model; Step S5.2: Use the pickle_deserialize() function to deserialize the compressed model difference data; Step S5.3: traverse each layer of model difference data, and perform dequantization processing on the quantized data of each layer of the compressed model difference data after deserialization processing: values ​​= quantized['values'].astype(np.float32) / scale Then construct the sparse coding data: dec_sparse_encoded = {'indices': quantized['indices'], 'values':values} Among them, dec_sparse_encoded is the sparse coding dictionary, indices is the index of the sparse element, and values ​​is the value vector of the sparse element.

[0070] Step S5.4: Decoding the sparsely coded data constructed in step S5.3. Specifically comprising: Step S5.4.1: Get the sparse coded data dec_sparse_encoded (dict), including 'indices' and 'values', and set the size of the decoded tensor.

[0071] Step S5.4.2: Initialize bucket_vector1 to an all-zero vector and fill it with the sparsely coded data: bucket_vector1[dec_sparse_encoded['indices']] = dec_sparse_encoded['values'] Among them, bucket_vector1 is the decoded vector.

[0072] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0073] In one embodiment, a distributed model distribution network system with cloud-assisted transparent compression is provided, the system comprising: a model publisher, a model compressor, a model forwarder, and a model subscriber, wherein: The model publisher is a node on the cloud side. The node is in the sending state and is used to publish the model.

[0074] The model compressor consists of dedicated sites on the cloud side, and the nodes are in a forwarding state for model compression.

[0075] The model forwarder is composed of nodes on the end side, providing multipath overlay network routing. The nodes are in the forwarding state and are used to forward the model.

[0076] The model subscriber consists of nodes on the end side. The node states include receiving, sleeping, and forwarding, which are used to upgrade and update the model.

[0077] When distributing the model, the model publisher prepares the AI ​​model, generates the AI ​​model file, and publishes the AI ​​model file; the model publisher and each model subscriber collaboratively calculate the difference part of the AI ​​model that needs to be updated, and the model compressor transparently compresses the difference part of the AI ​​model; the cloud-side node transmits the compressed difference part of the AI ​​model to the end-side node through P2P multipath transmission; the end-side node decompresses the received compressed difference part of the AI ​​model, and then assembles it with the same part locally into a complete updated model.

[0078] In one embodiment, the sleep state is only responsible for the communication of metadata and not for the distribution of the model, and the forwarding state can be used in conjunction with sending and receiving.

[0079] In one embodiment, the cloud-side node transmits the compressed difference part of the AI ​​model to the terminal-side node through a P2P multipath transmission method, including: The cloud side starts the BitTorrent transmission protocol, and the nodes on the end side join the BitTorrent protocol, and the difference parts of the compressed AI model are transmitted to the nodes on the end side through the collaboration of the nodes on the end side.

[0080] For the specific limitations of the distributed model distribution network system with cloud-assisted transparent compression, please refer to the limitations of the distributed model distribution method with cloud-assisted transparent compression above, which will not be repeated here. Each module in the above-mentioned distributed model distribution network system with cloud-assisted transparent compression can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0081] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A distributed model distribution method with cloud-assisted transparent compression, characterized in that: The method comprises: The cloud-side node prepares the AI ​​model, generates the AI ​​model file, and the cloud-side node publishes the AI ​​model file; The nodes on the cloud side and the nodes on the client side calculate the differences in the AI ​​model that need to be updated; The cloud-side nodes transparently compress the differences in the AI ​​model; The cloud-side nodes transmit the compressed difference parts of the AI ​​model to the client-side nodes through P2P multipath transmission. The node on the end side decompresses the difference part of the received compressed AI model, and then assembles it with the same part locally into a complete updated model.

2. The distributed model distribution method of cloud-assisted transparent compression according to claim 1 is characterized in that: The cloud-side node prepares the AI ​​model and generates the AI ​​model file. The cloud-side places the AI ​​model file on the cloud. The steps include system initialization, which include: The cloud-side server and the end-side are connected to the network, prepare the local computing, storage, and network environment, and start the corresponding model publisher, model compressor, model forwarder, and model subscriber processes; among which: The model publisher is used to prepare and publish the AI ​​model and is in the sending state; The model compressor is used to calculate model differences and compress them, is in a forwarding state, and decouples the model publisher and the model receiver in a proxy manner; The model forwarder is used to optimize transmission delay through multi-path transmission and is in a forwarding state; The model subscriber is used to receive, decompress and assemble models, and is in receiving, sleeping and forwarding states.

3. The distributed model distribution method of cloud-assisted transparent compression according to claim 1 is characterized in that: The differences between the AI ​​models that need to be updated when computing between cloud-side nodes and client-side nodes include: The publisher of the model and the subscribers of each model calculate the difference between the cloud and local AI models by comparing the model's tensor differences. If there is a difference between the remote and local bucket arrays, the tensor needs to be transmitted, otherwise it is skipped.

4. The distributed model distribution method of cloud-assisted transparent compression according to claim 1 is characterized in that: The difference part of the AI ​​model includes several tensors to be transmitted; The cloud-side nodes transparently compress the differences in the AI ​​model, including: The model compressor on the cloud side compresses each tensor to be transmitted using a transparent compression method. The transparent compression method refers to using a set of bucket vectors to splice the tensors into bucket vectors, and then using sparse encoding on the bucket vectors, including only the top-k elements. At the same time, quantization is used to represent each non-zero value through low-bit quantization; the compressed model is serialized and stored in the file system through pickle.

5. The distributed model distribution method of cloud-assisted transparent compression according to claim 1 is characterized in that: The cloud-side nodes transmit the compressed difference parts of the AI ​​model to the client-side nodes through P2P multipath transmission, including: The cloud side starts the BitTorrent transmission protocol, and the nodes on the end side join the BitTorrent protocol, and the difference parts of the compressed AI model are transmitted to the nodes on the end side through the collaboration of the nodes on the end side.

6. The distributed model distribution method of cloud-assisted transparent compression according to claim 4 is characterized in that: The client node decompresses the difference part of the compressed AI model and then assembles it with the local identical part into a complete updated model, including: The model subscriber of the node on the end side reversely decompresses the sparse and quantized bucket vectors, restores each tensor, and combines them with the local unupdated parts to form a dictionary of the AI ​​model, which is serialized and stored in a local file through pickle.

7. The distributed model distribution method of cloud-assisted transparent compression according to claim 1 is characterized in that: AI models are organized as dictionaries, including metadata at each level of the model and the corresponding parameter tensors.

8. A distributed model distribution network system with cloud-assisted transparent compression, characterized in that: The system includes: a model publisher, a model compressor, a model forwarder and a model subscriber; The model publisher is a node on the cloud side, which is in a sending state and is used to publish the model; The model compressor is composed of dedicated sites on the cloud side, and the nodes are in a forwarding state for model compression; The model forwarder is composed of nodes on the end side, providing multipath overlay network routing, and the nodes are in a forwarding state for forwarding the model; The model subscriber is composed of nodes on the end side, and the states of the nodes include receiving, sleeping and forwarding, which are used to upgrade and update the model; When distributing the model, the model publisher prepares the AI ​​model, generates the AI ​​model file, and publishes the AI ​​model file; the model publisher and each model subscriber collaboratively calculate the difference part of the AI ​​model that needs to be updated, and the model compressor transparently compresses the difference part of the AI ​​model; the cloud-side node transmits the compressed difference part of the AI ​​model to the end-side node through P2P multipath transmission; the end-side node decompresses the received compressed difference part of the AI ​​model, and then assembles it with the same part locally into a complete updated model.

9. The distributed model distribution network system with cloud-assisted transparent compression according to claim 1, characterized in that: The sleep state is only responsible for the communication of metadata, not the distribution of the model. The forwarding state can be used in conjunction with sending and receiving.

10. The distributed model distribution network system with cloud-assisted transparent compression according to claim 1, characterized in that: The cloud-side nodes transmit the compressed difference parts of the AI ​​model to the client-side nodes through P2P multipath transmission, including: The cloud side starts the BitTorrent transmission protocol, and the nodes on the end side join the BitTorrent protocol, and the difference parts of the compressed AI model are transmitted to the nodes on the end side through the collaboration of the nodes on the end side.

Images