Model data distribution system and method, storage medium and electronic equipment

Abstract

The invention relates to a model data distribution system and method, a storage medium and electronic equipment. The system comprises a cloud service agent used for dividing model data of a target model into multiple pieces of fragmented data according to fragmentation logic, uploading each piece of fragmented data to a cloud server in sequence, and sending corresponding uploading information to a distribution gateway while uploading of each piece of fragmented data is completed; the distribution gateway is used for acquiring configuration information of the cluster from the database, generating a distribution task of the target fragment data according to the configuration information and target uploading information when the target uploading information of the target fragment data is received, and sending the distribution task to the cluster; and the cluster is used for acquiring the target fragmented data from the cloud server when receiving the distribution task of the target fragmented data, and caching the target fragmented data. The technical problems that large model transmission consumes long time and the updating iteration frequency is influenced are solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a distribution system, method, storage medium and electronic device for model data. Background Art

[0002] With the maturity and popularity of large models, their technology has been widely used in cloud computing, edge computing, the Internet of Things, and enterprise-level applications. Especially in scenarios such as advertising and content recommendation that require extremely high real-time and accuracy, the rapid update and iteration frequency of the model directly affect business results and corporate competitiveness. However, these models are usually trained with massive data, ranging from several GB to hundreds of GB. How to efficiently distribute and iterate the model has become a core technical challenge. At present, the technical directions for rapid iteration of models include model compression and quantization, distributed storage and transmission, edge computing and caching, and optimization of model distribution platforms. Model compression technology reduces the number of model parameters through pruning and knowledge distillation, while quantization further reduces the model size and reduces the computational complexity by converting parameters into low-precision integers. Distributed storage uses distributed file systems or object storage systems to significantly improve storage and access efficiency; at the same time, transmission optimization technologies such as sharding transmission and breakpoint resumption effectively shorten transmission time. Edge caching reduces the computing pressure and data transmission delay of the central node by directly storing and reasoning on edge devices. Despite the existence of many optimization technologies, the problems of large model size and long transmission time are still prominent. Especially in real-time business, the frequency of model iteration directly determines the accuracy and profitability of the recommendation system, but delays in model updates may lead to decreased corporate profits. Summary of the invention

[0003] The present application provides a model data distribution system, method, storage medium and electronic device to solve the technical problem that the transmission of large models takes a long time and affects the update iteration frequency.

[0004] In a first aspect, the present application provides a model data distribution system, comprising: a cloud service agent, for dividing the model data of a target model into multiple shard data according to the shard logic, uploading each of the above shard data to the cloud server in turn, and sending the corresponding upload information to the above distribution gateway when each of the above shard data is uploaded; the above distribution gateway, for obtaining the configuration information of the cluster from the database, and upon receiving the target upload information of the target shard data, generating a distribution task for the above target shard data according to the above configuration information and the above target upload information, and sending the above distribution task to the above cluster, wherein the above target shard data is any one of all the shard data of the above model data; the above cluster, for obtaining the above target shard data from the above cloud server and caching the above target shard data when receiving the distribution task for the above target shard data.

[0005] As an optional example, the above-mentioned cluster includes at least one distribution component, and the above-mentioned distribution gateway sends the generated distribution task to the corresponding at least one distribution component, and the above-mentioned distribution component is used to: when receiving the distribution task of the above-mentioned target shard data, obtain the above-mentioned target shard data from the above-mentioned cloud server; cache the above-mentioned target shard data in the corresponding local memory.

[0006] As an optional example, the distribution component is also used to: when all shard data of the model data are not cached, obtain the missing shard data from other distribution components through the target protocol to cache all shard data of the model data.

[0007] As an optional example, the cluster further includes at least one inference node, which is used to: determine a target distribution component from all distribution components of the cluster; and obtain model data of the target model from a local memory corresponding to the target distribution component.

[0008] As an optional example, the inference node is further used to: determine, among all the distribution components of the cluster, a distribution component for which the inference node is available and which is closest to the inference node as the target distribution component.

[0009] As an optional example, the target inference node and any distribution component available to the target inference node are deployed on the same machine, or deployed on two different machines whose distance is less than the target threshold, wherein the target inference node is any inference node of the cluster.

[0010] In a second aspect, the present application provides a model data distribution method, which is applied to a model data distribution system, including: obtaining cluster configuration information from a database; upon receiving target upload information of target shard data sent by a cloud service agent, generating a distribution task for the target shard data based on the above configuration information and the above target upload information, wherein the target shard data is any one of a plurality of shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each of the shard data to the cloud server in turn; sending the distribution task to the cluster, so that the cluster obtains the target shard data from the cloud server and caches the target shard data.

[0011] In a third aspect, the present application provides a model data distribution device, which is applied to a model data distribution system, including: an acquisition module, which is used to obtain the configuration information of the cluster from a database; a generation module, which is used to generate a distribution task for the target shard data based on the above configuration information and the above target upload information when receiving the target upload information of the target shard data sent by the cloud service agent, wherein the above target shard data is any one of the multiple shard data into which the model data of the target model is divided by the above cloud service agent according to the sharding logic, and the above target upload information is the upload information generated by the above cloud service agent when the upload of the above target shard data is completed in the process of uploading each of the above shard data to the cloud server in turn; a sending module, which is used to send the above distribution task to the above cluster, so that the above cluster obtains the above target shard data from the above cloud server and caches the above target shard data.

[0012] In a fourth aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the above-mentioned model data distribution method when executed by a processor.

[0013] In a fifth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned model data distribution method through the computer program.

[0014] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0015] The present application adopts a cloud service agent, which is used to divide the model data of the target model into multiple shard data according to the shard logic, upload each of the above shard data to the cloud server in turn, and send the corresponding upload information to the above distribution gateway when each of the above shard data is uploaded; the above distribution gateway is used to obtain the configuration information of the cluster from the database, and when receiving the target upload information of the target shard data, generate the distribution task of the above target shard data according to the above configuration information and the above target upload information, and send the above distribution task to the above cluster, wherein the above target shard data is any one of all the shard data of the above model data; the above cluster is used to obtain the above target shard data from the above cloud server when receiving the distribution task of the above target shard data, and cache the above target shard data. Since in the above system, the cloud service agent, the distribution gateway and the cluster work together, the cloud service agent divides the model into multiple shard data according to the custom logic, and uploads them to the cloud server one by one, and sends the upload information to the distribution gateway in real time. The distribution gateway generates distribution tasks and distributes them to cluster nodes according to the cluster configuration information and upload information. After receiving the task, the cluster node pulls the target shard from the cloud server and caches it to provide support for subsequent reasoning or computing tasks. This achieves efficient distribution of large-scale models, improves model distribution efficiency, and shortens transmission time, thereby solving the technical problem of large model transmission taking a long time and affecting the frequency of update iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 is a schematic diagram of the structure of an optional model data distribution system according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of an optional model data distribution system according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of an optional method for distributing model data according to an embodiment of the present application;

[0022] Figure 4 is a schematic structural diagram of an optional model data distribution device according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0026] According to a first aspect of an embodiment of the present application, a model data distribution system is provided. Optionally, as Figure 1 As shown, the above system includes:

[0027] The cloud service agent 102 is used to divide the model data of the target model into multiple shard data according to the shard logic, upload each shard data to the cloud server in turn, and send the corresponding upload information to the distribution gateway when each shard data is uploaded;

[0028] The distribution gateway 104 is used to obtain the configuration information of the cluster from the database, and when receiving the target upload information of the target shard data, generate a distribution task of the target shard data according to the configuration information and the target upload information, and send the distribution task to the cluster, wherein the target shard data is any shard data of all the shard data of the model data;

[0029] Cluster 106 is used to obtain the target shard data from the cloud server and cache the target shard data when receiving a distribution task of the target shard data.

[0030] Optionally, in this embodiment, the distribution system is intended to efficiently manage and distribute large-scale model data to solve the problems of long time consumption and high resource usage in the traditional model transmission process. The system is mainly composed of the following three core modules: cloud service agent (OSS SDK Proxy), distribution gateway and cluster. The main functions of the cloud service agent: receive the complete data of the target model (i.e. Checkpoint); divide the model data into multiple shards according to the custom sharding logic. For example, a 10GB model may be divided into multiple 100MB shards. Upload the shard data to the cloud server (OSS, ObjectStorage Service) in sequence to ensure that the shard data is stored independently, improve parallelism and transmission efficiency, and immediately send the upload information (such as shard identification, shard data information, storage path, etc.) to the distribution gateway when each shard is uploaded. The main functions of the distribution gateway: obtain the configuration information of the cluster (such as the number of nodes, storage strategy, load status, etc.) from the database, receive the shard upload information sent by the cloud service agent, generate the distribution task of the target shard according to the configuration information and upload information, and send the task to the corresponding cluster node. The main functions of the cluster are: receiving distribution tasks, actively pulling target shard data from the cloud server, caching the pulled shard data in the node's memory or disk, and preparing for subsequent reasoning or other computing tasks.

[0031] Optionally, in this embodiment, the overall implementation process of the distribution system is as follows: the cloud service agent divides the model data into shards and uploads them to the cloud server. After each shard is uploaded, the agent notifies the distribution gateway. The distribution gateway parses the uploaded information and generates a distribution task in combination with the cluster configuration. The cluster pulls and caches the shard data from the cloud server according to the task.

[0032] Optionally, in this embodiment, the cloud service agent allows multiple shards to be uploaded simultaneously according to the sharding logic, reducing the overall transmission time. If an interruption occurs in the middle, only the failed shards need to be re-uploaded, without the need to repeatedly upload the entire model, and the distribution gateway is notified immediately after the upload is completed, thereby improving the response speed of subsequent tasks. The distribution gateway dynamically generates distribution tasks based on cluster configuration and upload information, optimizes resource allocation, and ensures that the sharded data can be reasonably distributed to multiple nodes in the cluster to avoid resource bottlenecks. After the cluster caches the sharded data, the inference node can access it directly, reducing the cloud transmission delay for each calculation. Through distributed caching, single-point storage bottlenecks are avoided, and the scalability of the system is improved. This model data distribution system realizes the efficient distribution of large-scale models through the collaborative work of cloud service agent, distribution gateway and cluster. Through technologies such as sharding transmission, parallel uploading, and dynamic task scheduling, the system greatly improves the efficiency of model distribution, shortens the transmission time, and supports breakpoint continuation, reducing the risk of failure. Distributed caching reduces bandwidth and storage pressure, optimizes resource utilization, and meets the high-frequency iteration and real-time requirements of the model. It not only improves the distribution speed and reliability, but also enhances the flexibility of the system, providing strong technical support for high real-time scenarios such as advertising recommendation and content distribution.

[0033] As an optional example, the cluster includes at least one distribution component, and the distribution gateway sends the generated distribution task to the corresponding at least one distribution component, and the distribution component is used to:

[0034] When receiving the distribution task of the target shard data, obtain the target shard data from the cloud server;

[0035] Cache the target shard data in the corresponding local memory.

[0036] Optionally, in this embodiment, in the system, the cluster includes at least one distribution component (that is, distribution Agent), which plays a key role in the process of distributing model data, and is responsible for obtaining shard data from the cloud server and caching it locally, providing efficient data access support for reasoning nodes. The following is the specific working mechanism: the distribution component receives the distribution task sent by the distribution gateway, and the distribution task contains detailed information of the target shard data, such as shard identification, cloud server storage path, task priority, etc. The distribution gateway allocates tasks to specific distribution components according to the cluster configuration and node load to achieve balanced resource usage. The distribution component actively pulls the corresponding shard data according to the cloud server path provided by the distribution task, supports breakpoint continuation of shard data, ensures rapid recovery even in the event of transmission interruption, and improves system reliability. The distribution component can use technologies such as parallel pulling and batch transmission to speed up the speed of large-scale data distribution, support caching while pulling, and reduce waiting time after pulling is completed. The distribution component caches the pulled shard data into the local memory. If needed, it can also be cached to the local disk as an extension or backup of the memory, to achieve efficient organization and indexing of cached data, facilitate rapid access by inference nodes, support on-demand elimination strategy to clean up expired or infrequently used data, and optimize resource utilization. Multiple distribution components in the cluster work together to share shard data through distributed cache, forming a P2P-like distribution network. Inference nodes can directly pull the required shard data from nearby distribution components, reducing transmission delays and bandwidth pressure on cloud servers.

[0037] Optionally, in this embodiment, by dynamically allocating distribution tasks and pulling distribution components in parallel, the acquisition time of model shard data is significantly shortened to meet the high-frequency iteration requirements of large models. After the data shards are cached in local memory, the inference node can quickly access the required data, reducing the time consumption of reading directly from the cloud server. Distributed caching and breakpoint resume support reduce the impact of single point failures on distribution tasks and ensure the stable operation of the system.

[0038] As an optional example, the distribution component is also used to:

[0039] In the case where all shard data of the model data are not cached, the missing shard data are obtained from other distribution components through the target protocol to cache all shard data of the model data.

[0040] Optionally, in this embodiment, in the model data distribution system, in addition to obtaining the target shard data from the cloud server, the distribution component can also obtain uncached shard data from other distribution components through internal communication in the cluster. This function enhances the flexibility and efficiency of data distribution and ensures that the cluster can quickly complete the complete caching of model data. Specifically, when receiving a distribution task, the distribution component first checks whether the local cache has completely stored all the shard data of the model. If all the shard data is not cached, the distribution component will identify the missing shards and record the list of data to be obtained. The distribution component uses a specific protocol (such as P2P protocol or RPC communication) to collaborate with other distribution components in the cluster. The distribution component broadcasts the request for the missing shard to other components in the cluster. The distribution component that has cached the missing shard data responds to the request and provides a data path or directly transmits the shard data. Data is transmitted directly between two distribution components, reducing the pressure on the cloud server, supporting the simultaneous pulling of missing shards from multiple distribution components, improving the transmission speed, supporting breakpoint continuation, and ensuring that data acquisition can be completed reliably even in the case of network fluctuations. The distribution component merges the shard data obtained from other components with the existing cached data to ensure the integrity of the model data, stores the newly obtained shard data in local memory or disk, and updates the local shard data index table to facilitate subsequent rapid data location.

[0041] Optionally, in this embodiment, the shard sharing mechanism within the cluster is used to reduce repeated requests to the cloud server and speed up the complete caching of the model data.

[0042] As an optional example, the cluster further includes at least one inference node, where the inference node is used to:

[0043] Determine a target distribution component from all distribution components of the cluster;

[0044] Get the model data of the target model from the local memory corresponding to the target distribution component.

[0045] Optionally, in this embodiment, in the distributed model distribution system, the inference node is responsible for performing the model inference task and needs to efficiently obtain the complete model data. By selecting a suitable distribution component in the cluster and directly obtaining data from its local cache, the inference node can significantly reduce the transmission delay and improve the real-time performance of the inference task. The inference node initiates a request from all available distribution components in the cluster to understand which distribution components have completely cached the target model data. The target distribution component is selected according to the following criteria: give priority to the nearest distribution component to reduce network latency; comprehensively consider the load of the current distribution component to avoid excessive occupation of the resources of a component; ensure that the target distribution component has complete model shard data. The inference node obtains the model data from the local memory of the target distribution component through an efficient transmission protocol. The inference node directly obtains the model data from the distribution component, avoiding repeated requests to the cloud server, and significantly reducing the cloud load and bandwidth occupancy. The inference node gives priority to the distribution component that caches the data, avoiding long link transmission and reducing delays. The inference node selects the target distribution component according to the real-time situation, balances the resource usage of the distribution components in the cluster, and improves the overall system stability. The cache sharing mechanism of the distribution component is combined with the intelligent selection strategy of the inference node to ensure that the inference node can quickly obtain complete model data to meet real-time inference requirements.

[0046] As an optional example, the Inference Node is also used to:

[0047] A distribution component in which the inference node is available and closest to the inference node among all distribution components in the cluster is determined as a target distribution component.

[0048] Optionally, in this embodiment, in the distributed model distribution system, in order to improve the real-time performance and efficiency of the reasoning task, the reasoning node selects the available and closest distribution component from all the distribution components in the cluster as the target distribution component through a specific strategy to obtain the model data. This selection mechanism makes full use of the shard sharing and geographical distribution characteristics within the cluster. The reasoning node will first filter out components that meet the following conditions from all the distribution components in the cluster: the component is running normally and is not under maintenance or downtime; the distribution component caches the model data required by the reasoning node; the reasoning node will measure the communication delay, bandwidth and other network indicators between each distribution component according to the network topology, and give priority to the distribution component with lower load among multiple equidistant distribution components to ensure stable data transmission.

[0049] As an optional example, the target inference node and any distribution component available to the target inference node are deployed on the same machine, or deployed on two different machines whose distance is less than the target threshold, wherein the target inference node is any inference node of the cluster.

[0050] Optionally, in this embodiment, in order to improve the performance of the model distribution system, the target inference node and its available distribution components follow the following optimization strategy when deploying: both can be deployed on the same machine, or on different machines whose distance is less than a set threshold. This mechanism improves the real-time performance and reliability of the system by reducing data transmission overhead and network latency. Whether deployed on the same machine or adjacent to each other, the target inference node can quickly obtain the required data from the distribution component, greatly reducing the model loading time.

[0051] An example is used to illustrate a model data distribution system, which aims to efficiently manage and distribute large-scale model data to solve the problems of long time consumption and high resource consumption in the traditional model transmission process, such as Figure 2 As shown in the figure, the system mainly consists of the following three core modules: cloud service proxy (OSSSDK Proxy), distribution gateway and cluster. The cluster can be multiple clusters. The overall implementation process is as follows:

[0052] 1. Generate the latest target model (Checkpoint) through business logic (such as scheduled tasks, manual triggering, etc.). The data of this model is usually large.

[0053] 2. The cloud service agent (OSSSDK Proxy) first divides the data of the target model into multiple shards according to the custom sharding logic, and uploads each shard to the cloud server (OSS). When each shard is uploaded, it sends the relevant information of the upload completion (such as shard number, upload status, etc.) to the distribution gateway.

[0054] 3. After the distribution gateway receives the shard upload completion information sent by the cloud service agent, it obtains the cluster configuration information from the database (DB), generates the distribution task of the shard data based on the configuration information and the uploaded target shard information, and sends the distribution task to the relevant distribution component in the cluster, instructing it to obtain the target shard data from the cloud server.

[0055] 4. After receiving the task sent by the distribution gateway, the distribution component (distribution agent) obtains the target shard data from the cloud server and caches the target shard data in the local memory of the distribution component. If the distribution component does not cache all the complete shard data, it will make data requests with other distribution components in the cluster through P2P and obtain the missing shard data from other distribution components.

[0056] 5. The inference node selects the required model data and obtains the data provided by the distribution component according to system requirements. The inference node obtains the required target model data by finding the available and nearest distribution component in the cluster. Once the inference node obtains the required data from the target distribution component, it can perform the inference task. At this time, all model shard data has been completely cached and can be efficiently inferred.

[0057] 6. After the inference node completes the task, it returns the result to the user application or other processing module. If there is a new model version that needs to be inferred, the system will automatically check and update the model distribution to ensure that the inference node always obtains the latest model data.

[0058] Through the implementation of this distribution system, the model distribution and reasoning process have been significantly optimized, ensuring efficient model updates and real-time reasoning task execution, helping enterprises maintain their competitive advantage in an environment with large-scale data and frequent iterations.

[0059] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0060] According to a second aspect of the embodiment of the present application, a method for distributing model data is also provided, which is applied to a distribution system for model data, such as Figure 3 As shown, the above method includes:

[0061] S302, obtaining cluster configuration information from a database;

[0062] S304, upon receiving the target upload information of the target shard data sent by the cloud service agent, generating a distribution task for the target shard data according to the configuration information and the target upload information, wherein the target shard data is any one of the multiple shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each shard data to the cloud server in sequence;

[0063] S306, sending the distribution task to the cluster, so that the cluster obtains the target shard data from the cloud server and caches the target shard data.

[0064] Optionally, in this embodiment, it is composed of the following three core modules: cloud service proxy (OSS SDK Proxy), distribution gateway and cluster. The main functions of the cloud service proxy: receive the complete data of the target model (i.e. Checkpoint); divide the model data into multiple shards according to the custom sharding logic. For example, a 10GB model may be divided into multiple 100MB shards. Upload the shard data to the cloud server (OSS, Object Storage Service) in sequence to ensure independent storage of the shard data, improve parallelism and transmission efficiency, and immediately send the upload information (such as shard identification, shard data information, storage path, etc.) to the distribution gateway when each shard is uploaded. The main functions of the distribution gateway: obtain the configuration information of the cluster (such as the number of nodes, storage strategy, load conditions, etc.) from the database, receive the shard upload information sent by the cloud service proxy, generate the distribution task of the target shard according to the configuration information and upload information, and send the task to the corresponding cluster node. The main functions of the cluster are: receiving distribution tasks, actively pulling target shard data from the cloud server, caching the pulled shard data in the node's memory or disk, and preparing for subsequent reasoning or other computing tasks.

[0065] Optionally, in this embodiment, the overall implementation process is as follows: the cloud service agent divides the model data into shards and uploads them to the cloud server. After each shard is uploaded, the agent notifies the distribution gateway. The distribution gateway parses the uploaded information and generates a distribution task in combination with the cluster configuration. The cluster pulls and caches the shard data from the cloud server according to the task.

[0066] Optionally, in this embodiment, the cloud service agent allows multiple shards to be uploaded simultaneously according to the sharding logic, reducing the overall transmission time. If an interruption occurs in the middle, only the failed shards need to be re-uploaded, without the need to repeatedly upload the entire model, and the distribution gateway is notified immediately after the upload is completed, thereby improving the response speed of subsequent tasks. The distribution gateway dynamically generates distribution tasks based on the cluster configuration and upload information, optimizes resource allocation, and ensures that the shard data can be reasonably distributed to multiple nodes in the cluster to avoid resource bottlenecks. After the cluster caches the shard data, the inference node can access it directly, reducing the cloud transmission delay for each calculation. Through distributed caching, single-point storage bottlenecks are avoided, and the scalability of the system is improved. Through the collaborative work of the cloud service agent, the distribution gateway and the cluster, the efficient distribution of large-scale models is achieved. Through technologies such as sharding transmission, parallel uploading, and dynamic task scheduling, the model distribution efficiency is greatly improved, the transmission time is shortened, and breakpoint continuation is supported to reduce the risk of failure. Distributed caching reduces bandwidth and storage pressure, optimizes resource utilization, and meets the high-frequency iteration and real-time requirements of the model. It not only improves the distribution speed and reliability, but also enhances the flexibility of the system, providing strong technical support for high-real-time scenarios such as advertising recommendation and content distribution.

[0067] For other examples of this embodiment, please refer to the examples and will not be repeated here.

[0068] According to a third aspect of the embodiment of the present application, a model data distribution device is also provided, which is applied to a model data distribution system, such as Figure 4 As shown, including:

[0069] The acquisition module 402 is used to acquire the configuration information of the cluster from the database;

[0070] A generation module 404 is used to generate a distribution task for the target shard data according to the configuration information and the target upload information when receiving the target upload information of the target shard data sent by the cloud service agent, wherein the target shard data is any one of the multiple shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each shard data to the cloud server in sequence;

[0071] The sending module 406 is used to send the distribution task to the cluster, so that the cluster obtains the target shard data from the cloud server and caches the target shard data.

[0072] It should be noted that the acquisition module 402 in this embodiment can be used to execute step S302 in the embodiment of the present application, the generation module 404 in this embodiment is used to execute step S304 in the embodiment of the present application, and the sending module 406 in this embodiment can be used to execute step S306 in the embodiment of the present application.

[0073] For other examples of this embodiment, please refer to the examples and will not be repeated here.

[0074] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506 and a communication bus 508, wherein the processor 502, the communication interface 504 and the memory 506 communicate with each other through the communication bus 508, wherein,

[0075] Memory 506, used for storing computer programs;

[0076] The processor 502 is used to execute the computer program stored in the memory 506 to implement the following steps:

[0077] Get the cluster configuration information from the database;

[0078] When receiving the target upload information of the target shard data sent by the cloud service agent, generating a distribution task of the target shard data according to the configuration information and the target upload information, wherein the target shard data is any one of the multiple shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each shard data to the cloud server in turn;

[0079] Send the distribution task to the cluster so that the cluster can obtain the target shard data from the cloud server and cache the target shard data.

[0080] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface is used for communication between the above electronic device and other devices.

[0081] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0082] As an example, the memory 506 may include but is not limited to the acquisition module 402, the generation module 404 and the sending module 406 in the distribution system device of the model data. In addition, it may also include but is not limited to other module units in the distribution device of the model data, which will not be repeated in this example.

[0083] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0084] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0085] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only. The device for implementing the above-mentioned model data distribution method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 5 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0086] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0087] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned model data distribution method when executed by a processor.

[0088] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0089] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0090] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0091] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0092] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0095] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model data distribution system, characterized in that: include: The cloud service agent is used to divide the model data of the target model into multiple shard data according to the shard logic, upload each of the shard data to the cloud server in turn, and send the corresponding upload information to the distribution gateway when each of the shard data is uploaded; The distribution gateway is used to obtain the configuration information of the cluster from the database, and when receiving the target upload information of the target shard data, generate the distribution task of the target shard data according to the configuration information and the target upload information, and send the distribution task to the cluster, wherein the target shard data is any shard data of all the shard data of the model data; The cluster is used to obtain the target shard data from the cloud server and cache the target shard data when receiving a distribution task for the target shard data.

2. The system according to claim 1, characterized in that The cluster includes at least one distribution component, and the distribution gateway sends the generated distribution task to the corresponding at least one distribution component, and the distribution component is used to: Upon receiving the distribution task of the target shard data, obtaining the target shard data from the cloud server; The target shard data is cached in the corresponding local memory.

3. The system according to claim 2, characterized in that The distribution component is also used to: In the case that all the shard data of the model data are not cached, the missing shard data are obtained from other distribution components through the target protocol to cache all the shard data of the model data.

4. The system according to claim 3, characterized in that The cluster further comprises at least one inference node, wherein the inference node is configured to: Determine a target distribution component from all distribution components of the cluster; The model data of the target model is obtained from the local memory corresponding to the target distribution component.

5. The system according to claim 4, characterized in that The inference node is also used to: A distribution component in which the inference node is available and which is closest to the inference node among all distribution components in the cluster is determined as the target distribution component.

6. The system according to claim 5, characterized in that The target inference node and any distribution component available to the target inference node are deployed on the same machine, or deployed on two different machines whose distance is less than a target threshold, wherein the target inference node is any inference node of the cluster.

7. A method for distributing model data, characterized in that: A distribution system for model data, including: Get the cluster configuration information from the database; Upon receiving the target upload information of the target shard data sent by the cloud service agent, generating a distribution task for the target shard data according to the configuration information and the target upload information, wherein the target shard data is any one of the multiple shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each of the shard data to the cloud server in sequence; The distribution task is sent to the cluster, so that the cluster obtains the target shard data from the cloud server and caches the target shard data.

8. A distribution device for model data, characterized in that: A distribution system for model data, including: The acquisition module is used to obtain the configuration information of the cluster from the database; A generation module, configured to generate a distribution task for the target shard data according to the configuration information and the target upload information when receiving the target upload information of the target shard data sent by the cloud service agent, wherein the target shard data is any one of the multiple shard data into which the model data of the target model is divided by the cloud service agent according to the sharding logic, and the target upload information is the upload information generated by the cloud service agent when the upload of the target shard data is completed in the process of uploading each of the shard data to the cloud server in sequence; The sending module is used to send the distribution task to the cluster, so that the cluster obtains the target shard data from the cloud server and caches the target shard data.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to claim 7 is performed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method of claim 7 through the computer program.

Images