Large model segmentation and deployment method and device based on cloud edge-end collaboration, and medium

Abstract

The invention provides a large model segmentation and deployment method and device based on cloud edge end collaboration and a medium, and the method comprises the steps: giving a pre-trained large model as input, and evaluating the performance and resource consumption conditions of the large model during cloud edge end deployment; determining a segmentation point of the large model according to a performance evaluation result and a resource consumption condition; decomposing the large model into a plurality of sub-models according to the segmentation points; deploying each sub-model at a cloud end and an edge end; and establishing a cloud side end cooperative work mechanism, so that the sub-models cooperatively work at the cloud side and the edge side. The method can reduce the demand for computing resources and improve the response speed while ensuring the model performance, and is suitable for the demand in the field of intelligent manufacturing.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a large model segmentation and deployment method, device, and readable storage medium based on cloud-edge-end collaboration. Background Art

[0002] In the field of intelligent manufacturing, with the expansion of production scale and the increase of product complexity, traditional manufacturing models have been unable to meet the needs of high-precision and high-efficiency production. With the rapid development of artificial intelligence and cloud computing technologies, large neural network models are widely used in various aspects of intelligent manufacturing to achieve intelligent decision-making, optimize production processes and improve production efficiency. The complexity of large neural network models and the limited computing resources have become the main problems restricting their application in intelligent manufacturing. On the one hand, large models require huge computing resources to support their training and reasoning processes, which is a huge challenge for edge devices with limited resources; on the other hand, intelligent manufacturing has very high requirements for real-time performance and response speed, and large models often have delays and bandwidth limitations when reasoning in the cloud. Summary of the invention

[0003] In response to the shortcomings of the prior art, the present invention proposes a large model segmentation and deployment method, device, and readable storage medium based on cloud-edge-end collaboration. This method can reduce the demand for computing resources and improve response speed while ensuring model performance, and is suitable for the needs of the intelligent manufacturing field.

[0004] In order to achieve the above objectives, the present invention provides a large model segmentation and deployment method based on cloud-edge-end collaboration, comprising: Given a pre-trained large model as input, evaluate the performance and resource consumption of the large model when deployed on the cloud edge. Determine the split point of the large model according to the performance and resource consumption; Decomposing the large model into multiple sub-models according to the segmentation point; Deploy each sub-model on the cloud and edge; Establish a cloud-edge collaborative working mechanism so that each sub-model can work collaboratively in the cloud and edge.

[0005] In some embodiments, based on the computational complexity of the large model, the resource consumption of the large model when deployed on the cloud edge is evaluated. The computational complexity is determined by the theoretical amount of computation and the number of parameters of the large model.

[0006] In some embodiments, the theoretical computation amount is represented by a theoretical floating-point computation speed, which is obtained by multiplying the main frequency, the number of floating-point computations per clock cycle, the number of cores per CPU, and the number of CPUs per node; The number of parameters is determined by the number of input channels and the number of output channels, and is expressed as: Among them, Co represents the number of input channels, Ci represents the number of output channels, is the size of the convolution kernel.

[0007] In some embodiments, a knowledge decomposition technique is used to identify multiple segmentation components in the large model according to the determined segmentation points, wherein the multiple segmentation components include key components, modules or layers; Evaluate the independence, reusability, and complexity of each segment to determine whether it is suitable as an independent sub-model.

[0008] In some embodiments, each sub-model is deployed on the cloud and the edge, including: Determine the resources required by each sub-model in the deployment environment based on its functions and task characteristics; Assess the resource situation in the cloud and at the edge; According to the resources of each sub-model and the resource evaluation results of the cloud and edge, determine the sub-model suitable for deployment on the cloud and the sub-model suitable for deployment on the edge; The sub-model deployed in the cloud is a sub-model whose computational complexity exceeds a complexity threshold and whose data processing volume exceeds a processing volume threshold; The sub-model deployed at the edge is a sub-model whose computational complexity does not exceed the complexity threshold, and / or the data processing volume does not exceed the processing volume threshold, or a sub-model that requires real-time data collection and rapid response.

[0009] In some embodiments, establishing a cloud-edge-end collaborative working mechanism includes: Select a suitable distributed computing framework based on the system data processing requirements and task computing type; In this distributed computing framework, the tasks and roles of each sub-model are defined, and the collaboration mode and relationship between the sub-models are determined; Formulate communication methods and collaboration rules between sub-models; Develop a task scheduling and allocation mechanism to reasonably allocate tasks to each sub-model for processing based on system requirements and resource conditions; Establish a data exchange and sharing mechanism to enable real-time sharing of data and information between sub-models.

[0010] In some embodiments, it further comprises: Adaptively optimize each sub-model deployed at the edge. Specifically, in one embodiment, the performance indicators of each sub-model at the edge are monitored in real time; and the sub-model parameters and resource allocation are dynamically adjusted according to the performance indicators of each sub-model.

[0011] On the other hand, the present invention further provides a large model segmentation and deployment device based on cloud-edge-end collaboration, adopting the above-mentioned large model segmentation and deployment method based on cloud-edge-end collaboration, and the large model segmentation and deployment device based on cloud-edge-end collaboration includes: The evaluation module is used to evaluate the performance and resource consumption of a pre-trained large model when it is deployed on the cloud edge. A segmentation module, used to determine the segmentation points of the large model according to the performance evaluation results and resource consumption; and Decomposing the large model into multiple sub-models according to the determined split points; Deployment module, used to deploy each sub-model on the cloud and edge; The collaboration module is used to establish a cloud-edge collaborative working mechanism, allowing each sub-model to work collaboratively in the cloud and edge.

[0012] On the other hand, the present invention also provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the steps of the above-mentioned large model segmentation and deployment method based on cloud-edge-end collaboration are implemented, and the same technical effect can be achieved.

[0013] It can be seen from the above scheme that the advantages of the present invention are: The large model segmentation and deployment method based on cloud-edge-end collaboration provided by the present invention divides the large model into multiple sub-models through cloud-edge-end collaboration, with some sub-models deployed in the cloud and some sub-models deployed in the edge, thus realizing efficient segmentation and deployment of large neural network models. While ensuring model performance, it reduces the demand for computing resources, improves resource utilization, and improves response speed, which is suitable for the needs of intelligent manufacturing. At the same time, the method can also perform adaptive optimization according to actual needs, improve the processing capacity and stability of the edge terminal model, and further meet the production needs of intelligent manufacturing for high precision and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The overall process diagram of the large model segmentation and deployment method based on cloud-edge-end collaboration provided by an embodiment of the present invention is shown; Figure 2 The overall architecture diagram of the large model segmentation and deployment device based on cloud-edge-end collaboration is shown; The symbols are: 10: Large model segmentation and deployment device based on cloud-edge-end collaboration 11: Evaluation Module 12: Segmentation module 13: Deployment Module 14: Collaboration Module DETAILED DESCRIPTION

[0015] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0016] The present invention provides a large model segmentation and deployment method based on cloud-edge collaboration, in view of the fact that intelligent manufacturing has very high requirements for real-time and response speed, and large models often have delays and bandwidth limitations when reasoning in the cloud; and that large models require huge computing resources to support their training and reasoning processes, while the resources of edge devices in the field of intelligent manufacturing are limited. This method divides the large model into multiple sub-models, deploys some sub-model computing tasks in the cloud, and uses the powerful computing power and data storage capabilities that the cloud can provide to support model training and complex reasoning tasks; some sub-models are deployed at the edge, and the real-time data processing and response capabilities provided by the edge are used to optimize the utilization of edge resources and respond quickly, thereby meeting the real-time requirements of intelligent manufacturing. The following will refer to Figure 1-Figure 2 , which introduces in detail the large model segmentation and deployment method based on cloud-edge-end collaboration.

[0017] See also Figure 1 , Figure 1 An overall flow chart of a large model segmentation and deployment method based on cloud-edge-end collaboration provided by an embodiment of the present invention is shown.

[0018] The large model segmentation and deployment method based on cloud-edge-end collaboration includes at least the following steps: Step S1: Given a pre-trained large model as input, evaluate the performance and resource consumption of the large model when deployed on the cloud edge.

[0019] In one embodiment, a pre-trained large model is selected as input, and the resource consumption of the large model in the cloud-edge deployment is monitored, including memory usage, computing resource utilization, network bandwidth, etc. These resource consumption data can help you determine whether the large model needs to be split into smaller sub-models. At the same time, the performance of the large model under different deployment conditions is evaluated, including computational complexity indicators. Based on the performance evaluation results, determine whether the large model needs to be split to improve performance.

[0020] The computational complexity is determined by the theoretical amount of computation and the number of parameters of the large model. In some embodiments, the theoretical amount of computation is the number of computations required for model inference, which is represented by the theoretical floating-point computing speed, which is obtained by multiplying the main frequency, the number of floating-point calculations per clock cycle, the number of cores per CPU, and the number of CPUs per node, that is: Among them, CR is the main frequency, c_CPU represents the number of floating-point calculations per clock cycle, n_CPU represents the number of cores per CPU, and n_CPU_node represents the number of CPUs per node.

[0021] In order to more accurately evaluate the computational complexity of large models, the number of parameters of the model also needs to be considered. The number of parameters directly determines the size of the model and also affects the amount of memory used during inference. The number of parameters is determined by the number of input channels and the number of output channels, and is expressed as: Among them, Co represents the number of input channels, Ci represents the number of output channels, is the size of the convolution kernel.

[0022] Step S2: Determine the segmentation points of the large model according to the performance evaluation results and resource consumption.

[0023] Specifically, by analyzing the structure and hierarchy of the large model, determine which parts can be run independently or split into sub-models. Generally, you can consider splitting the large model into sub-models with a certain degree of independence to reduce resource consumption and improve operation efficiency. According to the resource consumption and the structure of the large model, divide the tasks of the large model and determine the appropriate split point. Consider dividing the large model tasks into independent sub-tasks so that they can be assigned to different computing nodes or devices. After determining the split point, you can further conduct experimental verification to evaluate the performance and effect of the split sub-model to adjust and optimize the split point to achieve the best performance and resource utilization efficiency.

[0024] Step S3: decompose the large model into multiple sub-models according to the segmentation points.

[0025] In some embodiments, knowledge decomposition technology is used to identify multiple segmentation components in the large model based on determined segmentation points, and the multiple segmentation components include key components, modules or layers; the independence, reusability and complexity of each segmentation component are evaluated to determine whether the segmentation component is suitable as an independent sub-model.

[0026] Specifically, first, the large model needs to be analyzed, and multiple segmentation components in the large model are identified based on the determined segmentation points. The multiple segmentation components contain key components, modules or layers. Based on the determined segmentation components, the independence, reusability and complexity of each segmentation component are evaluated to determine whether the segmentation component is suitable as an independent sub-model, so as to decompose the large model into multiple sub-models, each of which corresponds to a segmentation component. These sub-models can be independent neural network models or other forms of models, such as rule models or traditional machine learning models.

[0027] Then, if a segmented component is determined to be a sub-model, the corresponding parameters and structures are extracted according to the segmented component to create an independent sub-model. Finally, the extracted sub-models are jointly trained to ensure that they can work together and achieve the overall function. In addition, the trained sub-models are integrated into the overall system, and tested and verified to ensure that the segmented sub-models can work properly.

[0028] Therefore, splitting a large model through knowledge decomposition can make the model easier to understand and manage, and also help improve the flexibility and scalability of the model. This approach is particularly useful when building complex large-scale model systems, and the model can be customized and extended as needed.

[0029] Step S4: deploy each sub-model on the cloud and edge.

[0030] In this embodiment, after the large model is divided into several sub-models, the sub-models are deployed on the cloud and edge according to the needs. Specifically, first, according to the functions and task characteristics of each sub-model, the resources required by the sub-model in the deployment environment are determined, including computational complexity, memory consumption, network communication, etc. And, further evaluate the resource conditions of the cloud and edge, including computing resources, storage resources, memory resources, network resources, security resources, etc. Among them, the sub-model requires sufficient computing resources in the deployment environment to run model reasoning or training tasks. This includes hardware resources such as CPU, GPU or other dedicated accelerators. Ensure that the deployment environment has sufficient computing resources to support the operation of the model. Storage resources: The deployment environment requires sufficient storage resources to store the model itself, input data, output results and other necessary files. Ensure that the storage resources can accommodate the model and related data, while considering the data read and write speed and storage stability. Memory resources: The sub-model needs to use memory to store intermediate results, parameters, gradients and other data during reasoning or training. The deployment environment requires sufficient memory resources to support the operation of the model to avoid performance degradation or program crashes due to insufficient memory. Network resources: If the sub-model needs to communicate with other models or services, the deployment environment needs sufficient network resources to support data transmission and communication. Ensure that the network connection is stable and reliable with low latency to ensure the normal operation of the model. Security resources: Security resources need to be considered in the deployment environment, including firewalls, access control, data encryption and other measures to protect the security of models and data. Ensure that the deployment environment meets security standards to prevent potential security threats and data leaks.

[0031] Then, based on the resources of each sub-model and the resource evaluation results of the cloud and edge, determine the sub-model suitable for deployment on the cloud and the sub-model suitable for deployment on the edge.

[0032] Specifically, since the cloud usually has greater computing resources, it can support more complex models and larger-scale data processing; while the computing resources at the edge are limited, it is necessary to select lightweight models or perform model compression according to the actual situation. In addition, deploying models in the cloud can facilitate management and maintenance, which is suitable for scenarios that require large-scale data processing. Therefore, sub-models whose computational complexity exceeds a complexity threshold and whose data processing volume exceeds a processing volume threshold can be deployed in the cloud, and sub-models whose computational complexity does not exceed the complexity threshold and / or whose data processing volume does not exceed the processing volume threshold can be deployed at the edge.

[0033] In addition, since deploying models at the edge can reduce network latency and improve real-time performance, and is suitable for scenarios that require quick response, sub-models that require real-time data collection and quick response can also be deployed at the edge, such as various sensors.

[0034] In addition, data security can also be considered. Sub-models with higher data security requirements can be deployed on the cloud, because the cloud usually has more complete security protection measures, while deploying models on the edge may require additional consideration of security and privacy protection.

[0035] Step S5: Establish a cloud-edge-end collaborative working mechanism so that each sub-model can work collaboratively on the cloud and edge.

[0036] In this embodiment, after each sub-model is deployed on the cloud and edge according to the needs, a collaborative working mechanism of the cloud and edge is further constructed to ensure that each sub-model works collaboratively on the cloud and edge. Specifically, first, according to the system data processing requirements and the task calculation type, a suitable distributed computing framework is selected. For example, in some embodiments, if the system requirement is to batch process large-scale data, Apache Hadoop or Apache Spark can be considered. If you need to process real-time data streams and have low latency requirements, you can choose Apache Flink or Apache Storm. If the task calculation type requires iterative calculations, such as iterative training in machine learning algorithms, Apache Spark can be selected. If you need to process complex event processing logic, you can consider using Apache Flink. In addition, programming environment factors can be further considered, for example, Python-based frameworks such as PySpark or TensorFlow can be selected. In addition, the present invention is not limited to this, and a framework with good scalability and high performance can also be selected based on the expected load and performance requirements of the system.

[0037] After determining the distributed computing framework, further define the tasks and roles of each sub-model in the distributed computing framework, determine the collaboration mode and relationship between each sub-model; at the same time, formulate the communication mode and collaboration rules between each sub-model; and formulate the task scheduling and allocation mechanism, and reasonably allocate tasks to each sub-model for processing according to system requirements and resource conditions; establish a data exchange and sharing mechanism so that each sub-model can share data and information in real time. Through the above process, a cloud-edge collaborative working mechanism is constructed, so that each sub-model deployed on the cloud and edge can work together according to this collaborative working mechanism.

[0038] In this embodiment, the collaborative working mechanism of the cloud-edge is used to ensure that the various sub-models can collaborate efficiently, thereby improving the overall processing capability and response speed.

[0039] In addition, in some embodiments, each sub-model deployed at the edge can be further adaptively optimized according to actual needs.

[0040] Specifically, in one embodiment, the performance indicators of each sub-model at the edge are monitored in real time, and the sub-model parameters and resource allocation are dynamically adjusted according to the performance indicators of each sub-model. In this embodiment, the processing capacity and stability of the edge terminal model are improved by dynamically adjusting the parameters and structure of the sub-model deployed at the edge, further meeting the production requirements of intelligent manufacturing for high precision and high efficiency.

[0041] In addition, in some embodiments, each sub-model deployed at the edge can be further adaptively optimized according to demand.

[0042] In summary, the present invention is based on the large model segmentation and deployment method of cloud-edge-end collaboration. Through cloud-edge-end collaboration, the large model is segmented into multiple sub-models, some of which are deployed in the cloud, and some are deployed in the edge, achieving efficient segmentation and deployment of large neural network models. While ensuring model performance, it reduces the demand for computing resources, improves resource utilization, and improves response speed, which is suitable for the needs of intelligent manufacturing. At the same time, the method can also perform adaptive optimization according to actual needs, improve the processing power and stability of the edge terminal model, and further meet the production needs of intelligent manufacturing for high precision and high efficiency.

[0043] In addition, the present invention also provides a large model segmentation and deployment device 10 based on cloud-edge-end collaboration, adopting the above-mentioned large model segmentation and deployment method based on cloud-edge-end collaboration, and the large model segmentation and deployment device 10 based on cloud-edge-end collaboration includes: An evaluation module 11 is used to evaluate the resource consumption of a pre-trained large model when it is deployed on the cloud edge. A segmentation module 12, for determining segmentation points of the large model according to the resource consumption; and Decomposing the large model into multiple sub-models according to the determined split points; A deployment module 13 is used to deploy each sub-model on the cloud and edge; The collaboration module 14 is used to establish a cloud-edge collaborative working mechanism so that each sub-model can work collaboratively in the cloud and the edge.

[0044] In addition, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the module described can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0045] In addition, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the above-mentioned large model segmentation and deployment method based on cloud-edge-end collaboration are implemented, and the same technical effect can be achieved.

[0046] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0047] In addition, an embodiment of the present application also provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the steps of the above-mentioned large model segmentation and deployment method based on cloud-edge-end collaboration are implemented, and the same technical effect can be achieved.

[0048] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be applied, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0049] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0050] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A large model segmentation and deployment method based on cloud-edge-end collaboration, characterized in that: Include: Given a pre-trained large model as input, evaluate the performance and resource consumption of the large model when deployed on the cloud edge. Determine the split point of the large model based on the performance evaluation results and resource consumption; Decomposing the large model into multiple sub-models according to the segmentation point; Deploy each sub-model on the cloud and edge; Establish a cloud-edge collaborative working mechanism so that each sub-model can work collaboratively in the cloud and edge.

2. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 1 is characterized in that: According to the computational complexity of the large model, the performance of the large model when deployed on the cloud edge is evaluated. The computational complexity is determined by the theoretical amount of computation and the number of parameters of the large model.

3. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 2 is characterized in that: The theoretical computing capacity is represented by the theoretical floating-point computing speed, which is obtained by multiplying the main frequency, the number of floating-point calculations per clock cycle, the number of cores per CPU, and the number of CPUs per node. The number of parameters is determined by the number of input channels and the number of output channels, and is expressed as: Among them, Co represents the number of input channels, Ci represents the number of output channels, is the size of the convolution kernel.

4. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 1 is characterized in that: Using knowledge decomposition technology, based on the determined segmentation points, multiple segmentation components in the large model are identified, and the multiple segmentation components include key components, modules or layers; Evaluate the independence, reusability, and complexity of each segment to determine whether it is suitable as an independent sub-model.

5. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 1 is characterized in that: Deploy each sub-model on the cloud and edge, including: Determine the resources required by each sub-model in the deployment environment based on its functions and task characteristics; Assess the resource situation in the cloud and at the edge; According to the resources of each sub-model and the resource evaluation results of the cloud and edge, determine the sub-model suitable for deployment on the cloud and the sub-model suitable for deployment on the edge; The sub-model deployed in the cloud is a sub-model whose computational complexity exceeds a complexity threshold and whose data processing volume exceeds a processing volume threshold; The sub-model deployed at the edge is a sub-model whose computational complexity does not exceed the complexity threshold, and / or the data processing volume does not exceed the processing volume threshold, or a sub-model that requires real-time data collection and rapid response.

6. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 1 is characterized in that: Establish a collaborative working mechanism between the cloud, edge and end, including: Select a suitable distributed computing framework based on the system data processing requirements and task computing type; In this distributed computing framework, the tasks and roles of each sub-model are defined, and the collaboration mode and relationship between the sub-models are determined; Formulate communication methods and collaboration rules between sub-models; Develop a task scheduling and allocation mechanism to reasonably allocate tasks to each sub-model for processing based on system requirements and resource conditions; Establish a data exchange and sharing mechanism to enable real-time sharing of data and information between sub-models.

7. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 1 is characterized in that: Also includes: Adaptively optimize each sub-model deployed at the edge.

8. The large model segmentation and deployment method based on cloud-edge-end collaboration according to claim 7 is characterized in that: Real-time monitoring of the performance indicators of each sub-model at the edge; According to the performance indicators of each sub-model, the sub-model parameters and resource allocation are dynamically adjusted.

9. A large model segmentation and deployment device based on cloud-edge-end collaboration, characterized in that: Right to take The method for segmenting and deploying a large model based on cloud-edge-end collaboration according to any one of claims 1 to 8, wherein the device for segmenting and deploying a large model based on cloud-edge-end collaboration comprises: The evaluation module is used to evaluate the performance and resource consumption of a pre-trained large model when it is deployed on the cloud edge. A segmentation module, used to determine the segmentation points of the large model according to the performance evaluation results and resource consumption; as well as Decomposing the large model into multiple sub-models according to the determined split points; Deployment module, used to deploy each sub-model on the cloud and edge; The collaboration module is used to establish a cloud-edge collaborative working mechanism, allowing each sub-model to work collaboratively in the cloud and edge.

10. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by the processor, the steps of the large model segmentation and deployment method based on cloud-edge-end collaboration as described in any one of claims 1-8 are implemented.

Images