Large network model inference method, system and apparatus, communication device and storage medium

Abstract

The present application relates to a large network model inference method, system and apparatus, a communication device and a storage medium. The method comprises: receiving a large network model issued by a cloud, wherein the large network model is trained by the cloud; optimizing the large network model on the basis of a service scenario, so as to obtain an optimized large network model; and sending the optimized large network model to a user end, so that the user end performs inference on user data on the basis of the optimized large network model.

Description

Network large model inference method, system, device, communication device and storage medium

[0001] Related applications

[0002] The present application claims priority to the Chinese patent application No. 202411386211.3, filed on September 30, 2024, entitled "Network large model inference method, system, device, communication device and storage medium", the contents of which are hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the field of artificial intelligence, and in particular to a network large model inference method, system, device, communication device and storage medium. BACKGROUND

[0004] As a machine learning model or a deep learning model with huge scale and complexity, a network large model usually contains tens of billions or even hundreds of billions of parameters, can handle complex tasks and exhibit excellent performance, but at the same time, it also needs huge computing resources and storage space to support its training and running.

[0005] As the main type of intelligent computing power at present, a graphics processing unit (GPU) can provide sufficient computing resources for a network large model. In view of the gradual popularization of the application of network large models in the existing network, the inference ability of the large model is pushed down to the edge / user side, which can meet the business needs more quickly on the one hand, and is conducive to supporting the on-demand customization and optimization of the business and improving the user experience on the other hand.

[0006] However, the GPU has constraints such as limited procurement, high price, high power consumption, and the need for a dedicated server, and is not widely deployed in the edge / user side. In addition, the efficiency of constructing a network large model and using a network large model for inference is not high in the case of insufficient computing resources in the edge / user side. SUMMARY

[0007] The embodiments of the present application provide a network large model inference method, system, device, communication device, storage medium and computer program product, which can improve the efficiency of applying a network large model in the edge / user side.

[0008] In a first aspect, the present application provides a network large model inference method applied to an edge side, the method comprising:

[0009] receiving a network large model issued by a cloud side; the network large model is obtained by training of the cloud side;

[0010] According to the business scene, the network large model is optimized to obtain an optimized network large model; and

[0011] The optimized network large model is sent to the user end, so that the user end infers user data according to the optimized network large model.

[0012] In a second aspect, the present application provides a network large model inference method applied to a cloud end, the method comprising:

[0013] A network large model is trained; and

[0014] The network large model is distributed to an edge end, so that the edge end optimizes the network large model according to a business scene, and an optimized network large model obtained is sent to a user end, so that the user end infers user data.

[0015] In a third aspect, the present application provides a network large model inference method applied to a user end, the method comprising:

[0016] An optimized network large model sent by an edge end is received; the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scene; and

[0017] User data is inferred according to the optimized network large model.

[0018] In a fourth aspect, the present application provides a network large model inference system, the system comprising a cloud end, an edge end and a user end;

[0019] The cloud end is configured to train a network large model and distribute the network large model to the edge end;

[0020] The edge end is configured to optimize the network large model according to a business scene, obtain an optimized network large model, and send the optimized network large model to the user end; and

[0021] The user end is configured to infer user data according to the optimized network large model.

[0022] In a fifth aspect, the present application provides a network large model inference device applied to an edge end, the device comprising:

[0023] A receiving module is configured to receive a network large model distributed by a cloud end, the network large model being trained by the cloud end;

[0024] A processing module is configured to optimize the network large model according to a business scene, and obtain an optimized network large model; and

[0025] The sending module is configured to send the optimized network large model to a user end, so that the user end performs inference on user data according to the optimized network large model.

[0026] The present application provides, in a sixth aspect, a network large model inference apparatus applied to a cloud end, the apparatus comprising:

[0027] The training module is configured to train a network large model; and

[0028] The migration module is configured to distribute the network large model to an edge end, so that the edge end optimizes the network large model according to a business scenario, sends the obtained optimized network large model to a user end, and the user end performs inference on user data.

[0029] The present application provides, in a seventh aspect, a network large model inference apparatus applied to a user end, the apparatus comprising:

[0030] The obtaining module is configured to receive an optimized network large model sent by an edge end; the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scenario; and

[0031] The inference module is configured to perform inference on user data according to the optimized network large model.

[0032] The present application provides, in an eighth aspect, a communication device comprising a receiver, a processor and a transmitter;

[0033] The receiver is configured to receive a network large model distributed by a cloud end; the network large model is trained by the cloud end;

[0034] The processor is configured to optimize the network large model according to a business scenario, to obtain an optimized network large model; and

[0035] The transmitter is configured to send the optimized network large model to a user end, so that the user end performs inference on user data according to the optimized network large model.

[0036] The present application provides, in a ninth aspect, a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to cause the processor to implement the following method:

[0037] Receiving a network large model distributed by a cloud end; the network large model is trained by the cloud end;

[0038] Optimizing the network large model according to a business scenario, to obtain an optimized network large model; and

[0039] send the optimized network large model to the user end, so that the user end infers user data according to the optimized network large model.

[0040] The present application provides a computer program product in the tenth aspect, comprising a computer program, which is executed by a processor to enable the processor to implement the following method:

[0041] receiving a network large model issued by the cloud end; the network large model is obtained by training by the cloud end;

[0042] optimizing the network large model according to a business scenario to obtain an optimized network large model;

[0043] sending the optimized network large model to the user end, so that the user end infers user data according to the optimized network large model.

[0044] The network large model inference method, system, device, communication equipment, storage medium and computer program product described above, by receiving a network large model issued by the cloud end, optimizing the network large model according to a business scenario to obtain an optimized network large model, and sending the optimized network large model to the user end, can train a general network large model using the sufficient computing resources of GPU in the cloud end, and migrate the network large model to the edge end. The edge end does not need to train the model, and only uses CPU to optimize the network large model according to the business scenario, so as to obtain a personalized network large model adapted to the actual business scenario, for the user end to infer. Since the edge end and the user end do not need to deploy GPU and train the network large model, the efficiency of the network large model application is improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technical solutions, the drawings needed to be used in the description of the embodiments of the present application or the related technical solutions will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0046] Fig. 1 is a schematic diagram of the application scenario of the network large model inference method in an embodiment of the present application.

[0047] Fig. 2 is a flowchart of the network large model inference method in an embodiment of the present application.

[0048] Fig. 3 is a schematic diagram of the three-level architecture of cloud, edge and end in an embodiment of the present application.

[0049] Fig. 4 is a flowchart of the network large model inference method based on general computing power in an embodiment of the present application.

[0050] FIG. 5 is an interaction flowchart of a network large model inference method in an embodiment of the present application.

[0051] FIG. 6 is an internal structure diagram of a communication device in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0053] FIG. 1 is a schematic diagram of an application scenario of a network large model inference method according to an embodiment of the present application. As shown in FIG. 1, the scenario includes a cloud end 102, an edge end 104 and a user end 106. The cloud end 102 and the edge end 104 perform data transmission through a network, and the edge end 104 and the user end 106 perform data transmission through a network.

[0054] Specifically, the cloud end 102 can be a cloud server providing cloud computing services, and a GPU can be deployed thereon. The edge end 104 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, and a central processing unit (CPU) can be deployed thereon. The user end 106 can be, but is not limited to, an Open Telecom IT Infrastructure (OTII) server, which is designed to develop and define standardized, modular and high-performance IT infrastructure suitable for the telecommunications industry, and a CPU can be deployed thereon.

[0055] Taking network operation management as an example, the cloud end 102, the edge end 104 and the user end 106 can correspond to a first department (e.g., a head office), a second department (e.g., a branch) and a third department (e.g., an edge or user site) of network operation management, respectively. The first department can be configured with a standard physical server (standard server) having a GPU training resource pool for training a network large model; the second department can also be configured with a standard physical server (standard server) having a CPU inference resource pool for network large model inference; and the third department can be configured with an OTII server having a CPU inference resource for network large model inference.

[0056] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0057] In one embodiment, as shown in FIG. 2, a network large model inference method is provided. Taking the edge end 104 in FIG. 1 as an example, the method includes the following steps S202 to S206.

[0058] Step S202: receiving a network large model issued by the cloud, wherein the network large model is trained by the cloud.

[0059] Specifically, the network large model can be a large model used in the network, which can be constructed based on a Transformer model (e.g., a neural network model based on a self-attention mechanism) or a Bidirectional Encoder Representations from Transformers (BERT) model.

[0060] In a specific implementation, the cloud can train large-scale data based on a GPU to obtain a network large model, and issue the network large model to the edge end. The edge end receives the network large model issued by the cloud.

[0061] In actual application, the first department standard server can train large-scale data based on a GPU training resource pool to obtain a network large model, and smoothly migrate the network large model to the second department standard server. Specifically, the first department standard server can obtain a model file of the network large model, transmit the model file to the second department standard server according to a certain data transmission protocol, load the model file into the second department standard server, and obtain the network large model according to the model file.

[0062] Step S204: optimizing the network large model according to a business scenario to obtain an optimized network large model.

[0063] Specifically, the business scenario can be a use scenario of the network large model in a business field, including but not limited to network planning, network construction, network maintenance, network optimization, operation decision, etc. The optimized network large model can be a network large model with adjusted structure, parameters, etc.

[0064] In a specific implementation, after receiving the network large model issued by the cloud, the edge end can optimize the network large model based on a CPU according to a business scenario to obtain an optimized network large model adapted to the business scenario.

[0065] In actual application, the second department standard server can optimize the network large model based on the CPU inference resource pool according to the inference performance of the network large model in the actual business scenario, to obtain an optimized network large model.

[0066] Step S206: sending the optimized network large model to the user end, so that the user end infers the user data according to the optimized network large model.

[0067] The user data can be various data related to the user, such as call duration, network access duration, traffic size, etc.

[0068] In specific implementation, the edge end can send the optimized network large model to the user end, and after the user end obtains the optimized network large model, the user end can use the optimized network large model to infer the real-time user data based on the CPU.

[0069] In actual application, the second department standard server can deploy the optimized network large model on the OTII server of the third department, and the OTII server can use the optimized network large model to infer the user data in real time based on the CPU inference resource, to obtain the inference result of the user data.

[0070] The above network large model inference method receives the network large model issued by the cloud end, optimizes the network large model according to the business scenario, obtains the optimized network large model, and sends the optimized network large model to the user end. Thus, the cloud end can train a general network large model using sufficient computing resources of the GPU, and the network large model can be smoothly migrated to the edge end. The edge end does not need to train the model, and only needs to optimize the network large model according to the business scenario using the CPU, so as to obtain a personalized network large model suitable for the actual business scenario, for the user end to infer. Since the edge end and the user end do not need to deploy the GPU and train the network large model, the efficiency of the network large model application is improved.

[0071] In one embodiment, the above step S202 can specifically include: receiving a model file of the network large model sent by the cloud end; and obtaining the network large model according to the model file.

[0072] Specifically, the model file can be a file recording parameters, structures, etc. of the trained network large model.

[0073] In specific implementation, the cloud end can generate the model file according to the trained network large model, send the model file to the edge end, and the edge end generates the network large model according to the model file after receiving the model file.

[0074] In actual application, the first department standard server can transmit the model file to the second department standard server according to a certain data transmission protocol, and the second department standard server loads the model file and generates the network large model according to the model file.

[0075] In this embodiment, by receiving the model file of the network large model sent by the cloud, the network large model is obtained according to the model file, and the network large model trained by the cloud can be smoothly migrated to the edge end, so that the edge end does not need to perform model training, and the computing resources of the edge end are saved.

[0076] In one embodiment, the edge end is configured with a CPU inference resource pool, and the step S204 can specifically include: based on the CPU resources in the CPU inference resource pool, optimizing the network large model according to the business scenario and the pre-set knowledge base to obtain the optimized network large model.

[0077] Specifically, the CPU inference resource pool can be a CPU resource pool for inference using the network large model. The CPU resource can be a CPU-based computing resource. The knowledge base can be a database storing the association relationship between the business scenario, user data, inference result, inference speed, inference accuracy and other data.

[0078] In specific implementation, the edge end can be configured with a CPU inference resource pool, and the edge end can obtain user data, inference result, inference speed, inference accuracy and other data associated with the specified business scenario from the knowledge base based on the CPU resources in the CPU inference resource pool, and optimize the network large model according to these data to obtain the optimized network large model.

[0079] For example, the second department standard server can test the inference speed and inference accuracy of the network large model by using the user data, inference result and the like associated with the specified business scenario, and obtain a network large model with faster inference speed and higher accuracy by adjusting the parameters and structure of the network large model as the optimized network large model.

[0080] In this embodiment, by optimizing the network large model according to the business scenario and the pre-set knowledge base based on the CPU resources in the CPU inference resource pool, the optimized network large model can be obtained, which can adjust the network large model according to the actual business scenario at the edge end to obtain a personalized network large model adapted to the actual business scenario, and improve the applicability of the network large model.

[0081] In one embodiment, the cloud is configured with a GPU training resource pool for the cloud to train large-scale data sets to obtain the network large model.

[0082] Specifically, the GPU training resource pool can be a GPU resource pool for training a network large model. The GPU resource can be a GPU-based computing resource. The large-scale dataset can be a collection of large-scale training sample data.

[0083] In a specific implementation, the cloud can be configured with a GPU training resource pool. The cloud can input training sample data in the large-scale dataset into the network large model to be trained based on the GPU resources in the GPU training resource pool, and adjust the parameters, structure, etc. of the network large model to be trained according to the inference result of the network large model to the training sample data until the preset convergence condition is met, thereby obtaining the trained network large model.

[0084] In actual application, the Transformer or BERT can be selected as the model architecture of the network large model, the parameters and structure of the network large model are initialized, and the complexity and performance target of the network large model are defined. Then, the first department standard server can train the large-scale dataset using the GPU cluster, monitor the training state during the training, and periodically evaluate the model performance, thereby adjusting the parameters and structure of the network large model until the parameters and structure no longer change (for example, the difference between adjacent two iterations is less than a preset value) or the preset iteration number is reached. The network large model obtained at this time is taken as the trained network large model.

[0085] In the embodiment, the network large model is obtained by training the large-scale dataset in the cloud, and the high-performance network large model can be trained in the cloud to ensure the reliability of model inference.

[0086] In one embodiment, the user end is configured with an OTII server and a CPU inference resource. The OTII server is used to load the optimized network large model, and the optimized network large model is used to infer user data based on the CPU inference resource, thereby obtaining the inference result of the user data.

[0087] Specifically, the CPU inference resource can be a CPU resource for inference using the network large model. The inference result can be a result obtained by the network large model, such as fault warning, balance reminder, etc.

[0088] In a specific implementation, the user end can be configured with an OTII server, and the CPU inference resource is set on the OTII server. After the optimized network large model is smoothly migrated to the user end, the OTII server can load the optimized network large model. When the user data is input into the OTII server, the OTII server can use the optimized network large model to infer the user data based on the CPU inference resource, thereby obtaining the inference result of the user data.

[0089] In this embodiment, the optimized network large model is loaded by the OTII server, and based on the CPU inference resource, the user data is inferred using the optimized network large model to obtain the inference result of the user data, so that the user end can directly apply the personalized network large model adapted to the actual business scenario to infer the user data and improve the efficiency of applying the network large model.

[0090] In one embodiment, the OTII server is further configured to adjust the optimized network large model according to the feedback information of the inference result, to obtain an adjusted network large model.

[0091] Specifically, the feedback information can be the feedback of the user on the inference result, for example, the generation speed, accuracy, etc. of the inference result. The adjusted network large model can be a network large model with further adjusted structure, parameters, etc.

[0092] In a specific implementation, the OTII server can obtain the feedback information of the inference result, adjust the optimized network large model according to the feedback information, obtain the adjusted network large model, and subsequently use the adjusted network large model to infer the user data when the user data is input to obtain the inference result.

[0093] For example, the OTII server can continuously adjust the structure, parameters, etc. of the optimized network large model according to the inference speed and inference accuracy fed back by the user, to improve the inference speed and inference accuracy thereof.

[0094] In this embodiment, the OTII server adjusts the optimized network large model according to the feedback information of the inference result to obtain an adjusted network large model, which can continuously improve the inference performance of the network large model at the user end.

[0095] In order to facilitate those skilled in the art to further understand the embodiments of the present application, the following will be described in conjunction with a specific example.

[0096] The development of network large models has made significant breakthroughs in recent years, especially in the fields of natural language processing and computer vision. However, the training and running of network large models cannot be separated from high-performance computing resources, especially high-end GPUs optimized for deep learning. For users, it is a huge expense to purchase a sufficient number of GPUs to meet the training needs of large models, so GPU deployment is not widespread, which has affected the large-scale application of large models.

[0097] With the gradual development of CPU's artificial intelligence (AI) acceleration capability, the CPU is no longer just a widely used general-purpose computing power, but also a potential computing power support for large models. First, the CPU is also an important form of intelligent computing power. In the current AI intelligent computing power shortage, the CPU provides intelligent computing power, which undoubtedly provides a new development idea and breaks the shackles of only GPU being the intelligent computing power. For large model inference, the deployment and practice efficiency of CPU can easily form a scale effect. In any large or small scenario, as long as the CPU-based optimization achieves a single-point breakthrough, it can quickly achieve accurate and rapid replication or expansion, enabling more users to apply AI to the same or similar scenarios at a faster speed and lower cost. Second, CPU resources are easily accessible and almost unrestricted. As a standard component of computers, almost all servers and computers are equipped with CPUs, and there are already a large number of ready-to-use CPU-based applications in traditional businesses. CPU supply is stable, and selecting CPU for inference is easy to obtain, does not require the design of a heterogeneous hardware platform or the possession of related talent reserves, and is also easier to obtain technical support and maintenance. Finally, deploying AI inference using CPU resources can significantly reduce construction and later operation costs. Deploying AI is not only limited to GPU. Compared with GPU, CPU is a general-purpose computing resource with fair prices. The cost of building a CPU-based inference resource pool is significantly lower than that of a GPU inference pool, and the ability of CPU to support large model inference is not inferior to that of GPU in many scenarios, and even superior to that of GPU in some scenarios.

[0098] Therefore, in order to solve the urgent demand for large model inference computing power, the present application proposes a different user end using large model way of a three-level architecture design of cloud edge and end. The edge / user site uses the OTII server for inference to achieve a balance in performance and total cost of ownership (TCO). Through the use of a three-level architecture design, the head office, branch office and edge / user side form an efficient whole. Cloud, edge and end linkage, together improve the overall efficiency of the system and user experience.

[0099] FIG. 3 provides a schematic diagram of a three-level cloud edge and end architecture, including a head office 302 in the cloud, a branch office 304 in the edge, and an edge / user site 306 in the user end, wherein,

[0100] The head office layer in the cloud includes:

[0101] The network large model training module is responsible for processing large-scale datasets and training deep learning models. It utilizes a GPU training resource pool to efficiently complete complex training tasks, ensuring that the trained network large model has high precision and high performance.

[0102] The resources and infrastructure are equipped with standard servers and liquid-cooled servers, ensuring the stability and efficiency of the system and providing necessary support for data transmission and task distribution between layers.

[0103] The branch layer at the edge includes:

[0104] The digital platform and knowledge base serve as the storage and management center of information, providing a solid data foundation for network large model reasoning.

[0105] The reasoning module and resource pool, in close cooperation with the CPU reasoning resource pool, are responsible for receiving and processing network large models from the head office layer and applying them to actual business scenarios. This process involves steps such as loading, optimizing, and executing network large models.

[0106] Standard servers ensure that the branch layer can efficiently process data and tasks from the head office while providing timely support and response to edge / user sites.

[0107] The edge / user site layer at the user end includes:

[0108] The digital platform and reasoning module are configured to adapt to the needs of user sites, directly interacting with users, processing user data, and providing immediate reasoning results.

[0109] The resources and infrastructure, including CPU reasoning resources and OTII servers, enable on-site devices to quickly respond to user needs by receiving and applying optimized models from the branch office, achieving efficient reasoning and computation.

[0110] The processing flow of the cloud-edge three-level architecture can include the following steps S311 to S315.

[0111] Step S311: Model training, in the head office layer at the cloud end, the network large model training module uses large-scale datasets and a GPU training resource pool to train models, generating high-precision and high-performance network large models.

[0112] Step S312: Model migration, the trained network large model is smoothly migrated to the branch office's network large model reasoning module through data transmission mechanisms.

[0113] Step S313: Model inference and optimization. After receiving the network large model from the head office, the network large model inference module first loads and optimizes the network large model. With the support of the digital platform and knowledge base, the network large model can be further optimized according to specific application scenarios.

[0114] Step S314: Model application. The optimized network large model is migrated to the edge / user site. At this level, the optimized network large model is loaded into the digital platform and network large model inference module, and the on-site CPU inference resources and OTII server are used for fast inference and response.

[0115] Step S315: Final application and feedback. At the user side, the digital platform and network large model inference module process the user input data in real time, providing immediate inference results. These results can be further optimized based on user feedback and new data, forming a closed loop to continuously improve the system's intelligence and response capability.

[0116] In one embodiment, as shown in FIG. 4, a network large model inference method based on general computing power is provided, which includes the following steps S401-S404.

[0117] Step S401: In the preparation stage, resource assessment, software environment establishment and data preparation are performed. Among them,

[0118] Resource assessment includes: specifying the specifications of the CPU resource pool, covering server models, CPU categories (such as Intel Xeon series, etc.), memory size, etc., establishing network connections to ensure smooth data flow between levels (head office, branch, edge / user site).

[0119] Software environment establishment includes: deploying operating systems and necessary tools on CPU servers, installing and setting up network large models (such as large-scale language model xFasterTransformer (XFT), etc.) to realize distributed inference optimization, and adapting to specific hardware platforms.

[0120] Data preparation includes: integrating large data sets and uploading them to the GPU training cluster at the head office level, and performing cleaning and preprocessing work on the data, such as removing noise, standardizing and format conversion, etc., to enhance training efficiency.

[0121] Step S402: Model training stage, model selection and design, training execution, model evaluation and verification. Among them,

[0122] Model selection and design includes: selecting appropriate large-scale network models such as Transformer or BERT according to business requirements; planning specific parameters and structure of the model, defining its complexity and performance goals.

[0123] Training execution includes: training large-scale data sets using GPU clusters, applying distributed training frameworks such as TensorFlow or PyTorch to speed up training progress, monitoring training status, regularly evaluating model performance, and making corresponding adjustments.

[0124] Model evaluation and verification includes: using test sets to measure model performance, including accuracy, recall, F1 value, etc., testing the stability and generalization ability of the model, and ensuring its good performance in real environment.

[0125] Step S403: Model migration and optimization phase (branch level), model migration, training loading and optimization, performance testing. Among them,

[0126] Model migration includes: smoothly transferring the trained network large model to the branch's CPU inference resource pool, sending the model file to the branch server through the data transmission protocol.

[0127] Training loading and optimization includes: loading the model file on the branch server, testing the model performance at the branch level, confirming that the inference speed and accuracy meet the business requirements, adjusting the optimization strategy, and addressing possible performance problems.

[0128] Step S404: Model application and feedback phase (edge / user site level), including model deployment, inference execution and feedback optimization. Among them,

[0129] Model deployment includes: deploying the optimized network large model on the OTII server at the edge / user end, configuring the inference component and digital platform, ensuring that the optimized network large model can respond to user requests in real time.

[0130] Inference execution includes: using CPU resources on the OTII server to perform real-time inference tasks, monitoring the inference process, maintaining system stability and handling abnormal conditions in a timely manner.

[0131] Feedback optimization includes: continuously improving model performance based on user feedback and new data, creating a feedback loop mechanism to continuously improve the intelligence level and response speed of the system.

[0132] The network large model inference system and method based on general computing power breaks the traditional GPU inference idea, migrates the network large model to the CPU to realize efficient inference and supports end-to-end processing flow. In order to better improve the CPU inference performance, the present application puts forward an open source distributed inference optimization scheme XFT. The optimization scheme is based on a series of instruction sets provided by the processor platform. XFT has effectively optimized the performance of large language models, and supports distributed deployment scheme between multiple CPU nodes, so that the deployment of super large models on CPU becomes possible. The data preparation, knowledge base storage and processing work in the early stage of large model application are generally based on CPU general computing power. Using CPU for large model inference can realize the connection of end-to-end processing flow, and does not need to transmit and schedule between general computing and intelligent computing resource pools, reducing resource consumption and system complexity.

[0133] Secondly, the present application introduces an edge server to accelerate the inference application of edge and end, adopts an edge server type OTII, and the deployment environment of the edge server requires to be higher than that of the data center, and sometimes even needs to be deployed in some extreme harsh conditions. This means that the edge server needs to meet specific requirements in terms of power consumption, volume, durability, TCO, etc., and can keep the normal operation of the device under extreme high and low temperature, dust, corrosion, vibration impact and electromagnetic interference. OTII formulates corresponding standards in terms of configuration specifications, physical form, power supply and environmental adaptability, basic input output system (BIOS), baseboard management controller (BMC) and hardware management, etc. According to the work load, the infrastructure function is finely adjusted to support the edge measurement of large model inference, and the balance of performance and TCO is realized.

[0134] Finally, the present application proposes a different user end using large model mode of a "cloud edge end" three-level architecture design. The head office layer uses GPU resource pool to process large-scale data set and train deep learning model, so as to ensure that the model has high precision and high performance. This reflects the advantages of centralized and high computing power training in the cloud, which can quickly iterate the model and improve the model quality. After the model training is completed, it can be smoothly migrated to the branch office layer to load and preliminarily optimize the model. With the support of digital platform and knowledge base, the model can be further optimized for specific business scenarios to realize the scenario application of the model. The edge / user site layer can quickly respond to user demand through CPU inference resources and OTII server, and provide instant inference results. From the perspective of safety and privacy protection, data processing and model inference are distributed in different levels, which can not only train big data in the cloud, but also keep sensitive data in the edge, which is conducive to protecting data security and user privacy.

[0135] In summary, at the business process level, maintenance personnel at all levels can access the network large model and use the inference results of the network large model directly through interactive methods such as knowledge question and answer, information screening and summarization. In the scenarios of network planning, network construction, network maintenance, network optimization, and operation decision-making, the inference output results of the network large model can help the grassroots maintenance personnel to accurately understand and analyze the rules, automatically check the rules, and provide professional domain knowledge to help the industry to analyze and generate solutions. The answer of the network large model shows that it can deeply understand the user needs and intentions of the maintenance personnel and provide personalized and accurate services.

[0136] In terms of product form and resource requirements, since data needs to be processed immediately or near immediately at the edge side, in order to better adapt to the demand for computing power at the edge side, the embodiments of the present application use an edge server that meets the OTII standard (equipped with a CPU as a large model inference computing power). The OTII server has typical advantages of being deployed in edge scenarios: strong environmental adaptability, capable of coping with edge harsh challenges, such as being able to work at a temperature below 55 degrees, supporting IP65 (protection level of foreign matter intrusion for electrical equipment housings), meeting electromagnetic compatibility level B design, supporting 9-level seismic intensity, small body, supporting 1U / 2U / 4U and other models, supporting wall-mounted installation. Support for modular design, comprehensive coverage of scenarios. The server also supports front maintenance, adapting to the maintenance needs of the edge. A large number of edge machine rooms do not need any modification to deploy standard servers based on the OTII standard, and the physical environment for large model inference at the edge side is relatively easy to implement.

[0137] In terms of performance, the industry generally considers that a generation delay of less than 100 milliseconds is an acceptable performance threshold for users (considering that in chat robots, a single word is generated within 100 milliseconds, which basically exceeds the reading speed of humans, and the conversation appears smooth and natural). Through related software optimization, the network large model (for example, Qiming) is smoothly migrated and deployed on the CPU for inference. It has been detected that in multiple application scenarios such as regulatory inquiry, troubleshooting, network perception, network configuration, network optimization, security risk identification, and accident reporting, the first character delay is within 1 second, and the generation delay is around 50 milliseconds, which can meet the functional and performance requirements of large model inference.

[0138] In one embodiment, a network large model inference method is provided. Taking the cloud 102 in FIG. 1 as an example, the method includes the following steps S512 to S514.

[0139] Step S512: training to obtain a network large model.

[0140] Step S514: The network large model is delivered to the edge end, and the edge end optimizes the network large model according to the business scenario, and sends the obtained optimized network large model to the user end for the user end to infer the user data.

[0141] In a specific implementation, the cloud end can train a large-scale data set based on a GPU to obtain a network large model, and smoothly migrate the obtained network large model to the edge end. After the edge end obtains the network large model, the edge end can optimize the network large model based on a CPU according to a business scenario to obtain an optimized network large model that is adapted to the business scenario. The edge end can smoothly migrate the optimized network large model to the user end. After the user end obtains the optimized network large model, the user end can use the optimized network large model to infer real-time user data based on the CPU.

[0142] The network large model inference method described above trains a network large model, and delivers the network large model to the edge end. The cloud end can train a general network large model using sufficient computing resources of a GPU, and smoothly migrate the network large model to the edge end. The edge end does not need to train a model, and only needs to optimize the network large model according to a business scenario using a CPU to obtain a personalized network large model that is adapted to an actual business scenario for the user end to infer. Since the edge end and the user end do not need to deploy a GPU or train a network large model, the efficiency of application of the network large model is improved.

[0143] In one embodiment, the step S512 described above can specifically include: training a large-scale data set based on a pre-configured GPU training resource pool to obtain a network large model.

[0144] In a specific implementation, the cloud end can be configured with a GPU training resource pool. The cloud end can input training sample data in a large-scale data set into a network large model to be trained based on a GPU resource in the GPU training resource pool, adjust parameters, structures, and the like of the network large model to be trained according to an inference result of the network large model to be trained on the training sample data, until a preset convergence condition is met, and obtain a trained network large model.

[0145] In this embodiment, a network large model is obtained by training a large-scale data set based on a pre-configured GPU training resource pool. A high-performance network large model can be trained in the cloud end to ensure the reliability of model inference.

[0146] In one embodiment, a network large model inference method is provided. The method is applied to the user end 106 in FIG. 1 as an example for illustration, and includes the following steps S612 to S614.

[0147] Step S612: receiving the optimized network large model sent by the edge end; the optimized network large model is obtained by the edge end based on the network large model trained by the cloud end according to the business scenario.

[0148] Step S614: inferring the user data according to the optimized network large model.

[0149] In a specific implementation, the cloud end can train a network large model based on a GPU, and smoothly migrate the obtained network large model to the edge end. After the edge end obtains the network large model, the edge end can optimize the network large model based on a CPU according to a business scenario to obtain an optimized network large model that is adapted to the business scenario, and the edge end can smoothly migrate the optimized network large model to the user end. After the user end obtains the optimized network large model, the user end can use the optimized network large model to infer real-time user data based on the CPU.

[0150] The network large model inference method described above receives the optimized network large model sent by the edge end, and infers the user data according to the optimized network large model. The cloud end can train a general network large model using sufficient computing resources of a GPU, and smoothly migrate the network large model to the edge end. The edge end does not need to train a model, but only needs to optimize the network large model according to a business scenario using a CPU to obtain a personalized network large model that is adapted to the actual business scenario, for the user end to infer. Since the edge end and the user end do not need to deploy a GPU or train a network large model, the efficiency of applying the network large model is improved.

[0151] In one embodiment, the step S614 described above can specifically include: inferring the user data using the optimized network large model based on pre-configured CPU inference resources, to obtain an inference result of the user data.

[0152] In a specific implementation, the user end can be configured with an OTTI server, and the OTTI server is provided with CPU inference resources. After the optimized network large model is smoothly migrated to the user end, the OTTI server can load the optimized network large model. When the user data is input to the OTTI server, the OTTI server can use the optimized network large model to infer the user data based on the CPU inference resources, to obtain an inference result of the user data.

[0153] In this embodiment, the user end can directly apply the personalized network large model that is adapted to the actual business scenario to infer the user data, and the efficiency of applying the network large model is improved, by inferring the user data using the optimized network large model based on pre-configured CPU inference resources to obtain an inference result of the user data.

[0154] In an embodiment, the method further includes adjusting the optimized network large model according to feedback information of the inference result, to obtain an adjusted network large model.

[0155] In a specific implementation, the OTII server can obtain feedback information of the inference result, adjust the optimized network large model according to the feedback information, to obtain an adjusted network large model, and subsequently use the adjusted network large model to infer user data when the user data is input, to obtain an inference result.

[0156] In this embodiment, by adjusting the optimized network large model according to feedback information of the inference result, to obtain an adjusted network large model, the inference performance of the network large model at the user end can be continuously improved.

[0157] In an embodiment, a network large model inference system is provided, including a cloud end, an edge end, and a user end.

[0158] The cloud end is configured to train a network large model and distribute the network large model to the edge end.

[0159] The edge end is configured to optimize the network large model according to a business scenario, to obtain an optimized network large model, and send the optimized network large model to the user end.

[0160] The user end is configured to infer user data according to the optimized network large model.

[0161] In a specific implementation, the cloud end can train a network large model based on GPU for large-scale data, and smoothly migrate the obtained network large model to the edge end. After the edge end obtains the network large model, the edge end can optimize the network large model based on CPU according to a business scenario, to obtain an optimized network large model that is adapted to the business scenario, and the edge end can smoothly migrate the optimized network large model to the user end. After the user end obtains the optimized network large model, the user end can use the optimized network large model to infer real-time user data based on CPU.

[0162] Since the specific processing procedures of the cloud end, the edge end, and the user end have been described in detail in the foregoing embodiments, they will not be described here again.

[0163] The network large model inference system trains a network large model on a cloud side, distributes the network large model to an edge side, optimizes the network large model according to a business scenario on the edge side to obtain an optimized network large model, sends the optimized network large model to a user side, and infers user data according to the optimized network large model. The network large model can be trained on the cloud side using sufficient computing resources of a GPU, and is smoothly migrated to the edge side. The edge side does not need to train a model, but only needs to optimize the network large model according to the business scenario using a CPU to obtain a personalized network large model that is adapted to the actual business scenario, for inference by the user side. Since the edge side and the user side do not need to deploy a GPU or train a network large model, the efficiency of application of the network large model is improved.

[0164] In one embodiment, as shown in FIG. 5, a network large model inference method is provided, which includes the following steps S701-S703.

[0165] Step S701: A network large model is trained on a cloud side, and the network large model is distributed to an edge side.

[0166] Step S702: The network large model is optimized according to a business scenario on the edge side to obtain an optimized network large model, and the optimized network large model is sent to a user side.

[0167] Step S703: User data is inferred according to the optimized network large model on the user side.

[0168] In a specific implementation, a network large model can be trained based on a GPU on a cloud side, and the obtained network large model is smoothly migrated to an edge side. After the edge side obtains the network large model, the network large model can be optimized based on a CPU according to a business scenario to obtain an optimized network large model that is adapted to the business scenario. The edge side can smoothly migrate the optimized network large model to a user side. After the user side obtains the optimized network large model, the user side can infer real-time user data using the optimized network large model based on a CPU.

[0169] Since the specific processing procedures of the cloud side, the edge side, and the user side have been described in detail in the foregoing embodiments, they will not be described again here.

[0170] The network large model inference method trains a network large model on a cloud side, distributes the network large model to an edge side, optimizes the network large model according to a business scenario on the edge side, obtains an optimized network large model, sends the optimized network large model to a user side, and infers user data according to the optimized network large model on the user side. The general network large model can be trained on the cloud side by using sufficient computing resources of a GPU, and the network large model is smoothly migrated to the edge side. The edge side does not need to train a model, and only uses a CPU to optimize the network large model according to a business scenario, so that a personalized network large model suitable for an actual business scenario is obtained, and the network large model is used for inference on the user side. Since the edge side and the user side do not need to deploy a GPU and train a network large model, the efficiency of application of the network large model is improved.

[0171] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0172] In one embodiment, a network large model inference apparatus applied to an edge side is provided, comprising a receiving module, a processing module and a sending module. Wherein:

[0173] The receiving module is configured to receive a network large model distributed by a cloud side; the network large model is obtained by training of the cloud side;

[0174] The processing module is configured to optimize the network large model according to a business scenario, and obtain an optimized network large model;

[0175] The sending module is configured to send the optimized network large model to a user side, so that the user side infers user data according to the optimized network large model.

[0176] In one embodiment, the receiving module is further configured to receive a model file of the network large model sent by the cloud side; and obtain the network large model according to the model file.

[0177] In one embodiment, the processing module is further configured to optimize the network large model based on CPU resources in the CPU inference resource pool according to the business scenario and a preconfigured knowledge base, to obtain the optimized network large model.

[0178] In one embodiment, the cloud end is configured with a GPU training resource pool for the cloud end to train a large-scale data set to obtain the network large model.

[0179] In one embodiment, the user end is configured with an OTII server and CPU inference resources, the OTII server is configured to load the optimized network large model, and the optimized network large model is used to infer the user data based on the CPU inference resources to obtain an inference result of the user data.

[0180] In one embodiment, the OTII server is further configured to adjust the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

[0181] In one embodiment, a network large model inference device is provided, applied to a cloud end, comprising a training module and a migration module. Wherein:

[0182] The training module is configured to train a network large model;

[0183] The migration module is configured to distribute the network large model to an edge end, so that the edge end optimizes the network large model according to a business scenario, and sends the obtained optimized network large model to a user end for the user end to infer user data.

[0184] In one embodiment, the training module is further configured to train a large-scale data set based on a preconfigured GPU training resource pool to obtain the network large model.

[0185] In one embodiment, a network large model inference device is provided, applied to a user end, comprising an acquisition module and an inference module, wherein:

[0186] The acquisition module is configured to receive an optimized network large model sent by an edge end, the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scenario;

[0187] The inference module is configured to infer user data according to the optimized network large model.

[0188] In one embodiment, the inference module is further configured to infer the user data based on a preconfigured CPU inference resource using the optimized network large model to obtain an inference result of the user data.

[0189] In an embodiment, the network large model inference apparatus further comprises an adjusting module configured to adjust the optimized network large model according to the feedback information of the inference result, to obtain an adjusted network large model.

[0190] The specific limitations of the network large model inference apparatus can refer to the limitations of the network large model inference method described above, which will not be repeated here. Each module in the network large model inference apparatus described above can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0191] FIG. 6 is a structural schematic diagram of a communication device according to an embodiment of the present application. The communication device can include a receiver 81, a memory 82, a processor 83, at least one communication bus 84, and a transmitter 85. The communication bus 84 is used to realize the communication connection between the elements. The memory 82 can include a high-speed RAM memory, and can also include a non-volatile storage NVM, such as at least one disk memory. The memory 82 can store various programs for completing various processing functions and implementing the method steps of the present embodiment. In the present embodiment, the transmitter 85 can be a radio frequency processing module or a baseband processing module in the communication device, and the receiver 81 can also be a radio frequency processing module or a baseband processing module in the communication device. The transmitter 85 and the receiver 81 can be integrated together to realize a transceiver. Both the transmitter 85 and the receiver 81 can be coupled to the processor 83, and can realize the receiving or transmitting action under the indication or control of the processor 83.

[0192] In the present embodiment, the receiver 81 is configured to receive a network large model issued by a cloud. The network large model is trained by the cloud.

[0193] The processor 83 is configured to optimize the network large model according to a business scenario, to obtain an optimized network large model.

[0194] The transmitter 85 is configured to send the optimized network large model to a user end, so that the user end infers user data according to the optimized network large model.

[0195] In an embodiment, the receiver 81 is further configured to receive a model file of the network large model sent by the cloud; and obtain the network large model according to the model file.

[0196] In an embodiment, the processor 83 is further configured to optimize the network large model according to the business scenario and a pre-set knowledge base based on CPU resources in the CPU inference resource pool, to obtain the optimized network large model.

[0197] In an embodiment, the cloud end is configured with a GPU training resource pool for the cloud end to train a large-scale data set to obtain the network large model.

[0198] In an embodiment, the user end is configured with an OTII server and CPU inference resources, the OTII server is configured to load the optimized network large model, and the optimized network large model is used to infer the user data based on the CPU inference resources to obtain an inference result of the user data.

[0199] In an embodiment, the OTII server is further configured to adjust the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

[0200] In an embodiment, another communication device is provided, comprising: a processor configured to train a network large model; and a transmitter configured to send the network large model to an edge end, so that the edge end optimizes the network large model according to a business scenario, and sends an obtained optimized network large model to a user end for the user end to infer user data.

[0201] In an embodiment, the processor is further configured to train a large-scale data set based on a pre-configured GPU training resource pool to obtain the network large model.

[0202] In an embodiment, another communication device is provided, comprising: a receiver configured to receive an optimized network large model sent by an edge end, wherein the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scenario; and a processor configured to infer user data according to the optimized network large model.

[0203] In an embodiment, the processor is further configured to infer the user data using the optimized network large model based on a pre-configured CPU inference resource to obtain an inference result of the user data.

[0204] In an embodiment, the processor is further configured to adjust the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

[0205] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, causes the processor to implement the following steps:

[0206] receiving a network large model issued by the cloud; the network large model is obtained by training of the cloud;

[0207] optimizing the network large model according to a business scenario to obtain an optimized network large model; and

[0208] sending the optimized network large model to a user end for the user end to perform inference on user data according to the optimized network large model.

[0209] In one embodiment, the computer program, when executed by the processor, further implements the following steps: receiving a model file of the network large model sent by the cloud; and obtaining the network large model according to the model file.

[0210] In one embodiment, the computer program, when executed by the processor, further implements the following steps: based on CPU resources in the CPU inference resource pool, optimizing the network large model according to the business scenario and a pre-set knowledge base to obtain the optimized network large model.

[0211] In one embodiment, the computer program, when executed by the processor, further implements the following steps: training a network large model; and issuing the network large model to an edge end for the edge end to optimize the network large model according to a business scenario, sending an optimized network large model obtained to a user end for the user end to perform inference on user data.

[0212] In one embodiment, the computer program, when executed by the processor, further implements the following steps: based on a pre-configured GPU training resource pool, training a large-scale data set to obtain the network large model.

[0213] In one embodiment, the computer program, when executed by the processor, further implements the following steps: receiving an optimized network large model sent by an edge end; the optimized network large model is obtained by the edge end optimizing a network large model trained by the cloud according to a business scenario; and performing inference on user data according to the optimized network large model.

[0214] In one embodiment, the computer program, when executed by the processor, further implements the following steps: based on a pre-configured CPU inference resource, using the optimized network large model to perform inference on the user data to obtain an inference result of the user data.

[0215] In an embodiment, the computer program, when executed by the processor, further implements the following steps: adjusting the optimized network large model according to feedback information of the inference result, to obtain an adjusted network large model.

[0216] The embodiment of the present application further provides a computer program product containing instructions, which, when executed by a processor, cause the processor to perform the following steps:

[0217] receiving a network large model issued by the cloud; the network large model is obtained by training of the cloud;

[0218] optimizing the network large model according to a business scenario, to obtain an optimized network large model; and

[0219] sending the optimized network large model to a user end, so that the user end performs inference on user data according to the optimized network large model.

[0220] In an embodiment, when the computer program product is executed by the processor, the processor further performs the following steps: receiving a model file of the network large model sent by the cloud; and obtaining the network large model according to the model file.

[0221] In an embodiment, when the computer program product is executed by the processor, the processor further performs the following steps: based on CPU resources in the CPU inference resource pool, optimizing the network large model according to the business scenario and a pre-set knowledge base, to obtain the optimized network large model.

[0222] In an embodiment, when the computer program product is executed by the processor, the processor further performs the following steps: training a network large model; issuing the network large model to an edge end, so that the edge end optimizes the network large model according to a business scenario, sends an optimized network large model obtained to a user end, and the user end performs inference on user data.

[0223] In an embodiment, when the computer program product is executed by the processor, the processor further performs the following steps: based on a pre-configured GPU training resource pool, training a large-scale data set, to obtain the network large model.

[0224] In an embodiment, when the computer program product is executed by the processor, the processor further performs the following steps: receiving an optimized network large model sent by an edge end; the optimized network large model is obtained by optimizing a network large model trained by the cloud according to a business scenario by the edge end; and performing inference on user data according to the optimized network large model.

[0225] In an embodiment, when the computer program product is run by the processor, the processor is further caused to perform the following step: performing inference on the user data using the optimized network large model based on pre-configured CPU inference resources to obtain an inference result of the user data.

[0226] In an embodiment, when the computer program product is run by the processor, the processor is further caused to perform the following step: adjusting the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

[0227] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0228] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description simple, each technical feature of the above embodiments is not described in all possible combinations, but as long as the combination of technical features does not exist, it should be considered as the scope of the present application.

[0229] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A network large model inference method applied to an edge end, the method comprising: receiving a network large model issued by a cloud end; the network large model being trained by the cloud end; optimizing the network large model according to a business scenario to obtain an optimized network large model; and sending the optimized network large model to a user end for the user end to perform inference on user data according to the optimized network large model.

2. The method of claim 1, wherein the receiving the network large model issued by the cloud end comprises: receiving a model file of the network large model sent by the cloud end; and obtaining the network large model according to the model file.

3. The method of claim 1, wherein the edge end is configured with a CPU inference resource pool; and the optimizing the network large model according to the business scenario to obtain the optimized network large model comprises: optimizing the network large model according to the business scenario and a pre-set knowledge base based on CPU resources in the CPU inference resource pool to obtain the optimized network large model.

4. The method of claim 1, wherein the cloud end is configured with a GPU training resource pool for the cloud end to train a large-scale data set to obtain the network large model.

5. The method of claim 1, wherein the user end is configured with an OTII server and a CPU inference resource, the OTII server being used to load the optimized network large model, and the CPU inference resource being used to perform inference on the user data using the optimized network large model to obtain an inference result of the user data.

6. The method of claim 5, wherein the OTII server is further used to adjust the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

7. A network large model inference method applied to a cloud end, the method comprising: training a network large model; and issuing the network large model to an edge end for the edge end to optimize the network large model according to a business scenario, and sending an optimized network large model obtained to a user end for the user end to perform inference on user data.

8. The method of claim 7, wherein the training the network large model comprises: training a large-scale data set based on a pre-configured GPU training resource pool to obtain the network large model.

9. A network large model inference method applied to a user end, the method comprising: receiving an optimized network large model sent by an edge end, wherein the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scenario; and performing inference on user data according to the optimized network large model.

10. The method of claim 9, wherein the performing inference on user data according to the optimized network large model comprises: performing inference on the user data using the optimized network large model based on a pre-configured CPU inference resource to obtain an inference result of the user data. ​ ​ ​ 11. The method of claim 9, further comprising: adjusting the optimized network large model according to feedback information of the inference result to obtain an adjusted network large model.

12. A network large model inference system, the system comprising a cloud end, an edge end and a user end, wherein the cloud end, the edge end and the user end are configured to perform the steps of the method of any one of claims 1-11.

13. A network large model inference apparatus applied to an edge end, the apparatus comprising: a receiving module configured to receive a network large model issued by a cloud end; the network large model is trained by the cloud end; a processing module configured to optimize the network large model according to a business scenario to obtain an optimized network large model; and a sending module configured to send the optimized network large model to a user end for the user end to perform inference on user data according to the optimized network large model.

14. A network large model inference apparatus applied to a cloud end, the apparatus comprising: a training module configured to train a network large model; and a migration module configured to issue the network large model to an edge end for the edge end to optimize the network large model according to a business scenario and send an optimized network large model obtained to a user end for the user end to perform inference on user data.

15. The apparatus of claim 14, wherein the training module is further configured to train a large-scale data set based on a pre-configured GPU training resource pool to obtain the network large model.

16. A network large model inference apparatus applied to a user end, the apparatus comprising: a receiving module configured to receive an optimized network large model sent by an edge end; the optimized network large model is obtained by the edge end optimizing a network large model trained by a cloud end according to a business scenario; and an inference module configured to perform inference on user data according to the optimized network large model.

17. The apparatus of claim 16, wherein the inference module is further configured to perform inference on the user data using the optimized network large model based on a pre-configured CPU inference resource to obtain an inference result of the user data.

18. A communication device comprising a receiver, a processor and a transmitter; wherein: the receiver is configured to receive a network large model issued by a cloud end, the network large model being trained by the cloud end; the processor is configured to optimize the network large model according to a business scenario to obtain an optimized network large model; and the transmitter is configured to send the optimized network large model to a user end for the user end to perform inference on user data according to the optimized network large model.

19. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, causes the processor to implement the steps of the method of any one of claims 1-11. ​ ​ ​ ​ 20. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, causes the processor to carry out the steps of the method of any one of claims 1 to 11.

Images