Model deployment method and device, computer equipment and readable storage medium

Abstract

The invention relates to a model deployment method and device, computer equipment and a readable storage medium. The method comprises the steps that based on a deployment request for a target model, current hardware performance parameters of multiple edge devices are acquired, and the deployment request comprises required hardware performance parameters when the target model is deployed; according to each current hardware performance parameter and the required hardware performance parameter, selecting a target edge device from the plurality of edge devices to establish a target resource pool required by a target model; and sending a configuration file of the target model to a target edge device in the target resource pool, the configuration file being used for indicating the target edge device in the target resource pool to deploy the target model. By adopting the method, challenges, such as selection of proper edge equipment and configuration of a correct operation environment, in a local deployment process can be solved, and effective deployment of the model can be realized on the premise of ensuring data security, so that the purposes of improving efficiency, reducing cost and enhancing privacy protection are achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model deployment method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] Many models with large parameter counts are run in the cloud. This approach is not suitable for privacy-critical applications such as banking, as using cloud services may pose a risk of data leakage. In such cases, the model needs to be deployed locally.

[0003] However, the biggest challenge with local deployment is configuring the appropriate operating environment and downloading the most appropriate model for the current environment. This process involves multiple steps and technical details, such as selecting a model version suitable for device performance and ensuring all dependencies are correctly installed. These steps require users to invest a lot of time and effort, which can easily lead to waste of manpower and financial resources. Summary of the Invention

[0004] Based on this, it is necessary to provide a model deployment method, device, computer equipment, computer-readable storage medium and computer program product that can improve efficiency, reduce costs and enhance privacy protection in response to the above technical problems.

[0005] In a first aspect, the present application provides a model deployment method, comprising:

[0006] Obtaining current hardware performance parameters of multiple edge devices based on a deployment request for the target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed;

[0007] Selecting a target edge device from the plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model;

[0008] The configuration file of the target model is sent to the target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0009] In one embodiment, selecting a target edge device from the plurality of edge devices to form a target resource pool required for the target model according to each of the current hardware performance parameters and the required hardware performance parameters includes:

[0010] Determine the current hardware performance parameters required for configuring the target model based on the required hardware performance parameters when the target model is deployed; when there are current hardware performance parameters among the multiple edge devices that match the current hardware performance parameters required by the target model, use the edge device to which the matching current hardware performance parameters belong as the target edge device;

[0011] Based on the selected target edge devices, a target resource pool required for the target model is constructed.

[0012] In one of the embodiments, when the current hardware performance parameters of the multiple edge devices do not match the current hardware performance parameters required by the target model, at least two edge devices are selected from the multiple edge devices as target edge devices, and the total amount of the current hardware performance parameters of the at least two edge devices matches the current hardware performance parameters required by the target model.

[0013] In one embodiment, sending the configuration file of the target model to the target edge device in the target resource pool includes:

[0014] According to the current hardware performance parameters of each target edge device, the configuration file of the target model is split into multiple configuration sub-files, and each current hardware performance parameter corresponds to the multiple configuration sub-files one by one; each configuration sub-file is sent to the corresponding target edge device, and each configuration sub-file is used to instruct the corresponding target edge device to deploy some functional modules of the target model.

[0015] In one embodiment, based on a deployment request for a target model, obtaining current hardware performance parameters of multiple edge devices includes:

[0016] Receive a deployment request for a target model sent by a user terminal; based on the deployment request, send a parameter test instruction to multiple edge devices so that the multiple edge devices each perform a parameter test; receive current hardware performance parameters fed back by the multiple edge devices in response to the parameter test instruction.

[0017] In one of the embodiments, configuration completion instructions fed back by each of the multiple target edge devices are received; in response to the configuration completion instructions fed back by each of the multiple target edge devices, the calling permission of the model operation programming interface is opened to the user terminal, so that the user terminal can implement the relevant functions of the target model by calling the model operation programming interface after obtaining the calling permission.

[0018] In one embodiment, before obtaining current hardware performance parameters of the plurality of edge devices, the method further includes:

[0019] A deployment request for a target model is obtained, where the deployment request is determined by a deployment task for the target model, where the deployment task is created by a user based on allocated resources and unallocated resources of respective multiple current resource pools.

[0020] In a second aspect, the present application further provides a model deployment device, comprising:

[0021] An acquisition module, configured to acquire current hardware performance parameters of multiple edge devices based on a deployment request for a target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed;

[0022] A selection module is configured to select a target edge device from the plurality of edge devices to form a target resource pool required for the target model according to each of the current hardware performance parameters and the required hardware performance parameters;

[0023] A sending module is used to send a configuration file of the target model to a target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0024] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Obtaining current hardware performance parameters of multiple edge devices based on a deployment request for the target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed;

[0026] Selecting a target edge device from the plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model;

[0027] The configuration file of the target model is sent to the target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0029] Obtaining current hardware performance parameters of multiple edge devices based on a deployment request for the target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed;

[0030] Selecting a target edge device from the plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model;

[0031] The configuration file of the target model is sent to the target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0032] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0033] Obtaining current hardware performance parameters of multiple edge devices based on a deployment request for the target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed;

[0034] Selecting a target edge device from the plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model;

[0035] The configuration file of the target model is sent to the target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0036] The model deployment method, apparatus, computer device, computer-readable storage medium, and computer program product described above obtain the current hardware performance parameters of multiple edge devices based on a deployment request for a target model. The deployment request includes the required hardware performance parameters for the target model during deployment. This provides a basis for subsequent selection of the most suitable device for running the target model, ensuring that the selected device meets the model's operational requirements. Automated device information acquisition reduces the need for manual hardware compatibility checks, saving time and effort.

[0037] Based on the current and required hardware performance parameters, target edge devices are selected from multiple edge devices to build the target resource pool required for the target model. This avoids operational failures or poor performance due to insufficient device performance. Furthermore, users do not need to manually compare the specifications of different devices with model requirements, reducing operational difficulty and the probability of errors.

[0038] The target model's configuration file is sent to the target edge device in the target resource pool. This configuration file instructs the target edge device in the target resource pool to deploy the target model. The configuration file contains all necessary parameters and dependency information, enabling the target edge device to automatically deploy the model without requiring manual configuration, greatly simplifying the deployment process. Furthermore, the model is deployed directly on the local edge device, and data processing is also performed locally, effectively preventing the risk of sensitive information leakage. This approach is particularly suitable for applications with high privacy requirements.

[0039] Through the automation of the above steps, the challenges faced during local deployment, including selecting appropriate edge devices and configuring the correct operating environment, are resolved. In addition, the model can be effectively deployed while ensuring data security, thereby achieving the goals of improving efficiency, reducing costs, and enhancing privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 An application environment diagram of a model deployment method in one embodiment;

[0042] Figure 2 A schematic diagram of a flow chart of a model deployment method in one embodiment;

[0043] Figure 3 Schematic diagram of the architecture of a model deployment method in one embodiment

[0044] Figure 4 A terminal deployment flow chart of a model deployment method in one embodiment;

[0045] Figure 5 A flowchart of a model deployment method in another embodiment;

[0046] Figure 6 It is a structural block diagram of a model deployment device in one embodiment;

[0047] Figure 7 It is a structural block diagram of a model deployment device in one embodiment;

[0048] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] The model deployment method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, device 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104 or located on another local network server. Terminal 102 is used to generate a model deployment request and send the model deployment request to server 104, causing server 104 to send the target model configuration file to the target edge device in the target resource pool. The configuration file is used to instruct the target edge device in the target resource pool to deploy the target model. Device 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart car devices, projectors, etc. Portable wearable devices can include smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 is a master node, which is the central node responsible for coordinating, managing, and allocating tasks within the entire distributed system. It can be deployed on a server with strong computing power and abundant storage resources. It is understandable that the main responsibilities of the master node include: receiving information from edge devices and analyzing and processing it. For example, selecting appropriate models and operating parameters based on the configuration performance test results reported by the edge device. Make decisions based on the collected data, such as deciding which edge device is most suitable for deploying a specific target model. Send configuration files or instructions to the selected edge device to guide it to complete tasks such as environment setup, model download and deployment. Regularly receive operating status information uploaded by edge devices to monitor the overall operation status of the cluster and perform necessary maintenance work. The edge device in this application is close to the data source or the computing device on the user side, and is used to perform functions such as data analysis, processing tasks, and model reasoning, rather than sending all data to the cloud for processing. These devices may include but are not limited to smartphones, IoT devices, embedded computers, etc.

[0051] In an exemplary embodiment, Figure 2 As shown, a model deployment method is provided, which is applied to Figure 1The server 104 in the example is used as an example to illustrate the process, including the following steps 202 to 206.

[0052] Step 202: Based on a deployment request for the target model, current hardware performance parameters of multiple edge devices are obtained. The deployment request includes the required hardware performance parameters when the target model is deployed.

[0053] The target model refers to the AI ​​model that needs to be deployed locally or on edge devices. For example, it can be a model trained for a specific task, such as text generation or image recognition.

[0054] A deployment request is a request sent from the user to deploy a specific target model on a specific edge device. This request contains some configuration parameters required to run the model.

[0055] Edge devices are computing devices located close to data sources or end users, used to perform data analysis and processing tasks. All data is processed locally, rather than being sent to the cloud for processing. In the application, the edge device is responsible for actually running the deployed target model and reporting its hardware configuration so that the system can select the model and operating parameters that best suit the current environment.

[0056] Current hardware performance parameters refer to the hardware and software configuration information of the edge device, including but not limited to the current computing power, current video memory, etc. Understandably, these parameters are crucial in determining the maximum model size and complexity that can be run on the device.

[0057] Required hardware performance parameters refer to the configuration conditions required for the target model to run. Examples include required computing power and required video memory. Understandably, these parameters affect the model's execution efficiency and effectiveness under given hardware conditions.

[0058] Specifically, the user first submits a deployment request for a specific target model on the terminal. This deployment request not only contains information about the target model that the user wants to deploy, but also includes the required hardware performance parameters (such as required computing power, required video memory, etc.) required for the model to run. Performance test instructions are then sent to all available edge devices, so that each edge device that receives the performance test instructions performs a performance test on its own hardware and software configuration parameters. These current hardware performance parameters include but are not limited to: current computing power and current video memory. Finally, each edge device reports the configuration performance test status (i.e., current hardware performance parameters) obtained from the above test to the master node (i.e., server). It can be understood that the purpose of this step is to evaluate the status of the current hardware environment and provide a basis for selecting appropriate operating parameters and models.

[0059] In step 204 , a target resource pool required for constructing a target model is selected from a plurality of edge devices based on the current hardware performance parameters and the required hardware performance parameters.

[0060] Among them, the target edge device refers to the edge device that is most suitable for deploying and running a specific target model, which is selected from multiple available edge devices after analysis and matching based on its current hardware performance parameters (such as current video memory, current computing power, etc.) and the required hardware performance parameters (such as required computing power, required video memory, etc.) required when the target model runs.

[0061] The target resource pool refers to a logical collection of computing resources selected and combined from multiple edge devices based on the required hardware performance parameters in the deployment request. It can be understood that this computing resource collection is not measured in units of the number of physical devices (for example, the number of computing devices), but in units of computing power. For example, a 150TOPS (tera operations per second) device and a 50TOPS device can be combined to form a 200TOPS target resource pool. Specifically, refer to Figure 3 First, after receiving data (i.e., device configuration parameters) from each edge device, the master node (i.e., server) begins to analyze the device configuration parameters of each edge device, including but not limited to the current computing power and current video memory.

[0062] Based on the model configuration parameters (required computing power and memory) in the user's deployment request, the master node further selects the target resource pool required by the target model for edge device components that not only meet the hardware requirements but also optimally support the model's execution. This ensures that the selected edge devices meet the minimum hardware requirements while also optimizing the model's execution efficiency and effectiveness.

[0063] Optional, reference Figure 3 The target resource pool is a collection of computing resources that combines multiple edge devices and meets both the required computing power and required video memory. For example, the deployment request for the target model includes a required computing power of 20TOPS and a required video memory of 5GB.

[0064] The master node will automatically query the child nodes that meet the requirements, for example, the combined current computing power reaches 20TOPS, and the combined current video memory reaches 5GB. Based on the query results, an edge device A with a current computing power of 15 TOPS and a current video memory of 4GB is found, and another edge device B with a current computing power of 10TOPS and a current video memory of 2GB is found. Edge devices A and B are combined to form a target resource pool, and the required task environment is configured. Step 206: Send the configuration file of the target model to the target edge device in the target resource pool. The configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0065] A configuration file contains all the necessary information to instruct the target edge device on how to deploy and run a specific target model. It's like a detailed "manual," guiding the target edge device through the entire process, from environment preparation to model deployment and ultimately service release. This ensures the target model runs smoothly on the selected device and performs its intended function.

[0066] Specifically, a configuration file is first prepared for the selected target edge device. Optionally, the configuration file contains all necessary information, such as environment setup instructions, dependency list, model weight download address, model running parameters, etc., to ensure that the target model can be correctly deployed and run on the target edge device.

[0067] The master node then sends the configuration file to the target edge device in the previously selected target resource pool through the TCP / IP protocol or other appropriate communication methods. This step is completed automatically, reducing the possibility and complexity of manual intervention. Once the configuration file is received, the target edge device begins to parse the contents. This process includes understanding specific operational guidelines such as how to configure the operating environment, which dependencies need to be installed or updated, and where to obtain model weights. According to the instructions in the configuration file, the target edge device will automatically perform the corresponding environment configuration tasks, such as installing or updating dependencies such as software libraries and frameworks. At the same time, according to the model weight download address given in the configuration file, the target edge device will automatically download all required model files and resources.

[0068] After the environment is configured, the target edge device uses the information in the configuration file to build an independent and optimized operating environment and deploy the target model within it. Finally, the target edge device provides an API interface through the server service, allowing clients to interact with the deployed model through these interfaces and realize related functional requirements.

[0069] In one of the embodiments, the current hardware performance parameters required to configure the target model are determined based on the required hardware performance parameters when the target model is deployed; when there are current hardware performance parameters among multiple edge devices that match the current hardware performance parameters required by the target model, the edge device to which the matching current hardware performance parameters belong is used as the target edge device; based on the selected target edge device, the target resource pool required for the target model is established.

[0070] Specifically, the system first understands and analyzes the specific runtime requirements of the target model provided by the user, including core parameters such as computing power and video memory. Based on these required hardware performance parameters, the system estimates the basic computing resources required to run the model, such as the required video memory and computing power. The master node then automatically searches for child nodes that meet these requirements. For example, if the combined current computing power reaches 20 TOPS and the combined current video memory reaches 5GB, the master node finds an edge device A with a current computing power of 15 TOPS and 4GB of video memory, and another edge device B with a current computing power of 10 TOPS and 2GB of video memory. These devices are combined to form a target resource pool, configuring the required task environment. By selecting the most powerful device as the target edge device, performance bottlenecks caused by hardware limitations are mitigated, reducing subsequent maintenance costs. Furthermore, by selecting the edge device with the highest current hardware performance parameters, hardware resources are optimally utilized, avoiding resource waste or shortages.

[0071] In one of the embodiments, when current hardware performance parameters of multiple edge devices do not match the current hardware performance parameters required by the target model, at least two edge devices are selected from the multiple edge devices as target edge devices, and the total amount of the current hardware performance parameters of the at least two edge devices matches the current hardware performance parameters required by the target model.

[0072] Optionally, the current hardware performance parameters required to configure the target model are determined based on the required hardware performance parameters when the target model is running; the current hardware performance parameters of multiple edge devices are compared with the current hardware performance parameters required by the target model; if there is no edge device with current hardware performance parameters greater than or equal to those required by the target model among the multiple edge devices, multiple edge devices with current hardware performance parameters greater than zero are used as target edge devices.

[0073] Specifically, we first understand and analyze the specific requirements provided by the user regarding the target model's runtime, including core parameters such as batch size, learning rate, and number of threads / processes. Based on the aforementioned hardware performance parameters, we estimate the basic computing resources required to run the model, such as video memory (for GPUs), memory, computing power (CPU / GPU performance), storage space, and network bandwidth. We then determine the operating system version, specific software libraries, and framework versions required to run the model, ensuring that these dependencies can be correctly installed and run on potential edge devices. Based on the above analysis results, we develop a minimum hardware configuration standard that meets the efficient and stable operation of the target model. This includes, but is not limited to, minimum memory capacity, video memory, computing power, and other necessary hardware or software requirements.

[0074] The master node then receives the configuration parameters from each edge device and compares each edge device's actual configuration parameters with the current hardware performance parameters required by the target model, checking to see if any device meets or exceeds all the configuration criteria of the target model. If no edge device fully meets the requirements of the target model (i.e., all edge devices have configuration parameters lower than the target model's requirements), a suboptimal strategy is adopted. At this point, multiple edge devices whose current hardware performance parameters are greater than zero and as close as possible to the target model's requirements are considered as target edge devices.

[0075] In the event that no edge device fully meets the target model requirements, a flexible selection method is provided: multiple edge devices whose current hardware performance parameters are greater than zero and as close to the target model requirements as possible are considered. This increases the adaptability and flexibility of the system, allowing model deployment even with limited resources. Even in resource-constrained situations, existing computing resources can be fully utilized, avoiding deployment failures caused by an inability to find fully qualified devices.

[0076] In one of the embodiments, the configuration file of the target model is split into multiple configuration sub-files based on the current hardware performance parameters of each target edge device, and each current hardware performance parameter corresponds to the multiple configuration sub-files one by one; each configuration sub-file is sent to the corresponding target edge device, and each configuration sub-file is used to instruct the corresponding target edge device to deploy part of the functional modules of the target model. Among them, the configuration sub-file refers to the multiple independent parts that are split from the original complete configuration file according to specific standards or requirements. Each configuration sub-file may contain specific configuration information for a certain aspect of the deployment and operation of the target model. Optionally, the configuration sub-files can be regarded as modules of the entire configuration file, and each module is responsible for describing a specific aspect or function in the model deployment process. For example, some sub-files may focus on defining environmental settings (such as which software libraries to install), while other sub-files focus on specifying the model weight download address, adjusting the model operation parameters, etc.

[0077] Specifically, first, the complete configuration file is split into multiple independent configuration sub-files based on the correspondence between the configuration sub-files and the current hardware performance parameters. Each configuration sub-file is responsible for handling a specific aspect or function in the target model deployment process. For example, some sub-files may focus on defining environmental settings (such as which software libraries to install), while other sub-files focus on specifying the model weight download address, adjusting the model running parameters, etc. Each configuration sub-file is analyzed to clarify the hardware and software configuration parameters required for each of them at runtime. These requirements may include but are not limited to memory size, video memory capacity, computing power, etc. Determine the minimum device configuration requirements corresponding to each configuration sub-file to ensure that the appropriate edge device can be accurately matched later.

[0078] It then automatically scans all available target edge devices and records their current hardware performance parameters, including but not limited to available memory, video memory, computing power (CPU / GPU computing power), storage space, network bandwidth, operating system version, and installed software versions. For each target edge device, it compares its actual current hardware performance parameters with the current hardware performance parameters required by each configuration subfile. It then determines which configuration subfiles can run on the current edge device, specifically whether the device's configuration meets or exceeds the requirements of the configuration subfile.

[0079] Finally, based on the comparison results, configuration subfiles that match the current hardware performance parameters of the target edge device are screened out. This means that the selected configuration subfile can correctly perform its intended function on the device. The selected configuration subfiles that match the current hardware performance parameters of the target edge device are sent to the corresponding edge device. This process is automated, ensuring that each edge device only receives and processes the tasks it is capable of performing. After receiving the configuration subfile, the target edge device begins to parse the contents and execute the corresponding configuration tasks according to the instructions. This includes but is not limited to installing or updating software libraries and frameworks, downloading model weight files, and configuring model operating parameters.

[0080] By allowing different tasks to be assigned based on the actual capabilities of edge devices, even if some devices cannot fully meet the requirements of the entire model, they can still take on some of the work. This approach greatly improves the system's adaptability and flexibility in the face of diverse hardware conditions. Furthermore, if the resources required by a configuration subfile are unavailable on the current edge device, the problem can be resolved by adjusting the functionality of that part or finding a more suitable device, without having to reconsider the entire model deployment strategy. This simplifies troubleshooting and solution implementation.

[0081] In one embodiment, a deployment request for a target model is received from a user terminal; based on the deployment request, a parameter test instruction is sent to multiple edge devices so that the multiple edge devices each perform a parameter test; and current hardware performance parameters fed back by the multiple edge devices in response to the parameter test instruction are received.

[0082] A parameter test instruction is a command or set of instructions generated by a master node (typically a server) based on a received deployment request for a target model and sent to multiple edge devices. The primary purpose of this instruction is to instruct edge devices to perform a series of predefined hardware and software configuration checks. Optionally, these parameter test instructions may include specific test items, such as requiring edge devices to report key performance indicators such as their currently available memory size, graphics memory capacity (if applicable), CPU / GPU computing power, and other information, including the operating system version and installed software and its versions.

[0083] Parameter testing is a series of hardware and software configuration self-tests performed by edge devices after receiving a parameter test command. This process aims to gain a comprehensive understanding of the actual operating environment of the edge device and collect relevant information to feed back to the master node.

[0084] Specifically, a user first sends a deployment request for a specific target model to the system through a terminal (such as a personal computer or mobile device). This request includes specific information about the target model's operation, such as the required hardware performance parameters (batch size, learning rate, etc.), as well as any special requirements or preferences. After receiving the user's deployment request, the master node (usually the server) generates a series of parameter test instructions based on the information in the request. These instructions are designed to guide the edge device to perform the necessary hardware and software configuration checks.

[0085] The master node then sends these parameter test instructions to multiple edge devices in the network. Each edge device receives the same set of instructions, ensuring that the configuration status of each device can be collected in a unified and standardized manner. The edge device that receives the parameter test instructions begins to automatically execute a series of predefined test tasks. This includes but is not limited to: measuring key performance indicators such as currently available memory, video memory, computing power, and checking the operating system version and installed related software and its versions, especially those libraries and frameworks that are closely related to model deployment. Optionally, after completing the above tests, the edge device will record all relevant result data to form a detailed report on the current hardware performance parameters.

[0086] Finally, each edge device reports its current hardware performance parameters back to the master node. This data is crucial for selecting the most suitable edge device for deploying the target model. After receiving feedback from all edge devices, the master node aggregates and analyzes this data to facilitate the selection of the most suitable edge device based on the requirements of the target model.

[0087] By sending parameter test instructions to edge devices and obtaining detailed current hardware performance parameters, we can accurately understand the actual performance and resource availability of each edge device. This enables the system to accurately match the edge device most suitable for deploying the target model based on its specific requirements (such as memory, graphics memory, computing power, etc.). This approach is particularly useful for workload distribution across multiple edge devices. It ensures that each device only receives the portion of the task that matches its capabilities, thereby achieving efficient operation of the entire system, which is particularly suitable for deploying large or complex AI models.

[0088] In one embodiment, a configuration completion instruction is received from each of the multiple target edge devices; in response to the configuration completion instruction from each of the multiple target edge devices, the user terminal is given the right to call the model operation programming interface, so that the user terminal can implement the relevant functions of the target model by calling the model operation programming interface after obtaining the calling permission. The configuration completion instruction refers to a confirmation signal or message sent by the target edge device to the main node (usually the server) after successfully completing all necessary environmental configuration, model download and deployment steps. It indicates that the edge device is ready to run the target model and can start receiving requests from the user terminal. Optionally, this configuration completion instruction usually contains some metadata information, such as device ID, model version number, current status (such as "ready"), possible error reports, etc., to inform the system that the device has completed the configuration and the model can run normally.

[0089] The model operation programming interface (MAPI) is a set of interfaces provided by the master node to user terminals. Through these interfaces, users can operate the deployed target model on their terminals. It serves as a bridge between users and the model deployed on edge devices, allowing users to initiate queries, updates, and other forms of interaction. As can be understood, the MAPI provides users with an operational entry point, allowing them to conveniently interact with the model. For example, users can use the API to submit data for prediction, obtain model output, and adjust certain operational parameters.

[0090] Specifically, after completing all necessary tasks, such as environment configuration and model download, the target edge device sends a configuration completion command to the master node. This command indicates that the device is ready to run the specified model. The master node (i.e., the server) is responsible for listening for and receiving configuration completion commands from multiple target edge devices. This step confirms that the operating environment has been successfully set up and that each edge device is ready for operation.

[0091] After receiving configuration completion instructions from all target edge devices, the master node verifies this information to ensure that each device has been correctly configured. Once verified, the master node begins the authorization process for the user terminal, providing the user with access and control rights to the target models on these edge devices.

[0092] Finally, once the authorization process is complete, the master node grants the user terminal access to the model operation programming interface (API) specific to the deployed model. This means the user terminal can now call these APIs to perform various functions related to the target model, such as initiating inference and adjusting parameter settings. Once granted access, the user terminal can call the corresponding model operation API as needed. This allows users to directly interact with the model on the edge device, performing desired tasks or obtaining analytical results.

[0093] Once all target edge devices report that their configuration is complete, the model operation programming interface (API) is immediately available to users. This means users can immediately begin using newly deployed models without waiting for additional manual confirmation or debugging, improving user satisfaction and experience. This approach is particularly useful for application scenarios that require distributed deployment across multiple edge devices. It ensures that each device is correctly configured, and users can easily manage and operate models distributed across different devices through a unified API, promoting the development of large-scale distributed applications.

[0094] In one embodiment, a deployment request for a target model is obtained, the deployment request being determined by a deployment task for the target model, the deployment task being created by a user based on allocated resources and unallocated resources of respective multiple current resource pools.

[0095] Specifically, the user sees the allocation of the current target resource pool and memory pool, including the allocated computing power, unallocated idle computing power, the memory size of each pool, and the remaining memory available for allocation. Users create new inference or training tasks based on actual needs. For example, when deploying an object detection task, a 20TOPS target resource pool and a 5Gb memory pool are allocated to it. After the task is issued, a deployment request for the target model will be generated, and the master node will automatically interconnect the sub-nodes that meet the requirements to form the target resource pool required for the task. It can be understood that the target resource pool: is not based on the number of physical devices, such as 1 computing device. Instead, it is based on computing power, such as 100TOPS. For example, a 150TOPS device and a 50TOPS device can form a 200TOPS target resource pool.

[0096] In one embodiment, reference Figure 4 , Figure 4 This is the terminal deployment flowchart of the model deployment method.

[0097] First, the user opens the terminal application. The terminal checks for previously saved environmental assessment results. If so, it searches the cloud for a suitable model. If not, it performs environmental and performance tests on the current device to determine whether it meets the basic requirements for running the model. Based on the environmental assessment results, the terminal selects a suitable model from the cloud. The terminal then checks whether the basic environment required to run the selected model exists on the device. If so, it downloads the model-specific environment. If not, it downloads and installs the basic environment from the cloud. Finally, the inference program is started and an inference interface is created. The inference interface is the entry point for user interaction with the model. Users call the inference interface through the terminal to begin using the model for operations such as inference or prediction. When all operations are completed, the terminal terminates the process and releases occupied resources.

[0098] Thanks to the terminal deployment process, users can easily complete all operations from model selection to inference startup on their terminal, providing a smooth and consistent user experience. The inference interface allows users to immediately interact with the model and receive immediate feedback, enhancing interactivity and satisfaction. Furthermore, the modular design approach (such as the distinction between basic environments and model-specific environments) ensures cross-platform compatibility and adaptability to different terminal device types and configurations.

[0099] In an exemplary embodiment, Figure 3 As shown, it includes steps 502 to 508. Among them:

[0100] Step 502: Receive a deployment request for a target model sent by a user terminal; based on the deployment request, send parameter test instructions to multiple edge devices so that the multiple edge devices each perform parameter testing; receive current hardware performance parameters fed back by the multiple edge devices in response to the parameter test instructions.

[0101] Step 504: Determine the current hardware performance parameters required for configuring the target model based on the required hardware performance parameters when the target model is deployed; when the current hardware performance parameters of multiple edge devices match the current hardware performance parameters required by the target model, select the edge device to which the matching current hardware performance parameters belong as the target edge device; based on the selected target edge device, establish the target resource pool required by the target model; when the current hardware performance parameters of multiple edge devices do not match the current hardware performance parameters required by the target model, select at least two edge devices from the multiple edge devices as target edge devices, and the total amount of the current hardware performance parameters of the at least two edge devices matches the current hardware performance parameters required by the target model;

[0102] Step 506: Split the target model configuration file into multiple configuration sub-files based on the current hardware performance parameters of each target edge device, with each current hardware performance parameter corresponding to each of the multiple configuration sub-files. Send each configuration sub-file to the corresponding target edge device, and each configuration sub-file is used to instruct the corresponding target edge device to deploy part of the functional modules of the target model.

[0103] Step 508: receive configuration completion instructions fed back by multiple target edge devices; in response to the configuration completion instructions fed back by multiple target edge devices, open the calling permission of the model operation programming interface to the user terminal, so that the user terminal can implement the relevant functions of the target model by calling the model operation programming interface after obtaining the calling permission.

[0104] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0105] Based on the same inventive concept, the present application also provides a model deployment device for implementing the aforementioned model deployment method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more model deployment device embodiments provided below can be found in the above-mentioned limitations of the model deployment method and will not be repeated here.

[0106] In an exemplary embodiment, Figure 6 As shown, a model deployment device 600 is provided, including: an acquisition module 602, a selection module 604 and a sending module 606, wherein:

[0107] An acquisition module 602 is configured to acquire current hardware performance parameters of multiple edge devices based on a deployment request for a target model, where the deployment request includes required hardware performance parameters when the target model is deployed.

[0108] A selection module 604 is configured to select a target edge device from a plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model;

[0109] The sending module 606 is used to send the configuration file of the target model to the target edge device in the target resource pool. The configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

[0110] In one embodiment, the selection module 604 is used to determine the current hardware performance parameters required to configure the target model based on the required hardware performance parameters when the target model is deployed; when there are current hardware performance parameters among multiple edge devices that match the current hardware performance parameters required by the target model, the edge device to which the matching current hardware performance parameters belong is used as the target edge device; based on the selected target edge device, the target resource pool required for the target model is established.

[0111] In one embodiment, the selection module 604 is used to select at least two edge devices from the multiple edge devices as target edge devices when the current hardware performance parameters of the multiple edge devices do not match the current hardware performance parameters required by the target model, and the total amount of the current hardware performance parameters of the at least two edge devices matches the current hardware performance parameters required by the target model.

[0112] In one embodiment, the sending module 606 is used to split the configuration file of the target model into multiple configuration sub-files according to the current hardware performance parameters of each target edge device, and each current hardware performance parameter corresponds to the multiple configuration sub-files one by one; each configuration sub-file is sent to the corresponding target edge device, and each configuration sub-file is used to instruct the corresponding target edge device to deploy part of the functional modules of the target model.

[0113] In one embodiment, the acquisition module 602 is used to receive a deployment request for a target model sent by a user terminal; based on the deployment request, send parameter test instructions to multiple edge devices so that the multiple edge devices each perform parameter testing; and receive current hardware performance parameters fed back by the multiple edge devices in response to the parameter test instructions.

[0114] In one embodiment, the model deployment device also includes a providing module 608 for receiving configuration completion instructions fed back by multiple target edge devices; in response to the configuration completion instructions fed back by multiple target edge devices, the calling permission of the model operation programming interface is opened to the user terminal, so that the user terminal can implement the relevant functions of the target model by calling the model operation programming interface after obtaining the calling permission.

[0115] In one embodiment, the model deployment device further includes a request module 610 for obtaining a deployment request for the target model. The deployment request is determined by a deployment task for the target model, and the deployment task is created by the user based on the allocated resources and unallocated resources of multiple current resource pools. Each module in the above-mentioned model deployment device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0116] In another embodiment, if Figure 7 As shown, Figure 7 The block diagram of a model deployment apparatus 600 in one embodiment includes an acquisition module 602, a selection module 604, a sending module 606, a providing module 608, and a requesting module 610. The providing module 608 is configured to receive configuration completion instructions fed back by multiple target edge devices; in response to the configuration completion instructions fed back by the multiple target edge devices, the providing module 608 grants a user terminal access to a model operation programming interface, so that the user terminal, with access permission, can implement relevant functions of the target model by calling the model operation programming interface.

[0117] The request module 610 is used to obtain a deployment request for a target model. The deployment request is determined by a deployment task for the target model. The deployment task is created by a user based on the allocated resources and unallocated resources of multiple current resource pools.

[0118] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data related to model deployment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external device through a network connection. When the computer program is executed by the processor, a model deployment method is implemented.

[0119] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0120] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0121] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0122] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0123] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0124] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0125] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0126] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A model deployment method, characterized in that: The method comprises: Obtaining current hardware performance parameters of multiple edge devices based on a deployment request for the target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed; Selecting a target edge device from the plurality of edge devices according to the current hardware performance parameters and the required hardware performance parameters to form a target resource pool required for the target model; The configuration file of the target model is sent to the target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

2. The method according to claim 1, characterized in that The selecting, according to each of the current hardware performance parameters and the required hardware performance parameters, a target edge device from the plurality of edge devices to form a target resource pool required for the target model includes: Determining current hardware performance parameters required for configuring the target model based on the required hardware performance parameters when the target model is deployed; When there are current hardware performance parameters among the multiple edge devices that match the current hardware performance parameters required by the target model, the edge device to which the matching current hardware performance parameters belong is used as the target edge device; Based on the selected target edge devices, a target resource pool required for the target model is constructed.

3. The method according to claim 2, characterized in that The method further comprises: When the current hardware performance parameters of the multiple edge devices do not match the current hardware performance parameters required by the target model, at least two edge devices are selected from the multiple edge devices as target edge devices, and the total amount of the current hardware performance parameters of the at least two edge devices matches the current hardware performance parameters required by the target model.

4. The method according to claim 3, characterized in that The sending the configuration file of the target model to the target edge device in the target resource pool includes: Splitting the configuration file of the target model into multiple configuration sub-files according to the current hardware performance parameters of each target edge device, where each current hardware performance parameter corresponds to each of the multiple configuration sub-files; Each of the configuration sub-files is sent to a corresponding target edge device, and each of the configuration sub-files is used to instruct the corresponding target edge device to deploy part of the functional modules of the target model.

5. The method according to any one of claims 1 to 4, characterized in that The step of obtaining current hardware performance parameters of multiple edge devices based on a deployment request for a target model includes: receiving a deployment request for a target model sent by a user terminal; Based on the deployment request, sending parameter test instructions to multiple edge devices, so that the multiple edge devices each perform a parameter test; Receive current hardware performance parameters fed back by the multiple edge devices in response to the parameter test instruction.

6. The method according to claim 5, characterized in that The method further comprises: receiving configuration completion instructions fed back by each of the plurality of target edge devices; In response to the configuration completion instructions fed back by each of the multiple target edge devices, the calling permission of the model operation programming interface is opened to the user terminal, so that the user terminal can implement the relevant functions of the target model by calling the model operation programming interface after obtaining the calling permission.

7. The method according to claim 1, characterized in that Before obtaining current hardware performance parameters of the plurality of edge devices, the method further includes: A deployment request for a target model is obtained, where the deployment request is determined by a deployment task for the target model, where the deployment task is created by a user based on allocated resources and unallocated resources of respective multiple current resource pools.

8. A model deployment device, characterized in that: The device comprises: An acquisition module, configured to acquire current hardware performance parameters of multiple edge devices based on a deployment request for a target model, wherein the deployment request includes required hardware performance parameters when the target model is deployed; A selection module is configured to select a target edge device from the plurality of edge devices to form a target resource pool required for the target model according to each of the current hardware performance parameters and the required hardware performance parameters; A sending module is used to send a configuration file of the target model to a target edge device in the target resource pool, where the configuration file is used to instruct the target edge device in the target resource pool to deploy the target model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images