Execution method of AI model task of terminal equipment, cloud server and storage medium

Abstract

The embodiment of the invention discloses an execution method and device for an AI model task of terminal equipment, a cloud server and a storage medium, which are used for executing the AI model task of the terminal equipment in parallel through end-cloud collaboration, effectively shortening the iteration period of an AI model of the terminal equipment and improving the development efficiency of the AI model. The method provided by the embodiment of the invention comprises the following steps: acquiring a plurality of tasks related to a terminal equipment AI model, wherein the plurality of tasks comprise an end-cloud cooperation task; determining unoccupied terminal equipment in a terminal resource pool, wherein the terminal equipment in the terminal resource pool is terminal equipment connected with a cloud server; and executing the plurality of tasks in parallel according to the unoccupied terminal equipment.

Description

Technical Field

[0001] The present application relates to the field of deep learning, and in particular to a method, device, cloud server and storage medium for executing AI model tasks of a terminal device. Background Art

[0002] The iteration process of each model version in the entire development life cycle of the terminal device AI model, including model training, testing, and delivery and deployment, requires frequent manual switching between the two different environments of the cloud platform and the terminal device. The relative separation of the testing and development processes leads to a relatively long iteration cycle. In addition, the entire AI model development process is a process of continuous iterative optimization, and offline local testing and delivery and deployment are difficult to expand. At the same time, the limitation of computing resources in the offline environment leads to the serial execution of testing and delivery and deployment processes as a single task, which also seriously blocks the delivery and use of the terminal device AI model. Summary of the invention

[0003] The embodiments of the present application provide a method, device, cloud server and storage medium for executing AI model tasks of a terminal device, which are used to execute AI model tasks of a terminal device in parallel through end-cloud collaboration, effectively shortening the iteration cycle of the AI ​​model of the terminal device and improving the efficiency of AI model development.

[0004] The first aspect of the present application provides a method for executing an AI model task of a terminal device, which may include:

[0005] Acquire multiple tasks about the AI ​​model of the terminal device, wherein the multiple tasks include a terminal-cloud collaborative task;

[0006] Determine unoccupied terminal devices in a terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to a cloud server;

[0007] The plurality of tasks are executed in parallel according to the unoccupied terminal devices.

[0008] The second aspect of the present application provides an execution device of an AI model task of a terminal device, which may include:

[0009] An acquisition module, used to acquire multiple tasks about the AI ​​model of the terminal device, wherein the multiple tasks include a terminal-cloud collaborative task;

[0010] A determination module, used to determine unoccupied terminal devices in a terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to a cloud server;

[0011] An execution module is used to execute the multiple tasks in parallel according to the unoccupied terminal devices.

[0012] A third aspect of the present application provides a cloud server, which may include:

[0013] A memory storing executable program code;

[0014] a processor coupled to the memory;

[0015] The processor is used to execute the method described in the first aspect of the present application.

[0016] Another aspect of an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a processor, enables the processor to execute the method described in the first aspect of the present application.

[0017] Yet another aspect of an embodiment of the present application discloses a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the method described in the first aspect of the present application.

[0018] Another aspect of an embodiment of the present application discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes the method described in the first aspect of the present application.

[0019] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0020] In an embodiment of the present application, multiple tasks about the terminal device AI model are obtained, and the multiple tasks include end-cloud collaborative tasks; unoccupied terminal devices in the terminal resource pool are determined, and the terminal devices in the terminal resource pool are terminal devices connected to the cloud server; and the multiple tasks are executed in parallel according to the unoccupied terminal devices. It is used to execute the terminal device AI model tasks in parallel through end-cloud collaboration, which effectively shortens the iteration cycle of the terminal device AI model and improves the efficiency of AI model development. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments and the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained based on these drawings.

[0022] Figure 1 This is a system architecture diagram used in the embodiments of the present application;

[0023] Figure 2 A schematic diagram of an embodiment of a method for executing a terminal AI model task in an embodiment of the present application;

[0024] Figure 3A A schematic diagram of the connection between a local mobile phone and a cloud platform in an embodiment of the present application;

[0025] Figure 3B A schematic diagram of multi-threaded parallel pipelining in an embodiment of the present application;

[0026] Figure 3C A schematic diagram of parallel flow control in an embodiment of the present application;

[0027] Figure 3D A schematic diagram of a process for performing indicator polling in an embodiment of the present application;

[0028] Figure 3E A flowchart of a method for executing an AI model task of a terminal device in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of an execution device of an AI model task of a terminal device in an embodiment of the present application;

[0030] Figure 5 This is a schematic diagram of an embodiment of a cloud server in an embodiment of the present application;

[0031] Figure 6 This is a schematic diagram of an embodiment of a terminal device in an embodiment of the present application;

[0032] Figure 7 This is another schematic diagram of a cloud server in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application provide a method, device, cloud server and storage medium for executing AI model tasks of a terminal device, which are used to execute AI model tasks of a terminal device in parallel through end-cloud collaboration, effectively shortening the iteration cycle of the AI ​​model of the terminal device and improving the efficiency of AI model development.

[0034] In order to make the technical personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, they should all fall within the scope of protection of the present application.

[0035] The following is a brief description of some of the terms involved in this application, as follows:

[0036] End-cloud collaboration refers to the collaborative completion of computing resources and tasks between cloud platforms (i.e., cloud servers) and terminal devices in the context of distributed computing and edge computing, so as to achieve more efficient computing and data processing. In this mode, cloud servers and terminal devices work closely together to complete various computing tasks through the powerful computing power of cloud computing and the high responsiveness of terminal devices. In end-cloud collaboration, cloud servers are responsible for large-scale data storage and complex computing tasks, while terminal devices are responsible for real-time perception, data collection, and low-latency response. When a task needs to be processed, the cloud server can analyze the nature of the task and assign part of the calculation to the terminal device to complete, thereby reducing data transmission and computing delays. After the terminal device completes the task, it transmits the result back to the cloud platform, achieving efficient end-cloud collaboration.

[0037] Get request: Requests data from a specified resource to obtain data, generally used for operations such as search, sorting, and filtering.

[0038] Post request: Submits data to be processed to the specified resource, used to send data to the server, generally used to modify and write data.

[0039] With the continuous enhancement of the intelligent computing power of terminal devices (the mobile phone is used as an example in this article), the smart scenarios supported by the mobile phone side are becoming more and more abundant, including 1) model reasoning capabilities for basic scenarios such as classification, detection, and face; 2) complex scenario capabilities with large computational loads such as segmentation, super-resolution, and semantics; 3) Natural language processing (NLP) supported by flagship mobile phone chip platforms and artificial intelligence generated content (AIGC) supported by some platforms. Supported by the huge computing power scale of the cloud, there is a huge space for the utilization of intelligent computing power on the mobile phone side, and more and more models are directly deployed on the mobile phone side to enable business.

[0040] For the entire development cycle of massive artificial intelligence (AI) models for terminal devices, including tasks such as training, testing, and delivery and deployment of AI models for terminal devices, whether the AI ​​model can be successfully applied to the terminal device side ultimately depends on a series of indicators such as the performance, accuracy, and power consumption of the AI ​​model on the terminal device side. However, the training part of the AI ​​model is currently mostly concentrated in the cloud for iteration, and usually lacks the perception capability of the hardware on the terminal device side. At the same time, the subsequent testing and delivery and deployment are highly dependent on the hardware equipment of the terminal device and need to be completed offline in the local terminal device service environment.

[0041] In the embodiment of the present application, the terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self driving, a wireless terminal device in remote medical, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.

[0042] As an example but not limitation, in the embodiments of the present application, the terminal device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0043] This application proposes a method and device for executing terminal device AI model tasks in terminal-cloud cooperative (TCC). This application supports end-to-end AI model development, testing, and delivery deployment on the cloud platform by building a data interaction link between the terminal device and the cloud platform (also called a cloud server), effectively shortening the iteration cycle of the terminal device AI model and improving the efficiency of AI model development.

[0044] In the embodiments of the present application, by adopting the end-cloud collaboration technology, the terminal device can complete multi-region cloud sharing without affecting the user experience; by utilizing the effective segmentation of the terminal device AI model tasks and the elastic scaling characteristics of cloud resources, fine-grained control of the end-cloud resources is performed on each module of the task, thereby improving the resource utilization efficiency of the end-cloud collaboration tasks; multi-threading and pipeline control are used to realize the parallel pipeline of multiple tasks; as a whole, the full cycle process of the terminal device AI model development is moved from cloud-centralization to cloud + end collaborative integration, and from manual and frequent online and offline switching to automated parallelization and pipeline, thereby building a ubiquitous intelligent computing power multi-task parallel computing architecture for terminal devices.

[0045] like Figure 1 The system architecture diagram shown in FIG. Figure 1 As shown, the cloud server includes a terminal resource management module, a parallel pipeline module, a parallel experiment management module, a log storage and parsing module, and a metric polling module.

[0046] The terminal resource management module is used to support local terminal devices and cloud terminal devices to access the cloud platform, form a terminal resource pool, and support real-time query of the terminal device status in the terminal resource pool.

[0047] The parallel pipeline module is used to execute multiple tasks in parallel with multiple threads through reasonable task segmentation and fine-grained resource control, or to execute each subtask in multiple tasks in parallel with multiple threads, or to execute multiple tasks in parallel pipeline with multiple threads, or to execute each subtask in multiple tasks in parallel pipeline with multiple threads.

[0048] The parallel experiment management module is used to manage parallel tasks and visualize and compare indicators.

[0049] The log storage and parsing module is used to effectively record logs during the execution of parallel tasks and support fast parsing of specific indicators.

[0050] The metric polling module is used to poll the API (Application Programming Interface) through the log, poll the metrics generated in the log and feed them back to the parallel pipeline module to achieve effective control during the execution of parallel tasks.

[0051] The technical solution of the present application is further described below by way of embodiments. Figure 2 As shown, it is a schematic diagram of an embodiment of a method for executing a terminal AI model task in an embodiment of the present application, which may include:

[0052] 201. Connect multiple terminal devices to a cloud server to form a terminal resource pool.

[0053] Optionally, the terminal device may include a local terminal device and a cloud terminal device.

[0054] Optionally, the method may further include: querying the status of each terminal device in the terminal resource pool.

[0055] Optionally, the state of the terminal device includes an occupied state and an unoccupied state.

[0056] Optionally, in the case that the multiple terminal devices are multiple local terminal devices, connecting the multiple terminal devices to the cloud server to form a terminal resource pool may include: connecting the multiple terminal devices to the cloud server through an industrial computer and a jump server to form a terminal resource pool.

[0057] It is understandable that in this application, the terminal device is described by taking a mobile phone as an example, and the cloud server can also be called a cloud platform. The terminal resource management module can include a local industrial computer access submodule and a terminal device status polling submodule, which are described below respectively:

[0058] 1) Local industrial computer access submodule

[0059] For example, the "ssh authentication + adb direct connection" solution can be adopted, using the ssh configuration key method for authentication, and using open STF or adb tool (adb kit) to perform protocol conversion on the industrial computer. The industrial computer refers to the local industrial computer, that is, the personal computer (PC) with the agent installed. Among them, open STF is a mobile device management platform that can perform remote management, debugging, remote mobile desktop monitoring and other operations on mobile phones.

[0060] The technical solution of this application uses an industrial computer and a jump server to realize a secure interactive link between the Internet and the office network. The industrial computer opens the Internet Protocol (IP) of the office network (intranet) and connects to the mobile phone through the Universal Serial Bus (USB); the jump server opens the Internet (public network) IP, and configures the user key and the industrial computer to access the network.

[0061] like Figure 3A As shown, it is a schematic diagram of the connection between the local mobile phone and the cloud platform in the embodiment of the present application. The connection process between the mobile phone and the cloud platform can be divided into the following steps:

[0062] (1) The industrial computer is directly connected to the mobile phone via USB;

[0063] (2) Install the adb kit on the industrial computer and perform protocol conversion;

[0064] (3) Perform port mapping on the jump server;

[0065] (4) Generate public and private keys (i.e. user keys) for the user;

[0066] (5) Configure user keys and port permissions on the jump server;

[0067] (6) Use the user key generated in (4) to map the local port (i.e., the port of the industrial computer) to the port of the jump server;

[0068] (7) Connect to the local port through adb.

[0069] 2) Terminal device status polling submodule

[0070] Through the terminal device access to the cloud platform, each user is provided with a customized terminal resource pool according to different permissions, that is, each user has different permissions, and the information obtained from the terminal resource pool is also different. In the process of parallel execution of multi-tasks in end-cloud collaboration, the parallel use of idle terminal devices in the terminal resource pool is involved; the terminal device status polling submodule provides a corresponding application programming interface (Application Programming Interface, API), which can query the identity (Identity, ID) of all idle terminal devices in the current terminal resource pool. At the same time, in order to prevent multiple tasks from accessing the same terminal device at the same time and causing a "deadlock" problem, it is necessary to update the real-time status of the terminal device while the terminal device is occupied to prevent it from being applied for by other tasks.

[0071] 202. Acquire multiple tasks about the AI ​​model of the terminal device, wherein the multiple tasks include terminal-cloud collaborative tasks.

[0072] Optionally, the multiple tasks may be tasks of the same terminal device AI model or tasks of different terminal device AI models, which is not specifically limited in this application.

[0073] Optionally, the multiple tasks may be multiple test tasks or multiple development tasks, which is not specifically limited in this application.

[0074] 203. Determine unoccupied terminal devices in a terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to a cloud server.

[0075] Optionally, determining the unoccupied terminal devices in the terminal resource pool may include: querying the status of each terminal device in the terminal resource pool; and determining the unoccupied terminal devices according to the status of each terminal device. It is understandable that the status of the terminal device includes an occupied state and an unoccupied state.

[0076] 204. Execute the multiple tasks in parallel according to the unoccupied terminal devices.

[0077] Optionally, executing the multiple tasks in parallel according to the unoccupied terminal devices may include: executing the multiple tasks in parallel and in a pipeline according to the unoccupied terminal devices.

[0078] Optionally, executing the multiple tasks in parallel according to the unoccupied terminal devices may include: dividing the multiple tasks according to configuration requirements of the multiple tasks to obtain subtasks; and executing the subtasks in parallel according to the unoccupied terminal devices.

[0079] Optionally, executing the subtasks in parallel according to the unoccupied terminal devices may include: executing the subtasks in parallel and pipeline according to the unoccupied terminal devices.

[0080] It is understandable that the role of pipeline is to control the order in which tasks are executed.

[0081] Optionally, executing the subtasks in parallel according to the unoccupied terminal devices may include: configuring resources required for each subtask in sequence according to the number of the subtasks, the resources required for each subtask including a graphics processing unit GPU, a central processing unit CPU, memory, video memory and at least one of a task type, the task types including end-cloud collaborative subtasks and cloud subtasks; configuring the degree of parallelism; executing the subtasks in parallel according to the unoccupied terminal devices and the resources required for each subtask.

[0082] Among them, the end-cloud collaboration subtask requires the cloud server and terminal devices to cooperate in execution, and the end-cloud collaboration subtask requires the cloud server to send and execute to the available terminal devices. The cloud subtask only needs the cloud server to execute.

[0083] In order to solve the problem of limited terminal cloud resources, based on the task type of the cloud platform and combining the configuration requirements of each task for different resources (graphics processing unit (GPU) / central processing unit (CPU) / memory / video memory, whether the terminal device resources need to be coordinated), the fine-grained control of the pipeline is implemented in the automation process, and the scalability of the cloud platform resources is combined to improve the overall utilization of terminal devices and cloud platform resources.

[0084] It can be understood that the parallel pipeline module can include a task segmentation submodule, a parallel pipeline and resource configuration submodule, a multi-threaded parallel pipeline construction submodule and a Task management submodule, which are described below respectively:

[0085] 1) Task segmentation submodule

[0086] The life cycle of the cloud platform task type will automatically apply for and release cloud computing resources. According to the configuration requirements of different end cloud resources, the running context requirements, and whether terminal device resources are needed for the specific task steps, the entire task process is reasonably divided into subtasks; then the entire task is scheduled based on this granularity to improve the end cloud resource efficiency in the parallel flow process.

[0087] For example, for a specific task, the startup shell script is modified, conditional branches are used to implement the segmentation of task execution dimensions, and different input parameters are used to support the startup of different subtasks. For example, the following shell script is the startup entry for a specific task, which is divided into three subtasks: quant, infer, and visandbench. Different subtasks can be started by controlling parameters.

[0088]

[0089]

[0090] It should be noted that the above-mentioned task segmentation is achieved by segmenting the task startup shell script to achieve segmentation and pipelining of the task execution dimension. This application is not limited to the form of shell, but can also be py and other languages, etc. This application does not make specific limitations.

[0091] 2) Parallel pipeline and resource configuration submodule

[0092] Based on the number of subtasks divided by the task segmentation submodules, the graphics processing unit (GPU), central processing unit (CPU), memory, video memory required for each subtask are configured in turn, as well as whether the subtask requires terminal device resource coordination; at the same time, the parallelism of multiple tasks can be configured according to the task input.

[0093] 3) Multi-threaded parallel pipeline construction submodule

[0094] Based on the access of the terminal resource management module, a development and testing environment for the terminal device AI model with end-to-end integration is built; the task segmentation submodule further refines the end-to-end cloud resources of each module according to the business logic of each module in the development of a single AI model, and performs reasonable splitting and packaging to form each subtask; the parallel pipeline and resource configuration submodule configures the parallelism of the development of multiple AI models and the end-to-end cloud resources required for the subtasks in each task.

[0095] For example, Figure 3BThe figure is a schematic diagram of a multi-threaded parallel pipeline in an embodiment of the present application. The multi-threaded parallel pipeline construction submodule builds a general parallel pipeline of a multi-task pipeline based on the above priors:

[0096] (1) Build a multi-threaded parallel pool according to the parallelism setting value;

[0097] (2) Obtain all available terminal device IDs in the terminal resource pool under the current authority;

[0098] (3) According to the subtask resource settings of the parallel and resource configuration submodule, a custom pipeline class is automatically built to ensure the running sequence and task type (end-cloud collaborative task / cloud task) of each subtask in the pipeline;

[0099] (4) According to the segmented subtasks and configured end-cloud resources, post requests are sent to the specified API structure in parallel to execute cloud-type and end-cloud collaboration-type subtasks.

[0100] In the embodiments of the present application, the entire process of AI model task execution is decoupled from user code, and there is no user code intrusion; the entire process executes multiple end-cloud collaborative tasks (Task) in parallel and interactively, effectively improving the utilization of resources on the terminal device side and the cloud platform side, realizing parallel pipeline execution of multiple tasks, and improving the production efficiency of the end AI model.

[0101] The serial delivery time of a single AI model with end-cloud integration is T 0 , the parallelism of the AI ​​model is N, and the current bottleneck time of the whole process is T 3 4 ,express Figure 3C 3.4, the number of idle terminal devices in the terminal resource pool is d, then the prime time T satisfies:

[0102]

[0103] For example, Figure 3C FIG. 1 is a schematic diagram of parallel flow control in an embodiment of the present application. Figure 3C In the figure, d = 2. Each row can be considered as a task, so Figure 3C In the figure, 6 tasks are used as an example. 0.0, 0.1, 0.2, 1.1, 1.2, 1.3, 2.0, 3.1, 3.2, 3.3, 3.4, 4.1, 4.2, 4.3 and 5.0 included in each row can be considered as multiple subtasks included in each task. The width of the small grid in each row can represent the execution time of the subtask.

[0104] Assume that step 3.4 and step 5.0, which require the coordination of terminal device resources in task segmentation, are separately encapsulated and resource scheduled. In the first time period, 2 subtasks (including 0.0 and 0.0) are executed in parallel on the cloud server. In the second time period, 4 subtasks (including 0.1, 0.1, 0.0 and 0.0) are executed in parallel on the cloud server. In the third time period, 6 subtasks (including 0.2, 0.2, 0.1, 0.1, 0.0 and 0.0) are executed in parallel on the cloud server. In the fourth time period, 6 subtasks (including 1.1, 1.1, 0.2, 0.2, 0.1 and 0.1) are executed in parallel on the cloud server. In the fifth time period, 6 subtasks (including 1.2, 1.2, 1.1, 1.1, 0.2 and 0.2) are executed in parallel on the cloud server. In the sixth duration, six subtasks (including 1.3, 1.3, 1.2, 1.2, 1.1 and 1.1) are executed in parallel on the cloud server. In the seventh duration, six subtasks (including 2.0, 2.0, 1.3, 1.3, 1.2 and 1.2) are executed in parallel on the cloud server. In the eighth duration, six subtasks (including 3.1, 3.1, 2.0, 2.0, 1.3 and 1.3) are executed in parallel on the cloud server. In the ninth duration, six subtasks (including 3.2, 3.2, 3.1, 3.1, 2.0 and 2.0) are executed in parallel on the cloud server. In the tenth duration, six subtasks (including 3.3, 3.3, 3.2, 3.2, 3.1 and 3.1) are executed in parallel on the cloud server. During the eleventh time period, 2 subtasks (including 3.4 and 3.4) are executed in parallel on the terminal device, and 4 subtasks (including 3.3, 3.3, 3.2 and 3.2) are executed in parallel on the cloud server. During the twelfth time period, 2 subtasks (including 3.4 and 3.4) are executed in parallel on the terminal device, and 4 subtasks (including 4.1, 4.1, 3.3 and 3.3) are executed in parallel on the cloud server. During the thirteenth time period, 2 subtasks (including 3.4 and 3.4) are executed in parallel on the terminal device, and 4 subtasks (including 4.2, 4.2, 4.1 and 4.1) are executed in parallel on the cloud server. During the fourteenth time period, 6 subtasks (including 4.3, 4.3, 4.2, 4.2, 4.1 and 4.1) are executed in parallel on the cloud server. During the fifteenth time period, two subtasks (including 5.0 and 5.0) are executed in parallel on the terminal device, and four subtasks (including 4.3, 4.3, 4.2, and 4.2) are executed in parallel on the cloud server. During the sixteenth time period, two subtasks (including 5.0 and 5.0) are executed in parallel on the terminal device, and two subtasks (including 4.3 and 4.3) are executed in parallel on the cloud server. During the seventeenth time period, two subtasks (including 5.0 and 5.0) are executed in parallel on the terminal device.

[0105] It should be noted that the first duration, the second duration, ..., the seventeenth duration may be the same or different in size, and no specific limitation is made here.

[0106] 4)Task management submodule

[0107] The Task Management submodule provides API interfaces for two types of tasks: end-cloud collaborative tasks and cloud tasks based on the cloud platform. The multi-threaded parallel pipeline submodule queries the available terminal devices from the terminal resource pool according to the end-cloud resource configuration of each subtask in the task segmentation, so as to trigger different Tasks to complete the parallel pipeline tasks. The parameter configuration content in the task is received in the form of an interface.

[0108] 205. Record indicator logs during the execution of the multiple tasks.

[0109] The user service code only needs to use the print standard indicator log. For example, the standard indicator log printing format is as follows:

[0110] print('--Metric--step:[N],key1:[value1],key2:[value2]')

[0111] Among them, --Metric-- is the identifier, step is the number of iteration steps, key is the metric name, and value is the metric value.

[0112] On this basis, the log storage and analysis module analyzes the standardized indicator log output in the log, stores the analyzed indicator content in TsHouse, MongoDB, or ChubaoFS, and sends the latest indicator information to the user interface (UI) for indicator display and comparison. At the same time, the log query function provides a corresponding indicator query interface for the parallel pipeline module to perform metric polling to obtain the running status and metric of each task in the experiment.

[0113] 206. Perform the parallel task management according to the indicator logs during the execution of the multiple tasks.

[0114] Optionally, the parallel task management based on the indicator logs during the execution of the multiple tasks may include: displaying task status, querying task details, performing visual comparison of task indicators, stopping tasks, deleting tasks, etc. based on the indicator logs during the execution of the multiple tasks.

[0115] The parallel experiment management module uses experiments to associate tasks, and updates the task status and the status of the experiment itself. It supports visual comparison of indicators of different tasks in the experiment and supports manual intervention in the experiment status and task status. Specific operations may include but are not limited to display of experiment status, experiment details, visual comparison of experimental task indicators, manual task stop function, task deletion function, and experiment deletion function. Among them, the experiment includes n tasks.

[0116] In addition, in the parallel experiment management module, a user field can be added to achieve isolation between users during use, ensure the information security of each user, and make the actual production belt safer and more reliable.

[0117] The Metric polling module uses a multi-threaded method to poll the indicator query interface. The interface returns content including interface status, Task running status, and indicator Metric. When the interface returns, the indicator Metric of the current Task is parsed according to the defined rules, and the Metric is uploaded to the parallel pipeline module; when the indicator parsing fails and the Task running status is unavailable, the Metric polling module is in a waiting state; when the Task is in a successful state, it continuously polls for a certain period of time, and the indicator parsed at the final time is sent to the parallel pipeline module as the final indicator.

[0118] For example, Figure 3D As shown, it is a flow chart of index polling in the embodiment of the present application. When starting index polling, first determine whether the index query interface is connected normally. If the connection is abnormal, re-query. In the case of normal connection, determine whether the task exists. If the task does not exist, it can be determined whether it is manually deleted. If it is manually deleted, exit directly. If it is not manually deleted, exit abnormally. If the task exists, determine whether the indicator exists. If the indicator does not exist, it can be determined whether the task is suspended. If it is suspended, exit, if it is not suspended, re-query. If the indicator exists, the task status can be determined; if the task status is task waiting, re-query. If the task status is task running, for example, you can poll for two minutes, the indicator cannot be found, exit; if the indicator is found and the polling exceeds two minutes, upload the indicator; if the polling time is not enough, re-query. If the task status is task success, if the indicator is found and there is an update, upload the indicator; if the indicator is found and there is no update, re-query; if the indicator cannot be found, re-query.

[0119] like Figure 3EAs shown, it is a flow chart of the execution method of the terminal device AI model task in the embodiment of the present application. The local terminal device and the cloud terminal device are connected to the cloud platform to form a terminal resource pool. The local terminal device can be connected to the cloud platform through the local industrial computer, and the cloud terminal device can be connected to the cloud platform through remote debugging. The terminal resource pool supports querying the status of each terminal device; the cloud server can also perform task segmentation, parallel and resource configuration, terminal cloud resource configuration, multi-threaded parallel pipeline construction, indicator polling, parallel experiment management, indicator log storage and analysis, etc.

[0120] It should be noted that steps 201, 205 and 206 are optional steps.

[0121] This application mainly proposes a development, testing and task scheduling method for an end-cloud collaborative terminal device AI model. This method builds an end-cloud integrated model development environment, avoids frequent switching between the cloud platform and the local during the development and testing of the terminal device AI model, and effectively shortens the development and testing cycle in the model iteration; forms a parallel pipeline framework without business code intrusion, supports end-cloud integrated multi-task automation, parallelization and pipeline execution, improves end-cloud resource utilization, and improves the delivery efficiency of multiple models; in addition, by standardizing the storage and query of logs, the information interaction between the parallel pipeline scheduling algorithm and the end-cloud collaborative tasks is realized in the form of an interface, thereby achieving effective control of the parallel process.

[0122] Based on the premise of no business code intrusion, and on the basis of being compatible with the entire development life cycle of the existing end-cloud collaborative terminal device AI model, this application also supports the use of multi-threaded parallel pipeline control to achieve automated integration and automated management of massive tasks, saving users time in repeated manual configuration. At the same time, the use of pipeline algorithms has also greatly saved end-cloud computing resource costs to a certain extent.

[0123] It is understandable that in this application, the business training tasks rely on the machine learning platform, and the calculation process is completely independent of any business framework. Therefore, it can make full use of various cluster resources such as Kubernetes to achieve distributed training, and the generalization ability of the platform is stronger than other solutions.

[0124] In an embodiment of the present application, multiple tasks about the terminal device AI model are obtained, and the multiple tasks include end-cloud collaborative tasks; unoccupied terminal devices in the terminal resource pool are determined, and the terminal devices in the terminal resource pool are terminal devices connected to the cloud server; and the multiple tasks are executed in parallel according to the unoccupied terminal devices. It is used to execute the terminal device AI model tasks in parallel through end-cloud collaboration, which effectively shortens the iteration cycle of the terminal device AI model and improves the efficiency of AI model development.

[0125] The technical solution of this application makes full use of the end-cloud collaboration technology to build an interactive link between the cloud platform and terminal devices under different networks, enabling users to perform end-cloud collaboration model development, testing, delivery and deployment tasks on the cloud platform; supports effective task segmentation according to the different demands for end-cloud resources at each stage of the task; utilizes a parallel pipeline control mechanism without code intrusion to achieve parallel pipeline execution of each segmented subtask of multiple tasks, effectively improving the utilization rate of end-cloud resources; and at the same time, through effective management of parallel tasks and effective analysis and feedback of logs, realizes intelligent control of parallel tasks.

[0126] like Figure 4 As shown, it is a schematic diagram of a device for executing an AI model task of a terminal device in an embodiment of the present application, which may include:

[0127] An acquisition module 401 is used to acquire multiple tasks related to the AI ​​model of the terminal device, wherein the multiple tasks include a terminal-cloud collaborative task;

[0128] The determination module 402 is used to determine unoccupied terminal devices in the terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to the cloud server;

[0129] The execution module 403 is used to execute the multiple tasks in parallel according to the unoccupied terminal devices.

[0130] Optionally, in some embodiments of the present application, the terminal device AI model task execution device may further include:

[0131] A recording module 404 is used to record the indicator logs during the execution of the multiple tasks;

[0132] The management module 405 is used to manage the parallel tasks according to the indicator logs during the execution of the multiple tasks.

[0133] Optionally, in some embodiments of the present application, the execution module 402 is specifically used to divide the multiple tasks into subtasks according to the configuration requirements of the multiple tasks; and execute the subtasks in parallel according to the unoccupied terminal devices.

[0134] Optionally, in some embodiments of the present application, the execution module 402 is specifically used to configure the resources required for each subtask in sequence according to the number of the subtasks, and the resources required for each subtask include a graphics processing unit GPU, a central processing unit CPU, memory, video memory, and at least one of the task types, and the task types include end-cloud collaborative subtasks and cloud subtasks; according to the unoccupied terminal devices and the resources required for each subtask, the subtasks are executed in parallel.

[0135] Optionally, in some embodiments of the present application, the terminal device AI model task execution device may further include:

[0136] The access module 406 is used to connect multiple terminal devices to the cloud server to form the terminal resource pool.

[0137] Optionally, in some embodiments of the present application, when the multiple terminal devices are multiple local terminal devices, the access module 406 is specifically used to connect the multiple terminal devices to the cloud server through the industrial computer and the jump server to form the terminal resource pool.

[0138] Optionally, in some embodiments of the present application, the management module 405 is specifically used to display the task status, query the task details, perform a visual comparison of the task indicators, stop the task, or delete the task according to the indicator logs during the execution of the multiple tasks.

[0139] like Figure 5 As shown, it is a schematic diagram of an embodiment of a cloud server in an embodiment of the present application, which may include: Figure 4 The execution device of the AI ​​model task of the terminal device shown.

[0140] like Figure 6 The figure is a schematic diagram of an embodiment of a terminal device in an embodiment of the present application. It may include:

[0141] Combine the following Figure 6 The following is a detailed introduction to the various components of the mobile phone in the terminal device:

[0142] The RF circuit 610 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 680 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 610 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 610 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0143] The memory 620 can be used to store software programs and modules. The processor 680 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 620. The memory 620 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 620 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0144] The input unit 630 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 631 or near the touch panel 631 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 631 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 680, and can receive and execute commands sent by the processor 680. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 631, the input unit 630 may also include other input devices 632. Specifically, the other input devices 632 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0145] The display unit 640 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 640 may include a display panel 641. Optionally, the display panel 641 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 631 may cover the display panel 641. When the touch panel 631 detects a touch operation on or near it, it is transmitted to the processor 680 to determine the type of touch event. Subsequently, the processor 680 provides a corresponding visual output on the display panel 641 according to the type of touch event. Although in Figure 6 In the embodiment, the touch panel 631 and the display panel 641 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 631 and the display panel 641 can be integrated to realize the input and output functions of the mobile phone.

[0146] The mobile phone may also include at least one sensor 650, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 641 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 641 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0147] The audio circuit 660, the speaker 661, and the microphone 662 can provide an audio interface between the user and the mobile phone. The audio circuit 660 can transmit the received audio data to the speaker 661 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 662 converts the collected sound signal into an electrical signal, which is received by the audio circuit 660 and converted into audio data, and then the audio data is output to the processor 680 for processing, and then sent to another mobile phone through the RF circuit 610, or the audio data is output to the memory 620 for further processing.

[0148] Wi-Fi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the Wi-Fi module 670, which provides users with wireless broadband Internet access. Figure 6 A Wi-Fi module 670 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0149] The processor 680 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 620, and calling data stored in the memory 620, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 680 may include one or more processing units; preferably, the processor 680 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 680.

[0150] The mobile phone also includes a power supply 690 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 680 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0151] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0152] like Figure 7 As shown, it is a schematic diagram of another embodiment of the cloud server in the embodiment of the present application, which may include:

[0153] A memory 701 storing executable program codes;

[0154] a processor 702 coupled to the memory 701;

[0155] Processor 702 is used to obtain multiple tasks about the terminal device AI model, and the multiple tasks include terminal-cloud collaborative tasks; determine unoccupied terminal devices in a terminal resource pool, and the terminal devices in the terminal resource pool are terminal devices connected to a cloud server; and execute the multiple tasks in parallel according to the unoccupied terminal devices.

[0156] Optionally, in some embodiments of the present application, the processor 702 is further used to record the indicator logs during the execution of the multiple tasks; and perform the parallel task management according to the indicator logs during the execution of the multiple tasks.

[0157] Optionally, in some embodiments of the present application, the processor 702 is specifically used to divide the multiple tasks into subtasks according to the configuration requirements of the multiple tasks; and execute the subtasks in parallel according to the unoccupied terminal devices.

[0158] Optionally, in some embodiments of the present application, the processor 702 is specifically used to configure the resources required for each subtask in sequence according to the number of the subtasks, and the resources required for each subtask include a graphics processing unit GPU, a central processing unit CPU, memory, video memory, and at least one of the task types, and the task types include end-cloud collaborative subtasks and cloud subtasks; according to the unoccupied terminal devices and the resources required for each subtask, the subtasks are executed in parallel.

[0159] Optionally, in some embodiments of the present application, the processor 702 is further configured to connect multiple terminal devices to the cloud server to form the terminal resource pool.

[0160] Optionally, in some embodiments of the present application, when the multiple terminal devices are multiple local terminal devices, the processor 702 is specifically used to connect the multiple terminal devices to the cloud server through the industrial computer and the jump server to form the terminal resource pool.

[0161] Optionally, in some embodiments of the present application, the processor 702 is specifically used to display the task status, query the task details, perform a visual comparison of the task indicators, stop the task, or delete the task according to the indicator logs during the execution of the multiple tasks.

[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0163] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0164] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0168] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for executing an AI model task of a terminal device, characterized in that: include: Acquire multiple tasks about the AI ​​model of the terminal device, wherein the multiple tasks include a terminal-cloud collaborative task; Determine unoccupied terminal devices in a terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to a cloud server; The plurality of tasks are executed in parallel according to the unoccupied terminal devices.

2. The method according to claim 1, characterized in that The method further comprises: Recording the indicator logs during the execution of the multiple tasks; The parallel task management is performed according to the indicator logs during the execution of the multiple tasks.

3. The method according to claim 1 or 2, characterized in that: The executing the plurality of tasks in parallel according to the unoccupied terminal devices comprises: According to the configuration requirements of the multiple tasks, the multiple tasks are divided into subtasks; The subtasks are executed in parallel according to the unoccupied terminal devices.

4. The method according to claim 3, characterized in that: The executing the subtasks in parallel according to the unoccupied terminal devices includes: According to the number of the subtasks, the resources required for the subtasks are sequentially configured, where the resources required for the subtasks include a graphics processor GPU, a central processing unit CPU, a memory, a video memory, and at least one of a task type, where the task type includes a terminal-cloud collaborative subtask and a cloud subtask; The subtasks are executed in parallel according to the unoccupied terminal devices and the resources required by the subtasks.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Connect multiple terminal devices to the cloud server to form the terminal resource pool.

6. The method according to claim 5, characterized in that In the case where the multiple terminal devices are multiple local terminal devices, the step of connecting the multiple terminal devices to the cloud server to form the terminal resource pool includes: Multiple terminal devices are connected to the cloud server through industrial computers and jump servers to form the terminal resource pool.

7. The method according to claim 2, characterized in that The performing the parallel task management according to the indicator logs during the execution of the multiple tasks includes: According to the indicator logs during the execution of the multiple tasks, the task status is displayed, the task details are queried, the task indicators are visually compared, the task is stopped, or the task is deleted.

8. A device for executing AI model tasks of a terminal device, characterized in that: include: An acquisition module, used to acquire multiple tasks about the AI ​​model of the terminal device, wherein the multiple tasks include a terminal-cloud collaborative task; A determination module, used to determine unoccupied terminal devices in a terminal resource pool, where the terminal devices in the terminal resource pool are terminal devices connected to a cloud server; An execution module is used to execute the multiple tasks in parallel according to the unoccupied terminal devices.

9. A cloud server, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, which, when executed on a processor, causes the processor to perform the method according to any one of claims 1 to 7.

Images