Task processing method, device, electronic device and computer readable storage medium

Abstract

The present disclosure provides a task processing method, device, electronic device and computer-readable storage medium, including: obtaining a target computing task; determining that the target computing task corresponds to a target model image, the target model image includes an operating environment configuration file, a model image slice description file, and multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; pulling the operating environment configuration file and the model image slice description file from a non-model file, and starting a target container according to the operating environment configuration file and the model image slice description file; determining that the target model slice is used to process the target computing task; pulling the target model slice from the target model image, and running the target model slice in the started target container to perform the target computing task.

Description

Technical Field

[0001] The present disclosure relates to the field of computer and Internet technology, and in particular to a task processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of computers, it has become a common technical means to perform reasoning calculations on processing tasks through NN (Neural Network) models.

[0003] The above NN model can be trained in one device (such as device A), and then inference and prediction can be performed on a certain task in another device (such as device B).

[0004] Generally speaking, when device B needs an NN model to perform inference and prediction for a task, it needs to pull the complete image of the NN model from device A, which is a challenge for device B (especially when device B is a terminal) with limited storage resources. With the increasing popularity of containerized AI applications, the large image size caused by the NN model will also become a bottleneck for image distribution and container deployment. Summary of the invention

[0005] The purpose of the present disclosure is to provide a task processing method, device, electronic device and computer-readable storage medium, which can independently deploy container startup and model operation, reduce the computing burden of the device, and thus improve the speed and efficiency of task processing.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] The present disclosure provides a task processing method, including: obtaining a target computing task; determining that the target computing task corresponds to a target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; pulling the operating environment configuration file and the model image slice description file from the non-model file, and starting a target container according to the operating environment configuration file and the model image slice description file; determining that the target model slice is used to process the target computing task; pulling the target model slice from the target model image, and running the target model slice in the started target container to perform the target computing task.

[0008] In some embodiments, the target model slice includes a first model slice and a second model slice, and the model image slice description file includes structural description information describing the structure of the target model; wherein, pulling the target model slice from the target model image and running the target model slice in the started target container to perform the target computing task includes: determining, according to the structural description information, that the execution order of the first model slice is before the second model slice in the target computing task; pulling and loading the first model slice from the target model image, and running the first model slice in the started target container; while running the first model slice, pulling and loading the second model slice from the target model image to perform the target computing task.

[0009] In some embodiments, the model image slice description file also includes location description information describing the location of each model slice in the target model image, and the location description information includes first location description information describing the location of the first model slice and second location description information describing the location of the second model slice; wherein, pulling and loading the first model slice from the target model image includes: pulling and loading the first model slice from the target model image according to the first location description information; wherein, while running the first model slice, pulling and loading the second model slice from the target model image includes: while running the first model slice, pulling and loading the second model slice from the target model image according to the second location description information.

[0010] In some embodiments, the target model image is used to process a target task to be processed, and the target computing task is a collaborative subtask of the target task to be processed, wherein the target model slice is pulled from the target model image, and the target model slice is run in the started target container to perform the target computing task, including: running the target model slice in the started target container to perform the target computing task and obtain a target computing result; wherein the method also includes: sending the target computing result to a target aggregation device so that the target aggregation device completes the target task to be processed based on the target computing result.

[0011] In some embodiments, obtaining a target computing task may include: obtaining the target computing task; obtaining target input data corresponding to the target computing task, wherein the target input data is an intermediate computing result of the target task to be processed; wherein, running the target model slice in the started target container to perform the target computing task and obtain the target computing result includes: processing the target input data by running the target model slice to obtain the target computing result.

[0012] In some embodiments, the target model image is obtained after the first device performs image processing on the target model, and the target model is a neural network model; before obtaining the target computing task, the method also includes: slicing the target model in units of neural network layers or neurons to obtain multiple model slices; storing the multiple model slices in the layer structure order of the target model to generate the model slice file, wherein each model slice is stored in a model image layer; generating the model image slice description file according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image; generating the operating environment configuration file according to the operating environment and configuration of the target model; and generating the target model image according to the model slice file, the model image slice description file and the operating environment configuration.

[0013] In some embodiments, the model image slice description file in the target model image also includes computing amount information corresponding to each model slice, and structural description information describing the structure of the target model; wherein the target model image is used to process the target task to be processed, and the target computing task is a collaborative subtask of the target task to be processed; wherein, before obtaining the target computing task, the method also includes: a second device obtains the target task to be processed; determines that the target model image is used to process the target task to be processed; determines the available computing amount of at least one device to be unloaded for collaborative unloading of the target task to be processed; according to the computing amount corresponding to each model slice of the available computing amount of the at least one device to be unloaded, performs task segmentation on the target task to be processed so as to divide the target task to be processed into multiple subtasks to be processed; determines the model slices corresponding to each subtask to be processed in the multiple model slices according to the structural description information, and the multiple subtasks to be processed include the target computing task; the second device sends each subtask to be processed to the at least one device to be unloaded, so that the at least one device to be unloaded completes the corresponding subtask to be processed according to the model slice.

[0014] In some embodiments, the non-model file and the model slice file are divided by a landmark file in the image file system, and the non-model file is located below the landmark file; wherein, pulling the operating environment configuration file and the model image slice description file from the non-model file includes: accessing the image file system storing the target model image; determining the landmark file corresponding to the target model image in the image file system; and searching for the operating environment configuration file and the model image slice description file below the landmark file.

[0015] In some embodiments, starting a target container according to the operating environment configuration file and the model image slice description file includes: creating a model slice directory for each model slice of the target model according to the model image slice description file; determining the non-model file layer where the operating environment configuration file and the model image slice description file are located; and mounting the model slice directory and the non-model file layer as the rootfs of the target container, so as to start the target container according to the rootfs composed of the model slice directory and the non-model file layer.

[0016] In some embodiments, the model image slice description file includes the file offset of each model slice in the target model image; wherein, pulling the target model slice from the target model image includes: determining the target model image layer where the target model slice is located; determining the target file offset of the target model image layer in the model slice file according to the file offset of each model slice in the target model image; and pulling the target model slice from the target model image according to the target file offset.

[0017] The embodiment of the present disclosure provides a task processing device, including: a computing task acquisition module, an image determination module, a container startup module, an image slice determination module, and a slice pulling module.

[0018] Among them, the computing task acquisition module is used to acquire the target computing task; the image determination module can be used to determine that the target computing task corresponds to the target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; the container startup module can be used to pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; the image slice determination module can be used to determine that the target model slice is used to process the target computing task; the slice pulling module can be used to pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task.

[0019] An embodiment of the present disclosure provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the task processing methods described above.

[0020] The embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the task processing method as described in any one of the above items is implemented.

[0021] The embodiment of the present disclosure provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the above-mentioned task processing method.

[0022] The task processing method, device, electronic device and computer-readable storage medium provided by the embodiments of the present disclosure, on the one hand, execute the startup of the container and the operation of the model separately, reducing the load of the device in the same time period and improving the efficiency and success rate of task processing; on the other hand, the present disclosure divides the target model image into multiple slices so that the target device can pull the corresponding image slices in a targeted manner according to the target computing task, without pulling redundant parts, saving storage resources and communication resources and improving task processing efficiency.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram showing an exemplary system architecture that can be applied to a task processing method or a task processing device according to an embodiment of the present disclosure is shown.

[0026] Figure 2 The figure is a flowchart of a model image building method according to an exemplary embodiment.

[0027] Figure 3 The figure is a flowchart of a task processing method according to an exemplary embodiment.

[0028] Figure 4 The figure is a flow chart of a slice pulling method according to an exemplary embodiment.

[0029] Figure 5 It is a schematic diagram showing a task processing method according to an exemplary embodiment.

[0030] Figure 6 A task segmentation processing method is shown according to an exemplary embodiment.

[0031] Figure 7 The figure is a schematic diagram of a model mirror according to an exemplary embodiment.

[0032] Figure 8 The figure is a schematic diagram of file storage according to an exemplary embodiment.

[0033] Fig. 9 It is a schematic diagram of a task segmentation method according to an exemplary embodiment.

[0034] Fig.10 The figure is a schematic diagram showing a task offloading processing method according to an exemplary embodiment.

[0035] Fig.11 The figure is a block diagram of a task processing device according to an exemplary embodiment.

[0036] Fig.12 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0038] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0039] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0041] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0042] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0043] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 A schematic diagram showing an exemplary system architecture that can be applied to a task processing method or a task processing device according to an embodiment of the present disclosure is shown.

[0045] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0046] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, virtual reality devices, smart homes, etc.

[0047] Terminals 101, 102, and 103 may, for example, obtain a target computing task; terminals 101, 102, and 103 may, for example, determine that the target computing task corresponds to a target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; terminals 101, 102, and 103 may, for example, pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; terminals 101, 102, and 103 may, for example, determine that the target model slice is used to process the target computing task; terminals 101, 102, and 103 may, for example, pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task.

[0048] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received request data, and feed back the processing results to the terminal device.

[0049] The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as basic cloud computing services such as big data and artificial intelligence platforms, etc. The present disclosure does not impose any restrictions on this.

[0050] The server 105 may, for example, obtain a target computing task; the server 105 may, for example, determine that the target computing task corresponds to a target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; the server 105 may, for example, pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; the server 105 may, for example, determine that the target model slice is used to process the target computing task; the server 105 may, for example, pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task.

[0051] It should be understood that Figure 1 The number of terminal devices, networks and servers in the figure is only for illustration. The server 105 may be a physical server or may be composed of multiple servers. According to actual needs, it may have any number of terminal devices, networks and servers.

[0052] Before introducing the present disclosure, the related technologies involved in the present disclosure may be explained.

[0053] 1. Container technology

[0054] Containerization is a method of software development whereby an application or service, its dependencies, and its configuration (abstracted as a deployment manifest file) are packaged together as a container image. Containerized applications can be tested as a unit and deployed to a host as a container image instance. Containers serve as a standard unit for software deployment, allowing developers and IT (Information Technology) professionals to deploy them to different environments with little or no modification. This makes containers portable, enabling build once, deploy anywhere.

[0055] Another advantage of containerization is scalability. By creating new containers for short-term tasks through the container orchestration management platform, the scale of application deployment can be quickly expanded, and the platform can manage the containers flexibly.

[0056] In summary, containers provide the following advantages throughout the application lifecycle workflow: isolation, portability, flexibility, scalability, and controllability. The most important advantage is that they provide isolation between development and operations, speeding up application release and deployment.

[0057] 2. Container images

[0058] In terms of data types, the essence of an image is metadata (such as environment variables, mount information, etc.) and a rootfs (the file contents of the Docker image and some configuration files required to run the Docker container constitute the static file system rootfs that the Docker container runs on). The metadata information will eventually be integrated when the container is started; the rootfs provides the file system required for the container to run. Usually, an image consists of the following four parts: 1) ImageIndex (optional): Contains a list of Manifest descriptors, each Manifest can provide different implementations of the image, such as images suitable for different platform attributes; 2) ImageManifest: Contains ImageConfig and image layer descriptors; 3) ImageConfig: Contains image layer order and image configuration information; 4) Filesystem (rootfs) Layers: base image layer and image layer change set. The descriptor usually includes a hash code and file size for the content of the described file.

[0059] 3.NN model segmentation and computational task offloading

[0060] In recent years, the computing intensity, memory consumption and power consumption of AI mobile applications have been increasing. However, terminal devices and some edge devices usually have strict energy consumption, computing and memory cost restrictions for running complete NN model reasoning offline. Therefore, by splitting the NN model, all or part of the model calculation tasks can be offloaded to other nodes (cloud devices, edge devices or terminal devices). By means of collaborative reasoning and calculation between devices, the pressure on the original device's computing, memory usage, storage, power and required data rate can be reduced, while reducing reasoning latency and improving the service quality of AI applications.

[0061] Depending on the current task and environment, the NN model can be divided into multiple slices, with the goal of offloading the computationally intensive and energy-intensive parts to other nodes, leaving the privacy-sensitive and latency-sensitive parts on the terminal device. The device performs the inference calculation of a specific part of the model, and then sends the remaining computing tasks and intermediate results to other nodes. One or more nodes that receive the tasks and intermediate results complete the remaining inference calculations and finally return the inference results to the device (see Figure 1 ).

[0062] 4.NN model calculation offloading algorithm

[0063] The key to the NN model computation offloading algorithm is to select the optimal segmentation mode and segmentation point to ensure that the required resources are lower than the upper limit of the available resources of the mobile device and optimize the consumption of computing, storage / memory, power, and communication resources of the device and each participating node. According to different optimization goals, the NN model can be segmented at different points.

[0064] In the present disclosure, NN model calculation offloading will use NN layers as the smallest unit for slicing and segmentation. Common NN layers include convolutional layers, pooling layers, etc. Of course, neurons in the NN model can also be used as the smallest unit for segmentation, but the present disclosure is limited to this step.

[0065] The improved technical solution disclosed in the present invention first requires mirroring the target model on the construction side (e.g., the first device) to construct a target model image, wherein the target model may be a neural network model.

[0066] Figure 2 The figure is a flowchart of a model image building method according to an exemplary embodiment.

[0067] refer to Figure 2 , the above-mentioned model image construction method may include the following steps.

[0068] Step S202, slice the target model in units of neural network layers or neurons to obtain multiple model slices.

[0069] In some embodiments, if the target model is sharded in units of neural network layers, then a model shard may include one or more neural network layers, which is not limited in the present disclosure.

[0070] In some other embodiments, if the target model is fragmented in units of neurons, then a model fragment may include one or more neurons, which is not limited in the present disclosure.

[0071] In some embodiments, after obtaining multiple model slices, it is also necessary to record the structure and association relationship between each model slice (for example, record the slice of the previous layer of each model slice), so that when the slices are used later, the flow direction of the data processed by each slice can be determined according to the structure and association relationship between each model slice. In essence, the above process is recording the model structure.

[0072] Step S204 , storing the multiple model slices in the layer structure order of the target model to generate a model slice file, wherein each model slice is stored in a model image layer.

[0073] Step S206 , generating a model image slice description file according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image.

[0074] Step S208: Generate an operating environment configuration file according to the operating environment and configuration of the target model.

[0075] Step S210, generating a target model image for the mirror according to the model slicing file, the model image slicing description file and the operating environment configuration.

[0076] After the build side completes the construction of the target model image, the run side can Figure 3 The illustrated embodiment uses slices in a target model image to process a target computing task.

[0077] Among them, the operation side and the construction side can be the same device or different devices, and the present disclosure does not limit this. Among them, the operation side can process the complete task corresponding to the target model image, and can also process the partial task corresponding to the target model image. For example, the target model image may correspond to a task that requires computational offloading, then the task corresponding to the operation side may be a collaborative task in the computational offloading process, and the task corresponding to the operation side may also be the main task corresponding to the offloading side in the computational offloading process, and the present disclosure does not limit this.

[0078] The construction side can be any electronic device with computing and processing capabilities, for example, Figure 1 The server or terminal device in the embodiment may also be composed of a server and a terminal device, and the present disclosure does not limit this.

[0079] Figure 3 The figure is a flowchart of a task processing method according to an exemplary embodiment.

[0080] The following embodiments may be executed by the operation side, which may be any electronic device with computing and processing capabilities, such as the above-mentioned Figure 1The server or terminal device in the embodiment may also be composed of a server and a terminal device, and the present disclosure does not limit this.

[0081] Reference Figure 3 The task processing method provided by the embodiment of the present disclosure may include the following steps.

[0082] Step S302, obtaining the target computing task.

[0083] The target computing task may be a complete computing task or a partial task in a complete task, and the present disclosure does not impose any limitation on this.

[0084] For example, the target model mirror can be used to process the target task to be processed, the target computing task can be a collaborative subtask of the target task to be processed (for example, a subtask executed by the unloading end, or a subtask executed by the collaborative end), or the target computing task can be the target task to be processed, and the present disclosure does not impose any restrictions on this.

[0085] Step S304, determine that the target computing task corresponds to the target model image, the target model image includes non-model files and model slice files, the non-model files include operating environment configuration files and model image slice description files, the model slice files include multiple model slices, the multiple model slices include target model slices, and the multiple model slices are obtained after slicing the target model.

[0086] Step S306, pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file.

[0087] In some embodiments, the non-model files and model slice files described above may be isolated in the image file system through landmark files.

[0088] In some embodiments, the non-model file may be located below the landmark file (of course, the non-model file may also be located below, to the left, etc. of the landmark file).

[0089] Then, pulling the operating environment configuration file and the model image slice description file from the non-model file may include: accessing the image file system that stores the target model image; determining the landmark file corresponding to the target model image in the image file system; and searching for the operating environment configuration file and the model image slice description file below (or above) the landmark file.

[0090] In some embodiments, starting a target container according to an operating environment configuration file and a model image slice description file may include: creating a model slice directory for each model slice of the target model according to the model image slice description file; determining the non-model file layer where the operating environment configuration file and the model image slice description file are located; and mounting the model slice directory and the non-model file layer as the rootfs of the target container, so as to start the target container according to the rootfs composed of the model slice directory and the non-model file layer.

[0091] Step S308, determining the target model slice for processing the target computing task.

[0092] Among them, the target model slice can be the above-mentioned multiple model slices, and the target model slice can also be a partial slice of the above-mentioned multiple model slices, which is not limited in the present disclosure.

[0093] Step S310, pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task.

[0094] In some embodiments, the model image slice description file includes the file offset of each model slice in the target model image; then pulling the target model slice from the target model image may include: determining the target model image layer where the target model slice is located; determining the target file offset of the target model image layer in the model slice file based on the file offset of each model slice in the target model image; and pulling the target model slice from the target model image based on the target file offset.

[0095] In some embodiments, when pulling the target model slice, all model slices can be pulled at one time, or each model slice can be pulled in batches. For example, assuming that the above target model slice includes two model slices, the two model slices can be pulled at one time, or one model slice can be pulled first, and then another model slice can be pulled while the previous model slice is running, so that the other model slice can be run after the previous model slice is completed. Through this method, it can be ensured that all target model slices can be pulled and run, and the storage resources required by the device for storing image slices can be reduced.

[0096] The technical solution provided by the embodiment of the present disclosure, on the one hand, independently deploys the startup of the container and the operation of the model, saving storage resources and startup resources; on the other hand, pulls the target model image in sequence, so that it can be pulled while running, thereby reducing the occupancy of storage resources.

[0097] Figure 4 The figure is a flow chart of a slice pulling method according to an exemplary embodiment.

[0098] In some embodiments, the target model slice may include a first model slice and a second model slice, and the model image slice description file includes structural description information that describes the structure of the target model.

[0099] refer to Figure 4 The above-mentioned slice pulling method may include the following steps.

[0100] Step S402: Determine, according to the structure description information, that in the target computing task, the execution order of the first model slice is before the second model slice.

[0101] Step S404: pull and load the first model slice from the target model image, and run the first model slice in the started target container.

[0102] In some embodiments, the model image slice description file may further include location description information describing the location of each model slice in the target model image, wherein the location description information may include first location description information describing the location of the first model slice. Then, pulling and loading the first model slice from the target model image may include: pulling and loading the first model slice from the target model image according to the first location description information.

[0103] Step S406, while running the first model slice, pull and load the second model slice from the target model image to perform the target computing task.

[0104] In some embodiments, the above-mentioned location description information may include second location description information describing the location of the second model slice. Then, while running the first model slice, pulling and loading the second model slice from the target model image may include: while running the first model slice, pulling and loading the second model slice from the target model image according to the second location description information, so as to continue to run the second image slice after the first image slice is completed. In some embodiments, when the first image slice is completed, the first image slice can be directly deleted to save storage resources.

[0105] The above process can be achieved by Figure 5 The embodiment shown is specifically described. Figure 5As shown, the target computing task to be run by the operation side device is computing task A' in computing task A, and computing task A' corresponds to slices FC_1 and FC_2 in the target model image. Then, the operation side device can first pull the non-model files of the target model image to create a directory corresponding to the model slice and start the container; after the container is started, the operation side device can then read the model slice FC_1 for operation; while slice FC_1 is running, the operation side device can also pull slice FC_2 so that slice FC_2 can continue to run when slice FC_1 is finished.

[0106] For terminals with limited computing power, the reasoning (prediction) of AI models can be completed by computing offloading. In related technologies, in order to adapt to the dynamic offloading scheme of NN (Neural Network) computing tasks during device collaborative reasoning, it is usually necessary to store the complete NN model on all participating devices, which is a challenge for devices with limited storage resources, especially terminals. With the gradual popularization of containerized AI applications, the large-volume images caused by NN models will also become a bottleneck for image distribution and container deployment.

[0107] Therefore, the present application combines the above-mentioned model slicing scheme with the computing task offloading scheme, which can be specifically illustrated by the following embodiments.

[0108] Figure 6 A task segmentation processing method is shown according to an exemplary embodiment.

[0109] In some embodiments, the model image slice description file in the target model image may also include computational information corresponding to each model slice, and structural description information describing the structure of the target model.

[0110] In some embodiments, the target model image can be used to process the target to-be-processed task, wherein the target computing task can be a collaborative subtask of the target to-be-processed task (i.e., the target to-be-processed task is a task that needs to be split and unloaded, and the target computing task can refer to the split subtask).

[0111] In some embodiments, Figure 6 The task splitting method in the illustrated embodiment may be executed by the operation side, which may be the unloading end of the collaborative unloading task (ie, the end that needs to perform task splitting and unloading).

[0112] refer to Figure 6 The above-mentioned collaborative subtask processing method may include the following steps.

[0113] Step S602: The second device obtains the target task to be processed.

[0114] In some embodiments, the second device may be the operating side described above.

[0115] Step S604: determine the target model image for processing the target task to be processed.

[0116] The target task to be processed may be a complete task.

[0117] Step S606: determine the available computing capacity of at least one device to be offloaded for collaborative offloading of the target task to be processed.

[0118] Step S608, based on the available computing capacity of at least one device to be offloaded and the computing capacity corresponding to each model slice, the target task to be processed is split into multiple subtasks to be processed.

[0119] The at least one device to be unloaded may refer to a device that accepts the unloading task sent by the second device to cooperate with the second device to complete the target task to be processed.

[0120] The above-mentioned available computing capacity may refer to the computing capacity that the to-be-offloaded device can provide for the offloading task of the target to-be-processed task.

[0121] Step S610, determining the model slice corresponding to each to-be-processed subtask in the multiple model slices according to the structure description information, the multiple to-be-processed subtasks including the target computing task.

[0122] Step S612: The second device sends each subtask to be processed to at least one device to be uninstalled, so that at least one device to be uninstalled completes the corresponding subtask to be processed according to the model slice.

[0123] In some embodiments, the above-mentioned device to be offloaded may include any device that processes the target computing task.

[0124] After completing the task division, the second device will send each pending task to at least one device to be uninstalled and the second device itself, so that the second device and each device to be uninstalled can complete the task independently in parallel or in sequence. After receiving the task to be processed, each device to be unloaded can specifically perform the following steps (taking the target computing task as an example for explanation): obtain the target computing task; determine that the target computing task corresponds to the target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; pull the target model slice from the target model image; pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; determine that the target model slice is used to process the target computing task; pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task and obtain the target computing result; send the target computing result to the target aggregation device so that the target aggregation device completes the target task to be processed based on the target computing result.

[0125] In some embodiments, the second device may also send some intermediate data when sending a task; or, if there is an execution order between the tasks of each device to be uninstalled, then when a device to be uninstalled completes the uninstallation task, the intermediate processing result will be sent to the device corresponding to the next task.

[0126] Therefore, in some embodiments, the above-mentioned acquisition of the target computing task may include: acquiring the target computing task; acquiring the target input data corresponding to the target computing task, wherein the target input data is the intermediate computing result of the target task to be processed. Then, running the target model slice in the started target container to perform the target computing task, and obtaining the target computing result may include: processing the target input data by running the target model slice to obtain the target computing result.

[0127] A specific application scenario of the present application will be given below in conjunction with specific embodiments.

[0128] The technical solution provided by the present disclosure first requires that a target model image be created on the construction side (e.g., the first device) to separate the model slice files from the non-model files:

[0129] 1. Model slice file

[0130] like Figure 7As shown, the NN model (neural network model) can be automatically sliced ​​in units of neural network layers or neurons to generate multiple model slices, and the layer order of the NN model can be recorded while slicing. After obtaining multiple model slices, the model slices can also be stored in layers in the mirror file system in layer structure order.

[0131] like Figure 8 As shown, while storing different slices in layers, the slices can also be stored according to different data types. For example, the weight parameter information (i.e., model parameters) in the slice can be stored separately, and the structure file of the slice can be stored separately to facilitate updating and changing the weight parameters at any time.

[0132] 2. Non-model files

[0133] In some embodiments, Image Manifest, Image Config and other configurations can be used to generate an operating environment configuration file. A model image slice description file can also be generated based on the model slice image layer name and the NN layer structure order, and its identifier and descriptor are added in the image Manifest for addressing. The model image slice description file includes at least: the slice image layer name corresponding to each layer of the model, the file offset of each model slice image layer, the complete model structure represented by a directed acyclic graph, etc.; other additional information can also be written, such as the machine learning framework information applicable to the model, etc., to check the operating environment of the model container, such as the amount of calculation involved in each model slice, etc., in order to plan the model unloading plan.

[0134] 3. Landmark File

[0135] In the image file system, model slice image files can be separated from non-model files by using landmarks.

[0136] The construction side may be any device, and may be the same device as the operation side, or may be a different device, and the present disclosure does not impose any limitation on this.

[0137] After building the target model image, the task offloading end (such as the second device) can Fig. 9Complete the segmentation of the target task to be processed, for example, the second device obtains the target task to be processed; determines the target model image for processing the target task to be processed; determines the available computing capacity of at least one device to be unloaded for collaborative unloading of the target task to be processed; performs task segmentation on the target task to be processed according to the available computing capacity of at least one device to be unloaded and the computing capacity corresponding to each model slice to divide the target task to be processed into multiple subtasks to be processed; determines the model slice corresponding to each subtask to be processed in multiple model slices according to the structural description information, and the multiple subtasks to be processed include the target computing task; the second device sends each subtask to be processed to at least one device to be unloaded, so that at least one device to be unloaded completes the corresponding subtask to be processed according to the model slice.

[0138] After completing the task division, the operation side can Fig.10 The schematic diagram shown completes task unloading, and each end receiving the task (including the end that unloads the task and the end that receives the unloaded task) can complete the task according to the following steps.

[0139] Step 1: According to the NN model image specified by the computing task, read the image Manifest (a file name), Config (a file name), model image slice description file and other non-model files separated by landmarks from the image repository or other nodes in the computing cluster.

[0140] Step 2: Create a corresponding directory for each layer of model slice according to the model image slice description file, mount it together with the non-model file layer as the rootfs of the container, and start the container according to the rootfs composed of non-model files and model slice directories.

[0141] Step 3: The running side device determines the name of the required model slice image layer according to the current computing task, and determines the position of the image layer in the image according to the offset recorded in the model image slice description file, reads the model slice according to the layer order of the model structure, and loads the read slice into the corresponding directory of the container rootfs.

[0142] Step 4: When reading multiple layers of model slices, after one layer of slices is read and loaded, the container can start calculating the layer and pull other layers while calculating.

[0143] Currently, AI-related containers need to read the entire model image. The large image volume caused by the deep NN model is a challenge for devices with limited storage resources, especially terminals. It is also easy to cause network congestion and is a bottleneck for image distribution and container deployment.

[0144] This patented model slicing containerization solution innovatively embeds landmarks and model image slice description files that can be addressed by Manifest in the image to build an AI model image that can efficiently complete computing tasks.

[0145] By distinguishing model image layers from non-model image layers through landmark files, it is possible to support the separation of models from their configurations, operating environments, and applications, and to enable container startup based on non-model files and model slice directories on the container operation side, as well as the separation of the calculation and reading processes of model slices.

[0146] The image model image slice description file can realize the positioning and assembly of model slices, and can also provide support for task offloading solutions by adding additional information. In model calculation and calculation offloading scenarios, the specified model slices can be read according to the computing task requirements, effectively saving the operation side equipment resources and network resources.

[0147] Compared with existing technologies, the main advantages are:

[0148] 1. The model slice image is read on demand. In the model calculation offloading scenario, the participating computing nodes do not need to download the entire model, which can save node device and network resources, reduce the delay caused by reading the entire model image, and improve the execution efficiency of computing tasks. It is especially suitable for application on terminals.

[0149] 2. Supports the separation of the model from its configuration, operating environment, and application, which can improve the startup speed of the container and the efficiency of application deployment.

[0150] 3. Supports the separation of the calculation process and the reading process of model slicing. By running the two in parallel, the execution efficiency of computing tasks can be further improved in model calculation and computing offloading scenarios.

[0151] In addition, the weight files of each slice can be stored separately to achieve separate storage of model slice weight files, which can facilitate the update of model weight files on the device operation side. That is, after the model weights are updated, the operation side only needs to pull the new weight file of the model slice specified by the computing task.

[0152] In addition, through the separate storage of the above-mentioned model slice weight files, the separate training of the target model can also be achieved. For example, different model slices are deployed in different devices. During the model training process, only the intermediate training results and weight files need to be sent to each device to achieve the training of the target model.

[0153] The solution proposed in this patent aims to solve the above problems. When the running side (end / edge / cloud) can perform NN model slice calculation according to the current offloading solution, there is no need to download the complete NN model. It only needs to read the corresponding model slice image layer to complete the model calculation in the container. At the same time, when using multi-layer NN model calculation, the calculation can also be accelerated by loading and calculating the model layer in the container in parallel.

[0154] Fig.11 is a block diagram of a task processing device according to an exemplary embodiment. Fig.11 The task processing device 1100 provided in the embodiment of the present disclosure may include: a computing task acquisition module 1101, an image determination module 1102, a container startup module 1103, an image slice determination module 1104, and a slice pulling module 1105.

[0155] Among them, the computing task acquisition module 1101 can be used to acquire the target computing task; the image determination module 1102 can be used to determine that the target computing task corresponds to the target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; the container startup module 1103 can be used to pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; the image slice determination module 1104 can be used to determine that the target model slice is used to process the target computing task; the slice pulling module 1105 can be used to pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task.

[0156] In some embodiments, the target model slice includes a first model slice and a second model slice, and the model image slice description file includes structural description information that describes the structure of the target model; wherein the slice pulling module 1105 may include: an execution order determination submodule, a first model slice pulling submodule, and a second model slice pulling submodule.

[0157] Among them, the execution order determination submodule is used to determine that in the target computing task, the execution order of the first model slice is before the second model slice according to the structure description information; the first model slice pulling submodule is used to pull and load the first model slice from the target model image, and run the first model slice in the started target container; the second model slice pulling submodule is used to pull and load the second model slice from the target model image while running the first model slice to perform the target computing task.

[0158] In some embodiments, the model image slice description file also includes location description information describing the location of each model slice in the target model image, the location description information includes first location description information describing the location of the first model slice and second location description information describing the location of the second model slice; wherein the first model slice pulling submodule includes: a first pulling unit, and the second model slice pulling submodule includes: a second pulling unit.

[0159] Among them, the first pulling unit is used to pull and load the first model slice from the target model image according to the first location description information; the second pulling unit is used to pull and load the second model slice from the target model image according to the second location description information while running the first model slice.

[0160] In some embodiments, the target model image is used to process the target to-be-processed task, and the target computing task is a collaborative subtask of the target to-be-processed task; wherein, the target model slice is pulled from the target model image, and the target model slice is run in the started target container to perform the target computing task, including: running the target model slice in the started target container to perform the target computing task and obtain the target computing result; wherein the task processing device 1100 is also used to send the target computing result to the target aggregation device, so that the target aggregation device completes the target to-be-processed task based on the target computing result.

[0161] In some embodiments, obtaining a target computing task may include: obtaining a target computing task; obtaining target input data corresponding to the target computing task, wherein the target input data is an intermediate computing result of the target task to be processed; wherein, running a target model slice in the started target container to perform the target computing task and obtaining a target computing result includes: processing the target input data by running the target model slice to obtain the target computing result.

[0162] In some embodiments, the target model image is obtained after the first device performs image processing on the target model, and the target model is a neural network model; wherein the task processing device 1100 includes a slicing processing module, a model slicing file generation module, a slicing description file generation module, an operating environment configuration file generation module, and a target model image generation module.

[0163] Among them, the slicing processing module is used to slice the target model in units of neural network layers or neurons before obtaining the target computing task to obtain multiple model slices; the model slice file generation module is used to store multiple model slices in the layer structure order of the target model to generate a model slice file, wherein each model slice is stored in a model image layer respectively; the slice description file generation module is used to generate a model image slice description file according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image; the operating environment configuration file generation module is used to generate an operating environment configuration file according to the operating environment and configuration of the target model; the target model image generation module is used to generate a target model image according to the model slice file, the model image slice description file and the operating environment configuration.

[0164] In some embodiments, the model image slice description file in the target model image also includes the computing amount information corresponding to each model slice, and the structural description information describing the structure of the target model; wherein, the target model image is used to process the target task to be processed, and the target computing task is a collaborative subtask of the target task to be processed; wherein, the task processing device 1100 may include: a target task to be processed acquisition module, a target model image determination module, an available computing amount determination module, a task segmentation module, a slice matching module, and a task sending module.

[0165] Among them, the target task to be processed acquisition module is used to acquire the target task to be processed through the second device before acquiring the target computing task; the target model image determination module is used to determine the target model image for processing the target task to be processed; the available computing amount determination module is used to determine the available computing amount of at least one device to be unloaded for collaborative offloading processing of the target task to be processed; the task segmentation module is used to perform task segmentation on the target task to be processed according to the computing amount corresponding to each model slice of the available computing amount of at least one device to be unloaded, so as to divide the target task to be processed into multiple sub-tasks to be processed; the slice matching module is used to determine the model slice corresponding to each sub-task to be processed in multiple model slices according to the structural description information, and the multiple sub-tasks to be processed include the target computing task; the task sending module is used for the second device to send each sub-task to be processed to at least one device to be unloaded, so that at least one device to be unloaded completes the corresponding sub-task to be processed according to the model slice.

[0166] In some embodiments, non-model files and model slice files are segmented in the image file system through a landmark file, and the non-model files are located below the landmark file; wherein, the container startup module 1103 may include: an image file system access submodule, a segmented file positioning submodule, and a search submodule.

[0167] Among them, the image file system access submodule is used to access the image file system that stores the target model image; the split file positioning submodule is used to determine the landmark file corresponding to the target model image in the image file system; the search submodule is used to search for the operating environment configuration file and the model image slice description file below the landmark file.

[0168] In some embodiments, the container startup module 1103 may include: a directory creation submodule, a non-model file layer location submodule, and a mounting submodule.

[0169] Among them, the directory creation submodule is used to create a model slice directory for each model slice of the target model according to the model image slice description file; the non-model file layer positioning submodule is used to determine the non-model file layer where the operating environment configuration file and the model image slice description file are located; the mounting submodule is used to mount the model slice directory and the non-model file layer as the rootfs of the target container, so as to start the target container according to the rootfs composed of the model slice directory and the non-model file layer.

[0170] In some embodiments, the model image slice description file includes the file offset of each model slice in the target model image; wherein the slice pulling module 1105 may include: a slice layer determination submodule, a file offset determination submodule, and a pulling submodule.

[0171] Among them, the slice layer determination submodule is used to determine the target model image layer where the target model slice is located; the file offset determination submodule is used to determine the target file offset of the target model image layer in the model slice file according to the file offset of each model slice in the target model image; the pulling submodule is used to pull the target model slice from the target model image according to the target file offset.

[0172] Since the functions of the device 1100 have been described in detail in the corresponding method embodiments, the present disclosure will not elaborate on them here.

[0173] The modules and / or submodules involved in the embodiments described in the present application may be implemented by software or hardware. The modules and / or submodules described may also be arranged in a processor. The names of these modules and / or submodules do not, in certain cases, constitute limitations on the modules and / or submodules themselves.

[0174] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0175] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0176] Fig.12 The structure diagram of the electronic device suitable for implementing the embodiment of the present disclosure is shown. It should be noted that: Fig.12 The electronic device 1200 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0177] like Fig.12 As shown, the electronic device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0178] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0179] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1209, and / or installed from a removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the present application are executed.

[0180] It should be noted that the computer-readable storage medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0181] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by a device, the device can realize functions including: obtaining a target computing task; determining that the target computing task corresponds to a target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes multiple model slices, the multiple model slices include a target model slice, and the multiple model slices are obtained after slicing the target model; pulling the operating environment configuration file and the model image slice description file from the non-model file, and starting the target container according to the operating environment configuration file and the model image slice description file; determining that the target model slice is used to process the target computing task; pulling the target model slice from the target model image, and running the target model slice in the started target container to perform the target computing task.

[0182] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above-mentioned embodiments.

[0183] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for a computing device (which can be a personal computer, a server, a mobile terminal, or a smart device, etc.) to execute the method according to the embodiment of the present disclosure, for example Figure 3 Steps shown.

[0184] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not applied for by the present disclosure. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0185] It should be understood that the present disclosure is not limited to the detailed structures, drawings or implementations shown herein, but rather the present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A task processing method, characterized in that: include: Get the target computing task; Determining that the target computing task corresponds to a target model image, the target model image includes a non-model file and a model slice file, the non-model file includes an operating environment configuration file and a model image slice description file, the model slice file includes a plurality of model slices, the plurality of model slices include a target model slice, and the plurality of model slices are obtained after slicing the target model; Pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; wherein the model image slice description file is generated according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image; Determining that the target model slice is used to process the target computing task; Pulling the target model slice from the target model image, and running the target model slice in the started target container to perform the target computing task; Wherein, starting the target container according to the operating environment configuration file and the model image slice description file includes: Creating a model slice directory for each model slice of the target model according to the model image slice description file; Determine the non-model file layer where the operating environment configuration file and the model image slice description file are located; The model slice directory and the non-model file layer are mounted as the rootfs of the target container, so as to start the target container according to the rootfs composed of the model slice directory and the non-model file layer.

2. The method according to claim 1, characterized in that: The target model slice includes a first model slice and a second model slice, and the model image slice description file includes structure description information describing the structure of the target model; wherein the target model slice is pulled from the target model image, and the target model slice is run in the started target container to perform the target computing task, including: Determining, according to the structure description information, that in the target computing task, the execution order of the first model slice is before the execution order of the second model slice; Pull and load the first model slice from the target model image, and run the first model slice in the started target container; While running the first model slice, the second model slice is pulled and loaded from the target model image to perform the target computing task.

3. The method according to claim 2, characterized in that: The model image slice description file also includes position description information describing the position of each model slice in the target model image, and the position description information includes first position description information describing the position of the first model slice and second position description information describing the position of the second model slice; wherein pulling and loading the first model slice from the target model image includes: Pulling and loading the first model slice from the target model image according to the first location description information; Wherein, while running the first model slice, pulling and loading the second model slice from the target model image includes: While running the first model slice, the second model slice is pulled and loaded from the target model image according to the second location description information.

4. The method according to claim 1, characterized in that: The target model image is used to process the target task to be processed, and the target computing task is a collaborative subtask of the target task to be processed; wherein, pulling the target model slice from the target model image and running the target model slice in the started target container to perform the target computing task includes: Running the target model slice in the started target container to perform the target computing task and obtain a target computing result; Wherein, the method further comprises: The target calculation result is sent to a target aggregation device so that the target aggregation device completes the target to-be-processed task based on the target calculation result.

5. The method according to claim 4, characterized in that: Get the target computing task, including: Obtaining the target computing task; Acquire target input data corresponding to the target computing task, wherein the target input data is an intermediate computing result of the target task to be processed; The target model slice is run in the started target container to perform the target computing task and obtain the target computing result, including: The target input data is processed by running the target model slice to obtain the target calculation result.

6. The method according to claim 1, characterized in that: The target model image is obtained after the first device performs image processing on the target model, and the target model is a neural network model; Before acquiring the target computing task, the method further includes: Slice the target model in units of neural network layers or neurons to obtain the multiple model slices, wherein one model slice includes one or more neural network layers, or one model slice includes one or more neurons; Storing the plurality of model slices in the order of the layer structure of the target model to generate the model slice file, wherein each model slice is stored in a model image layer respectively; Generate the model image slice description file according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image; Generate the operating environment configuration file according to the operating environment and configuration of the target model; The target model image is generated according to the model slicing file, the model image slicing description file and the operating environment configuration.

7. The method according to claim 1, characterized in that: The model image slice description file in the target model image also includes the calculation amount information corresponding to each model slice and the structure description information describing the structure of the target model; wherein the target model image is used to process the target task to be processed, and the target calculation task is a collaborative subtask of the target task to be processed; wherein before obtaining the target calculation task, the method further includes: The second device obtains the target task to be processed; Determine that the target model image is used to process the target task to be processed; Determine the available computing capacity of at least one device to be offloaded for collaborative offloading of the target task to be processed; According to the available computing capacity of the at least one device to be offloaded and the computing capacity corresponding to each model slice, the target task to be processed is split into multiple subtasks to be processed; Determine, according to the structure description information, a model slice corresponding to each to-be-processed subtask in the multiple model slices, wherein the multiple to-be-processed subtasks include the target computing task; The second device sends each subtask to be processed to the at least one device to be unloaded, so that the at least one device to be unloaded completes the corresponding subtask to be processed according to the model slice.

8. The method according to claim 1, characterized in that: The non-model files and the model slice files are isolated in the image file system through landmark files.

9. The method according to claim 8, characterized in that: The model image slice description file includes the file offset of each model slice in the target model image; wherein pulling the target model slice from the target model image includes: Determine the target model image layer where the target model slice is located; Determine the target file offset of the target model image layer in the target model image according to the file offset of each model slice in the target model image; The target model slice is pulled from the target model image according to the target file offset.

10. A task processing device, characterized in that: include: A computing task acquisition module is used to acquire target computing tasks; An image determination module is used to determine that the target computing task corresponds to a target model image, wherein the target model image includes a non-model file and a model slice file, wherein the non-model file includes an operating environment configuration file and a model image slice description file, wherein the model slice file includes a plurality of model slices, wherein the plurality of model slices includes a target model slice, and wherein the plurality of model slices are obtained after slicing the target model; A container startup module, used to pull the operating environment configuration file and the model image slice description file from the non-model file, and start the target container according to the operating environment configuration file and the model image slice description file; wherein the model image slice description file is generated according to the structure description information describing the structure of the target model and the position description information describing the position of each model slice in the target model image; A mirror slice determination module, used to determine the target model slice for processing the target computing task; A slice pulling module, used to pull the target model slice from the target model image, and run the target model slice in the started target container to perform the target computing task; Wherein, starting the target container according to the operating environment configuration file and the model image slice description file includes: Creating a model slice directory for each model slice of the target model according to the model image slice description file; Determine the non-model file layer where the operating environment configuration file and the model image slice description file are located; The model slice directory and the non-model file layer are mounted as the rootfs of the target container, so as to start the target container according to the rootfs composed of the model slice directory and the non-model file layer.

11. An electronic device, characterized in that: include: Memory; and a processor coupled to the memory, wherein the processor is configured to execute the task processing method according to any one of claims 1 to 9 based on instructions stored in the memory.

12. A computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, the task processing method according to any one of claims 1 to 9 is implemented.

Images