Intelligent application migration method and system oriented to edge heterogeneous system

Abstract

The embodiment of the invention provides an intelligent application migration method and system oriented to an edge heterogeneous system, and the method comprises the steps: carrying out the quantification processing of an original model in response to a deployment request of an intelligent application, and carrying out the distillation unit division of the models before and after the quantification processing; taking the original model as a teacher model, and performing supervised training on the quantized student model until all distillation units complete distillation; splitting the intelligent application cascade relation into configuration components, and determining a configuration file according to the topological relation and internal parameters of the configuration components; and calling the configuration component to construct an intelligent application processing assembly line oriented to the edge heterogeneous system, and initializing computing resources of the edge heterogeneous system.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent application migration method and system for edge heterogeneous systems. Background Art

[0002] With the development of artificial intelligence (AI) technology, deep learning-based solutions are widely used in fields such as computer vision, natural language processing, and intelligent gaming. As application requirements and business scenarios continue to expand, the demand for deploying intelligent applications in resource-constrained edge environments is also growing.

[0003] In the existing technology, there are two major problems when migrating intelligent applications to edge heterogeneous systems. On the one hand, there is a significant contradiction between the resource requirements for model deployment and the limited resources under edge deployment conditions. Models often need to be lightweight before they can be deployed on the edge. Traditional lightweight methods include quantization and distillation, which face problems such as severe accuracy degradation, long training cycles, and even difficulty in convergence. On the other hand, the heterogeneous system adopts a master-slave architecture system. The master device is the CPU, which is responsible for task allocation, resource scheduling, monitoring and coordination; the slave device is usually one or more combinations of GPUs, FPGAs, NPUs and other coprocessors, responsible for performing specific computing tasks. The scheduling implementation of heterogeneous computing systems is complex, resource utilization is low, the processing flow of intelligent applications is frequently updated and iterated, and the migration and adaptation cycle is long. Summary of the Invention

[0004] In response to the problems existing in the prior art, embodiments of the present invention provide a method and system for intelligent application migration for edge heterogeneous systems.

[0005] An embodiment of the present invention provides a method for intelligent application migration for edge heterogeneous systems, the method comprising: In response to deployment requests from intelligent applications, the original model is quantized and the models before and after quantization are divided into distillation units. Using the original model as the teacher model, the quantized student model is supervised and trained until all distillation units have completed distillation; In combination with the processing unit characteristics of the edge heterogeneous system, the intelligent application cascade relationship is split into configuration components, and the configuration file is determined with reference to the topological relationship and internal parameters of the configuration components; The configuration component is called to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

[0006] In one embodiment, the method further comprises: Quantizing the original model as a teacher model to obtain a student model, and dividing several cascade layers of the teacher model and the student model into a distillation unit; The distillation unit output of the teacher unit is used as a soft label, the output loss of the distillation unit of the teacher model and the student model is calculated, and the student model is supervised and trained until all distillation units complete distillation.

[0007] In one embodiment, the method further comprises: Decoupling the intelligent application processing flow components and constructing a topological relationship of the configuration components through a cascade relationship; The configuration file of the intelligent application is written with reference to the topological relationship and internal parameters of the configuration components, and the data cache size and the number of parallel processing threads of the configuration components are dynamically adjusted by adjusting the file parameters of the configuration file.

[0008] In one embodiment, the method further comprises: The configuration file describes the topological relationship and internal parameters in a field manner, and specifies the downstream components through the description fields; The configuration file describes the number of parallel processing threads and data distribution mode of the configuration component in a field manner, wherein the data distribution includes polling transmission and multi-channel parallel transmission.

[0009] In one embodiment, the method further comprises: Call the multi-component parallel stream processing framework to provide component registration and component implementation development interfaces, and complete component function development based on component function requirements; After the component function development is completed, call the initialization interface in the multi-component parallel stream processing framework, specify the configuration file path, and complete the registration and topological association of various configuration components.

[0010] In one embodiment, the method further comprises: After receiving the processed data, the processed data is transferred between the configuration components through the pipeline, wherein the transfer process transfers data through a depth-controllable secure data queue, and each configuration component executes and transfers data in parallel. An embodiment of the present invention provides an intelligent application migration system for edge heterogeneous systems, the system comprising: A quantization module is used to quantize the original model in response to deployment requests from intelligent applications, and to divide the models before and after quantization into distillation units; A distillation module is used to supervise the training of the quantized student model using the original model as a teacher model until all distillation units have completed distillation; A configuration module, configured to split the intelligent application cascade relationship into configuration components based on the processing unit characteristics of the edge heterogeneous system, and determine a configuration file by referring to the topological relationship and internal parameters of the configuration components; The pipeline module is used to call the configuration component to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

[0011] In one embodiment, the system further comprises: a partitioning module, configured to quantize the original model as a teacher model to obtain a student model, and to partition several cascaded layers of the teacher model and the student model into a distillation unit; The training module is used to use the distillation unit output of the teacher unit as a soft label, calculate the output loss of the distillation unit of the teacher model and the student model, and perform supervised training on the student model until all distillation units complete distillation. An embodiment of the present invention provides an electronic device, including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method described in one or more embodiments.

[0012] An embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned intelligent application migration method for edge heterogeneous systems are implemented.

[0013] In view of the above, in one or more embodiments of the present specification, in response to a deployment request of an intelligent application, the original model is quantized, and the models before and after quantization are divided into distillation units; the original model is used as a teacher model to supervise the training of the quantized student model until all distillation units have completed distillation; the intelligent application cascade relationship is split into configuration components, and the configuration file is determined with reference to the topological relationship and internal parameters of the configuration components; the configuration component is called to build an intelligent application processing pipeline for edge heterogeneous systems, and the computing resources of the edge heterogeneous systems are initialized. In this way, on the one hand, the model can be lightweight processed by combining quantization and distillation, reducing resource requirements while taking into account accuracy; on the other hand, a multi-component parallel stream processing framework for heterogeneous systems is proposed and designed, which improves resource utilization and processing performance through multi-dimensional, fine-grained parallel design and scheduling optimization; further, the development process can be greatly simplified and development efficiency can be improved by editing configuration files and providing template processing classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 This is a flowchart of an intelligent application migration method for edge heterogeneous systems provided by an embodiment of this specification.

[0016] Figure 2 This is a network topology diagram provided by an embodiment of this specification.

[0017] Figure 3 This is a schematic diagram of multi-threaded processing of a serial application provided by an embodiment of this specification.

[0018] Figure 4 This is a schematic diagram of an inference application framework encapsulated by a multi-stage pipeline mechanism provided in an embodiment of this specification.

[0019] Figure 5 This is a schematic diagram comparing a multi-stage pipeline mechanism and serial processing provided by an embodiment of this specification.

[0020] Figure 6 This is a structural diagram of an intelligent application migration system for edge heterogeneous systems provided by an embodiment of this specification.

[0021] Figure 7 This is a structural diagram of an electronic device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0022] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope of protection, applicability, or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of this specification. Various examples may omit, replace, or add various processes or components as needed. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted, or combined. In addition, features described relative to some examples may also be combined in other examples.

[0023] As used herein, the term "includes" and its variants are meant to be an open term, meaning the inclusion of but not limited to. The term "based on" means "based, at least in part, on." The terms "one embodiment" and "an embodiment" mean "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "a first," "a second," etc. do not require that there be only one of each, but rather that there are at least one of each. The following detailed description can include other definitions, either explicitly or implicitly. Unless specifically stated otherwise, as apparent from context, terms defined herein shall be understood to be consistent with definitions in the field at the time of the embodiments.

[0024] As Figure 1 illustrated, the embodiment of the present application provides an intelligent application migration method for edge heterogeneous systems, comprising: Step S102, in response to the deployment request of the intelligent application, quantizing the original model, and dividing the model before and after quantization into distillation units.

[0025] Specifically, when receiving the migration request of the intelligent application, i.e. migrating the intelligent application from the cloud / data center system to the edge heterogeneous system, the request of deploying the intelligent application to the edge heterogeneous system. In response to the deployment request of the intelligent application to the edge heterogeneous system, the deep learning model of the intelligent application is lightened, wherein the step of lightening includes quantizing the original model. The neural network model before quantization is the teacher model, which generally uses 32-bit floating point type. The quantization process can be low-bit (such as INT8) processing of floating point model weight and data, so as to save model storage space, improve calculation efficiency, realize model performance improvement and optimization. The quantization after training has relatively poor quantization accuracy because the numerical range of the data cannot be accurately obtained, and the quantization-aware training is more dependent on the label in the training data to achieve relatively small quantization error. The model after quantization is the student model.

[0026] Further, because the student model is quantized from the teacher model and has the same module structure as the teacher model, when the model before and after quantization (teacher model and student model) is divided into distillation units, the teacher model and the student model are one-to-one corresponding to the distillation unit module structure.

[0027] Step S104, taking the original model as the teacher model, supervising and training the student model after quantization until all distillation units are completed.

[0028] Specifically, the output of each unit in the teacher model is used as a soft label, and the corresponding unit of the student model is progressively fine-tuned for supervised training. The supervised training process includes: using soft labels to represent the probability distribution of each unit in the model, that is, the probability that the sample belongs to each category, and using the output of the teacher model through forward propagation as the training target (soft label) to supervise the student model. During the supervised training process, the output value loss of the teacher model and the student model is calculated and gradient updated after each distillation unit to obtain a more accurate student model. The training steps are then repeated until all distillation units are completely distilled. In the distillation process, the control starts from the lowest layer unit (such as Conv1) and distills upward step by step to the highest layer (Layer4). Because the underlying features (such as edge texture) are the basis of high-level semantics, prioritizing repair can improve convergence efficiency.

[0029] Furthermore, the distillation process for a single unit includes the following data. The soft label is defined as the feature map output by Layer 1 of the teacher model (not the final classification result). The loss function can be: Loss = 0.7*||F_teacher - F_student||2 (feature map L2 distance) or 0.3*KL(teacher output probability||student output probability) (output distribution similarity). In the iterative control of supervised training, after every three epochs, if the student model output accuracy improves by less than 0.1%, the distillation of that unit can be stopped. Then, after the distillation of that unit is completed, its weight is fixed and distillation of the next unit is carried out.

[0030] Step S106 : splitting the intelligent application cascade relationship into configuration components, and determining a configuration file by referring to the topological relationship and internal parameters of the configuration components.

[0031] Specifically, after completing the lightweight processing of the neural network model, the configuration components are split and decoupled based on the master-slave system characteristics of the edge heterogeneous system and the cascade relationship of the intelligent application, and the corresponding network topology structure representation processing process is formed. Among them, the network topology includes linear and nonlinear, covering the application requirements of serial processing, parallel processing, etc. in typical intelligent processing applications. The network topology is as follows: Figure 2 As shown, for the network topology, the first processing node is designated as the source component, that is, the data processing component, which realizes the input access function.

[0032] Furthermore, intelligent application configuration files are compiled based on the network topology and the internal parameters of each configuration component (including internal parallelism, queue depth, and mapping method). By adjusting the configuration file parameters, data cache size and the number of parallel processing threads of the component can be dynamically adjusted to adapt to the data throughput requirements of different heterogeneous edge systems, reducing repetitive development work and accelerating the efficiency of intelligent application migration.

[0033] The intelligent application configuration file, formatted in JSON, describes the component topology and parameters using fields. Each component name is customizable, and the "next_modules" field within the component description specifies its downstream components. This field is in array format and supports multiple component names to implement parallel topologies. Ultimately, the intelligent application's network topology is described in a JSON configuration file.

[0034] Configure the internal parallelism of the component. Each component in the configuration file provides a "parallelism" field. Smart applications use this field to apply for the number of component processing threads, distribute data, and maintain it. This eliminates the need for application development to perceive the field, improving the efficiency of smart application migration. Figure 3 Typical serial application multithreading shown.

[0035] Furthermore, data distribution includes two types: polling sending and multi-channel parallel sending. Among them, polling sending means that when the data source sends data, the frame data is cyclically sent to multiple parallel processing threads of the downstream component for parallel processing. It is mostly suitable for application scenarios where there is no time sequence correlation between the processing of the previous and next frames of the data. In this embodiment, the input 1080P resolution image is cut into two small images with a resolution of 1080*1080. The fusion of the inference results of the two small frames has no requirements for time sequence, so the data distribution adopts polling sending. Multi-channel parallel sending means that the data source accepts multiple channels of data, and each channel of data is sent to the downstream component for parallel processing at the same time. It is mostly suitable for scenarios where multiple channels of data are processed simultaneously. In this embodiment, the data source inputs 4 channels of monitoring video streams and sends data in a multi-channel parallel manner. The processing of each channel of data does not interfere with each other, and data parallel processing is achieved at the same time.

[0036] Step S108: Call the configuration component to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

[0037] Specifically, the multi-component parallel stream processing framework provides a development interface for component registration and implementation, and component functional development is completed based on component functional requirements. The deep learning model inference component is divided into three parts: preprocessing, model inference, and post-processing. Model inference runs on the GPU or NPU, while preprocessing and post-processing are mostly run on the CPU, using the NEON instruction set and OpenMP for multi-core parallel acceleration.

[0038] Furthermore, the multi-component parallel stream processing framework is an inference application framework encapsulated based on a component-orchestrated multi-stage pipeline mechanism. This framework is based on the inference framework usage paradigm of pipeline and event bus mechanisms, utilizing a hybrid development approach of the factory and bridge patterns, introducing a variety of efficient and secure mechanisms, and constructing a multi-stage processing model based on the concept of isolating computational description and resource scheduling. This multi-component parallel stream processing framework performs thread pool maintenance, thread-safe queue maintenance between components, message bus monitoring, data distribution, configuration file parsing, and pipeline construction. Through thread resource allocation and asynchronous callbacks, it achieves multi-dimensional, fine-grained parallel processing and scheduling optimization, improving system resource utilization.

[0039] By deriving from base class templates, business-based functional components can be implemented, giving developers great development flexibility to meet various business needs. The framework has features such as multi-channel concurrency, adaptive scheduling, and heterogeneous load balancing. It is compatible with linear and nonlinear topologies of component structures and supports building different pipelines by modifying configuration files without recompiling source code, flexibly meeting the performance and scalability requirements of various businesses. This stream processing framework can effectively improve the software's data throughput performance, resource utilization, reusability, and robustness. Framework Logic Description Figure 4 shown.

[0040] The deep learning model inference component is the core function in the migration of intelligent applications, including data preprocessing, model inference, and post-processing. The AI ​​processor supports hardware acceleration for image processing, but has relatively more restrictions on images. Atypical data and post-processing still need to be processed on the CPU. Model inference runs on the NPU or GPU of the heterogeneous system, and model inference acceleration is achieved through the AI ​​processor.

[0041] In this embodiment, in the typical application field of image target recognition, data preprocessing mainly includes processing steps such as image color gamut conversion, image scaling, and image normalization, and the image processing is accelerated in parallel through the NEON instruction set; for concurrent operation steps such as target sorting in the target recognition model NMS post-processing, OpenMP is used for concurrent acceleration processing to reduce module processing delay and improve intelligent application processing efficiency.

[0042] In this embodiment, the deep learning model inference component can be split into a preprocessing component, a model inference component, and a post-processing component. The three components are connected in series, and the processing delay between components is based on a multi-stage pipeline mechanism to hide part of the delay, thereby improving data throughput.

[0043] Further, in combination with the configuration file construction processing pipeline, the parallel processing environment is initialized, that is, the initialization interface in the multi-component parallel stream processing framework is called in the main process, the path of the configuration file is specified, the registration and topology association of each component are completed, and the directed acyclic graph of the specific application business is formed. Among them, the initialization interface is an initialization function provided in the multi-component parallel stream processing framework, which parses the generated configuration file, registers the components to the processing pipeline according to the parsed parameters, and configures the number of thread parallelism, data transmission direction between components, and queue depth. The deep learning model inference component is distributed to the edge heterogeneous system for model inference from the processor (NPU / GPU), and the model loading to the slave processor and the model input / output memory application are completed in the initialization stage. Finally, the mapping of the JSON configuration file to the intelligent application processing pipeline is completed.

[0044] Further, after the multi-component parallel stream processing framework of the heterogeneous system is completed, the heterogeneous system receives data, and the data is processed by the components in the pipeline constructed according to the configuration file. The data is transmitted between the components through the implicitly deep controllable secure data queue, and each component is executed in parallel and transmits data, forming a "pipeline" type data processing mode, which can hide the component processing delay and improve the data throughput, so as to achieve the effect of data processing acceleration. Among them, the deep controllable secure data queue is a connector for data transmission between components. The first-in-first-out mechanism of the queue maintains the data timing. Each component can connect multiple input connectors and multiple output connectors. Two components using the same connector as an input connector and an output connector can be automatically connected together. The data between the components is transmitted according to the distribution mechanism through the connector. Different connection modes of the connector can realize various network topologies. Finally, the data transmission between the components and the component execution are realized in a data-driven manner.

[0045] In addition, the queue depth of the secure data queue can be obtained from the configuration file. The data between the components uses a unified interface and data structure. The current component only needs to focus on receiving the data information of the unified data structure issued by the previous component and performing corresponding processing. The upper framework is responsible for the transmission of data to the next component. From the design level, the generality and ease of development of the component are required.

[0046] The multi-stage pipeline processing mechanism shields the calling relationship between components through the queue. Each component is driven by data flow and only needs to process the data information flowing into the component, realizes multi-component parallel processing, fully utilizes system resources, and improves data throughput. Figure 5 As shown in the multi-stage pipeline processing mechanism ( Figure 5 below the coordinate axis), compared with the serial processing ( Figure 5 above the coordinate axis), the relative time delay of the result output is shortened, the data throughput is obviously increased, and the processing speed of the intelligent application is improved.

[0047] An embodiment of the present invention provides a method for migrating intelligent applications for edge heterogeneous systems. In response to an intelligent application deployment request, the original model is quantized and the models before and after quantization are divided into distillation units. The original model is used as a teacher model to supervise the training of the quantized student model until all distillation units have completed distillation. The intelligent application cascade relationship is split into configuration components, and the configuration file is determined by referring to the topological relationship and internal parameters of the configuration components. The configuration components are called to build an intelligent application processing pipeline for edge heterogeneous systems and initialize the computing resources of the edge heterogeneous systems. This method can, on the one hand, lightweight the model by combining quantization and distillation, reducing resource requirements while ensuring accuracy. On the other hand, it proposes and designs a multi-component parallel stream processing framework for heterogeneous systems, which improves resource utilization and processing performance through multi-dimensional, fine-grained parallel design and scheduling optimization. Furthermore, the development process can be greatly simplified and development efficiency can be improved by editing configuration files and providing template processing classes.

[0048] See Figure 6 , Figure 6 This is a schematic diagram of the structure of an intelligent application migration system for edge heterogeneous systems provided by an embodiment of the present application. Figure 6 As shown, the system includes: Quantization module S602, configured to quantize the original model in response to a deployment request of the intelligent application, and divide the model before and after quantization into distillation units; A distillation module S604 is configured to use the original model as a teacher model to perform supervised training on the quantized student model until all distillation units complete distillation; Configuration module S606, configured to split the intelligent application cascade relationship into configuration components based on the processing unit characteristics of the edge heterogeneous system, and determine a configuration file by referring to the topological relationship and internal parameters of the configuration components; The pipeline module S608 is used to use the configuration components to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

[0049] In another embodiment, an intelligent application migration system for edge heterogeneous systems further includes: a partitioning module, configured to quantize the original model as a teacher model to obtain a student model, and to partition several cascaded layers of the teacher model and the student model into a distillation unit; The training module is used to use the distillation unit output of the teacher unit as a soft label, calculate the output loss of the distillation unit of the teacher model and the student model, and perform supervised training on the student model until all distillation units complete distillation.

[0050] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented by means of software and / or hardware. The "unit" and "module" in the specification refer to software and / or hardware capable of independently completing or cooperating with other components to complete a specific function, wherein the hardware may, for example, be a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), and the like.

[0051] The various processing units and / or modules of the embodiments of the present application can be implemented by means of analog circuits that implement the functions described in the embodiments of the present application, or can be implemented by means of software that executes the functions described in the embodiments of the present application. Referring to Figure 7 , a structural schematic diagram of an electronic device related to the embodiments of the present application is shown, which can be used to implement the method in the embodiments shown in Figure 1 . As shown in Figure 7 , the electronic device 700 can include at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702.

[0052] The communication bus 702 is used to realize the connection and communication between the components.

[0053] The user interface 703 can include a display screen (Display) and a camera (Camera), and the optional user interface 703 can further include a standard wired interface and a wireless interface.

[0054] The network interface 704 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0055] The processor 701 may include one or more processing cores. The processor 701 utilizes various interfaces and circuits to connect various components within the electronic device 700. It executes instructions, programs, code sets, or instruction sets stored in the memory 705, and accesses data stored in the memory 705 to perform various functions and process data within the electronic device 700. Optionally, the processor 701 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 701 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processor (NPU). The CPU is primarily responsible for task allocation, resource scheduling, monitoring, and coordination, and performs some data preprocessing operations. The GPU and NPU are used for model inference and intelligent processing.

[0056] Among them, the memory 705 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 705 includes a non-transitory computer-readable storage medium. The memory 705 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 705 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 705 may also be optionally at least one storage device located away from the aforementioned processor 701. As Figure 7 As shown, the memory 705 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0057] exist Figure 7In the electronic device 700 shown, the user interface 703 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 701 can be used to call the image-generated interactive application stored in the memory 705, and perform the following specific operations: in response to the deployment request of the intelligent application, the original model is quantized, and the model before and after the quantization is divided into distillation units; the original model is used as the teacher model, and the quantized student model is supervised and trained until all distillation units are completed; the intelligent application cascade relationship is split into configuration components, and the configuration file is determined with reference to the topological relationship and internal parameters of the configuration components; the configuration component is called to build an intelligent application processing pipeline for the edge heterogeneous system, and the computing resources of the edge heterogeneous system are initialized. The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0058] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0059] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0061] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0062] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0063] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disk, etc., various media that can store program code.

[0064] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program. The program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0065] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for intelligent application migration for edge heterogeneous systems, the method comprising: In response to deployment requests from intelligent applications, the original model is quantized and the models before and after quantization are divided into distillation units. Using the original model as the teacher model, the quantized student model is supervised and trained until all distillation units have completed distillation; In combination with the processing unit characteristics of the edge heterogeneous system, the intelligent application cascade relationship is split into configuration components, and the configuration file is determined with reference to the topological relationship and internal parameters of the configuration components; The configuration component is called to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

2. The method according to claim 1, characterized in that The method further comprises: Quantizing the original model as a teacher model to obtain a student model, and dividing several cascade layers of the teacher model and the student model into a distillation unit; The distillation unit output of the teacher unit is used as a soft label, the output loss of the distillation unit of the teacher model and the student model is calculated, and the student model is supervised and trained until all distillation units complete the distillation.

3. The method according to claim 1, characterized in that The step of splitting the intelligent application cascade relationship into configuration components and determining the configuration file by referring to the topological relationship and internal parameters of the configuration components includes: Decoupling the intelligent application processing flow components and constructing a topological relationship of the configuration components through a cascade relationship; The configuration file of the intelligent application is written with reference to the topological relationship and internal parameters of the configuration components, and the data cache size and the number of parallel processing threads of the configuration components are dynamically adjusted by adjusting the file parameters of the configuration file.

4. The method according to claim 3, characterized in that The method further comprises: The configuration file describes the topological relationship and internal parameters in a field manner, and specifies the downstream components through the description fields; The configuration file describes the number of parallel processing threads and data distribution mode of the configuration component in a field manner, wherein the data distribution includes polling transmission and multi-channel parallel transmission.

5. The method according to claim 3, characterized in that The calling of the configuration component to construct an intelligent application processing pipeline for an edge heterogeneous system and initializing computing resources of the edge heterogeneous system includes: Call the multi-component parallel stream processing framework to provide component registration and component implementation development interfaces, and complete component function development based on component function requirements; After the component function development is completed, call the initialization interface in the multi-component parallel stream processing framework, specify the configuration file path, and complete the registration and topological association of various configuration components.

6. The method according to claim 1, characterized in that The method further comprises: After receiving the processed data, the processed data is transferred between the configuration components through the pipeline, wherein the transfer process transfers data through a depth-controllable secure data queue, and each configuration component executes and transfers data in parallel.

7. An intelligent application migration system for edge heterogeneous systems, characterized by: The system comprises: A quantization module is used to quantize the original model in response to deployment requests from intelligent applications, and to divide the models before and after quantization into distillation units; A distillation module is used to supervise the training of the quantized student model using the original model as a teacher model until all distillation units have completed distillation; A configuration module, configured to split the intelligent application cascade relationship into configuration components based on the processing unit characteristics of the edge heterogeneous system, and determine a configuration file by referring to the topological relationship and internal parameters of the configuration components; The pipeline module is used to call the configuration component to build an intelligent application processing pipeline for the edge heterogeneous system and initialize the computing resources of the edge heterogeneous system.

8. The system according to claim 7, characterized in that The system further comprises: a partitioning module, configured to quantize the original model as a teacher model to obtain a student model, and to partition several cascaded layers of the teacher model and the student model into a distillation unit; The training module is used to use the distillation unit output of the teacher unit as a soft label, calculate the output loss of the distillation unit of the teacher model and the student model, and perform supervised training on the student model until all distillation units complete distillation.

9. An electronic device comprising a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images