Unified supporting method for intelligent sensing algorithm training framework

Abstract

The invention relates to an intelligent perception algorithm training framework unified support method, and belongs to the field of deep learning. The method comprises a mainstream domestic framework conversion compatible technology; a unified training system; training a framework multi-hardware compatible technology; multi-type computing resource virtualization techniques. According to the method, a domestic mainstream training framework PaddlePadd is integrated, and the unified conversion compatibility of a PyTorch training framework in a training code layer and a model layer is realized through an X2Padd tool; training environment management is carried out by a computing resource virtualization technology, compatible support for different training frameworks is realized in a mirroring manner, and access of mainstream domestic hardware computing equipment is realized; algorithm resource integration is realized through model resource management and scene development suite construction, so that the rapid development capability of different task common scenes is realized; through a distributed training technology, scheduling use of single-machine multi-card and multi-machine multi-card resources is supported. Through hyper-parameter tuning and model compression technologies, the training model effect and the training efficiency are optimized.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and specifically relates to a unified support method for an intelligent perception algorithm training framework. Background Art

[0002] The realization of intelligent perception models mainly relies on deep learning frameworks. In the early stages of the development of deep learning frameworks, developers and users mainly focused on performance optimization. As the types of deep learning frameworks continue to increase, the problem of insufficient compatibility between multiple deep learning frameworks has gradually emerged, that is, models developed using a certain deep learning framework are difficult to port to another specified framework. In addition, there are differences in the actual engineering implementation between the model training framework and the framework supported by the deployment inference platform, but there is currently no unified deep learning framework that can fully adapt to domestic hardware. Furthermore, the problem of conversion accuracy loss of models under different deep learning frameworks is also one of the current major research hotspots.

[0003] Overseas, the United States has implemented a variety of deep learning frameworks in the research of heterogeneous intelligent technology, such as Caffe, PyTorch and Tensorflow, which support server-side applications such as target detection, personnel detection, intelligent monitoring, and drone solutions. Non-localized frameworks are mainly dominated by PyTorch and Tensorflow. Foreign mainstream intelligent framework manufacturers work closely with inference hardware manufacturers, and there is usually no separation between model training frameworks and inference deployment. In addition, foreign manufacturers have launched the open neural network exchange standard ONNX (OpenNeuralNetwork Exchange) to make model conversion between different deep learning frameworks easier. ONNX defines an intermediate representation format standard for deep learning models and provides a runtime library to support multi-platform operation of ONNX models, but the standard is currently still limited in promotion among domestic intelligent inference card manufacturers. In addition, ONNX does not natively support major deep learning frameworks. For example, Tensorflow itself does not support ONNX, and model conversion requires third-party software.

[0004] Domestically, the research on deep learning frameworks started late. Driven by the trend of localization and self-control, the industry and academia have launched self-developed deep learning frameworks, such as Baidu's PaddlePaddle, Huawei's MindSpore and Tsinghua University's Jittor. Among them, PaddlePaddle is a deep learning framework with a relatively complete domestic product system. It supports a variety of deep learning applications, provides a series of deep learning tools including model libraries, data processing tools and deployment tools, and as a distributed training framework, it can support high-performance heterogeneous distributed training acceleration. In response to the model conversion problem between different deep learning frameworks, Baidu proposed the x2Paddle model conversion tool to convert mainstream deep learning framework models (including caffe, tensorflow, and onnx models) into models under the PaddlePaddle framework, but if the model framework corresponding to the model does not belong to the above three, it cannot be uniformly converted. In short, the current domestic deep learning framework is in a state of "a hundred flowers blooming", lacking support for various downstream reasoning hardware and compatibility with non-domestic frameworks. The model training framework and the reasoning deployment process urgently need to be connected in engineering implementation. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide a unified support method for the intelligent perception algorithm training framework to solve the problems faced by the current intelligent perception model training, such as the inconsistency of the training framework, the risk of non-domestic training framework, and the poor compatibility of the domestic framework.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes a unified support method for an intelligent perception algorithm training framework, which includes:

[0009] Mainstream domestic framework conversion compatibility technology: Use X2Paddle conversion tool to quickly migrate users of other deep learning frameworks to the PaddlePaddle framework;

[0010] Unified training system: Training is based on the PaddlePaddle training framework, and is compatible with multiple other framework access methods;

[0011] Training framework multi-hardware compatibility technology: Provides the ability to connect to a wide variety of training and prediction hardware through a general programming interface and an intermediate representation layer IR interface;

[0012] Multi-type training resource virtualization technology: Docker containerization technology and k8s cluster management technology are used to realize the virtualization of multi-type training resources.

[0013] (III) Beneficial effects

[0014] The present invention proposes a unified support method for intelligent perception algorithm training framework. The present invention proposes a unified support method for intelligent perception algorithm training framework, focusing on solving the problems of unified training framework support, model knowledge integration management, sufficient scheduling of computing resources, model debugging optimization design, etc. throughout the entire training task process in intelligent applications, so as to build the rapid R&D and iteration capabilities of intelligent algorithms. The unified support method for training framework adopted by the present invention integrates the domestic mainstream training framework PaddlePaddle, and realizes the unified conversion compatibility of the PyTorch training framework at the training code layer and the model layer through the X2Paddle tool; and uses computing resource virtualization technology to manage the training environment, and realizes compatible support for different training frameworks in a mirroring manner, and realizes the access of mainstream domestic hardware computing devices. In terms of model construction, algorithm resource integration is realized through model resource management and construction of scenario development kits to realize the rapid development capabilities of different task scenarios. Through distributed training technology, the scheduling and use of single-machine multi-card and multi-machine multi-card resources are supported. Through hyperparameter tuning and model compression technology, the optimization of training model effect and training efficiency is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is the overall architecture diagram of the unified support method of the intelligent perception algorithm training framework of the present invention;

[0016] Figure 2 This is a logical structure diagram of the training system of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0018] The present invention relates to a unified support method for an intelligent perception algorithm training framework, and in particular to a unified support method for an intelligent perception algorithm training framework for domestic hardware computing devices.

[0019] The present invention proposes a unified support method for an intelligent perception algorithm training framework for domestic hardware computing devices, the method comprising:

[0020] (1) Mainstream domestic framework conversion compatibility technology: mainly using X2Paddle conversion tool to quickly migrate users of other deep learning frameworks to the PaddlePaddle framework;

[0021] (2) Unified training system: Training is based on the PaddlePaddle training framework and is compatible with multiple other framework access methods;

[0022] (3) The training framework has multi-hardware compatibility technology, which provides the ability to connect to a wide variety of training and prediction hardware through a general programming interface and an intermediate representation layer (IR) interface;

[0023] (4) Multi-type training resource virtualization technology. The present invention uses docker containerization technology and k8s cluster management technology to realize the virtualization of multi-type training resources.

[0024] In (1), the mainstream frameworks involved include PaddlePaddle, PyTorch and MindSpore. Baidu PaddlePaddle is the earliest domestic open source deep learning framework and the only fully functional end-to-end open source deep learning platform in China. It integrates deep learning training and prediction framework, model library, tool components and service platform. It has flexible and efficient development mechanism, industrial application effect model, ultra-large-scale parallel deep learning capability, inference engine integrated design and systematic service support. It aims to bridge the gap between AI algorithm research and production deployment, and has a relatively complete ecosystem and rich functions. In the algorithm research stage, it provides developers with a unified programming experience to improve the efficiency of algorithm development; in the production stage, automatic parallelization can greatly speed up the development and debugging efficiency of distributed training, while fully tapping the computing power of heterogeneous hardware; in the deployment stage, based on the "end-edge-cloud" unified architecture, it meets the needs of industrial-level deployment.

[0025] Compared with other domestic deep learning frameworks, PaddlePaddle has the highest popularity in the open source community and has obvious advantages in tool chain completeness and domestic acceleration hardware support. Table 1 shows the statistics of the activeness of the open source community of domestic frameworks.

[0026] Table 1 Statistics on the activity of domestic framework open source communities

[0027] Github ranking Domestic framework name References Activity Number of Contributors Attention 1 PaddlePaddle 4300 33753 524 17500 2 MindSpore 514 37308 267 2700 3 Jittor 235 1266 31 2300

[0028] In addition, when selecting the framework, the present invention investigates and demonstrates factors such as mainstream framework compatibility, model conversion support, hardware support, and the number of model support, and focuses on comparing three training frameworks: PaddlePaddle, MindSpore, and Jittor. In terms of framework compatibility, except for PaddlePaddle, which can provide conversion tools to complete the automatic conversion of PyTorch code, the remaining frameworks require manual code conversion. For models exported by mainstream frameworks, PaddlePaddle and MindSpore can achieve automatic conversion support for PyTorch and TensorFlow models, while the Jittor framework has no corresponding support. In terms of the breadth of training hardware support, the PaddlePaddle framework is the most complete and can achieve access to 5 mainstream devices. In terms of the number of models supported, PaddlePaddle also has a large advantage. The comparative data is shown in Table 2.

[0029] Table 2 Comparison of domestic open source platform support

[0030]

[0031] In order to convert as many mainstream training frameworks as possible into the Baidu Paddle framework at the code or model level. The Paddle ecosystem has developed the X2Paddle conversion tool to quickly migrate users of other deep learning frameworks to the Paddle framework, and provide conversion capabilities at both the code and model levels. First, at the code conversion level, in theory, any deep learning framework can complete the migration by aligning the operator development code, but the cost and difficulty here are relatively high. X2Paddle is used to automatically migrate and convert the PyTorch training project code with one click. Considering the huge audience of PyTorch, this support function has considerable algorithm migration benefits and development convenience. Then at the model conversion level, X2Paddle is used to implement the direct import and conversion of the four major mainstream model files of Caffe, TensorFlow, ONNX, and PyTorch. With its support for most operators of Caffe, TensorFlow, ONNX, and PyTorch (see Table 3) and automatic analysis and comparison capabilities, it can support most model conversions on mainstream CV and NLP models, and provides detailed API comparison documents between different frameworks to reduce the time cost of developers migrating models to Paddle.

[0032] Table 3 X2Paddle supports statistics of mainstream model operators

[0033]

[0034]

[0035] In (2), the present invention integrates and supports multiple training methods, providing a one-main-multiple-auxiliary training capability, that is, the PaddlePaddle training framework is the main one, and it is also compatible with multiple other framework access methods. The overall training system logic is as follows Figure 2 Down:

[0036] Specifically, the system is designed with PaddlePaddle as the main basic training framework. It can provide the ability to integrate multiple levels such as algorithm code and model into the PaddlePaddle framework.

[0037] First, at the algorithm code development level, as shown in the green area of ​​the logic diagram, the training system supports and promotes the development and design of algorithm codes based on the PaddlePaddle API and provides a development and compilation environment for training and reasoning in the PaddlePaddle framework, completing the entire end-to-end process; for engineering code development based on PyTorch, the X2Paddle automatic code conversion tool provided by the system can be used to convert the PyTorch engineering code into the PaddlePaddle algorithm code, and then the PaddlePaddle framework provides a development and compilation environment for training and reasoning. For engineering codes developed based on frameworks other than the above, if the developer's capabilities allow, they can use the API comparison between different frameworks to perform manual code migration, or developers who prefer to use specific frameworks can use the system's virtualization capabilities to build a specific framework training development environment and directly perform native development.

[0038] Then, for models that have been trained based on other frameworks, such as the yellow area in the logic diagram, the usual practice is to use ONNX, an open intermediate representation format used for conversion in various deep learning frameworks, and use it to agree on the same model data storage interaction format to enable different artificial intelligence frameworks to complete conversion with each other. This method supports this conversion method.

[0039] In addition, with the help of the X2Paddle tool, Caffe models, Tensorflow models, and Pytorch models can be directly converted into PaddlePaddle models.

[0040] In (3), PaddlePaddle uses a layered design in its framework structure to ensure good access and compatibility with various types of computing hardware at multiple module levels. It currently supports multiple types of devices from seven mainstream hardware manufacturers. As shown in Table 4, PaddlePaddle is the domestic deep learning framework with the most complete hardware support.

[0041] Table 4 PaddlePaddle supports hardware manufacturers

[0042]

[0043]

[0044] The PaddlePaddle framework is designed to ensure this support capability at multiple levels. The unified expression of the internal IR layer enhances two capabilities: on the one hand, it facilitates computational optimization such as operator fusion, and on the other hand, it facilitates serialization for docking with different hardware and distributed training.

[0045] In terms of operator fusion, PaddlePaddle not only provides a complete operator library, but also has highly efficient operators that can be used directly for more than 99% of models. However, due to the continuous advancement of science, there are always new models that may use some new computing functions, so the framework also provides an interface for adding user-defined operators, and the user-defined operators have the same training and reasoning capabilities as native operators, which expands the framework's deployment capabilities on a wide range of hardware.

[0046] In terms of operator sequence execution, in addition to supporting efficient execution according to the forward and backward computing logic order when training on a single machine and a single card, for distributed training on a single machine with multiple cards and on multiple machines with multiple cards, PaddlePaddle is used to implement concurrent and asynchronous execution of computing processes, and to schedule heterogeneous devices and other support capabilities.

[0047] In terms of hardware docking, PaddlePaddle provides the ability to dock with a wide range of training and prediction hardware. Depending on the different levels of openness of these hardware interfaces, different access methods are provided, typically the general programming interface of the IR subgraph and the intermediate representation layer IR interface access method.

[0048] The general programming interface supports programming languages ​​provided by hardware manufacturers, such as NVIDIA's CUDA, Knorons' OpenCL, and Cambrian's BANG-C language, etc. It also includes high-performance library functions provided by hardware manufacturers, such as NVIDIA's cuDNN, Intel's MKL and MKL-DNN, etc. Its advantage is flexibility, but it tests the ability and experience of developers and their familiarity with hardware.

[0049] The intermediate representation layer (IR) interface supports network IR and runtime API access, such as NVIDIA's TensorRT, Intel's nGraph, Huawei HiAI IR, and Baidu Kunlun's XTCL interface. It can shield hardware details, and the optimization, generation, and execution of the model are completed by the runtime library.

[0050] In (4), deep learning training is based on the computing power support provided by hardware training resources. At present, artificial intelligence hardware and software are in a stage of rapid development, and the industry has not yet formed a unified industry standard, resulting in large differences in the design and implementation solutions of different models of existing domestic equipment, making them difficult to be compatible with each other. The intelligent computing servers or clusters used for training integrate various types of heterogeneous acceleration cards such as NPU and GPU. Various intelligent computing frameworks are tightly coupled with heterogeneous acceleration cards, and conflicts may also occur between dependent libraries. In addition, since it is impossible to predict the occupancy of heterogeneous card resources, the difficulty of application development increases, it is difficult to handle related pipeline tasks, and it is impossible to fully utilize the acceleration capabilities of heterogeneous cards. Therefore, in order to solve the problem of complex training resource management and use, the present invention applies container technology, encapsulates heterogeneous card drivers, intelligent computing frameworks and dependent libraries in images, builds isolated application development and operation environments, and realizes virtualization of heterogeneous cards by mapping heterogeneous cards into containers. The mapped heterogeneous cards and their environments are used as a whole heterogeneous resource for users. The state of the container is equivalent to the state of the heterogeneous card and can be perceived by the outside world, thereby converting the management and scheduling of heterogeneous cards into the management and scheduling of heterogeneous resource containers. The training resource virtualization technology is studied to build a virtual computing environment for tasks, map CPU, memory, and storage logical volume resources, and virtualize and map the co-processing module at the same time. Tasks can be designed in a way that they exclusively occupy these resources, shielding the management details of heterogeneous cards, allowing users to focus on application development, and giving full play to the intelligent computing capabilities of hardware resources. The present invention proposes to realize the virtualization of multiple types of training resources based on docker containerization technology and k8s cluster management technology.

[0051] Embodiment 1:

[0052] The present invention proposes a unified support method for intelligent perception algorithm training framework for domestic hardware computing devices, including the following modules:

[0053] 1. Mainstream domestic framework conversion compatible technology;

[0054] 2. Unified training system;

[0055] 3. Training framework multi-hardware compatibility technology;

[0056] 4. Multi-type computing resource virtualization technology.

[0057] In (1), the mainstream frameworks involved include PaddlePaddle, PyTorch and MindSpore. Baidu PaddlePaddle is the earliest domestic open source deep learning framework and the only fully functional end-to-end open source deep learning platform in China. It integrates deep learning training and prediction framework, model library, tool components and service platform. It has flexible and efficient development mechanism, industrial application effect model, ultra-large-scale parallel deep learning capability, inference engine integrated design and systematic service support. It aims to bridge the gap between AI algorithm research and production deployment, and has a relatively complete ecosystem and rich functions. In the algorithm research stage, it provides developers with a unified programming experience to improve the efficiency of algorithm development; in the production stage, automatic parallelization can greatly speed up the development and debugging efficiency of distributed training, while fully tapping the computing power of heterogeneous hardware; in the deployment stage, based on the "end-edge-cloud" unified architecture, it meets the needs of industrial-level deployment.

[0058] Compared with other domestic deep learning frameworks, PaddlePaddle has the highest popularity in the open source community and has obvious advantages in tool chain completeness and domestic acceleration hardware support. Table 1 shows the statistics of the activeness of the open source community of domestic frameworks.

[0059] Table 1 Statistics on the activity of domestic framework open source communities

[0060] Github ranking Domestic framework name References Activity Number of Contributors Attention 1 PaddlePaddle 4300 33753 524 17500 2 MindSpore 514 37308 267 2700 3 Jittor 235 1266 31 2300

[0061] In addition, when selecting the framework, the present invention investigates and demonstrates factors such as mainstream framework compatibility, model conversion support, hardware support, and the number of model support, and focuses on comparing three training frameworks: PaddlePaddle, MindSpore, and Jittor. In terms of framework compatibility, except for PaddlePaddle, which can provide conversion tools to complete the automatic conversion of PyTorch code, the remaining frameworks require manual code conversion. For models exported by mainstream frameworks, PaddlePaddle and MindSpore can achieve automatic conversion support for PyTorch and TensorFlow models, while the Jittor framework has no corresponding support. In terms of the breadth of training hardware support, the PaddlePaddle framework is the most complete and can achieve access to 5 mainstream devices. In terms of the number of models supported, PaddlePaddle also has a large advantage. The comparative data is shown in Table 2.

[0062] Table 2 Comparison of domestic open source platform support

[0063]

[0064] In order to convert as many mainstream training frameworks as possible into the Baidu Paddle framework at the code or model level. The Paddle ecosystem has developed the X2Paddle conversion tool, which can quickly migrate users of other deep learning frameworks to the Paddle framework and provide conversion capabilities at both the code and model levels. First, at the code conversion level, in theory, any deep learning framework can complete the migration by aligning the operator development code, but the cost and difficulty here are relatively high. X2Paddle provides a one-click automatic migration and conversion function for PyTorch training project codes. Considering the huge audience of PyTorch, this support function has considerable algorithm migration benefits and development convenience. Then at the model conversion level, X2Paddle currently supports the direct import and conversion of the four major mainstream model files of Caffe, TensorFlow, ONNX, and PyTorch. With its support for most operators of Caffe, TensorFlow, ONNX, and PyTorch (see Table 3) and automatic analysis and comparison capabilities, it can support most model conversions on mainstream CV and NLP models, and provides detailed API comparison documents between different frameworks to reduce the time cost of developers migrating models to Paddle.

[0065] Table 3 X2Paddle supports statistics of mainstream model operators

[0066] Mainstream model formats Supported Operators PyTorch 142 TensorFlow 97 ONNX 96 Caffe 33

[0067] In (2), the present invention integrates and supports multiple training methods, providing a one-main-multiple-auxiliary training capability, that is, the PaddlePaddle training framework is the main one, and it is also compatible with multiple other framework access methods. The overall training system logic is as follows Figure 2 Down:

[0068] Specifically, the system is designed with PaddlePaddle as the main basic training framework, and it can provide the ability to integrate into the PaddlePaddle framework at multiple levels such as algorithm code and model. First, at the algorithm code development level, as shown in the green area of ​​the logic diagram, the training system supports and promotes the development and design of algorithm code based on the PaddlePaddle API and provides a development and compilation environment for training and reasoning in the PaddlePaddle framework, completing the entire end-to-end process; for engineering code developed based on PyTorch, the X2Paddle automatic code conversion tool provided by the system can be used to convert the PyTorch engineering code into PaddlePaddle algorithm code, and then the PaddlePaddle framework provides a development and compilation environment for training and reasoning. For engineering codes developed based on other frameworks besides the above, if the developer's capabilities allow, they can use the API comparison between different frameworks to perform manual code migration, or developers who prefer to use specific frameworks can use the system's virtualization capabilities to build a specific framework training development environment and directly perform native development.

[0069] Then, for models that have been trained based on other frameworks, such as the yellow area in the logic diagram, the usual practice is to use ONNX, an open intermediate representation format for conversion between various deep learning frameworks. With the same model data storage interaction format, different artificial intelligence frameworks can be converted to each other. This method supports this conversion method. In addition, with the help of the X2Paddle tool, Caffe models, Tensorflow models, and Pytorch models can be directly transferred to Paddle models.

[0070] In (3), PaddlePaddle uses a layered design in its framework structure to ensure good access and compatibility with various types of computing hardware at multiple module levels. It currently supports multiple types of devices from seven mainstream hardware manufacturers. As shown in Table 4, PaddlePaddle is the domestic deep learning framework with the most complete hardware support.

[0071] Table 4 PaddlePaddle supports hardware manufacturers

[0072]

[0073] The internal framework design of the PaddlePaddle framework ensures this support capability at multiple levels. The unified expression of the internal IR layer enhances two capabilities: on the one hand, it facilitates computational optimization such as operator fusion, and on the other hand, it facilitates serialization for docking with different hardware and distributed training.

[0074] The operator sequence execution module not only supports the efficient execution of forward and backward calculation logic sequence when training on a single machine and a single card, but also provides support capabilities such as concurrent and asynchronous execution of calculation processes and scheduling of heterogeneous devices for distributed training on a single machine with multiple cards and on multiple machines with multiple cards.

[0075] In addition to providing a complete operator library, the operator library module has highly efficient operators that can be used directly for more than 99% of the models. However, due to the continuous advancement of science, there are always new models that may use some new calculation functions, so the framework also provides an interface for adding user-defined operators, and the user-defined operators have the same training and reasoning capabilities as native operators, which expands the framework's deployment capabilities on a wide range of hardware.

[0076] The back-end hardware access module provides the ability to connect to a wide range of training and prediction hardware. Depending on the different levels of openness of these hardware interfaces, different access methods are provided, typically the high-level access method of IR subgraphs and the low-level programming interface access method.

[0077] The general programming interface (Low Level) supports programming languages ​​provided by hardware manufacturers, such as NVIDIA's CUDA, Knorons' OpenCL, and Cambrian's BANG-C language, etc. It also includes high-performance library functions provided by hardware manufacturers, such as NVIDIA's cuDNN, Intel's MKL and MKL-DNN, etc. Its advantage is flexibility, but it tests the ability and experience of developers and their familiarity with hardware.

[0078] The Intermediate Representation (IR) interface (High Level) supports network IR and runtime API access, such as NVIDIA's TensorRT, Intel's nGraph, Huawei HiAI IR, and Baidu Kunlun's XTCL interface. It can shield hardware details, and the optimization, generation, and execution of the model are completed by the runtime library.

[0079] In (4), deep learning training is based on the computing power support provided by hardware computing resources. At present, artificial intelligence hardware and software are in a stage of rapid development, and the industry has not yet formed a unified industry standard, resulting in large differences in the design and implementation solutions of different models of existing weapons and equipment, making them difficult to be compatible with each other. The intelligent computing servers or clusters used for training integrate various types of heterogeneous acceleration cards such as NPU and GPU. Various intelligent computing frameworks are tightly coupled with heterogeneous acceleration cards, and conflicts may also occur between dependent libraries. In addition, since it is impossible to predict the occupancy of heterogeneous card resources, the difficulty of application development increases, it is difficult to handle related pipeline tasks, and it is impossible to fully utilize the acceleration capabilities of heterogeneous cards. Therefore, in order to solve the complex problem of training resource management and use, the present invention applies container technology, encapsulates heterogeneous card drivers, intelligent computing frameworks, and dependent libraries in images, and constructs mutually isolated application development and operation environments. Virtualization of heterogeneous cards is achieved by mapping heterogeneous cards into containers. The mapped heterogeneous cards and their environments are provided to users as a whole heterogeneous resource. The state of the container is equivalent to the state of the heterogeneous card and can be perceived by the outside world, thereby converting the management and scheduling of heterogeneous cards into the management and scheduling of heterogeneous resource containers. The training resource virtualization technology is studied to build a virtual computing environment for tasks, map CPU, memory, and storage logical volume resources, and virtualize and map the co-processing modules at the same time. Tasks can be designed in a way that they exclusively occupy these resources, shielding the management details of heterogeneous cards, allowing users to focus on application development, and giving full play to the intelligent computing capabilities of hardware resources. The present invention implements virtualization of multiple types of computing resources based on docker containerization technology and k8s cluster management technology.

[0080] In view of the problems faced by the current intelligent perception model training, such as the lack of unified training framework, the risk of "stuck neck" of non-domestic training framework, and the poor compatibility of domestic framework, the present invention proposes a unified support method for the intelligent perception algorithm training framework, focusing on solving the problems of unified training framework support, model knowledge integration management, sufficient scheduling of computing resources, model debugging and optimization design, etc. throughout the entire process of training tasks in intelligent applications, so as to build the rapid R&D and iteration capabilities of intelligent algorithms. The unified support method for the training framework adopted by the present invention integrates the domestic mainstream training framework PaddlePaddle, and realizes the unified conversion compatibility of the PyTorch training framework at the training code layer and the model layer through the X2Paddle tool; and manages the training environment by using computing resource virtualization technology, realizes the compatible support of different training frameworks in a mirroring manner, and realizes the access of mainstream domestic hardware computing devices. In terms of model construction, algorithm resource integration is realized through model resource management and construction of scenario development kits to realize the rapid development capabilities of different task scenarios. Through distributed training technology, the scheduling and use of single-machine multi-card and multi-machine multi-card resources are supported. Through hyperparameter tuning and model compression technology, the optimization of training model effect and training efficiency is realized.

[0081] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A unified support method for intelligent perception algorithm training framework, characterized in that: The method includes: Mainstream domestic framework conversion compatibility technology: Use X2Paddle conversion tool to quickly migrate users of other deep learning frameworks to the PaddlePaddle framework; Unified training system: Training is based on the PaddlePaddle training framework, and is compatible with multiple other framework access methods; Training framework multi-hardware compatibility technology: Provides the ability to connect to a wide variety of training and prediction hardware through a general programming interface and an intermediate representation layer IR interface; Multi-type training resource virtualization technology: Docker containerization technology and k8s cluster management technology are used to realize the virtualization of multi-type training resources.

2. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: The X2Paddle conversion tool is used to quickly migrate users of other deep learning frameworks to the PaddlePaddle framework and provide code-level conversion capabilities; at the code conversion level, X2Paddle is used to perform one-click automatic migration and conversion of PyTorch training project codes.

3. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: The X2Paddle conversion tool is used to quickly migrate users of other deep learning frameworks to the PaddlePaddle framework and provide model-level conversion capabilities. At the model conversion level, X2Paddle is used to directly import and convert the four major mainstream model files of Caffe, TensorFlow, ONNX, and PyTorch.

4. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: The unified training system supports and promotes the development and design of algorithm codes based on the PaddlePaddle API and provides a development and compilation environment for training and reasoning in the PaddlePaddle framework to complete the entire end-to-end process; for engineering code development based on PyTorch, the X2Paddle automatic code conversion tool is used to convert the PyTorch engineering code into PaddlePaddle algorithm code, and then the PaddlePaddle framework provides a development and compilation environment for training and reasoning; for engineering code developed based on other frameworks, if the developer's capabilities permit, manual code migration can be performed with the help of API comparison between different frameworks, or developers who prefer specific frameworks can use the system's virtualization capabilities to build a specific framework training development environment and directly perform native development.

5. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: In the unified training system, for models that have been trained based on other frameworks, based on ONNX, an open intermediate representation format used for conversion among various deep learning frameworks, the same model data storage interaction format is agreed upon, allowing different artificial intelligence frameworks to complete conversion between each other.

6. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: In the unified training system, the Caffe model, Tensorflow model and Pytorch model are directly converted into PaddlePaddle models with the help of the X2Paddle tool.

7. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: In the multi-hardware compatibility technology of the training framework, in terms of operator fusion, the operator library provided by PaddlePaddle is adopted, and the framework provides an interface for adding user-defined operators; in terms of operator sequence execution, PaddlePaddle is used to implement concurrent and asynchronous execution of computing processes and schedule heterogeneous devices.

8. The intelligent perception algorithm training framework unified support method according to claim 1, characterized in that: In the multi-hardware compatibility technology of the training framework, in terms of hardware docking, it supports programming languages ​​and high-performance library functions provided by hardware manufacturers; the intermediate presentation layer interface supports networking IR and runtime API access.

9. The unified support method for intelligent perception algorithm training framework according to claim 1, characterized in that: In the multi-type training resource virtualization technology, container technology is applied to encapsulate heterogeneous card drivers, intelligent computing frameworks and dependent libraries in images to build isolated application development and operating environments. The virtualization of heterogeneous cards is achieved by mapping heterogeneous cards into containers. The mapped heterogeneous cards and their environments are provided to users as a whole heterogeneous resource. The state of the container is equivalent to the state of the heterogeneous card and is perceived by the outside world. Therefore, the management and scheduling of heterogeneous cards is converted into the management and scheduling of heterogeneous resource containers.

10. The method for unified support of intelligent perception algorithm training framework according to claim 1, characterized in that: In the multi-type training resource virtualization technology, a virtual computing environment is constructed for the task, mapping the CPU, memory, and storage logical volume resources, and at the same time, the co-processing module is virtualized and mapped. The task is designed in a way to exclusively occupy these resources, shielding the management details of the heterogeneous cards.

Images