Artificial intelligence model generation method, electronic device, and storage medium

Abstract

The application provides an AI model generation method, an electronic device and a storage medium. The method comprises: obtaining a business requirement corresponding to a target AI model; determining a target model and target hyperparameter configuration information in a basic database according to the business requirement, or determining the target model and the target hyperparameter configuration information in a pre-trained model according to the business requirement and preset expert experience logic, or determining the target model in a network structure vector space according to the business requirement and network structure search logic and determining the target hyperparameter configuration information according to the business requirement and hyperparameter search logic; configuring the target model according to the target hyperparameter configuration information, and training the target model according to training data to generate the target AI model. The model and the hyperparameter configuration corresponding to the business requirement are determined in different modes, and the AI model generation method based on different modes can realize systematic and low-threshold cost reduction and benefit increase, and promote AI large-scale landing.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an AI model generation method, electronic device, and storage medium. Background Technology

[0002] Artificial intelligence (AI) is a technical science that studies, develops, and applies theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. In recent years, people's demand for artificial intelligence has been increasing.

[0003] The implementation of AI technology involves data processing, model design, model training, and model deployment, and the barriers to entry are relatively high. Although there are already mainstream open-source deep learning frameworks such as PyTorch, TensorFlow, and Caffe, strong development capabilities in AI algorithms are still required. This involves repeatedly debugging data, network structures, hyperparameters, etc., for different data and scenarios, which is time-consuming, labor-intensive, and requires the experience of expert engineers.

[0004] Against this backdrop, AutoML (Automated Machine Learning) technology has emerged, enabling the design of AI with AI and lowering the barrier to entry for AI technology. However, current AutoML-related products are either geared towards professional engineers or have limited functionality, such as offering only a publicly available, commonly used pre-trained model, which hinders the widespread adoption of AI with low barriers to entry. Summary of the Invention

[0005] This application provides an AI model generation method, electronic device, and storage medium, aiming to improve the adaptability of AI models generated by AutoML technology and lower the barrier to entry for their use.

[0006] In a first aspect, embodiments of this application provide an AI model generation method, including:

[0007] Obtain the business requirements corresponding to the target AI model;

[0008] Determine the model generation mode, wherein the model generation mode includes at least one of a first generation mode, a second generation mode, and a third generation mode;

[0009] If the model generation mode is determined to be the first generation mode, the target model and target hyperparameter configuration information are determined in the basic database according to the business requirements.

[0010] If the model generation mode is determined to be the second generation mode, the target model and the target hyperparameter configuration information are determined in the pre-trained model according to the business requirements and the preset expert experience logic.

[0011] If the model generation mode is determined to be the third generation mode, the target model is determined in the network structure vector space according to the business requirements and network structure search logic, and the target hyperparameter configuration information is determined according to the business requirements and hyperparameter search logic.

[0012] Configure the target model according to the target hyperparameter configuration information, and train the target model according to the training data to generate the target AI model.

[0013] Secondly, embodiments of this application provide an electronic device, including a memory and a processor;

[0014] The memory is used to store computer programs;

[0015] The processor is used to execute the computer program and, when executing the computer program, to implement the AI ​​model generation method described above.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the AI ​​model generation method described above.

[0017] This application provides an AI model generation method, electronic device, and storage medium. The method involves acquiring the business requirements corresponding to the target AI model; determining the target model and target hyperparameter configuration information in a basic database based on the business requirements, or determining the target model and target hyperparameter configuration information in a pre-trained model based on the business requirements and preset expert experience logic, or determining the target model in a network structure vector space based on the business requirements and network structure search logic, and determining the target hyperparameter configuration information based on the business requirements and hyperparameter search logic; configuring the target model based on the target hyperparameter configuration information, and training the target model based on training data to generate the target AI model. By determining the model and hyperparameter configuration corresponding to the business requirements through different modes, each mode possesses different characteristics and is coupled with each other. AI model generation methods based on different modes can achieve systematic, low-threshold cost reduction and efficiency improvement, promoting the large-scale deployment of AI.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the disclosure of the embodiments of this application. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an AI model generation method provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the application scenarios of AI model generation methods;

[0022] Figure 3 This is a schematic diagram of a directed acyclic graph of the base layer of an AutoML system;

[0023] Figure 4 This is a schematic diagram of a directed acyclic graph of the prefabricated layers in an AutoML system.

[0024] Figure 5 This is a schematic diagram of a directed acyclic graph of the customization layer of an AutoML system in one implementation.

[0025] Figure 6 This is a schematic diagram of a directed acyclic graph of the custom layer of an AutoML system in another implementation.

[0026] Figure 7 This is a schematic diagram of the underlying framework of an AutoML system in one implementation method;

[0027] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0030] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating an AI model generation method provided in an embodiment of this application. The AI ​​model generation method can be applied to electronic devices, such as terminal devices or servers, for processes such as generating AI models; wherein, the terminal device can be an electronic device such as a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant, and wearable device; the server can be a standalone server or a server cluster.

[0032] For example, such as Figure 2 The diagram illustrates a scenario where the AI ​​model generation method is applied to a server. The server can obtain business requirements from the terminal device, execute the AI ​​model generation method to generate the target AI model, and send the generated target AI model to the terminal device for model testing or deployment.

[0033] like Figure 1 As shown, the AI ​​model generation method of this application embodiment includes steps S110 to S160.

[0034] S110. Obtain the business requirements corresponding to the target AI model.

[0035] For example, a requirements acquisition interface is displayed on the terminal device's display screen, where users can input their business requirements. Understandably, the target AI model represents the AI ​​model the user needs.

[0036] In some implementations, the business requirements include at least one of the following: task type, terminal type, application scenario, and computing power requirements. Thus, users of the terminal device can set business requirements as needed, so that the server executes the AI ​​model generation method to generate a target AI model that meets the user's needs, making the generated target AI model more targeted.

[0037] The task type indicates the practical scenario for which the user needs the target AI model, such as the task the target AI model needs to handle, like classification, detection, video processing, or natural language processing. The terminal type indicates the deployment environment of the target AI model, such as the model of the terminal deploying the target AI model, the type and model of the terminal's processor, etc. For example, the terminal's processor type can include CPU (Central Processing Unit) and / or GPU (Graphics Processing Unit). The application scenario can include at least one of the following: few-shot detection, small object detection, and imbalanced sample detection.

[0038] S120. Determine the model generation mode, wherein the model generation mode includes at least one of the first generation mode, the second generation mode, and the third generation mode.

[0039] For example, different generation modes have different processes for generating target AI models. For instance, there are differences in the speed of model generation, the computing power consumption for generating target AI models, and / or the size and accuracy of target AI models, etc.

[0040] For example, the model generation speed of the first generation mode is faster than that of the second generation mode, and the model generation speed of the second generation mode is faster than that of the third generation mode; or the accuracy of the model generated by the first generation mode is lower than that of the second generation mode, and the accuracy of the model generated by the second generation mode is lower than that of the third generation mode.

[0041] For example, the first generation mode can be called the basic generation mode, the second generation mode can be called the prefab generation mode, and the first generation mode can be called the custom generation mode.

[0042] In some implementations, multiple generation modes can be displayed on the display device of the terminal device, and the model generation mode can be determined according to the user's mode selection operation.

[0043] In other implementations, the model generation mode can be determined based on the acquired business requirements.

[0044] In some implementations, the AI ​​model generation method can be used in an AutoML system that can be deployed on a server or terminal device.

[0045] For example, the AutoML system may include a base layer, a prefab layer, and a customization layer. The base layer is used to generate a model in a first generation mode, the prefab layer is used to generate a model in a second generation mode, and the customization layer is used to generate a model in a third generation mode.

[0046] S130. If the model generation mode is determined to be the first generation mode, the target model and target hyperparameter configuration information are determined in the basic database according to the business requirements.

[0047] In some implementations, the basic database of the base layer includes standard data modules, model selection modules, and hyperparameter configuration modules for different business needs, such as different tasks (classification, detection, video, natural language processing, etc.), which can be understood as a model repository module for models.

[0048] Taking classification tasks as an example, the standard data module can include various data augmentation units for performing data augmentation algorithms, such as flipping, pruning, and rotation; the model selection module can include mainstream neural network model architectures, such as VGG (Visual Geometry Group), ResNet (Residual Network), DenseNet (Densely Connected Convolutional Networks), Inception, etc.; the hyperparameter configuration can include various hyperparameters, including but not limited to Dropout, weight decay, and learning rate.

[0049] For example, the target model can be determined as a VGG architecture based on business requirements, and the configuration information of at least one hyperparameter can be determined, namely the target hyperparameter configuration information.

[0050] In some implementations, the base layer may also include a training module for training the model according to a defined training strategy. The training module may contain various training strategy settings, including but not limited to gradient descent algorithms and reinforcement learning algorithms such as SGD (stochastic gradient descent) and Adam (adaptive moment estimation).

[0051] In some implementations, the base layer may also include a deployment module, which is used to transform the trained model into a scenario task that can be used in the cloud or on the chip, by packaging the model and deploying it to the corresponding device.

[0052] In some implementations, the AutoML system includes a directed acyclic graph (DAG) used to describe the automated workflow of the AutoML system, thereby encapsulating the architecture of the automated workflow.

[0053] For example, the base layer of an AutoML system includes a directed acyclic graph (DAG) used to describe the automated workflow of the base layer, thus implementing the architectural encapsulation of the automated workflow. Of course, the DAG can also be configured in the prefab or custom layers of the AutoML system, or it can be configured independently of the base layer, prefab layer, and custom layer.

[0054] For example, such as Figure 3The diagram shows a directed acyclic graph of the base layer. Solid lines represent active workflows, while dashed lines represent inactive workflows. When generating a model using the first generation mode, the workflow represented by the solid lines is executed.

[0055] In some implementations, the directed acyclic graph includes identifiers of multiple AI models and identifiers of multiple hyperparameter configuration information in the underlying database.

[0056] For example, such as Figure 3 VGG, ResNet, DenseNet, and Inception are currently the mainstream AI models, also known as the names of neural network model architectures. Figure 3 Dropout, weights decay, and learning rate are optional hyperparameters, i.e., identifiers of hyperparameter configuration information.

[0057] For example, the step S130 of determining the target model and target hyperparameter configuration information in the basic database according to the business requirements includes: determining the identifier of at least one AI model in the directed acyclic graph as the identifier of the target model, and determining the identifier of at least one hyperparameter configuration information as the identifier of the target hyperparameter configuration information.

[0058] For example, such as Figure 3 As shown, the VGG architecture is determined as the target model; Dropout, weight decay, and learning rate are hyperparameters.

[0059] In some implementations, the training strategy can also be determined in the directed acyclic graph of the base layer, such as determining the training strategy as SGD.

[0060] like Figure 3 As shown, if the model generation mode is determined to be the first generation mode, the target model and target hyperparameter configuration information are determined in the basic database according to the business requirements. This includes: augmenting the training data through flipping, pruning, and rotating; training the determined target model based on the augmented data and the determined target hyperparameter configuration information, using the SGD training strategy during training; and obtaining the target AI model after training, which can then be deployed. Thus, one-click automated model training can be achieved based on the basic layer.

[0061] Specifically, a directed acyclic graph can be displayed in the form of a configuration table interface. The configuration table can be connected to the software front-end or web front-end, or it can be connected to command line operations.

[0062] S140. If the model generation mode is determined to be the second generation mode, the target model is determined in the pre-trained model according to the business requirements and the preset expert experience logic, and the target hyperparameter configuration information is determined.

[0063] Specifically, expert experience logic can include the experience logic accumulated by experts and professional engineers on scenario task data, such as pre-trained models and hyperparameter configurations corresponding to different terminals and scenarios.

[0064] For example, the expert experience logic encapsulated in the prefabricated layer can include various chip models (which can be refined to the corresponding models), computing power requirements, application scenarios (such as small sample detection, small object detection, imbalanced sample detection, task type), as well as the corresponding pre-trained model and target hyperparameter configuration information.

[0065] For example, the pre-trained model corresponding to the business requirement can be determined as the target model based on the preset expert experience logic, and the target hyperparameter configuration information can be determined. For example, based on the business requirements, the mainstream pre-trained model that can be implemented in the scenario and its suitable hyperparameters, the pre-trained self-developed and hand-designed model and its suitable hyperparameters, and the pre-trained AutoML searched model and its suitable hyperparameters can be obtained.

[0066] For example, the requirements of a scenario task can be summarized into three requirements: terminal model, application scenario, and computing power requirements.

[0067] Taking a classification scenario running on CPU and GPU as an example, if the user chooses to use the classification scenario on CPU and has no computational power constraints, due to the CPU-friendly nature of the depthwise separatable operator, a pre-trained MobileNet model can be started based on expert experience, and fine-tuned on the scene data. Here, the MobileNet model refers to a lightweight deep neural network model. If the user chooses to use the classification scenario on GPU and has no computational power constraints, due to the GPU-friendly nature of the conv33 operator, a pre-trained ResNet-50 model can be started based on expert experience, and fine-tuned on the scene data. If the user chooses CPU and the business requires the model to have less than 3 megabytes (M) of parameters, since the MobileNet model with 4.2 megabytes of parameters does not meet the requirement, a pre-trained, compressed MobileNet model (e.g., with 2M parameters) can be started based on expert experience, and fine-tuned on the scene data.

[0068] For example, such as Figure 4Model 1, Model 2, Model 3, and Model 4 are identifiers for pre-trained models. Hyperparameter configuration table 1, Hyperparameter configuration table 2, and Hyperparameter configuration table 3 are optional hyperparameters, i.e., identifiers for hyperparameter configuration information. Training strategy 1 and Training strategy 2 are identifiers for optional training strategies.

[0069] In some implementations, the prefabrication layer encapsulates the automated workflow based on the directed acyclic graph of the base layer, with the encapsulation form as follows: Figure 4 As shown, this is also an encapsulation based on a directed acyclic graph. It can reflect the business logic corresponding to business requirements and the corresponding automated training and deployment encapsulation. The logic encapsulation of the prefabricated layer is partly a secondary encapsulation based on the encapsulation of the base layer, and partly a customized R&D result encapsulation adapted to specific scenarios.

[0070] For example, such as Figure 4 As shown, Model 1 is determined as the target model, and Hyperparameter Configuration Table 1 is determined as the hyperparameter.

[0071] In some implementations, a training strategy can also be determined in the directed acyclic graph of the prefabricated layer, such as determining the training strategy as training strategy 1.

[0072] like Figure 4 As shown, if the model generation mode is determined to be the second generation mode, the target model is determined in the pre-trained model according to the business requirements and preset expert experience logic, and the target hyperparameter configuration information is determined, including: determining model 1 as the target model according to the terminal model, application scenario and computing power requirements; training the determined target model according to the determined target hyperparameter configuration information, using training strategy 1 during training; obtaining the target AI model after training is completed, and then deploying the target AI model. Thus, one-click automated model training can be achieved based on the prefabricated layer.

[0073] S150. If the model generation mode is determined to be the third generation mode, the target model is determined in the network structure vector space according to the business requirements and network structure search logic, and the target hyperparameter configuration information is determined according to the business requirements and hyperparameter search logic.

[0074] For example, the network structure vector space can be the union of multiple different network models. A network model can be described by a vector, which can at least describe the width, depth, activation function, loss function, etc. of the network model.

[0075] Specifically, when generating models using the third generation mode, there is a high degree of freedom, allowing for automated search processes such as network structure, hyperparameter settings, and data processing from scratch. It is a customized AI deployment service driven by data, scenarios, and computing power based on AutoML algorithms. Although the search is time-consuming, it is fully customized, resulting in models with higher accuracy and better suitability for specific scenarios.

[0076] For example, the target model and target hyperparameter configuration information can be determined by searching the automl algorithm based on Pretrian automl, which satisfies both time-saving and customization requirements.

[0077] For example, such as Figure 5 As shown, the model can be customized with a high degree of freedom, with 1M computational cost and network performance as optimization objectives, to seek the model with the highest performance under 1M parameters.

[0078] In some implementations, the AI ​​model generation method further includes: determining a data augmentation strategy for the training data based on the business requirements; and performing augmentation processing on the training data based on the data augmentation processing unit corresponding to the data augmentation strategy in the basic database to increase the information content of the training data, so that the trained model can be more accurate.

[0079] For example, the data augmentation strategy includes at least one of the following: data cleaning, data preprocessing, and data augmentation, wherein data augmentation may include flipping, cropping, rotating, etc.

[0080] For example, if the model generation mode is determined to be the third generation mode, the following can be performed: automated search for network structure, data augmentation strategies, and hyperparameters can be executed based on the terminal model, application scenario, and computing power requirements to determine the target model, data augmentation strategies, and target hyperparameter configuration information. The determined target model is then trained using training strategy 1. After training, the target AI model is obtained, and it can then be deployed. Thus, one-click automated model training can be achieved based on the customization layer.

[0081] In some implementations, determining the target model in the network structure vector space based on the business requirements and network structure search logic includes: determining a search initialization state based on the target model corresponding to the first generation mode or the pre-trained model corresponding to the second generation mode; and performing a network structure search in the network structure vector space based on the business requirements and network structure search logic, using the search initialization state, to obtain the target model.

[0082] To efficiently complete the search process, the base layer and prefabricated layers can be used to initialize the network model structure and hyperparameter configuration for automated searching of network structure, hyperparameters, and data. Customized AI deployment services are driven by data, scenario, and computing power constraints based on AutoML algorithms. The search can begin with either a publicly available model from the base layer as the initial search state, or a self-developed model from the prefabricated layers. Using either the publicly available model from the base layer or a self-developed model from the prefabricated layers allows for semi-customization of the model, resulting in a better initial search state, shorter search time, and faster search convergence.

[0083] For example, if a user selects CPU and the business requires a model with 1M parameters, but the MobileNet model with 4.2M parameters does not meet the requirements, and the expert-pre-compressed pretrained MobileNet model with 2M parameters also does not meet the requirements, then an automated compression algorithm can be started in the customization layer. The pre-compressed 2M parameter pretrained MobileNet model can be used as the initial state, and the network can be automatically compressed from 2M to 1M parameters. Then, the compressed pre-compressed model can be fine-tuned on the scene data.

[0084] For example, such as Figure 6 As shown, a pre-trained model can be determined based on business requirements such as terminal model, application scenario, and computing power needs, as well as preset expert experience logic. For example, pre-built model 1 is the target model. The search initialization state is determined using this pre-trained model. Based on this search initialization state, a network structure search is performed in the network structure vector space to obtain the target model. Automated search for data augmentation strategies and automated hyperparameter search are then performed to determine the data augmentation strategy and target hyperparameter configuration information. The determined target model is then trained using training strategy 1. After training, the target AI model is obtained and can be deployed. Thus, one-click automated model training can be achieved based on the customization layer.

[0085] Understandably, the functions of the customization layer include: internally providing automated AutoML algorithms based on the base layer and the prefabricated layer to realize automated invocation of the base layer and the prefabricated layer; and externally providing high-level AutoML development tool capabilities.

[0086] S160. Configure the target model according to the target hyperparameter configuration information, and train the target model according to the training data to generate a target AI model.

[0087] In some implementations, if the model generation mode is determined to be the first generation mode, the target model can be configured according to the automated workflow of the base layer and trained according to the training data; if the model generation mode is determined to be the second generation mode, the target model can be configured according to the automated workflow of the prefab layer and trained according to the training data; if the model generation mode is determined to be the third generation mode, the target model can be configured according to the automated workflow of the customization layer and trained according to the training data.

[0088] In some implementations, the base layer may include a training module for training the model according to a defined training strategy. The training module may include various training strategy settings, such as SGD, Adam, etc. For example, the automated workflow for the training phases of prefab and custom layers can also be executed by the training module of the base layer to simplify the AutoML system.

[0089] For example, in step S140, when determining the target model in the pre-trained model according to the business requirements and the preset expert experience logic, and determining the target hyperparameter configuration information, the method further includes: storing the pre-trained model determined as the target model in the basic database, adding the identifier of the pre-trained model to the directed acyclic graph; and storing the determined target hyperparameter configuration information in the basic database, and adding the identifier of the determined target hyperparameter configuration information to the directed acyclic graph.

[0090] Understandably, the target model, data augmentation strategy, and target hyperparameter configuration information determined in step S140 based on expert experience logic can be automatically pulled into the workflow of the base layer for training.

[0091] In some implementations, step S160, which involves configuring the target model according to the target hyperparameter configuration information and training the target model according to training data to generate a target AI model, includes: determining a workflow path in the directed acyclic graph of the base layer, and training the target model according to the training data based on the workflow path to generate a target AI model.

[0092] The directed acyclic graph (DAG) is used to describe the automated workflow of the base layer. The workflow path includes the identifier of the target model, the identifier of the target hyperparameter configuration information, the identifier of the training strategy, and the paths between the identifier of the target model and the identifier of the target hyperparameter configuration information, and between the identifier of the target hyperparameter configuration information and the identifier of the training strategy. For details, please refer to... Figures 3 to 6Defining workflow paths using directed acyclic graphs (DAGs) makes it easier to integrate model generation processes corresponding to various model generation modes within the workflow path. For example, the training and deployment processes within the workflow path can be reused.

[0093] For example, when determining the target model in the network structure vector space according to the business requirements and network structure search logic in step S150, and determining the target hyperparameter configuration information according to the business requirements and hyperparameter search logic, the method further includes: storing the determined target model in the basic database, adding the identifier of the target model to the directed acyclic graph; and storing the determined target hyperparameter configuration information in the basic database, and adding the identifier of the determined target hyperparameter configuration information to the directed acyclic graph.

[0094] Understandably, in step S150, the target model, data augmentation strategy, and target hyperparameter configuration information determined by searching can be automatically pulled into the workflow of the base layer for training.

[0095] In some implementations, the base layer may include a deployment module, which is used to transform the trained model into a scenario task that can be used in the cloud or on the chip, by packaging the model and deploying it to the corresponding device.

[0096] In some implementations, the AI ​​model generation method further includes: deploying the target AI model to a target device and / or publishing the target AI model to a model trading platform.

[0097] The target device can be the terminal device or other electronic devices besides the terminal device. For example, the server can obtain the target device specified by the terminal device through interaction with the terminal device. The target device is an electronic device used to deploy the target AI model, such as a mobile phone, tablet, laptop, desktop computer, personal digital assistant, and wearable device. The server can then deploy the target AI model to the target device so that the target device can apply the target AI model to perform preset tasks, such as image classification.

[0098] In some implementations, such as Figure 7 As shown, AutoML systems can use open-source frameworks such as TensorFlow, PyTorch, Caffe, and MXNet (but are not limited to these) as their underlying frameworks.

[0099] For example, the output from the underlying framework sequentially enters the NNCF framework, the ONNX framework, and the TNN framework, and after iteration, it is used for ARM, GPU, or CPU chips.

[0100] Specifically, through the aforementioned mainstream open-source frameworks, such as TensorFlow, TF-Lite can be seamlessly integrated, and then connected to numerous chips, such as MTK-APU and CPU. The outputs of TensorFlow, PyTorch, Caffe, and MXNet frameworks can be sequentially fed into the NNCF framework, ONNX framework, and TNN framework, seamlessly connecting to numerous chips such as ARM, GPU, and CPU.

[0101] Furthermore, the AutoML system can be linked to an AI model trading platform to provide a SaaS service model for the AI ​​system. A button for publishing models can be set on the product's web interface, and the button can be linked to the trading platform to enable the external publishing of automatically produced models.

[0102] Specifically, this embodiment provides the SaaS form of the AutoML system after productization. The SaaS is Software as a Service, which is used for external release after the model is automatically produced. A trading platform button is set on the web interface and linked to the AI ​​trading platform.

[0103] The AI ​​model generation method provided in this application involves: acquiring the business requirements corresponding to the target AI model; determining the target model and target hyperparameter configuration information in a basic database based on the business requirements, or determining the target model and target hyperparameter configuration information in a pre-trained model based on the business requirements and preset expert experience logic, or determining the target model in the network structure vector space based on the business requirements and network structure search logic, and determining the target hyperparameter configuration information based on the business requirements and hyperparameter search logic; configuring the target model based on the target hyperparameter configuration information, and training the target model based on training data to generate the target AI model. By determining the model and hyperparameter configuration corresponding to the business requirements through different modes, each mode possesses different characteristics and is coupled with each other. AI model generation methods based on different modes can achieve systematic, low-threshold cost reduction and efficiency improvement, promoting the large-scale deployment of AI.

[0104] In some implementations, determining the target model in the network structure vector space based on the business requirements and network structure search logic includes: determining the network structure vector space corresponding to the business requirements; searching for a network structure in the network structure vector space based on a random search strategy; determining the weight parameters of the network structure based on the shared weights corresponding to the network structure vector space to obtain a weight model; fine-tuning the weight model based on the search sample set to obtain a fine-tuned model; and determining the target AI model among multiple fine-tuned models.

[0105] In some embodiments, the network structure vector space includes a network structure vector space graph, which includes multiple network units and the connections between them. In other embodiments, the network structure vector space can also be described in the form of vectors, where each element represents a connection between network units. Of course, the embodiments of this application are not limited to these.

[0106] The network structure vector space can include many operations. Based on different connection methods, all or some of these operations can form different neural networks. In other words, based on different connection methods, the network structures of the neural networks formed by all or some of these operations can be different.

[0107] For example, the network structure vector space includes network units such as convolution (Conv) operations and depthwise (DW) convolution operations. Multiple network units and the connections between multiple network units can constitute a neural network.

[0108] For example, the step of searching for the network structure in the network structure vector space includes: searching in the network structure vector space graph to determine directed paths including a plurality of network units and connections between the plurality of network units, and determining the network structure based on the directed paths.

[0109] Understandably, compared to search methods based on gradient information, reinforcement learning, and evolutionary algorithms, random search methods are inherently unbiased and applicable to a variety of tasks. Therefore, by using a random search strategy to search for network structures in the network structure vector space, search biases across different tasks and data can be reduced or eliminated, ensuring the stability of the resulting network structure.

[0110] In some implementations, the step of searching for the network structure in the network structure vector space based on a random search strategy includes: searching for the network structure in the network structure vector space based on a randomly initialized search control network.

[0111] For example, the search control network can also be called a controller. It can be configured using an LSTM neural network, but it is not limited to this; an RNN neural network can also be used. The search control network is used to determine several network units and the connections between these network units in the network structure vector space, thus obtaining the network structure.

[0112] In this embodiment, the search control network is untrained and can maintain random sampling, thereby enabling a better search for the network structure in the network structure vector space based on a random search strategy.

[0113] Weight sharing is a strategy that involves sharing the weights of all network structures in the network structure vector space to accelerate the search process. By using weight sharing to reuse the weights of different network structures in the network structure vector space, search efficiency can be improved.

[0114] For example, by obtaining a training set and a test set, the training set can be split into a search set and a valid set. Of course, the search set and the valid set can also be obtained independently.

[0115] For example, the weight model can be fine-tuned until convergence to obtain a fine-tuned model; or the weight model can be fine-tuned for several epochs to obtain a fine-tuned model.

[0116] Specifically, after configuring the network structure and shared weights, fine-tuning the weight model by searching a sample set can improve the reliability of networks based on weight-shared parameters, prevent the use of network structures with weight-shared parameters to complete tasks, and prevent the predicted performance from failing to effectively reflect the actual performance of the network after training, thereby improving the efficiency of the search.

[0117] For example, determining the target model among the multiple fine-tuning models based on the evaluation parameters of each fine-tuning model can improve search performance.

[0118] Compared to weighted models with shared parameters, the performance of fine-tuned models can be closer to that of models trained in reality. The evaluation parameters of fine-tuned models are more correlated with those of models trained in reality.

[0119] Understandably, when using a random search strategy to search for network structures, each network structure found is assigned shared weights, and then fine-tuned for a predetermined number of steps on the search sample set. This can solve the problem of weak relevance caused by weight sharing. For example, the predetermined number of steps for fine-tuning could be one epoch, because experiments have shown that one epoch can significantly improve the relevance.

[0120] In some embodiments, determining the target model among the plurality of fine-tuned models includes: determining a first evaluation parameter for each of the plurality of fine-tuned models based on a validation sample set; determining a plurality of candidate models among the plurality of fine-tuned models based on the first evaluation parameters; training each of the candidate models based on a training sample set to obtain a training model corresponding to each candidate model; determining a second evaluation parameter for each of the training models based on a test sample set; and determining the target model among the training models corresponding to the plurality of candidate models based on the second evaluation parameters.

[0121] For example, each fine-tuned model is tested on a valid set to obtain the first evaluation parameter, which can be called the valid reward. In the case of a classification task, the reward is the accuracy.

[0122] The first evaluation parameter can be used to evaluate the performance of the fine-tuned model. Based on the first evaluation parameter, the fine-tuned models with better performance are selected as candidate models. For example, the K fine-tuned models with the best valid rewards are selected as candidate models, where K is a natural number greater than or equal to 1.

[0123] For example, candidate models are trained on a training sample set until convergence, resulting in the training model corresponding to each candidate model.

[0124] For example, each trained model is tested on a test set to obtain a second evaluation parameter, which may be called the test reward.

[0125] The evaluation parameters of the fine-tuned model can be closer to those of the trained model. By identifying candidate models from the fine-tuned model, more suitable candidate models can be obtained. Even after training, a trained model with good performance can still be obtained, thus improving search efficiency.

[0126] Understandably, the evaluation system for the effectiveness of a search method is the correlation between K valid rewards and K test rewards. It is generally believed that the higher the monotonic consistency between K valid rewards and K test rewards, the better the search method is, where K is a natural number greater than or equal to 1.

[0127] For example, the training model with the best test reward is selected as the target model.

[0128] In some implementations, although the step of fine-tuning the weight model is added, the time spent training the controller is eliminated, and the overall time does not increase, but the robustness of the search results is increased.

[0129] Using random search directly presents the following problems: 1) the search results have high variance, leading to low efficiency; 2) the search results are unreliable. Therefore, more structures need to be searched to find a better network structure. This application's embodiments improve search efficiency and reliability by using weight sharing, fine-tuning the weight model, and determining the target model from the fine-tuned model, thereby addressing the aforementioned technical problems.

[0130] For example, the fine-tuning period is shorter than the training period, and / or the search sample set used for fine-tuning is smaller than the training sample set used for training. For instance, fine-tuning training is typically performed on a single epoch because experiments have shown that one epoch can significantly improve relevance and search efficiency.

[0131] In some implementations, the network structure vector space is a pre-trained network structure vector space. For example, a dataset for a preset task is obtained, and based on a random training strategy, the global network structure corresponding to the network structure vector space is pre-trained using the dataset.

[0132] For example, using a large dataset corresponding to a pre-prepared task, such as ImageNet for classification or COCO for detection, a graph can be trained on the prepared open-source data using a random strategy (the random strategy ensures unbiased graph training), resulting in a pre-trained network structure vector space. When determining the network structure vector space corresponding to the business requirement, the pre-trained network structure vector space based on the task type of the business requirement can be determined, saving the step of training the graph and significantly improving search efficiency.

[0133] By using a pre-trained network structure vector space, the search process can utilize only one pre-trained network structure vector space for the same type of task, eliminating the need to train the network structure vector space from scratch each time. Taking classification tasks as an example, a graph pre-trained on ImageNet can be used as a pre-trained graph to implement most classification tasks. In actual classification scenarios, it is only necessary to search for the network structure on the pre-trained graph and then fine-tune and test the valid reward on the scenario data.

[0134] Please refer to the above embodiments. Figure 8 , Figure 8This is a schematic block diagram of the electronic device 600 provided in the embodiments of this application.

[0135] For example, electronic devices may include terminal devices or servers; wherein, terminal devices may be electronic devices such as mobile phones, tablets, laptops, desktop computers, personal digital assistants and wearable devices; servers may be standalone servers or server clusters.

[0136] The electronic device 600 includes a processor 601 and a memory 602.

[0137] For example, processor 601 and memory 602 are connected via bus 603, such as an I2C (Inter-integrated Circuit) bus.

[0138] Specifically, the processor 601 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0139] Specifically, the memory 602 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0140] The processor 601 is used to run a computer program stored in the memory 602, and implements the aforementioned AI model generation method when executing the computer program.

[0141] For example, the processor 601 is configured to run a computer program stored in the memory 602, and when executing the computer program, perform the following steps:

[0142] Obtain the business requirements corresponding to the target AI model;

[0143] Determine the model generation mode, wherein the model generation mode includes at least one of a first generation mode, a second generation mode, and a third generation mode;

[0144] If the model generation mode is determined to be the first generation mode, the target model and target hyperparameter configuration information are determined in the basic database according to the business requirements.

[0145] If the model generation mode is determined to be the second generation mode, the target model and the target hyperparameter configuration information are determined in the pre-trained model according to the business requirements and the preset expert experience logic.

[0146] If the model generation mode is determined to be the third generation mode, the target model is determined in the network structure vector space according to the business requirements and network structure search logic, and the target hyperparameter configuration information is determined according to the business requirements and hyperparameter search logic.

[0147] Configure the target model according to the target hyperparameter configuration information, and train the target model according to the training data to generate the target AI model.

[0148] The specific principles and implementation methods of the electronic devices provided in this application are similar to the AI ​​model generation methods in the foregoing embodiments, and will not be repeated here.

[0149] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the AI ​​model generation method provided in the above embodiments.

[0150] The computer-readable storage medium can be an internal storage unit of the electronic device described in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device.

[0151] It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0152] It should also be understood that the term “and / or” as used in this application and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An AI model generation method, characterized in that, include: Obtain the business requirements corresponding to the target AI model; Determine the model generation mode, wherein the model generation mode includes at least one of a first generation mode, a second generation mode, and a third generation mode; If the model generation mode is determined to be the first generation mode, the target model and target hyperparameter configuration information are determined in the basic database according to the business requirements. If the model generation mode is determined to be the second generation mode, the target model and the target hyperparameter configuration information are determined in the pre-trained model according to the business requirements and the preset expert experience logic. If the model generation mode is determined to be the third generation mode, the target model is determined in the network structure vector space according to the business requirements and network structure search logic, and the target hyperparameter configuration information is determined according to the business requirements and hyperparameter search logic. Configure the target model according to the target hyperparameter configuration information, and train the target model according to the training data to generate the target AI model; The AI ​​model generation method is used in an AutoML system, which includes a directed acyclic graph. The directed acyclic graph includes the identifiers of multiple AI models in the basic database and / or the identifiers of the pre-trained models and / or the historical target models determined under the third generation mode. The step of determining the target model in the network structure vector space based on the business requirements and network structure search logic includes: determining a search initialization state based on the target model corresponding to the first generation mode or the pre-trained model corresponding to the second generation mode; determining the network structure vector space corresponding to the business requirements; searching for a network structure in the network structure vector space based on a randomly initialized search control network and the search initialization state; determining the weight parameters of the network structure based on the shared weights corresponding to the network structure vector space to obtain a weight model; fine-tuning the weight model to obtain a fine-tuned model; and determining the target model from the fine-tuned model obtained by fine-tuning the weight model.

2. The AI ​​model generation method according to claim 1, characterized in that, The business requirements include at least one of the following: task type, terminal type, application scenario, and computing power requirements.

3. The AI ​​model generation method according to claim 1, characterized in that, The method further includes: Based on the business requirements, a data augmentation strategy for the training data is determined, and the data augmentation strategy includes at least one of the following: data cleaning, data preprocessing, and data augmentation. The training data is augmented based on the data augmentation processing unit in the base database that corresponds to the data augmentation strategy.

4. The AI ​​model generation method according to any one of claims 1-3, characterized in that, The step of configuring the target model according to the target hyperparameter configuration information and training the target model according to the training data to generate the target AI model includes: Workflow paths are determined in the directed acyclic graph, which is used to describe the automated workflow of the AutoML system. The workflow path includes the identifier of the target model, the identifier of the target hyperparameter configuration information, the identifier of the training strategy, and the paths between the identifier of the target model and the identifier of the target hyperparameter configuration information, and between the identifier of the target hyperparameter configuration information and the identifier of the training strategy. Based on the workflow path, the target model is trained using the training data to generate the target AI model.

5. The AI ​​model generation method according to claim 4, characterized in that, The step of determining the target model and target hyperparameter configuration information in the basic database according to the business requirements includes: Based on the business requirements, at least one AI model identifier is determined in the directed acyclic graph as the identifier of the target model, and at least one hyperparameter configuration information identifier is determined as the identifier of the target hyperparameter configuration information.

6. The AI ​​model generation method according to claim 4, characterized in that, When determining the target model in the pre-trained model based on the business requirements and preset expert experience logic, and when determining the target hyperparameter configuration information, the method further includes: The pre-trained model identified as the target model is stored in the base database, and an identifier for the pre-trained model is added to the directed acyclic graph; and the identified target hyperparameter configuration information is stored in the base database, and an identifier for the identified target hyperparameter configuration information is added to the directed acyclic graph. When determining the target model in the network structure vector space based on the business requirements and network structure search logic, and determining the target hyperparameter configuration information based on the business requirements and hyperparameter search logic, the method further includes: The determined target model is stored in the base database, and the identifier of the target model is added to the directed acyclic graph; and the determined target hyperparameter configuration information is stored in the base database, and the identifier of the determined target hyperparameter configuration information is added to the directed acyclic graph.

7. The AI ​​model generation method according to any one of claims 1-3, characterized in that, The method further includes: Deploy the target AI model to the target device and / or publish the target AI model to a model trading platform.

8. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the AI ​​model generation method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the AI ​​model generation method as described in any one of claims 1-7.

Images