Model obtaining method, electronic device, and storage medium

Abstract

The application provides a model obtaining method, an electronic device and a storage medium. The model obtaining method comprises the following steps: obtaining first information corresponding to hardware structures of a plurality of deployment terminals and model unit information of a plurality of network structure units used for constructing a model; when an adaptation instruction is received, each first information is adaptively associated with at least one model unit information according to the adaptation instruction to obtain adaptation information; obtaining model requirement information of a target model, and obtaining target first information of a target deployment terminal according to the model requirement information, the target deployment terminal being used for deploying the target model; determining target model unit information corresponding to a plurality of groups of target network structure units that can be used for constructing the target model according to the target first information and the adaptation information; constructing an initial model according to the target model unit information and the model requirement, and screening the target model from the initial model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model acquisition method, electronic device, and storage medium. Background Technology

[0002] Automated machine learning typically utilizes model search methods during model building, training, and evaluation to identify models that can be deployed on corresponding terminal devices, thereby enabling automatic optimization of model structure and parameters. Since there are various types of terminal devices for model deployment, and different terminal devices correspond to different hardware types—such as different hardware manufacturers, hardware platforms, and chip IPs—the optimal model structure and size for each terminal device will vary.

[0003] Existing model search methods involve selecting some models in the search space for training and obtaining the target model based on the training results. However, the target model obtained through this search method may not fully utilize the hardware energy efficiency of the terminal device, resulting in a poor user experience. Summary of the Invention

[0004] This application provides a model acquisition method, electronic device, and storage medium, which aim to acquire the optimal adapted model for the deployment terminal, so as to give full play to the hardware efficiency of the deployment terminal and improve the user experience.

[0005] In a first aspect, embodiments of this application provide a model acquisition method, including:

[0006] Obtain first information corresponding to the hardware structure of various deployment terminals, and model unit information of various network structure units used to build the model;

[0007] When an adaptation instruction is received, each piece of the first information is adapted and associated with at least one piece of the model unit information according to the adaptation instruction to obtain adaptation information;

[0008] Obtain the model requirement information of the target model, and obtain the target first information of the target deployment terminal based on the model requirement information, wherein the target deployment terminal is used to deploy the target model;

[0009] Based on the first target information and the adaptation information, target model unit information corresponding to multiple sets of target network structure units that can be used to construct the target model is determined;

[0010] An initial model is constructed based on the target model unit information and the model requirements, and the target model is selected from the initial model.

[0011] In this embodiment, adaptation information is obtained by matching the first information corresponding to the hardware structure of various deployment terminals with the model unit information corresponding to the corresponding network structure unit adapted to that hardware structure. When receiving the user's model requirement information, the target first information corresponding to the hardware structure of the target deployment terminal is parsed according to the model requirement information. Then, the target network structure unit information corresponding to the target first information is adapted using the adaptation information. Based on the target network structure unit information, the target network structure unit adapted to the hardware structure of the target deployment terminal is obtained. An initial model can be constructed based on the target network structure unit, and a target model that meets the preset requirements can be selected from the initial model. Based on the adaptation of the target network structure unit of the constructed model with the hardware structure of the target deployment terminal, the hardware energy efficiency of the hardware structure of the target deployment terminal can be fully utilized, thereby ensuring that the final target model can also fully utilize the hardware energy efficiency of the hardware structure of the target deployment terminal, and thus improving the running experience of the target model on the target deployment terminal.

[0012] In some implementations, obtaining the first information corresponding to the hardware structure of various deployment terminals includes:

[0013] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0014] The first information corresponding to the hardware structure is determined based on the second information.

[0015] In this embodiment, the second information includes, but is not limited to, information disclosed by the hardware structure that can be obtained from the network, such as the official website information of the hardware structure. The first information corresponding to the hardware structure of the target deployment terminal is comprehensively filtered by the information of the hardware structure of the corresponding deployment terminal disclosed by the network, thereby ensuring the accuracy and comprehensiveness of the obtained first information.

[0016] In some implementations, obtaining the first information corresponding to the hardware structure of various deployment terminals includes:

[0017] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0018] Based on the second information, obtain the corresponding testing tool for the hardware structure;

[0019] According to the test instructions, the corresponding hardware structure is tested using the test tool and the hardware test results are obtained.

[0020] The first information of the corresponding hardware structure is determined based on the hardware test results.

[0021] In this embodiment, by acquiring the corresponding hardware structure testing tool and using the testing tool to test the hardware structure, the required hardware information of the hardware structure is obtained, ensuring that the obtained first information has high credibility and accuracy.

[0022] In some embodiments, the method further includes, prior to receiving the adaptation instruction:

[0023] The hardware architecture characteristics of the hardware structure are determined based on the first information;

[0024] The hardware architecture features and the model unit information of the network structure unit are displayed on the preset display interface;

[0025] The adaptation instruction is generated by triggering the user's response to the first information and the model unit information.

[0026] In this embodiment, the user identifies the first information of the corresponding hardware structure through experience, and determines the model unit information of the network structure unit that is adapted to the hardware structure through the characteristics of the hardware architecture. Thus, the user obtains the corresponding first information and the model unit information that is adapted to the first information. The user associates the first information and the corresponding model unit information to generate the corresponding adaptation instructions, so that the adaptation of the first information and the model unit information is more accurate.

[0027] In some embodiments, the method further includes, prior to receiving the adaptation instruction:

[0028] The test network structure unit is obtained from the preset database, and multiple test models are constructed using the test network structure unit. The test model unit information of the test network structure unit that constitutes each test model is recorded.

[0029] The test model is run on a preset test terminal, and the test data of the test model and the test hardware information corresponding to the test terminal are recorded.

[0030] The compatibility of the test hardware information and the test model unit information is determined based on the running test data, and the adaptation instruction is generated based on the compatibility.

[0031] In this embodiment, the hardware structure of the test terminal is tested by constructing multiple test models based on the test network structure units. The matching degree between the test network structure units corresponding to the test models and the test hardware information corresponding to the test terminal is obtained based on the test results, thereby ensuring the accuracy of the adaptation instruction generation.

[0032] In some implementations, constructing an initial model based on the target model unit information and the model requirements includes:

[0033] The target model units that match the target model unit information are selected from the preset search space;

[0034] The initial network architecture of the model is constructed using the target model unit, and the relevant parameters of the initial network architecture are configured to obtain the initial model.

[0035] In this embodiment, the search space stores network structural units for building the model and defines various operations at the operation layer between every two nodes, i.e., two network structural units, in the model. By searching for network structural units in the search space, the initial model can be built quickly and effectively.

[0036] In some implementations, the step of filtering the target model from the initial model includes:

[0037] The initial model is trained using a preset training dataset, and the performance of each initial model is tested to obtain the performance test results of each initial model.

[0038] Based on the performance test results, target models that meet the preset requirements are selected from the initial models.

[0039] In this embodiment, the initial model whose performance test results meet the preset requirements is used as the target model to ensure that the obtained target model has good model performance.

[0040] In some embodiments, the method further includes:

[0041] If a target model that meets the preset requirements cannot be selected from the initial model, the steps of constructing the initial model based on the target model unit information and the model requirements, and selecting the target model from the initial model are repeated until the target model is obtained.

[0042] In this embodiment, by performing model building and testing multiple times, the target model obtained is ensured to meet the preset requirements, and the hardware efficiency of the target deployment terminal is fully utilized, thereby improving the running experience of the target model on the target deployment terminal.

[0043] Secondly, embodiments of this application provide an electronic device, which includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the model acquisition method in any embodiment of this application specification.

[0044] Thirdly, embodiments of this application provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the model acquisition method in any embodiment of this application.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a model acquisition method provided in an embodiment of this application;

[0048] Figure 2 This is a schematic diagram illustrating an application scenario of the model acquisition method provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram illustrating the mapping relationship between hardware structure and network structure units in the embodiments of this application;

[0050] Figures 4A-4B This is a schematic diagram of the neural network structure search process provided in an embodiment of this application;

[0051] Figure 4C This is a schematic diagram of the search results for the neural network structure provided in the embodiments of this application;

[0052] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specific Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0054] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0055] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0056] Machine learning is widely used in various fields; however, building machine learning models requires highly skilled machine learning experts who manually design and debug them, resulting in high human and time costs and increasing product iteration cycles. To make machine learning easier to apply, reduce the required expertise, and improve model performance, automated machine learning has emerged.

[0057] Automated machine learning (AutoML) provides a complete automated solution for all processes of machine learning, including data cleaning, feature engineering, model building, model training, and evaluation. It trades computing power for manpower and time, reducing reliance on machine learning engineers.

[0058] AutoML typically utilizes model search methods during model building, training, and evaluation to obtain models that can be deployed on corresponding terminal devices, thereby enabling automatic optimization of model structure and parameters.

[0059] The terminal devices deployed based on the model are of various types, and different terminal devices correspond to different hardware types. For example, different terminal devices correspond to different hardware manufacturers, different hardware platforms, and different chip running IPs, resulting in different model structures and model sizes corresponding to the optimal model adapted to different terminal devices.

[0060] Existing model search methods involve selecting some models in the search space for training and obtaining the target model based on the training results. However, the target model obtained through this search method may not fully utilize the hardware energy efficiency of the terminal device, resulting in a poor user experience.

[0061] To address the aforementioned technical problems, this application provides a model acquisition method, an electronic device, and a storage medium, aiming to acquire the optimal adapted model for the deployment terminal, thereby fully utilizing the hardware efficiency of the deployment terminal and improving the user experience. The following detailed description, in conjunction with the accompanying drawings, illustrates some embodiments of this application. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0062] Please see Figure 1 , Figure 1 This is a flowchart illustrating a model acquisition method provided in an embodiment of this application.

[0063] The model acquisition method can be applied to electronic devices, such as terminal devices, servers, or cloud servers. Terminal devices can be mobile phones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices, etc.; servers can be standalone servers or server clusters.

[0064] This application uses the application of the model acquisition method to a server as an example for illustration.

[0065] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the application of the model acquisition method on a server.

[0066] Server 40 runs a server-side component of a model training platform, and terminal device 20 runs a client-side component of the same platform. When a user wants to obtain certain target models, they can input the corresponding model requirement information through the user interface of the client running on terminal device 20 via an input device connected to terminal device 20, and then send the requirement information to the server-side component of server 40. The server-side component of server 40 parses the user requirement information, trains the target model based on the requirement information, and sends the target model to the corresponding deployment terminal or returns it to the corresponding terminal device. The input device includes, but is not limited to, a keyboard, mouse, and touch screen.

[0067] Specifically, the requirement information includes at least one of the following: task type, terminal type, application scenario, and computing power requirements. The task type indicates the practical scenario for which the user needs the target model, such as the tasks the target model needs to handle, such as classification, detection, video processing, and natural language processing. The terminal type indicates the deployment environment corresponding to the terminal deploying the target model, such as the model number, processor type, and model of the terminal. The processor type of the terminal can include CPU (Central Processing Unit) and / or GPU (Graphics Processing Unit). Application scenarios include, but are not limited to, the following: few-shot detection, small object detection, and imbalanced sample detection.

[0068] For example, taking a classification scenario running on CPU and GPU as an example, if the user chooses to use the classification scenario on CPU and there is no computing power constraint, then the model is constructed by sampling CPU-friendly operators (such as depthwise-separatable operators) or network structure units, and the model is trained to obtain the target model.

[0069] If the user chooses to use GPU for the classification scenario and there are no computing power constraints, then the model is constructed by sampling GPU-friendly operators (e.g., conv33 operator) or network structure units, and the model is trained to obtain the target model.

[0070] In the process of model building, the relevant operations between network structure units of the model can be obtained through network structure search and the relevant parameters of the model can be determined. Then, the corresponding training data can be sampled and trained through the preset training strategy to obtain the target model, and the target model can be sent to the target terminal for deployment.

[0071] like Figure 1 As shown, the model acquisition method in this application embodiment includes steps S11 to S15.

[0072] Step S11: Obtain the first information corresponding to the hardware structure of various deployment terminals, and the model unit information of various network structure units used to build the model.

[0073] The deployment terminal can be a portable terminal such as a mobile phone, tablet, or laptop, or a non-portable terminal such as a desktop computer or server. The hardware structure of the deployment terminal includes, but is not limited to, GPU (Graphics Processing Unit), CPU (Central Processing Unit), DSP (Digital Signal Processor), and NPU (neural-network processing unit).

[0074] The first piece of information includes the hardware information corresponding to the hardware structure of the deployed terminal. Different hardware structures correspond to different hardware architectures, and the corresponding network structure units that can fully utilize the performance of that hardware structure are also different. For example, hardware information includes hardware type, hardware manufacturer, hardware clock speed, turbo boost speed, number of cores, number of threads, cache level, memory type, etc.

[0075] Network structural units are the basic units used to build models. A neural network model includes multiple network structural units. When obtaining a target model, the corresponding network structural unit is first selected, and a candidate model is built based on the selected network structural unit. Then, the parameters are configured, and the model with the parameters configured is trained to obtain the corresponding target model.

[0076] In some implementations, obtaining the first information corresponding to the hardware structure of various deployment terminals includes:

[0077] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0078] The first information corresponding to the hardware structure is determined based on the second information.

[0079] For example, the second information is the official website address of the hardware structure or deployment terminal. By accessing the official website address of the corresponding hardware structure, the corresponding target text is identified by OCR (optical character recognition) technology, and keywords are extracted from the target text. The hardware parameters of the corresponding hardware structure are obtained by using the obtained keywords, and the first information corresponding to the hardware structure can be extracted through the hardware parameters.

[0080] For example, in order to obtain the first information corresponding to the hardware structure of a type A deployment terminal, one can go to the official website of the type A deployment terminal, obtain the keywords or key words corresponding to the hardware parameter description of the type A terminal, and use keyword recognition to obtain the hardware information corresponding to the hardware structure from the hardware parameter description, that is, the first information.

[0081] In this embodiment, the second information includes, but is not limited to, information disclosed by the hardware structure that can be obtained from the network, such as the official website information of the hardware structure. The first information corresponding to the hardware structure of the target deployment terminal is comprehensively filtered by the information of the hardware structure of the corresponding deployment terminal disclosed by the network, thereby ensuring the accuracy and comprehensiveness of the obtained first information.

[0082] In some implementations, obtaining the first information corresponding to the hardware structure of various deployment terminals includes:

[0083] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0084] Based on the second information, obtain the corresponding testing tool for the hardware structure;

[0085] According to the test instructions, the corresponding hardware structure is tested using the test tool and the hardware test results are obtained.

[0086] The first information of the corresponding hardware structure is determined based on the hardware test results.

[0087] For example, the second information is the official website address of the hardware structure or deployment terminal. By accessing the official website address of the corresponding hardware structure, the corresponding hardware structure testing tool is downloaded from the official website. The downloaded testing tool is used to test the hardware structure, thereby obtaining the test results corresponding to the hardware structure. Based on the test results, the required hardware information is filtered out, and then the first hardware information is obtained.

[0088] For example, in a type A deployment terminal, the hardware structures affecting model operation include hardware structures A11 and A12. Hardware structures A11 and A12 can be any two of a GPU, CPU, DSP, or NPU. The user downloads a testing tool E1 for hardware structure A11 from its official website and a testing tool E2 for hardware structure A12 from its official website. After installing and debugging the hardware testing tools, the user uses test commands to have testing tool T2 test hardware structure A12 and testing tool T1 test hardware structure A11, obtaining hardware test results Z1 for hardware structure A11 and Z2 for hardware structure A12. The hardware information corresponding to the hardware structure, i.e., the first piece of information, can be obtained from the hardware test results Z1 and Z2. These hardware test results include the hardware parameters of the hardware structure, from which the corresponding hardware information can be filtered.

[0089] In this embodiment, by acquiring the corresponding hardware structure testing tool and using the testing tool to test the hardware structure, the required hardware information of the hardware structure is obtained, ensuring that the obtained first information has high credibility and accuracy.

[0090] Step S12: When an adaptation instruction is received, each piece of the first information is adapted and associated with at least one piece of the model unit information according to the adaptation instruction to obtain adaptation information.

[0091] The system detects whether an adaptation instruction has been received. This adaptation instruction is used to associate the first information of each hardware structure with the model unit information corresponding to at least one network structure unit. In other words, the adaptation instruction is used to associate the hardware information corresponding to the hardware structure with one or more model unit information that are adapted to it. Thus, when the hardware information corresponding to the hardware structure is obtained, the system can analyze the model unit information corresponding to the network structure unit that can fully utilize the hardware information based on the hardware information.

[0092] For example, through empirical or experimental analysis, it is found that the network structure units adapted to the hardware structure (hardware structure A11, hardware structure A12) of a type A deployment terminal include A1, A2, A3, and A4 network structure units. The network structure units adapted to the hardware structure (hardware structure B11, hardware structure B12) of a type B deployment terminal include B1, B2, B3, and B4 network structure units. The network structure units adapted to the hardware structure (hardware structure C11, hardware structure C12) of a type C deployment terminal include C1, C2, C3, and C4 network structure units. If a model composed of network structure units of types B1 to B4 or C1 to C4 is run on a type A deployment terminal, it may not fully utilize the hardware structure performance of the type A deployment terminal, resulting in a poor user experience.

[0093] The adaptation information is obtained by associating the hardware information corresponding to hardware structures A11 and A12 of deployment terminals of type A with the model unit information corresponding to network structure units of types A1, A2, A3, and A4; associating the network structure units adapted to the hardware structures (hardware structures B11 and B12) of deployment terminals of type B with the model unit information corresponding to network structure units of types B1, B2, B3, and B4; and associating the network structure units adapted to the hardware structures (hardware structures C11 and C12) of deployment terminals of type C with the model unit information corresponding to network structure units of types C1, C2, C3, and C4.

[0094] In some implementations, the adaptation instructions are generated as follows:

[0095] The hardware architecture characteristics of the hardware structure are determined based on the first information;

[0096] The hardware architecture features and the model unit information of the network structure unit are displayed on the preset display interface;

[0097] The adaptation instruction is generated by triggering the user's response to the first information and the model unit information.

[0098] For example, the server extracts the hardware architecture characteristics of the hardware structure based on the first information corresponding to the hardware structure, and displays the hardware architecture characteristics corresponding to the hardware structure and the model unit information corresponding to the network structure unit searched based on the hardware structure characteristics on the corresponding display interface. The display interface displays an association option, which is used to associate the first information and the model unit information.

[0099] Users analyze the model unit information of the network structure unit that is compatible with the hardware structure through experience, associate the first information corresponding to the hardware structure and the model unit information that is compatible with the first information through the association option, and generate the adaptation instruction according to the association prompt information.

[0100] In this embodiment, the user identifies the first information of the corresponding hardware structure through experience, and determines the model unit information of the network structure unit that is adapted to the hardware structure through the characteristics of the hardware architecture. Thus, the user obtains the corresponding first information and the model unit information that is adapted to the first information. The user associates the first information and the corresponding model unit information to generate the corresponding adaptation instructions, so that the adaptation of the first information and the model unit information is more accurate.

[0101] In some implementations, the adaptation instructions are generated as follows:

[0102] The test network structure unit is obtained from the preset database, and multiple test models are constructed using the test network structure unit. The test model unit information of the test network structure unit that constitutes each test model is recorded.

[0103] The test model is run on a preset test terminal, and the test data of the test model and the test hardware information corresponding to the test terminal are recorded.

[0104] The compatibility of the test hardware information and the test model unit information is determined based on the running test data, and the adaptation instruction is generated based on the compatibility.

[0105] For example, given the hardware structure of the test terminal, the test network structure unit is used to build the test model. Different test models are installed and run on the same test terminal each time, and the running test data of each test model on the corresponding test terminal is recorded. By analyzing the running test data, the compatibility between the test hardware information corresponding to the test terminal and the test model unit information corresponding to the test network structure unit can be determined. Based on the compatibility, it can be determined whether the corresponding network structure unit can fully utilize the hardware energy efficiency of the corresponding hardware structure. Thus, an adaptation instruction is generated based on the adaptation information matching at least one network structure unit adapted to the hardware structure.

[0106] For example, if testing the model reveals that network structure units of types A1, A2, A3, and A4 can fully utilize the performance of hardware structure A11 when running on hardware structure A11, then the hardware information corresponding to hardware structure A11 is associated with the model unit information corresponding to the network structure units of types A1, A2, A3, and A4, and adaptation instructions are generated based on this association information.

[0107] In this embodiment, the hardware structure of the test terminal is tested by constructing multiple test models based on the test network structure units. The matching degree between the test network structure units corresponding to the test models and the test hardware information corresponding to the test terminal is obtained based on the test results, thereby ensuring the accuracy of the adaptation instruction generation.

[0108] Step S13: Obtain the model requirement information of the target model, and obtain the target first information of the target deployment terminal based on the model requirement information. The target deployment terminal is used to deploy the target model.

[0109] When receiving model requirement information from a user through the client, the system parses the model requirement information to obtain the first information corresponding to the hardware structure of the target deployment terminal to which the model is to be deployed.

[0110] For example, model requirements information includes the terminal type of the target deployment terminal, the application scenario of the target model, and its computing power requirements. Based on the requirements information, the server can obtain the hardware structure information of the target deployment terminal, i.e., the primary target information.

[0111] Step S14: Based on the first target information and the adaptation information, determine the target model unit information corresponding to multiple sets of target network structure units that can be used to construct the target model.

[0112] Step S15: Construct an initial model based on the target model unit information and the model requirements, and select the target model from the initial model.

[0113] After determining the hardware structure and adaptation information of the target deployment terminal, the corresponding target model unit information can be determined. Based on the target model unit information, the target network structure unit used to build the target model can be determined, and multiple models can be built using the target network structure unit.

[0114] For example, by using the target first information and adaptation information corresponding to the target hardware structure of the target deployment terminal, it can be known that there are N target network structure units that are adapted to the target hardware structure, and that the N target network structure units can better utilize the hardware energy efficiency of the target hardware structure.

[0115] After determining the target network structure unit, information such as the application scenario and computing power requirements of the model are obtained according to the model requirements. Then, the corresponding initial model is constructed using the obtained target network structure unit, and the initial model is trained using the corresponding sample data. After the initial model is trained, the target model that meets the preset requirements is selected from the initial model.

[0116] This application adapts the hardware information corresponding to the hardware structure of various deployment terminals and the model unit information corresponding to the network structure unit adapted to that hardware structure to obtain adaptation information. When receiving the user's model requirement information, it parses the target hardware information corresponding to the hardware structure of the target deployment terminal based on the model requirement information, and then adapts the corresponding target network structure unit information to the target hardware information using the adaptation information. Based on the target network structure unit information, it obtains the target network structure unit adapted to the hardware structure of the target deployment terminal. An initial model can then be constructed based on the target network structure unit, and a target model that meets the preset requirements can be selected from the initial model. Based on the adaptation of the target network structure unit of the constructed model to the hardware structure of the target deployment terminal, the hardware efficiency of the target deployment terminal's hardware structure can be fully utilized, ensuring that the final obtained target model can also fully utilize the hardware efficiency of the target deployment terminal's hardware structure, thereby improving the running experience of the target model on the target deployment terminal.

[0117] In some implementations, constructing an initial model based on the target model unit information and the model requirements includes:

[0118] The target model units that match the target model unit information are selected from the preset search space;

[0119] The initial network architecture of the model is constructed using the target model unit, and the relevant parameters of the initial network architecture are configured to obtain the initial model.

[0120] The step of selecting the target model from the initial model includes:

[0121] The initial model is trained using a preset training dataset, and the performance of each initial model is tested to obtain the performance test results of each initial model.

[0122] Based on the performance test results, target models that meet the preset requirements are selected from the initial models.

[0123] For example, after determining the requirements of the target network model, a search space is determined for the network structure search. The search space defines the scope of the network structure search, and this search space can be a pre-constructed search space containing various target network structure units. Here, a network structure unit can be the basic unit for building a neural network model, specifically a single network layer, such as a single convolutional layer or a fully connected layer; or it can be a structure unit formed by combining multiple network layers, such as a block structure formed by combining convolutional layers, batch normalization layers, and nonlinear layers (such as ReLU), without limitation here.

[0124] Each target network structure unit is assigned a code. The corresponding target network structure unit is searched from the search space using the corresponding code, and the initial network architecture is constructed using the target network structure unit.

[0125] like Figure 4A As shown, the initial network architecture includes 4 nodes, namely nodes 0, 1, 2 and 3, but the operations between the nodes are unknown, which is represented by the question mark "?" in the figure.

[0126] Nodes in a neural network model can be understood as feature layers within the model. For example, in Figure 4A The initial network architecture constructed includes one input feature layer, two intermediate feature layers, and one output feature layer. Node 0 represents the input feature layer, nodes 1 and 2 represent the intermediate feature layers, and node 3 represents the output feature layer. It should be understood that node 0 includes the feature data (feature vector or feature matrix, similar to below) from the input feature layer, node 1 includes the feature data from the first intermediate feature layer, node 2 includes the feature data from the second intermediate feature layer, and node 3 includes the feature data from the output feature layer. An operation between two nodes refers to the operation required to exchange feature data from one node for feature data from another node. The operations mentioned in this embodiment can be convolution, pooling, fully connected operations, or other neural network operations. The operations between two nodes can be considered as the operation layer between these two nodes. Typically, the operation layer between two nodes has multiple searchable operations, i.e., multiple candidate operations. The purpose of network architecture search is to determine an operation at each operation layer.

[0127] Network structure search can determine the operations between nodes 0, 1, 2, and 3 from the search space. Different combinations of operations between nodes 0, 1, 2, and 3 correspond to different network architectures. Therefore, multiple initial network architectures can be obtained through network structure search.

[0128] like Figure 4B As illustrated, the search space, exemplarily, defines multiple operations on the operation layer between every two nodes in the neural network model. The search space defines three operations for each operation layer, with different dashed lines representing operation 1, operation 2, and operation 3, respectively. For example, operation 1 is a convolution operation, operation 2 is a pooling operation, and operation 3 is a fully connected operation. The purpose of network structure search for an operation layer of the neural network is to select one operation from these three operations as the operation for that operation layer.

[0129] Using a pre-defined optimization algorithm, such as a gradient-based optimization algorithm, a network structure search is performed on the neural network model in the search space. This allows for the configuration of structural parameters for various operations at each operational layer of the neural network model, resulting in optimized structural parameters. These optimized structural parameters are then used to configure the initial network architecture and determine the corresponding initial model. Figure 4C As shown, the final initial model is obtained through network structure search.

[0130] At least one initial model is obtained through the search space. The initial model is trained using a pre-set training dataset, and the performance of each initial model is tested. The performance test results of each initial model are obtained, and the initial model with the best performance is selected as the target model.

[0131] In this embodiment, the search space stores network structural units for building the model and defines various operations at the operation layer between every two nodes, i.e., two network structural units, in the model. By searching for network structural units in the search space, the initial model can be built quickly and effectively.

[0132] In some embodiments, the method further includes:

[0133] If a target model that meets the preset requirements cannot be selected from the initial model, the steps of constructing the initial model based on the target model unit information and the model requirements, and selecting the target model from the initial model are repeated until the target model is obtained.

[0134] In one application scenario, we will take the hardware information corresponding to the hardware structure as an example for explanation.

[0135] The system acquires hardware information corresponding to the hardware structures of various deployment terminals, as well as model unit information for various network structure units used to build the model. For example, it acquires hardware information A corresponding to the hardware structure of a type A deployment terminal, hardware information B corresponding to the hardware structure of a type B deployment terminal, and hardware information C corresponding to the hardware structure of a type C deployment terminal. It also acquires model unit information for various network structure units used to build the model, such as type A1 network structure units, type A2 network structure units, type A3 network structure units, and type A4 network structure units.

[0136] Upon receiving an adaptation instruction, each piece of hardware information is associated with at least one model unit information to obtain adaptation information. This adaptation instruction includes adaptation guidelines between the hardware information and the model units. In other words, when the model unit runs on the adapted hardware structure, the energy efficiency of that hardware structure can be fully utilized. For example, the network structure units adapted to the hardware structure (hardware structure A11, hardware structure A12) of a Class A deployment terminal include Class A1, Class A2, Class A3, and Class A4 network structure units. If a model composed of non-Class A network structure units is run on a Class A deployment terminal, the hardware structure performance of the Class A deployment terminal may not be fully utilized, resulting in a poor user experience.

[0137] Then, by associating the hardware information corresponding to the hardware structure A11 and hardware structure A12 of the deployment terminal corresponding to the adaptation instruction A with the model unit information corresponding to the network structure units of type A1, type A2, type A3 and type A4, the information table corresponding to the adaptation information is obtained.

[0138] Upon receiving the model requirement information for acquiring the target model, the system acquires the target hardware information of the target deployment terminal based on the model requirement information. Based on the target hardware information and adaptation information, it determines the target model unit information corresponding to multiple sets of target network structure units that can be used to build the target model. The system then constructs an initial model based on the target model unit information and model requirements, and selects target models from the initial model.

[0139] For example, by using the target hardware information and adaptation information corresponding to the target hardware structure of the target deployment terminal, it can be known that there are N target network structure units that are compatible with the target hardware structure, and that these N target network structure units can better utilize the hardware energy efficiency of the target hardware structure.

[0140] After determining the target network structure unit, information such as the application scenario and computing power requirements of the model are obtained according to the model requirements. Then, the corresponding initial model is constructed using the obtained target network structure unit, and the initial model is trained using the corresponding sample data. After the initial model is trained, the target model that meets the preset requirements is selected from the initial model.

[0141] Please see Figure 5 , Figure 5 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device includes, but is not limited to, a server.

[0142] like Figure 5 As shown, the electronic device 30 includes a processor 301, a memory 302, and a network interface 303. The processor 301 and the memory 302 are connected by a bus and can communicate with external devices through the network interface 303 or a wireless communication module provided in the electronic device 30. The bus is, for example, an I2C (Inter-integrated Circuit) bus.

[0143] Specifically, the processor 301 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0144] The memory 302 can be a Flash chip, a read-only memory (ROM), a disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0145] The processor 301 is used to run a computer program stored in the memory, and when executing the computer program, implements any of the model acquisition methods provided in the embodiments of this application.

[0146] In some implementations, processor 301 is used for:

[0147] Obtain first information corresponding to the hardware structure of various deployment terminals, and model unit information of various network structure units used to build the model;

[0148] When an adaptation instruction is received, each piece of the first information is adapted and associated with at least one piece of the model unit information according to the adaptation instruction to obtain adaptation information;

[0149] Obtain the model requirement information of the target model, and obtain the target first information of the target deployment terminal based on the model requirement information, wherein the target deployment terminal is used to deploy the target model;

[0150] Based on the first target information and the adaptation information, target model unit information corresponding to multiple sets of target network structure units that can be used to construct the target model is determined;

[0151] An initial model is constructed based on the target model unit information and the model requirements, and the target model is selected from the initial model.

[0152] In some implementations, when the processor 301 acquires the first information corresponding to the hardware structure of various deployed terminals, it includes:

[0153] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0154] The first information corresponding to the hardware structure is determined based on the second information.

[0155] In some implementations, when the processor 301 acquires the first information corresponding to the hardware structure of various deployed terminals, it includes:

[0156] Determine the second information corresponding to the hardware structure of various deployment terminals;

[0157] Based on the second information, obtain the corresponding testing tool for the hardware structure;

[0158] According to the test instructions, the corresponding hardware structure is tested using the test tool and the hardware test results are obtained.

[0159] The first information of the corresponding hardware structure is determined based on the hardware test results.

[0160] In some implementations, the processor 301 is also used for:

[0161] The hardware architecture characteristics of the hardware structure are determined based on the first information;

[0162] The hardware architecture features and the model unit information of the network structure unit are displayed on the preset display interface;

[0163] The adaptation instruction is generated by triggering the user's response to the first information and the model unit information.

[0164] In some implementations, the processor 301 is also used for:

[0165] The test network structure unit is obtained from the preset database, and multiple test models are constructed using the test network structure unit. The test model unit information of the test network structure unit that constitutes each test model is recorded.

[0166] The test model is run on a preset test terminal, and the test data of the test model and the test hardware information corresponding to the test terminal are recorded.

[0167] The compatibility of the test hardware information and the test model unit information is determined based on the running test data, and the adaptation instruction is generated based on the compatibility.

[0168] In some implementations, when the processor 301 constructs an initial model based on the target model unit information and the model requirements, it includes:

[0169] The target model units that match the target model unit information are selected from the preset search space;

[0170] The initial network architecture of the model is constructed using the target model unit, and the relevant parameters of the initial network architecture are configured to obtain the initial model.

[0171] In some implementations, when the processor 301 filters the target model from the initial model, it includes:

[0172] The initial model is trained using a preset training dataset, and the performance of each initial model is tested to obtain the performance test results of each initial model.

[0173] Based on the performance test results, target models that meet the preset requirements are selected from the initial models.

[0174] In some implementations, the processor 301 is also used for:

[0175] If a target model that meets the preset requirements cannot be selected from the initial model, the steps of constructing the initial model based on the target model unit information and the model requirements, and selecting the target model from the initial model are repeated until the target model is obtained.

[0176] This application also provides a storage medium for computer-readable storage, which stores one or more programs that can be executed by one or more processors to implement the steps of any of the model acquisition methods provided in the embodiments of this application.

[0177] The storage medium can be an internal storage unit of the electronic device described in the foregoing embodiments, such as a hard drive or memory of the electronic device. Alternatively, the storage medium can be an external storage device of the electronic device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a Flash Card.

[0178] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, as well as the functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0179] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0180] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be defined by the scope of the claims.

Claims

1. A model acquisition method, characterized in that, include: Obtain first information corresponding to the hardware structure of various deployment terminals, and model unit information of various network structure units used to build the model; When an adaptation instruction is received, each piece of the first information is adapted and associated with at least one piece of the model unit information according to the adaptation instruction to obtain adaptation information; Obtain the model requirement information of the target model, and obtain the target first information of the target deployment terminal based on the model requirement information, wherein the target deployment terminal is used to deploy the target model; Based on the first target information and the adaptation information, target model unit information corresponding to multiple sets of target network structure units that can be used to construct the target model is determined; An initial model is constructed based on the target model unit information and the model requirements, and the target model is selected from the initial model; The method for generating the adaptation instructions includes: The test network structure unit is obtained from the preset database, and multiple test models are constructed using the test network structure unit. The test model unit information of the test network structure unit that constitutes each test model is recorded. The test model is run on a preset test terminal, and the test data of the test model and the test hardware information corresponding to the test terminal are recorded. The compatibility of the test hardware information and the test model unit information is determined based on the running test data, and the adaptation instruction is generated based on the compatibility. Alternatively, the method for generating the adaptation instructions includes: The hardware architecture characteristics of the hardware structure are determined based on the first information; The hardware architecture features and the model unit information of the network structure unit are displayed on the preset display interface; The adaptation instruction is generated by triggering an operation that obtains the user's response to the first information and the model unit information. The first information includes hardware information corresponding to the hardware structure of the deployment terminal. The hardware information includes any one of the following: hardware type, hardware manufacturer, hardware clock speed information, turbo frequency information, number of cores, number of threads, cache level, and memory type. The step of constructing an initial model based on the target model unit information and the model requirements includes: The target model units that match the target model unit information are selected from the preset search space; The initial network architecture of the model is constructed using the target model unit, and the relevant parameters of the initial network architecture are configured to obtain the initial model; The search space stores network structural units for building the model and defines various operations at the operation layer between every two nodes, i.e., two network structural units, in the model. By searching for network structural units within the search space, the initial model can be built quickly and efficiently.

2. The model acquisition method according to claim 1, characterized in that, The first information corresponding to the hardware structure of various deployment terminals includes: Determine the second information corresponding to the hardware structure of various deployment terminals; The first information corresponding to the hardware structure is determined based on the second information.

3. The model acquisition method according to claim 1, characterized in that, The first information corresponding to the hardware structure of various deployment terminals includes: Determine the second information corresponding to the hardware structure of various deployment terminals; Based on the second information, obtain the corresponding testing tool for the hardware structure; According to the test instructions, the corresponding hardware structure is tested using the test tool and the hardware test results are obtained. The first information of the corresponding hardware structure is determined based on the hardware test results.

4. The model acquisition method according to claim 1, characterized in that, The step of filtering the target model from the initial model includes: The initial model is trained using a preset training dataset, and the performance of each initial model is tested to obtain the performance test results of each initial model. Based on the performance test results, target models that meet the preset requirements are selected from the initial models.

5. The model acquisition method according to claim 4, characterized in that, The method further includes: If a target model that meets the preset requirements cannot be selected from the initial model, the steps of constructing the initial model based on the target model unit information and the model requirements, and selecting the target model from the initial model are repeated until the target model is obtained.

6. An electronic device, characterized in that, The electronic device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for enabling communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the model acquisition method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the model acquisition method as described in any one of claims 1-5.

Images