Model generation method and apparatus, and nonvolatile storage medium and electronic device

Abstract

Disclosed in the present application are a model generation method and apparatus, and a nonvolatile storage medium and an electronic device. The method comprises: determining feature attribution magnitudes of operation units in a hypernetwork, wherein the feature attribution magnitudes are used for indicating the importance degrees of the operation units, the hypernetwork comprises a plurality of feature processing cells, each feature processing cell comprises a plurality of nodes, and an edge between any two connected nodes consists of a plurality of operation units; on the basis of the feature attribution magnitudes of the operation units, determining a plurality of target operation units in the hypernetwork; and on the basis of the plurality of target operation units, determining a model architecture of a target neural network model.

Description

Model generation method, device, non-volatile storage medium and electronic device

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2024103799819, filed on March 29, 2024, entitled “Model generation method, device, non-volatile storage medium and electronic device”, the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of deep learning models, and specifically, to a model generation method, device, non-volatile storage medium and electronic device. Background Art

[0004] Related art provides a model retrieval method for constructing a target model by searching for possible structures of the target model in a search space. This method typically uses search algorithms such as evolutionary algorithms and genetic algorithms to find the optimal architecture from a large number of possible structures. However, this method suffers from poor interpretability, making it unclear why the selected architecture is optimal. Furthermore, using search algorithms to retrieve the optimal structure in a feature space consumes significant computing resources and cannot guarantee search results.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] According to a first aspect of an embodiment of the present application, a model generation method is provided, comprising: determining a feature attribution index for each operation unit in a hypernetwork, wherein the feature attribution index is used to indicate the importance of the operation unit, the hypernetwork includes a plurality of feature processing cells, the feature processing cells include a plurality of nodes, and the edge between any two connected nodes is composed of a plurality of operation units; determining a plurality of target operation units in the hypernetwork based on the feature attribution index of each operation unit; and determining a model architecture of a target neural network model based on the plurality of target operation units.

[0007] Optionally, the step of determining the characteristic attribution index of each operation unit in the hypernetwork includes: determining the actual input and reference input of the operation unit, and a preset interpolation function; performing integrated gradient calculation on the operation unit based on the actual input, reference input and preset interpolation function to obtain the characteristic attribution index of the operation unit.

[0008] Optionally, the integrated gradient calculation is performed on the operation unit based on the actual input, the reference input and the preset interpolation function, and the calculation formula for the characteristic attribution index of the operation unit is obtained as follows:

[0009] In the above formula Indicates the characteristic attribution index of the operation unit, Indicates the actual input, represents the reference input, S represents the shape and size of the actual input, α represents the preset interpolation function, γ x (α) indicates that the preset interpolation function is expanded to (x′+α(xx′)), l indicates the lth feature processing cell in the hypernetwork, i indicates the i-th edge in the l-th feature processing cell, j indicates the j-th operation unit in the i-th edge, and k indicates the k-th step of the interpolation calculation.

[0010] Optionally, the structures of the feature extraction cells in the hypernetwork are the same, and the structures include the number of nodes contained in the feature extraction cells, the number and connection mode of the edges between the nodes in the feature extraction cells, and the number and connection mode of the operation units in the edges; the step of determining multiple target operation units in the hypernetwork based on the feature attribution indicators of each operation unit includes: summarizing the feature attribution indicators of each operation unit in the hypernetwork to obtain a first feature attribution indicator set, wherein the dimension of the first feature attribution indicator set is N×P×Q, N represents the number of feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cells, Q represents the number of operation units in the edges, and the first feature attribution indicator set The method includes feature attribution indicators of each operation unit in the hypernetwork; the first feature attribution indicator set is aggregated to obtain a second feature attribution indicator set, wherein the dimension of the second feature attribution indicator set is P×Q, the rows in the second feature attribution indicator set correspond to the edges in the hypernetwork, the columns in the second feature attribution indicator set correspond to the operation units in the hypernetwork, and the edges with the same position in each feature extraction cell in the hypernetwork correspond to the same row in the second feature attribution indicator set, and the operation units with the same position in each edge in the hypernetwork correspond to the same column in the second feature attribution indicator set; the target operation unit is determined in each row of the second feature attribution indicator set to obtain multiple target operation units.

[0011] Optionally, the step of performing aggregation processing on the first feature attribution indicator set to obtain the second feature attribution indicator set includes: determining the intermediate feature matrix corresponding to the operation units with the same position in each feature processing cell, wherein the same position means that the edges corresponding to the operation units have the same sequence number in the feature processing cell, and the sequence number of the operation units in the edges is also the same; determining the values ​​of the corresponding elements in the second feature attribution indicator set based on the intermediate feature matrix, thereby obtaining the second feature attribution indicator set, wherein the edge sequence number corresponding to the intermediate feature matrix is ​​the row sequence number of the corresponding element in the second feature attribution indicator set, and the sequence number of the operation unit corresponding to the intermediate feature matrix is ​​the column sequence number of the corresponding element in the second feature attribution indicator set.

[0012] Optionally, the aggregation function for performing aggregation processing on the first feature attribution indicator set is as follows:

[0013] In the above aggregation function, represents the intermediate feature matrix, represents the feature attribution index in the g-th feature processing cell, It represents the feature attribution index in the hth feature processing cell, i represents the sequence number of the edge, and j represents the sequence number of the operation unit.

[0014] Optionally, the step of determining the value of the corresponding element in the second feature attribution indicator set according to the intermediate feature matrix to obtain the second feature attribution indicator set includes: determining the maximum feature attribution indicator in the intermediate feature matrix as the value of the corresponding element.

[0015] Optionally, the target operation unit is determined in each row of the second feature attribution indicator set, and the step of obtaining multiple target operation units includes: determining the largest feature attribution indicator in each row of the second feature attribution indicator set as the target feature attribution indicator; and determining the target operation unit in each row according to the row and column corresponding to the target feature attribution indicator.

[0016] According to the second aspect of the embodiment of the present application, a model generation device is also provided, including: a first processing module for determining the feature attribution index of each operation unit in a super network, wherein the feature attribution index is used to indicate the importance of the operation unit, the super network includes multiple feature processing cells, the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; a second processing module for determining multiple target operation units in the super network based on the feature attribution index of each operation unit; a third processing module for determining the model architecture of the target neural network model based on the multiple target operation units.

[0017] According to a third aspect of an embodiment of the present application, a non-volatile storage medium is further provided, in which a program is stored. When the program is running, the device where the non-volatile storage medium is located is controlled to execute the model generation method.

[0018] According to a fourth aspect of an embodiment of the present application, an electronic device is further provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the model generation method is executed when the program is run.

[0019] According to a fifth aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements the model generation method when executed by a processor.

[0020] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] FIG1 is a schematic structural diagram of a computer terminal (mobile terminal) provided according to an embodiment of the present application;

[0023] FIG2 is a flow chart of a model generation method according to an embodiment of the present application;

[0024] FIG3 is a schematic diagram of the architecture of a super network provided according to an embodiment of the present application;

[0025] FIG4 is a schematic diagram of a polymerization process according to an embodiment of the present application;

[0026] FIG5 is a schematic structural diagram of a model generation device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0030] IG (Integrated Gradients): IG is a method for explaining the predictions of deep learning models. It measures the importance of each feature to the model prediction by integrating the gradients of the input features. In Integrated Gradients, a path between a reference input and a target input is first determined, and the input is interpolated along this path. Then, the weight of each feature is obtained by calculating the gradient of the model for each interpolation point and multiplying it by the difference of the input features. Finally, these weights are integrated to obtain the contribution of each feature to the output. By applying Integrated Gradients, we can understand the degree to which the deep learning model depends on the input features when making predictions. It provides an interpretable way to explain the model's decisions and helps reveal which features have a positive or negative impact on the formation of the model's prediction results. This explanatory ability is very valuable for debugging models, detecting biases, verifying the reliability of models, and enhancing user trust in model predictions. The general formal formula of the IG algorithm is as follows:

[0031] In the above formula, x i Represents the actual input feature, x′ i is the reference input feature, α represents the interpolation function, and F represents the neural network.

[0032] NAS (Neural Architecture Search): NAS is a method for automatically searching for neural network architectures. In traditional neural network design, architectures are often manually designed and continuously optimized based on trial and error and experience. NAS architecture search aims to automate this process through algorithms. It uses search algorithms (such as evolutionary algorithms, genetic algorithms, reinforcement learning, etc.) to find the optimal architecture from a large space of possible neural network architectures. This search process includes selecting the type and number of layers, connection methods, and hyperparameters to achieve the best performance and efficiency of the network. NAS architecture search can significantly reduce the workload of manually designing network architectures and achieve better performance in various tasks. It can help discover novel network structures and improve the accuracy, robustness, and generalization ability of the model. In addition, NAS can search for specific hardware platforms or resource constraints to generate customized network architectures suitable for specific environments.

[0033] Integrated Gradients is a technique for explaining deep learning model predictions. It emerged to address the need to understand black-box models. Deep learning models have achieved remarkable success in various fields, but their internal mechanisms and decision-making processes are often considered opaque and difficult to understand. This uncertainty limits the models' credibility and scope of application. To overcome this challenge, researchers have begun exploring methods to explain deep learning model predictions and improve their interpretability. Integrated Gradients, as one such method, was proposed in this context. The core idea of ​​Integrated Gradients is to combine gradient computation with numerical integration. By interpolating the input along a path in the input feature space and calculating the gradient at each interpolation point, the contribution of each feature to the model's prediction can be determined. By integrating these weights, the overall importance of each feature relative to the output can be obtained. With the development of this technique, Integrated Gradients has made significant progress in explaining deep learning model predictions. It provides a highly interpretable and intuitive framework to help understand the degree to which a model relies on input features when making decisions. This technique provides greater insight into model behavior, revealing potential biases, anomalies, or errors, and enhancing trust in model predictions. The application of Integrated Gradients allows for a better understanding of the inner workings of deep learning models, facilitating model improvement and optimization. This is particularly important in areas involving sensitive data, medical diagnosis, and financial decision-making, as the transparency and explainability of models directly impact their reliability and acceptability.

[0034] Designing excellent neural network architectures is crucial in the field of deep learning. However, traditional manual design methods require extensive manpower and experience, and struggle to explore the entire space of network architectures. To address this issue, automated neural architecture search (NAS) techniques have been explored. The development of NAS can be traced back to early evolutionary and genetic algorithms. These algorithms employ principles similar to biological evolution, performing random searches and selecting the best candidates within the space of candidate network architectures. However, the efficiency and performance of this approach are limited by the search algorithm and computational resources. With the rise of reinforcement learning, researchers have begun to incorporate it into NAS. Reinforcement learning methods build a proxy model to learn a parameterized policy for network architectures and continuously improve the policy performance through interaction with the environment. This approach offers greater search efficiency and flexibility, enabling NAS to find better solutions in the complex space of network architectures. In addition to evolutionary algorithms and reinforcement learning, several other NAS techniques have emerged. For example, gradient-based methods use gradient information to search within the space of network architectures, such as optimizing architecture parameters in a differentiable hypernetwork. These methods fully utilize gradient information during the design process, offering more efficient and accurate search performance. In addition, some NAS technologies represent neural networks as graphs or search spaces, and use graph search algorithms or optimization algorithms to perform architecture search. This approach guides the search process by defining search operations and constraints to obtain the optimal network architecture that meets design requirements. The development of NAS encompasses a variety of technologies, including evolutionary algorithms, genetic algorithms, reinforcement learning, gradient-based methods, and graph search. These technologies make automated network architecture search possible and provide new possibilities for the development of deep learning. By utilizing these algorithms and methods, researchers can more quickly and efficiently design neural network architectures with excellent performance, promoting the widespread development of deep learning applications.

[0035] However, these neural network architecture search algorithms have the problems of poor interpretability and excessive consumption of computing resources during the search process.

[0036] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0037] According to an embodiment of the present application, a method embodiment of a model generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing the model generation method. As shown in Figure 1, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used in the figure to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in Figure 1, or have a configuration different from that shown in Figure 1.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the model generation method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned model generation method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0043] In the above operating environment, an embodiment of the present application provides a model generation method, as shown in FIG2 , which includes the following steps:

[0044] Step S202: determining a feature attribution index for each operation unit in the hypernetwork, wherein the feature attribution index is used to indicate the importance of the operation unit. The hypernetwork includes multiple feature processing cells, each feature processing cell includes multiple nodes, and an edge between any two connected nodes is composed of multiple operation units.

[0045] Specifically, the architecture of the hypernetwork provided in the embodiment of the present application is shown in Figure 3. The feature processing cells in the hypernetwork include Normal cells (feature extraction cells) and Reduction cells (feature downsampling cells), which are chain-stacked to form our image recognition model.

[0046] In addition, it can be seen from the first part of Figure 3 that the structures of the feature processing cells in the super network are the same, and they all include the same number of input nodes and the same number of inflow nodes, such as the two input nodes and four inflow nodes in Figure 3. And all the subsequent nodes will accept the data feature inflow from the previous nodes. It can be seen from the second part of Figure 3 that the edges between the nodes in the figure are obtained by adding and combining a preset number of operation units (operation). That is to say, the data features flowing into each operation unit in the same edge are the same, and the features processed by each operation unit will form new features and flow into the next node after the addition operation. The search space of the network architecture in the embodiment of the present application is the operation unit in each edge in the super network and the data inflow method between nodes. The map in the third part of Figure 3 is the feature output by each operation unit.

[0047] In the technical solution provided in step S202, the step of determining the feature attribution index of each operation unit in the super network includes: determining the actual input and reference input of the operation unit, as well as a preset interpolation function; performing an integrated gradient calculation on the operation unit based on the actual input, the reference input and the preset interpolation function to obtain the feature attribution index (Attribution Magnitude, abbreviated as Am in the embodiment of this application) of the operation unit.

[0048] As an optional implementation, the integrated gradient calculation is performed on the operation unit based on the actual input, the reference input, and the preset interpolation function to obtain the calculation formula of the characteristic attribution index of the operation unit as follows:

[0049] In the above formula Indicates the characteristic attribution index of the operation unit, Indicates the actual input, represents the reference input, S represents the shape and size of the actual input, α represents the preset interpolation function, γ x (α) represents the expansion of the preset interpolation function to (x′ + α(xx′)), l represents the lth feature processing cell in the hypernetwork, i represents the i-th edge in the lth feature processing cell, j represents the jth operation unit in the i-th edge, and k represents the kth step of the interpolation calculation. The above formula enables fine-grained analysis of the output features of each operation unit using an integrated gradient method, determining the importance of the output features of the operation unit, and thus determining the importance of the operation unit. The more important the output features, the more important the operation unit.

[0050] The above formula can be used to obtain the first feature attribution index set corresponding to each operation unit in the hypernetwork. Among them, the dimension of the first feature attribution indicator set is N×P×Q, where N represents the number of feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cell, and Q represents the number of operation units in the edge. The first feature attribution indicator set includes the feature attribution indicators of each operation unit in the hypernetwork, and i and j represent the serial number of the edge where the operation unit is located and the serial number of the operation unit in the edge, respectively.

[0051] Step S204, determining multiple target operation units in the super network based on the characteristic attribution index of each operation unit;

[0052] In the technical solution provided in step S204, the structure of each feature extraction cell in the hypernetwork is the same, and the structure includes the number of nodes contained in the feature extraction cell, the number and connection mode of the edges between the nodes in the feature extraction cell, and the number and connection mode of the operation units in the edges; the step of determining multiple target operation units in the hypernetwork based on the feature attribution index of each operation unit includes: summarizing the feature attribution index of each operation unit in the hypernetwork to obtain a first feature attribution index set, wherein the dimension of the first feature attribution index set is N×P×Q, N represents the number of feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cell, Q represents the number of operation units in the edge, and the first feature attribution index set is N×P×Q, where N represents the number of feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cell, Q represents the number of operation units in the edge, and The feature attribution indicator set includes feature attribution indicators of each operation unit in the hypernetwork; the first feature attribution indicator set is aggregated to obtain a second feature attribution indicator set, wherein the dimension of the second feature attribution indicator set is P×Q, the rows in the second feature attribution indicator set correspond to the edges in the hypernetwork, the columns in the second feature attribution indicator set correspond to the operation units in the hypernetwork, and the edges with the same position in each feature extraction cell in the hypernetwork correspond to the same row in the second feature attribution indicator set, and the operation units with the same position in each edge in the hypernetwork correspond to the same column in the second feature attribution indicator set; the target operation unit is determined in each row of the second feature attribution indicator set to obtain multiple target operation units.

[0053] In some embodiments of the present application, as shown in Figure 4, the step of performing aggregation processing on the first feature attribution indicator set to obtain the second feature attribution indicator set includes: determining the intermediate feature matrix corresponding to the operation units with the same position in each feature processing cell, wherein the same position means that the edges corresponding to the operation units have the same sequence number in the feature processing cell, and the sequence number of the operation units in the edge is also the same; determining the value of the corresponding element in the second feature attribution indicator set based on the intermediate feature matrix, thereby obtaining the second feature attribution indicator set, wherein the edge sequence number corresponding to the intermediate feature matrix is ​​the row sequence number of the corresponding element in the second feature attribution indicator set, and the sequence number of the operation unit corresponding to the intermediate feature matrix is ​​the column sequence number of the corresponding element in the second feature attribution indicator set.

[0054] In some embodiments of the present application, the aggregation function for performing aggregation processing on the first feature attribution indicator set is as follows:

[0055] In the above aggregation function, represents the intermediate feature matrix, represents the feature attribution index in the g-th feature processing cell, It represents the feature attribution index in the hth feature processing cell, i represents the sequence number of the edge, and j represents the sequence number of the operation unit.

[0056] As an optional implementation, determining the value of the corresponding element in the second feature attribution indicator set based on the intermediate feature matrix to obtain the second feature attribution indicator set includes: determining the maximum feature attribution indicator in the intermediate feature matrix as the value of the corresponding element.

[0057] In some embodiments of the present application, a target operation unit is determined in each row of the second feature attribution indicator set, and the step of obtaining multiple target operation units includes: determining the largest feature attribution indicator in each row of the second feature attribution indicator set as the target feature attribution indicator; and determining the target operation unit in each row according to the row and column corresponding to the target feature attribution indicator.

[0058] Specifically, A in FIG4 mN It represents the set of Am values ​​of each operation unit in the Nth cell. The rows in the set represent the edges in the cell, the columns represent the operation units in each edge, and any element in the set represents the Am value of the operation unit corresponding to the row and column. As can be seen from Figure 4, when the Am values ​​of the operation units at the same position in each cell are placed in the same After the matrix is ​​formed, it can be obtained from each The maximum value in the matrix is ​​selected as the value of the corresponding element, where the corresponding element refers to the element in the i-th row and j-th column in the second feature attribution indicator set (the final architecture in the figure). The superscript i in the matrix represents the index of the edge where the operation unit is located in the feature processing cell, and j represents the index of the operation unit in the edge. When determining the maximum feature attribution index from each row of the second feature attribution index set as the target feature attribution index, a DARTS (Differentiable Architecture Search) method can be used to determine the target feature attribution index.

[0059] In step S206 , a model architecture of a target neural network model is determined according to the plurality of target operation units.

[0060] In the technical solution provided in step S206, the final architecture result can be determined based on the determined target operation units, and then stacked into a target neural network model, and the model is pre-trained. The final architecture result can be determined based on the determined target operation units, and only the target operation units can be retained in the super network.

[0061] By adopting the model generation method provided by this application, the test results on the NAS basic time dataset are compared with the test results in related technologies as shown in the following table:

[0062] In the above table, proposal represents the model generation method provided by this application. It can be seen that the method provided by this application can effectively shorten the search time, and other indicators are also better than most model structure search methods.

[0063] By adopting a feature attribution index for each operation unit in a hypernetwork, wherein the feature attribution index is used to indicate the importance of the operation unit, the hypernetwork includes multiple feature processing cells, the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; multiple target operation units are determined in the hypernetwork based on the feature attribution index of each operation unit; the model architecture of the target neural network model is determined based on the multiple target operation units, and the importance of each operation unit is quantitatively reflected by determining the feature attribution index of each operation unit, thereby achieving the purpose of improving the interpretability of the model structure confirmation process, thereby realizing the technical effect of improving the efficiency of model structure generation, and further solving the technical problems of poor interpretability and excessive consumption of computing resources caused by the use of search algorithms to retrieve model architectures in related technologies.

[0064] In addition, the present embodiment utilizes IG technology for operation attribution in NAS, thereby proposing an indicator and calculation method for measuring the importance of each operation unit, thereby improving the interpretability of the model search process. Furthermore, the present embodiment also provides a scale-invariant conversion method that incorporates the feature attribution indicator provided by the present application into the operation selection process of DARTS.

[0065] An embodiment of the present application provides a model generation device, and Figure 5 is a structural schematic diagram of the device. As shown in Figure 5, the device includes: a first processing module 50, which is used to determine the feature attribution index of each operation unit in the super network, wherein the feature attribution index is used to indicate the importance of the operation unit, and the super network includes multiple feature processing cells, and the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; a second processing module 52, which is used to determine multiple target operation units in the super network based on the feature attribution index of each operation unit; and a third processing module 54, which is used to determine the model architecture of the target neural network model based on the multiple target operation units.

[0066] In some embodiments of the present application, the first processing module 50 is used to determine the step of characteristic attribution index of each operation unit in the super network, including: determining the actual input and reference input of the operation unit, and a preset interpolation function; performing integrated gradient calculation on the operation unit based on the actual input, reference input and preset interpolation function to obtain the characteristic attribution index of the operation unit.

[0067] In some embodiments of the present application, an integrated gradient calculation is performed on the operation unit based on the actual input, the reference input, and the preset interpolation function to obtain the calculation formula of the characteristic attribution index of the operation unit as follows:

[0068] In the above formula Indicates the characteristic attribution index of the operation unit, Indicates the actual input, represents the reference input, S represents the shape and size of the actual input, α represents the preset interpolation function, γ x (α) indicates that the preset interpolation function is expanded to (x′+α(xx′)), l indicates the lth feature processing cell in the hypernetwork, i indicates the i-th edge in the l-th feature processing cell, j indicates the j-th operation unit in the i-th edge, and k indicates the k-th step of the interpolation calculation.

[0069] In some embodiments of the present application, the structures of the feature extraction cells in the hypernetwork are the same, and the structures include the number of nodes contained in the feature extraction cells, the number and connection mode of the edges between the nodes in the feature extraction cells, and the number and connection mode of the operation units in the edges; the second processing module 52 determines the multiple target operation units in the hypernetwork based on the feature attribution indicators of each operation unit, including: summarizing the feature attribution indicators of each operation unit in the hypernetwork to obtain a first feature attribution indicator set, wherein the dimension of the first feature attribution indicator set is N×P×Q, N represents the number of feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cells, Q represents the number of operation units in the edges, and the first The feature attribution indicator set includes feature attribution indicators of each operation unit in the hypernetwork; the first feature attribution indicator set is aggregated to obtain a second feature attribution indicator set, wherein the dimension of the second feature attribution indicator set is P×Q, the rows in the second feature attribution indicator set correspond to the edges in the hypernetwork, the columns in the second feature attribution indicator set correspond to the operation units in the hypernetwork, and the edges with the same position in each feature extraction cell in the hypernetwork correspond to the same row in the second feature attribution indicator set, and the operation units with the same position in each edge in the hypernetwork correspond to the same column in the second feature attribution indicator set; the target operation unit is determined in each row of the second feature attribution indicator set to obtain multiple target operation units.

[0070] In some embodiments of the present application, the second processing module 52 performs aggregation processing on the first feature attribution indicator set, and the step of obtaining the second feature attribution indicator set includes: determining the intermediate feature matrix corresponding to the operation units with the same position in each feature processing cell, wherein the same position means that the edges corresponding to the operation units have the same serial number in the feature processing cell, and the serial numbers of the operation units in the edges are also the same; determining the values ​​of the corresponding elements in the second feature attribution indicator set based on the intermediate feature matrix, thereby obtaining the second feature attribution indicator set, wherein the edge serial number corresponding to the intermediate feature matrix is ​​the row serial number of the corresponding element in the second feature attribution indicator set, and the serial number of the operation unit corresponding to the intermediate feature matrix is ​​the column serial number of the corresponding element in the second feature attribution indicator set.

[0071] In some embodiments of the present application, the aggregation function for performing aggregation processing on the first feature attribution indicator set is as follows:

[0072] In the above aggregation function, represents the intermediate feature matrix, represents the feature attribution index in the g-th feature processing cell, It represents the feature attribution index in the hth feature processing cell, i represents the sequence number of the edge, and j represents the sequence number of the operation unit.

[0073] In some embodiments of the present application, the second processing module 52 determines the value of the corresponding element in the second feature attribution indicator set based on the intermediate feature matrix, thereby obtaining the second feature attribution indicator set, including: determining the largest feature attribution indicator in the intermediate feature matrix as the value of the corresponding element.

[0074] In some embodiments of the present application, the second processing module 52 determines the target operation unit in each row of the second feature attribution indicator set, and the step of obtaining multiple target operation units includes: determining the largest feature attribution indicator in each row of the second feature attribution indicator set as the target feature attribution indicator; and determining the target operation unit in each row according to the row and column corresponding to the target feature attribution indicator.

[0075] It should be noted that the various modules in the above-mentioned model generation device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0076] According to an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following model generation method: determining the feature attribution index of each operation unit in a super network, wherein the feature attribution index is used to indicate the importance of the operation unit, the super network includes multiple feature processing cells, the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; determining multiple target operation units in the super network based on the feature attribution index of each operation unit; and determining the model architecture of the target neural network model based on the multiple target operation units.

[0077] According to an embodiment of the present application, an electronic device is also provided, which includes a memory and a processor, the processor being used to run a program stored in the memory, wherein the following model generation method is executed when the program is running: determining the feature attribution index of each operation unit in a super network, wherein the feature attribution index is used to indicate the importance of the operation unit, the super network includes multiple feature processing cells, the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; determining multiple target operation units in the super network based on the feature attribution index of each operation unit; and determining the model architecture of the target neural network model based on the multiple target operation units.

[0078] According to an embodiment of the present application, a computer program product is also provided, including a computer program, which implements the following model generation method when executed by a processor: determining the feature attribution index of each operation unit in a super network, wherein the feature attribution index is used to indicate the importance of the operation unit, the super network includes multiple feature processing cells, the feature processing cells include multiple nodes, and the edge between any two connected nodes is composed of multiple operation units; determining multiple target operation units in the super network based on the feature attribution index of each operation unit; and determining the model architecture of the target neural network model based on the multiple target operation units.

[0079] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0082] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0084] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model generation method, comprising: Determining a feature attribution index for each operation unit in a hypernetwork, wherein the feature attribution index is used to indicate the importance of the operation unit, the hypernetwork includes a plurality of feature processing cells, the feature processing cells include a plurality of nodes, and an edge between any two connected nodes is composed of a plurality of operation units; determining a plurality of target operation units in the super network according to the characteristic attribution index of each of the operation units; A model architecture of a target neural network model is determined according to the plurality of target operation units.

2. The model generation method according to claim 1, wherein the step of determining the characteristic attribution index of each of the operating units in the hypernetwork comprises: determining an actual input and a reference input of the operating unit, and a preset interpolation function; An integrated gradient calculation is performed on the operation unit according to the actual input, the reference input and the preset interpolation function to obtain a feature attribution index of the operation unit.

3. The model generation method according to claim 2, wherein the calculation formula for the characteristic attribution index of the operation unit is obtained by performing integrated gradient calculation on the operation unit based on the actual input, the reference input and the preset interpolation function as follows: In the above formula represents a characteristic attribution index of the operation unit, represents the actual input, represents the reference input, S represents the shape and size of the actual input, α represents the preset interpolation function, γ x (α) represents the expansion of the preset interpolation function to (x′+α(xx′)), l represents the lth feature processing cell in the hypernetwork, i represents the i-th edge in the l-th feature processing cell, j represents the j-th operation unit in the i-th edge, and k represents the k-th step of the interpolation calculation.

4. The model generation method according to claim 1, wherein the structure of each of the feature extraction cells in the hypernetwork is the same, the structure including the number of nodes contained in the feature extraction cell, the number and connection mode of the edges between the nodes in the feature extraction cell, and the number and connection mode of the operation units in the edges; the step of determining multiple target operation units in the hypernetwork based on the feature attribution index of each of the operation units comprises: Summarizing the feature attribution indicators of each of the operation units in the hypernetwork to obtain a first feature attribution indicator set, wherein the dimension of the first feature attribution indicator set is N×P×Q, where N represents the number of the feature processing cells in the hypernetwork, P represents the number of edges in the feature processing cell, and Q represents the number of operation units in the edge, and the first feature attribution indicator set includes the feature attribution indicator of each of the operation units in the hypernetwork; Performing aggregation processing on the first feature attribution indicator set to obtain a second feature attribution indicator set, wherein the dimension of the second feature attribution indicator set is P×Q, the rows in the second feature attribution indicator set correspond to the edges in the hypernetwork, the columns in the second feature attribution indicator set correspond to the operation units in the hypernetwork, and the edges with the same position in each feature extraction cell in the hypernetwork correspond to the same row in the second feature attribution indicator set, and the operation units with the same position in each edge in the hypernetwork correspond to the same column in the second feature attribution indicator set; The target operation unit is determined in each row of the second feature attribution indicator set to obtain the multiple target operation units.

5. The model generation method according to claim 4, wherein the step of aggregating the first feature attribution indicator set to obtain the second feature attribution indicator set comprises: Determine the intermediate feature matrices corresponding to the operation units with the same position in each feature processing cell, wherein the same position means that the edges corresponding to the operation units have the same sequence numbers in the feature processing cell, and the sequence numbers of the operation units in the edges are also the same; Determine the value of the corresponding element in the second feature attribution indicator set based on the intermediate feature matrix, thereby obtaining the second feature attribution indicator set, wherein the edge number corresponding to the intermediate feature matrix is ​​the row number of the corresponding element in the second feature attribution indicator set, and the number of the operation unit corresponding to the intermediate feature matrix is ​​the column number of the corresponding element in the second feature attribution indicator set.

6. The model generation method according to claim 5, wherein the aggregation function for performing aggregation processing on the first feature attribution indicator set is as follows: In the above aggregation function, represents the intermediate feature matrix, represents the feature attribution index in the g-th feature processing cell, represents the feature attribution index in the hth feature processing cell, i represents the sequence number of the edge, and j represents the sequence number of the operation unit.

7. The model generation method according to claim 5, wherein the step of determining the values ​​of corresponding elements in the second feature attribution indicator set based on the intermediate feature matrix to obtain the second feature attribution indicator set comprises: Determine the largest feature attribution index in the intermediate feature matrix as the value of the corresponding element.

8. The model generation method according to claim 5, wherein the step of determining the target operation unit in each row of the second feature attribution indicator set and obtaining the plurality of target operation units comprises: Determining the largest feature attribution indicator in each row of the second feature attribution indicator set as a target feature attribution indicator; The target operation unit is determined in each row according to the row and column corresponding to the target feature attribution indicator.

9. A model generation device comprising: a first processing module, configured to determine a feature attribution index for each operation unit in a hypernetwork, wherein the feature attribution index is used to indicate the importance of the operation unit, the hypernetwork including a plurality of feature processing cells, each of the feature processing cells including a plurality of nodes, and an edge between any two connected nodes consisting of a plurality of operation units; a second processing module, configured to determine a plurality of target operation units in the super network according to the characteristic attribution index of each of the operation units; The third processing module is used to determine the model architecture of the target neural network model according to the multiple target operation units.

10. A non-volatile storage medium having a program stored therein, wherein: When the program is running, the device where the non-volatile storage medium is located is controlled to execute the model generation method according to any one of claims 1 to 8.

11. An electronic device comprising: A memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the model generation method according to any one of claims 1 to 8 is executed when the program is run.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the model generation method according to any one of claims 1 to 8.

Images