Network architecture search method, device, equipment and storage medium

Abstract

The present invention discloses a network architecture search method, device, equipment and storage medium, including: constructing a super network based on original image data; processing the super network to obtain a sub-network training set; creating a graph convolutional network model based on the sub-network training set; processing a target network based on the graph convolutional network model to obtain the network architecture of the target network. In an embodiment of the present invention, a large amount of original image data is first obtained, and then a super network is constructed based on the original image data to build a search space; in addition, in an embodiment of the present invention, the constructed super network is simplified based on integration rules, which greatly reduces the computational cost and the difficulty of data processing.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving technology, and in particular relates to a network architecture search method, device, equipment and storage medium. Background Art

[0002] At present, in the field of autonomous driving, deep learning technology is widely used for image perception, namely target detection and semantic segmentation.

[0003] Object detection and semantic segmentation require the manual selection or creation of appropriate deep neural networks based on the data. However, a high-precision and highly robust deep neural network requires a large number of experiments in data collection, and requires a lot of time and financial costs. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one object of the present invention is to provide a network architecture search method, device, equipment and storage medium.

[0005] In order to solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] A network architecture search method, comprising:

[0007] Based on the original image data, a hypernetwork is constructed;

[0008] Processing the super network to obtain a sub-network training set;

[0009] Based on the sub-network training set, creating a graph convolutional network model;

[0010] The target network is processed based on the graph convolutional network model to obtain the network architecture of the target network.

[0011] Optionally, constructing a hypernetwork based on the original image data includes:

[0012] Based on the original image data, obtaining a plurality of nodes;

[0013] A connection rule of a plurality of the nodes is established, and the super network is constructed.

[0014] Optionally, after constructing the super network, the method further includes:

[0015] The super network is simplified based on the integration rule to obtain a target super network; wherein the target super network includes multiple cells.

[0016] Optionally, the processing the super network to obtain a sub-network training set includes:

[0017] Sampling the target super network to obtain multiple sub-networks;

[0018] Screening the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks;

[0019] Based on the multiple target sub-networks, a sub-network training set is obtained.

[0020] Optionally, the screening of the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks includes:

[0021] Performing multiple training on each of the sub-networks based on the original image data to obtain the accuracy of each training;

[0022] Processing the accuracy of each training to determine the actual accuracy of each sub-network;

[0023] Comparing the actual accuracy rate with the preset screening threshold;

[0024] If the actual accuracy of the sub-network is greater than the preset screening threshold, the sub-network is determined as a target sub-network.

[0025] Optionally, creating a graph convolutional network model based on the subnetwork training set includes:

[0026] Obtaining an adjacency matrix for each of the subnetworks;

[0027] Processing each of the adjacency matrices to determine the state of each frequency domain convolutional layer;

[0028] Based on the state of each of the frequency domain convolutional layers, the graph convolutional network model is created.

[0029] Optionally, the state of the frequency domain convolutional layer is calculated based on the following formula:

[0030]

[0031] Where: is the state of the frequency domain convolution of the l+1th layer; is the state of the frequency domain convolution layer of the lth layer; σ is the activation function; U is a symmetric matrix; U T is the transpose of U; is a diagonal matrix.

[0032] Optionally, the value of the diagonal matrix is ​​calculated based on the following formula:

[0033]

[0034] in,

[0035] In the formula, g θ is the frequency domain convolution kernel, θ k is the first parameter; Λ is the characteristic matrix of the adjacency matrix; I N is the second parameter; T k is the kth term of the Chebyshev polynomial.

[0036] Optionally, processing the target network based on the graph convolutional network model to obtain the network architecture of the target network includes:

[0037] Processing the target network based on the graph convolutional network model to obtain states of multiple frequency domain convolutional layers of the target network;

[0038] Regression processing is performed on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network.

[0039] Optionally, after performing regression processing on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network, the method further comprises:

[0040] The network architecture is trained based on the original image data to obtain a target network architecture.

[0041] An embodiment of the present invention further provides a network architecture search device, comprising:

[0042] A construction module, used to construct a hypernetwork based on raw image data;

[0043] A processing module, used to process the super network to obtain a sub-network training set;

[0044] A creation module, used to create a graph convolutional network model based on the sub-network training set;

[0045] The output module is used to process the target network based on the graph convolutional network model to obtain the network architecture of the target network.

[0046] An embodiment of the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method described above when executing the computer program.

[0047] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method described above.

[0048] The embodiments of the present invention have the following technical effects:

[0049] The above technical solutions of the present invention, 1) in the embodiment of the present invention, firstly obtain a large amount of original image data, then construct a hypernetwork based on the original image data, and build a search space; in addition, in the embodiment of the present invention, the constructed hypernetwork is simplified based on the integration rules, which greatly reduces the calculation cost and the difficulty of data processing.

[0050] 2) In an embodiment of the present invention, a subnetwork training set is obtained by randomly sampling a supernetwork. At the same time, the embodiment of the present invention obtains the accuracy of each subnetwork, and based on the accuracy, the subnetwork is screened and the upper limit of the subnetwork accuracy is determined.

[0051] 3) In an embodiment of the present invention, a GCN model is obtained based on a sub-network training set. In the process of obtaining the GCN model, a relationship between adjacent frequency domain convolutional layers is obtained, and the connection between the subgraph of the target sub-network G and the adjacent subgraphs is retained (in the process of processing G, G is divided into multiple subgraphs, each subgraph is connected with the adjacent subgraphs, and correspondingly, each frequency domain convolution is also connected with the adjacent frequency domain convolutions); in addition, in an embodiment of the present invention, the solution of the feature matrix is ​​accelerated based on Chebyshev polynomials, which greatly reduces the amount of calculation.

[0052] 4) In an embodiment of the present invention, the original image data is divided into training data and verification data to prevent the obtained target network architecture from having an overfitting problem.

[0053] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of a network architecture search method provided by an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of a sub-network structure provided by an embodiment of the present invention;

[0056] Figure 3 It is a structural diagram of a network architecture search device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0058] In order to facilitate the understanding of the embodiments by those skilled in the art, some terms are explained:

[0059] (1) GCN: Graph Convolutional Network.

[0060] (2) Normal structure: The input feature maps are output with the same size.

[0061] (3) Reduction structure: The length and width of the input feature map are reduced by half and then output.

[0062] (4) Inception structure: The features of different receptive fields generated are fused and output.

[0063] (5) RLS algorithm: (Recursive Least Square) algorithm, that is, recursive least squares method, which is also called least squares method.

[0064] (6)Hadamard product, a matrix operation.

[0065] An embodiment of the present invention provides a network architecture search system, comprising:

[0066] Processor and memory;

[0067] The system acquires a large amount of raw image data during or related to autonomous driving based on the network, and stores the large amount of raw image data in the memory for easy access by the processor;

[0068] The processor calls the original image data in the memory and processes the original image data. Specifically, the processor builds a hypernetwork based on the original image data, wherein the hypernetwork is a special and huge network, a directed acyclic graph, with hundreds or thousands of nodes, each node in the graph represents a different meaning, and the node can represent but is not limited to input, output, convolution, pooling, activation, linking, filling, addition and other operations. The two special nodes in the hypernetwork are the input node with only out-degree and the output node with only in-degree.

[0069] The processor establishes connection rules between hypernetwork nodes to prevent nodes that should not be connected from connecting;

[0070] Then, the processor simplifies the hypernetwork to obtain the target hypernetwork, so as to reduce the processing intensity of subsequent processors;

[0071] The processor samples the target super network based on a random algorithm to obtain a sub-network training set; and stores the sub-network training set in a memory;

[0072] The processor calls the sub-network training set from the memory, and creates a graph convolutional network model by processing the sub-network training set;

[0073] The system inputs the target network to be processed into the graph convolutional network model, outputs the code, then decodes and regresses the code, and outputs the adjacency matrix of the target network structure.

[0074] like Figure 1 As shown, an embodiment of the present invention provides a network architecture search method, which is applied to the above system, including:

[0075] Step S1: construct a hypernetwork based on the original image data;

[0076] Specifically, the construction of a hypernetwork based on the original image data includes:

[0077] Based on the original image data, obtaining a plurality of nodes;

[0078] A connection rule of a plurality of the nodes is established, and the super network is constructed.

[0079] In actual application scenarios, not all nodes in a hypernetwork can be connected to each other, so it is necessary to establish connection rules, wherein the connection rules can be manually specified.

[0080] For example, the connection rules could be that activated nodes cannot be connected, pooling must be after convolution or between two convolutions, etc.

[0081] Furthermore, after constructing the super network, the method further includes:

[0082] The super network is simplified based on the integration rule to obtain a target super network; wherein the target super network includes multiple cells.

[0083] In actual application scenarios, a large number of nodes in the hypernetwork will increase the difficulty of subsequent work. Therefore, it is necessary to simplify the nodes of the hypernetwork and integrate some nodes together as one node; the integrated nodes are cells.

[0084] Specifically, the form of the cell can be a residual structure, a jump connection structure, a residual jump connection structure, an Inception structure, a normal structure, and a reduction structure.

[0085] For example: A hypernetwork can be composed based on the permutations and combinations of the above six structures.

[0086] The embodiments of the present invention first obtain a large amount of original image data, then construct a hypernetwork based on the original image data and build a search space; in addition, the embodiments of the present invention simplify the constructed hypernetwork based on integration rules, which greatly reduces the calculation cost and the difficulty of subsequent data processing.

[0087] Step S2: Processing the super network to obtain a sub-network training set;

[0088] Specifically, the processing of the super network to obtain a sub-network training set includes:

[0089] Sampling the target super network to obtain multiple sub-networks;

[0090] Screening the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks;

[0091] Based on the multiple target sub-networks, a sub-network training set is obtained.

[0092] like Figure 2 As shown, an example of a sub-network structure is shown, specifically a sub-network composed of nodes In, Ca1, Ca2, Cb, Cc1, Cc2, Cd and Out.

[0093] In actual application scenarios, each layer of the search space of the target super network is randomly sampled based on a random algorithm to obtain multiple sub-networks, that is, the sub-network is a sub-structure of the super network.

[0094] Specifically, in order to facilitate subsequent calculations, the obtained sub-network is output in the form of an adjacency matrix.

[0095] The structure of one of the sub-networks corresponds to the adjacency matrix represented by the output Table 1. The rows and columns of the table represent the nodes of the sub-network. The value of the cell in the table is "0" or "1", indicating that the row node corresponding to the cell cannot or can jump to the column node corresponding to the cell.

[0096] Table 1

[0097] In Ca1 Ca2 Cb Cc1 Cc2 Cd Out In 0 1 0 0 0 0 0 0 Ca1 0 0 1 1 0 0 0 0 Ca2 0 0 0 0 0 1 0 0 Cb 0 0 0 0 1 0 0 0 Cc1 0 0 0 0 0 1 1 0 Cc2 0 0 0 0 0 0 0 1 Cd 0 0 0 0 0 1 0 0 Out 0 0 0 0 0 0 0 0

[0098] When the value of a cell in the table is "1", it means that the row node corresponding to the cell can jump to the column node of the cell.

[0099] For example, if the value of the cell in the third row and fourth column is "1", it means that node Ca1 can jump to node Ca2;

[0100] When the value of a cell in the table is "0", it means that the row node corresponding to the cell cannot jump to the column node corresponding to the cell.

[0101] For example, if the value of the cell in the third row and sixth column is "0", it means that the node Ca1 cannot jump to the node Cc1.

[0102] Furthermore, the screening of the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks includes:

[0103] Performing multiple training on each of the sub-networks based on the original image data to obtain the accuracy of each training;

[0104] Processing the accuracy of each training to determine the actual accuracy of each sub-network;

[0105] Comparing the actual accuracy rate with the preset screening threshold;

[0106] If the actual accuracy of the sub-network is greater than the preset screening threshold, the sub-network is determined as a target sub-network.

[0107] In actual application scenarios, after obtaining the sub-network, it is necessary to evaluate the performance of the sub-network and use a small amount of original image data to train the sub-network for multiple rounds to obtain the accuracy of each round.

[0108] Then, the actual accuracy of each sub-network can be determined through certain algorithms.

[0109] For example, the RLS algorithm can be used to fit the accuracy curve, and the actual accuracy of the sub-network can be determined based on the coefficient of the accuracy curve.

[0110] Further, it is assumed that the preset screening threshold is 82%.

[0111] When the actual accuracy of the sub-network is determined to be 80% based on the RLS algorithm, the actual accuracy is compared with the preset screening threshold. If the actual accuracy is less than the preset screening threshold, the sub-network is deleted without subsequent operations.

[0112] When the actual accuracy of the subnetwork is determined to be 85% based on the RLS algorithm, the actual accuracy is compared with the preset screening threshold. If the actual accuracy is greater than the preset screening threshold, the subnetwork is retained as the target subnetwork for training the GSN.

[0113] Furthermore, based on the obtained actual accuracy of the sub-network, an upper limit of the actual accuracy of the sub-network can be determined.

[0114] For example, when the actual accuracy of a sub-network in the sub-network training set determined based on the RLS algorithm is at most 90%, the sub-network is deleted, and it can be determined that the accuracy of the sub-network training set does not exceed 90%.

[0115] In an optional embodiment of the present invention, after obtaining the sub-network, it is necessary to evaluate the performance of the sub-network and use a small amount of original image data to train the sub-network for 10 to 15 rounds to obtain the accuracy of each round, thereby avoiding the problem of large amount of calculation required for training each sub-network hundreds of times or even more times in the prior art, and greatly reducing the amount of calculation.

[0116] In an optional embodiment of the present invention, the above steps are repeated, and the sub-network training set finally obtained includes approximately 1000 target sub-networks.

[0117] Furthermore, the number of target subnetworks in the subnetwork training set can be adjusted according to actual training needs; for example: 500, 2000 or 3000, etc.

[0118] The embodiment of the present invention obtains a subnetwork training set by randomly sampling a supernetwork. In addition, the embodiment of the present invention obtains the accuracy of each subnetwork, and implements the screening of subnetworks and the determination of the upper limit of the subnetwork accuracy based on the accuracy.

[0119] Step S3: creating a graph convolutional network model based on the sub-network training set;

[0120] Specifically, creating a graph convolutional network model based on the subnetwork training set includes:

[0121] Obtaining an adjacency matrix for each of the subnetworks;

[0122] Processing each of the adjacency matrices to determine the state of each frequency domain convolutional layer;

[0123] Based on the state of each of the frequency domain convolutional layers, the graph convolutional network model is created.

[0124] Furthermore, the state of the frequency domain convolutional layer is calculated based on the following formula:

[0125]

[0126] Where: is the state of the frequency domain convolution of the l+1th layer; is the state of the frequency domain convolution layer of the lth layer; σ is the activation function; U is a symmetric matrix; U T is the transpose of U; is a diagonal matrix.

[0127] Furthermore, the value of the diagonal matrix is ​​calculated based on the following formula:

[0128]

[0129] in,

[0130] In the formula, g θ is the frequency domain convolution kernel, θ k is the first parameter; Λ is the characteristic matrix of the adjacency matrix; I N is the second parameter; T k is the kth term of the Chebyshev polynomial.

[0131] In actual application scenarios, since the subnetwork is a directed acyclic graph, a node updates its state after receiving a message from each of its neighbor nodes. However, since the direction of the subnetwork is clear, its direction can be ignored in actual operations, and then frequency domain convolution is used to learn node information.

[0132] Specifically, the frequency domain convolutional network adopted in the embodiment of the present invention is a Chebyshev network; when the direction is ignored, the adjacency matrix of the subnetwork becomes a symmetric matrix.

[0133] For example, assuming that the number of nodes in a target subnetwork G is n, it can be decomposed according to the following formula: L = UΛU T ,U=(u1,u2,…u n );

[0134]

[0135] Where L is the Laplace matrix of G, U is a symmetric matrix, and U T is the transpose of U, u1,u2,…u n is the n column vector of U, Λ is the characteristic matrix composed of the n eigenvectors of U; λ n is the eigenvalue of U, n is greater than or equal to 2.

[0136] Therefore, the formula for frequency domain convolution on G can be written as:

[0137] in, is the Fourier transform form of f(t) on G, and f(t) is the eigenvector of the tth node on G.

[0138] If we perform convolution on the n nodes of the entire graph together, the convolution of G is:

[0139]

[0140] Among them, f is the characteristic function, g is the convolution kernel, then:

[0141] (f*g)=F -1 [F[f]⊙F[g]]

[0142] Substitute the convolution of G into (f*g) G =F-1 [F[f]⊙F[g]], we can get:

[0143] (f*G g )=U(U T f T g)=U(U T g⊙U T f)

[0144] Among them, G g is the convolution kernel of G in the embodiment of the present application, F is the Fourier transform, and F -1 is the inverse Fourier transform, ⊙ is the Hadamard product; T g is regarded as the overall learnable convolution kernel, denoted as g θ , we have: o=(f*G g ) θ =Ug θ U T f

[0145] Assume that the hidden state of layer l is: Similarly, the l+1th layer is: Then the state of the frequency domain convolutional layer is updated as:

[0146]

[0147]

[0148] In the embodiment of the present invention, a relational expression between a previous frequency domain convolution layer and a next frequency domain convolution layer is obtained by eigendecomposing a Laplacian matrix.

[0149] Furthermore, since the basic frequency domain convolutional network needs to calculate all the eigenvalues ​​and eigenvectors of the Laplacian matrix, the amount of calculation is huge; in order to solve the above problem, the embodiment of the present invention applies Chebyshev polynomials to accelerate the solution of the characteristic matrix.

[0150] Specifically, assume that the kth term of the Chebyshev polynomial is T k ,The frequency domain convolution kernel is calculated as:

[0151]

[0152] in, T k (x) = 2xT k-1 (x)-T k-2 (x), T0(x)=1 and T1(x)=1; x is a variable.

[0153] In an embodiment of the present invention, a GCN model is obtained based on a sub-network training set. In the process of obtaining the GCN model, a relationship between adjacent frequency domain convolutional layers is obtained, and the connection between the subgraph of the target sub-network G and the adjacent subgraphs is retained (in the process of processing G, G is divided into multiple subgraphs, each subgraph is connected with the adjacent subgraphs, and correspondingly, each frequency domain convolution is also connected with the adjacent frequency domain convolution, and there is a certain connection).

[0154] In addition, the embodiment of the present invention accelerates the solution of the characteristic matrix based on Chebyshev polynomials, thereby greatly reducing the amount of calculation.

[0155] Step S4: Process the target network based on the graph convolutional network model to obtain the network architecture of the target network.

[0156] Specifically, processing the target network based on the graph convolutional network model to obtain the network architecture of the target network includes:

[0157] Processing the target network based on the graph convolutional network model to obtain states of multiple frequency domain convolutional layers of the target network;

[0158] Regression processing is performed on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network.

[0159] In actual application scenarios, the system obtains the network to be processed, processes the network based on the obtained GCN model, and outputs

[0160] Will The data is input to the regression layer and regression processing is performed based on LSTM to obtain the adjacency matrix of the predicted network structure.

[0161] Further, after regressing the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network, the following steps are included:

[0162] The network architecture is trained based on the original image data to obtain a target network architecture.

[0163] In actual application scenarios, the original image data is divided into two categories, the first category is training data, and the second category is verification data;

[0164] Train the network structure based on the original image data to obtain the target network architecture;

[0165] The target network architecture is verified based on the verification data to confirm the accuracy of the target network architecture.

[0166] Among them, the ratio of verification data to training data can be preset according to actual conditions.

[0167] In an embodiment of the present invention, the original image data is divided into training data and verification data to prevent the obtained target network architecture from having an overfitting problem.

[0168] like Figure 3 As shown, an embodiment of the present invention further provides a network architecture search device 300, comprising:

[0169] A construction module 301 is used to construct a hypernetwork based on the original image data;

[0170] A processing module 302 is used to process the super network to obtain a sub-network training set;

[0171] A creation module 303, used to create a graph convolutional network model based on the sub-network training set;

[0172] The output module 304 is used to process the target network based on the graph convolutional network model to obtain the network architecture of the target network.

[0173] Optionally, constructing a hypernetwork based on the original image data includes:

[0174] Based on the original image data, obtaining a plurality of nodes;

[0175] A connection rule of a plurality of the nodes is established, and the super network is constructed.

[0176] Optionally, after constructing the super network, the method further includes:

[0177] The super network is simplified based on the integration rule to obtain a target super network; wherein the target super network includes multiple cells.

[0178] Optionally, the processing the super network to obtain a sub-network training set includes:

[0179] Sampling the target super network to obtain multiple sub-networks;

[0180] Screening the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks;

[0181] Based on the multiple target sub-networks, a sub-network training set is obtained.

[0182] Optionally, the screening of the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks includes:

[0183] Performing multiple training on each of the sub-networks based on the original image data to obtain the accuracy of each training;

[0184] Processing the accuracy of each training to determine the actual accuracy of each sub-network;

[0185] Comparing the actual accuracy rate with the preset screening threshold;

[0186] If the actual accuracy of the sub-network is greater than the preset screening threshold, the sub-network is determined as a target sub-network.

[0187] Optionally, creating a graph convolutional network model based on the subnetwork training set includes:

[0188] Obtaining an adjacency matrix for each of the subnetworks;

[0189] Processing each of the adjacency matrices to determine the state of each frequency domain convolutional layer;

[0190] Based on the state of each of the frequency domain convolutional layers, the graph convolutional network model is created.

[0191] Optionally, the state of the frequency domain convolutional layer is calculated based on the following formula:

[0192]

[0193] Where: is the state of the frequency domain convolution of the l+1th layer; is the state of the frequency domain convolution layer of the lth layer; σ is the activation function; U is a symmetric matrix; U T is the transpose of U; is a diagonal matrix.

[0194] Optionally, the value of the diagonal matrix is ​​calculated based on the following formula:

[0195]

[0196] in,

[0197] In the formula, g θ is the frequency domain convolution kernel, θ k is the first parameter; Λ is the characteristic matrix of the adjacency matrix; I N is the second parameter; T k is the kth term of the Chebyshev polynomial.

[0198] Optionally, processing the target network based on the graph convolutional network model to obtain the network architecture of the target network includes:

[0199] Processing the target network based on the graph convolutional network model to obtain states of multiple frequency domain convolutional layers of the target network;

[0200] Regression processing is performed on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network.

[0201] Optionally, after performing regression processing on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network, the method includes:

[0202] The network architecture is trained based on the original image data to obtain a target network architecture.

[0203] An embodiment of the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method described above when executing the computer program.

[0204] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method described above.

[0205] In addition, other structures and functions of the device in the embodiment of the present invention are known to those skilled in the art and will not be described in detail here to reduce redundancy.

[0206] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0207] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0208] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0209] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0210] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0211] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0212] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0213] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A network architecture search method, characterized in that: include: Based on the original image data, a hypernetwork is constructed; Processing the super network to obtain a sub-network training set; Based on the sub-network training set, creating a graph convolutional network model; Processing the target network based on the graph convolutional network model to obtain the network architecture of the target network; The step of creating a graph convolutional network model based on the sub-network training set includes: Obtaining an adjacency matrix for each of the subnetworks; Processing each of the adjacency matrices to determine the state of each frequency domain convolutional layer; Based on the state of each of the frequency domain convolutional layers, creating the graph convolutional network model; The state of the frequency domain convolutional layer is calculated based on the following formula: Where: is the state of the frequency domain convolution of the l+1th layer; is the state of the frequency domain convolution layer of the lth layer; σ is the activation function; U is a symmetric matrix; U T is the transpose of U; is a diagonal matrix; The value of the diagonal matrix is ​​calculated based on the following formula: in, In the formula, g θ is the frequency domain convolution kernel, θ k is the first parameter; Λ is the characteristic matrix of the adjacency matrix; I N is the second parameter; T k is the kth term of the Chebyshev polynomial.

2. The method according to claim 1, characterized in that The method of constructing a hypernetwork based on the original image data includes: Based on the original image data, obtaining a plurality of nodes; A connection rule of a plurality of the nodes is established, and the super network is constructed.

3. The method according to claim 1, characterized in that After constructing the super network, the method further includes: The super network is simplified based on the integration rule to obtain a target super network; wherein the target super network includes multiple cells.

4. The method according to claim 3, characterized in that The step of processing the super network to obtain a sub-network training set includes: Sampling the target super network to obtain multiple sub-networks; Screening the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks; Based on the multiple target sub-networks, a sub-network training set is obtained.

5. The method according to claim 4, characterized in that The step of screening the plurality of sub-networks based on a preset screening threshold to obtain a plurality of target sub-networks includes: Performing multiple training on each of the sub-networks based on the original image data to obtain the accuracy of each training; Processing the accuracy of each training to determine the actual accuracy of each sub-network; Comparing the actual accuracy rate with the preset screening threshold; If the actual accuracy of the sub-network is greater than the preset screening threshold, the sub-network is determined as a target sub-network.

6. The method according to claim 1, characterized in that Processing the target network based on the graph convolutional network model to obtain the network architecture of the target network includes: Processing the target network based on the graph convolutional network model to obtain states of multiple frequency domain convolutional layers of the target network; Regression processing is performed on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network.

7. The method according to claim 6, characterized in that After performing regression processing on the states of the plurality of frequency domain convolutional layers to obtain the network architecture of the target network, the method further comprises: The network architecture is trained based on the original image data to obtain a target network architecture.

8. A network architecture search device, characterized in that: include: A construction module, used to construct a hypernetwork based on raw image data; A processing module, used to process the super network to obtain a sub-network training set; A creation module, used to create a graph convolutional network model based on the sub-network training set; An output module, used to process the target network based on the graph convolutional network model to obtain the network architecture of the target network; The step of creating a graph convolutional network model based on the sub-network training set includes: Obtaining an adjacency matrix for each of the subnetworks; Processing each of the adjacency matrices to determine the state of each frequency domain convolutional layer; Based on the state of each of the frequency domain convolutional layers, creating the graph convolutional network model; The state of the frequency domain convolutional layer is calculated based on the following formula: Where: is the state of the frequency domain convolution of the l+1th layer; is the state of the frequency domain convolution layer of the lth layer; σ is the activation function; U is a symmetric matrix; U T is the transpose of U; is a diagonal matrix; The value of the diagonal matrix is ​​calculated based on the following formula: in, In the formula, g θ is the frequency domain convolution kernel, θ k is the first parameter; Λ is the characteristic matrix of the adjacency matrix; I N is the second parameter; T k is the kth term of the Chebyshev polynomial.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images