Cooperative neural network search method and search device thereof

Abstract

The embodiment of the invention relates to the technical field of neural architecture search, in particular to a collaborative neural network search method and a search device thereof. The device comprises a first node used for providing a preset search space and a data set; the second node is used for generating a variational auto-encoder and dividing a search space; at least two third nodes cooperating with each other; the third node is used for determining the performance of a plurality of candidate neural architectures, training to obtain a performance predictor and updating an embedding position according to a gradient direction provided by a performance trainer; wherein data information is shared among different third nodes. Through the variational auto-encoder, the third node can collaboratively search the optimal neural architecture (by broadcasting the evaluation result of the neural architecture) on the basis of the hypothesis that similar unit structures / neural architectures have similar performance on the same data set during searching, so that effective evaluation of different neural architectures is realized.

Description

[Technical field]

[0001] The present invention relates to the field of neural architecture search technology, and in particular to a collaborative neural network search method and a search device thereof. [Background technology]

[0002] Neural Architecture Search (NAS) is a technique that studies how to automatically design high-performance deep neural network architectures without manual debugging, so that users do not need to have rich expert experience to design high-performance deep neural network architectures. NAS is essentially an optimization problem. It automatically obtains an optimal neural network structure through a suitable optimization algorithm.

[0003] Currently, most of the outputs from neural architecture search consist of stacks of small, repetitive mathematical models. Most current research focuses on finding the best structure of smaller repetitive units (“cells”) that repeat themselves in the overall structure. Among them, neural architecture optimization (a branch of NAS, also known as NAO) proposes a method to formulate neural architecture search as a mathematical optimization problem.

[0004] However, existing neural architecture search technologies all require evaluating the adaptability of different neural architectures to a specific data set. This critical operation step is very time-consuming and computationally resource-intensive, limiting the application and development of neural architecture search. [Summary of the invention]

[0005] The embodiments of the present application aim to provide a collaborative neural network search method and a search device thereof, which can solve at least some of the defects of existing neural architecture searches.

[0006] In the first aspect, the embodiment of the present application provides a collaborative neural architecture search device. The device includes: a first node for providing a preset search space and data set; a second node for generating a variational autoencoder and dividing the search space; wherein the variational autoencoder is used to map the neural architecture to the vector space so that the neural architectures with similar structures have adjacent embedding positions; at least two third nodes that cooperate with each other; each of the third nodes is used to: determine the performance of multiple candidate neural architectures, train to obtain a performance predictor, and update the embedding position according to the gradient direction provided by the performance trainer; wherein data information is shared between different third nodes; the data information includes: the performance of the candidate neural architecture.

[0007] Optionally, the variational autoencoder is trained by an encoder training data set and broadcast to the third node; wherein the encoder training data set includes: a number of randomly generated neural architectures that satisfy the search space; the distribution of the neural architectures in the vector space is selected from: a Gaussian distribution with a standard deviation of 1 or a distribution specified by the first node.

[0008] Optionally, updating the embedding position according to the gradient direction provided by the performance trainer specifically includes: determining an initial embedding position; calculating the gradient direction given by the performance predictor through a back-propagation algorithm; obtaining an update result of the initial embedding position in a gradient ascent manner; wherein the update result is: an optimized embedding position having better performance than the initial embedding position.

[0009] Optionally, the initial embedding position is the embedding position of the initial neural architecture in the vector space; the initial neural architecture is selected from: a neural architecture assigned by the first node or a randomly generated neural architecture; wherein the randomly generated neural architecture is obtained in the following manner: using an identifier as a random seed and randomly generating it according to the setting of the search space; the identifier is assigned by the second node, or automatically generated by the third node.

[0010] Optionally, the initial embedding position is the embedding position of the first N preferred neural architectures in the vector space; wherein N is a positive integer; the preferred neural architectures are sorted according to performance evaluation results, and the neural architectures with better performance evaluation results have earlier serial numbers.

[0011] Optionally, in order to evaluate different neural architectures simultaneously on different computing processes or nodes, a series of autoencoders (and corresponding autodecoders) recorded in a distributed public ledger and a public key infrastructure (PKI) are used to partition the neural architecture search space during the search process, so as to perform the neural architecture search in the search space partitioning process. The public key is used to determine the vector to be fed into the autoencoder, which in turn can generate the next neural architecture to be trained.

[0012] Optionally, the public key itself represents a point in the neural architecture search space. This allows for minimal communication in a peer-to-peer (or distributed) network. Once a node has generated the public and private keys, the node can use the distributed public ledger retrieved from its peers to perform the model evaluation process.

[0013] Alternatively, the node ID can be calculated as P mod N, where P is the integer representation of the public key and N is the maximum number of nodes in the distributed network. The search process starts with an initialization process among a large number of search nodes by broadcasting an autoencoder that encodes the neural architecture structure.

[0014] Alternatively, writing an effective autoencoder in the blockchain requires a lot of computation. This search process uses a probabilistic verification method to verify the amount of work involved in training the autoencoder. Figure 2 and Figure 3 Reveals one of the probabilistic methods)

[0015] Optionally, the autoencoders stored in the distributed public ledger make the search space evenly distributed among each node. The search node can then use the (verified) results from other peers and the (verified) autoencoders (and their IDs generated from their public keys as input to the autoencoders) in the distributed public ledger to determine the next neural architecture to search.

[0016] Optionally, the training to obtain the performance predictor specifically includes: obtaining k groups of training data to form a predictor training data set; training to obtain the performance predictor through the predictor training data set; wherein each group of training data includes: a candidate neural architecture and corresponding performance; the candidate neural architecture is formed by sampling positions located in the peripheral area of ​​the initial embedding position.

[0017] Optionally, the third node includes: a search sub-node for determining a candidate neural architecture; a working sub-node for determining the performance of the candidate neural architecture, training to obtain a performance predictor, and updating an embedding position; and a verification sub-node for executing a Merkle tree-based verification process in an untrusted environment to verify data information broadcast by other third nodes.

[0018] Optionally, the working sub-node is specifically used to: obtain the performance of the candidate neural architecture by testing the data set; or obtain the performance of the candidate neural architecture by collecting data information from other third nodes.

[0019] In a second aspect, an embodiment of the present application provides a collaborative neural architecture search method. The method includes: S1, determining and training an initial neural architecture; S2, mapping the initial neural architecture to an initial embedding position corresponding to a vector space through a variational autoencoder; S3, broadcasting data information of the initial neural architecture; S4, obtaining k groups of training data to form a predictor training data set, where k is a positive integer;

[0020] S5. The performance predictor is obtained by training the predictor training data set; S6. The embedded position is updated by gradient ascent based on the gradient direction provided by the performance predictor; the performance of the updated embedded position is better than the initial embedded position; S7. Steps S1 to S6 are repeated L times, where L is a positive integer.

[0021] Optionally, S8, jump to the embedding positions of the first N preferred neural architectures in the vector space for searching; wherein N is a positive integer, and the preferred neural architectures are sorted according to performance, and the neural architectures with better performance have earlier serial numbers.

[0022] One of the advantages of the embodiments of the present application is that the candidate neural architecture is mapped into a continuous encoding vector space through a suitable variational autoencoder. As a result, when performing a search, the third node can collaboratively search for the best neural architecture (by broadcasting the evaluation results of its neural architecture) based on the assumption that similar unit structures / neural architectures have similar performance on the same data set, thereby achieving effective evaluation of different neural architectures.

[0023] Another advantage of the embodiments of the present application is that the collaborative neural architecture search method can be transformed into any blockchain-related application, thereby encouraging people to invest their own computing resources. In this way, the search for neural architectures can continue when someone is willing to invest computing resources in exchange.

Brief Description of the Drawings

[0024] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0025] Figure 1 A schematic diagram of a collaborative neural architecture search device according to an embodiment of the present application;

[0026] Figure 2 A schematic diagram of a verification process of a collaborative neural architecture search device according to an embodiment of the present application;

[0027] Figure 3 Schematic diagram of the Merkle tree; the parameter adjustment process is the process responsible for training the neural architecture proposed by the search process. During the training process, weights can be shared between neural architectures instead of training from scratch. The hash is calculated based on the public key, the hash of the training dataset, the current model state (including parameters), and the gradient of each epoch. After the parameter adjustment process, all the hashes generated before are used to build the Merkle tree;

[0028] Figure 4This is a flow chart of a collaborative neural architecture search method according to an embodiment of the present application. [Specific implementation method]

[0029] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0030] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0031] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0032] Neural Architecture Optimization (also known as NAO) is a research program that uses gradient-based methods to optimize in a tighter vector space.

[0033] Specifically, it first trains the encoder and decoder using a set of randomly generated architectures and encodes the set of architectures into a set of vectors. Each architecture is evaluated for performance using a given dataset.

[0034] The architectures and their corresponding evaluated performance results are then used to train a set of vectors and predictors. A new set of vectors is generated by gradient ascent of the previous set of vectors using the gradient of the input with respect to the predictor output. Finally, the decoder trained in the first step can be used to decode the new set of architectures from these vectors. This whole process can be repeated any number of times to achieve better results.

[0035] However, in the process of implementing this application, the applicant found that this approach has the following technical problems: the operation steps of evaluating the adaptability of different neural architectures to a given data set are relatively time-consuming and computationally resource-intensive. In addition, in order to evaluate different neural architectures at the same time, an additional central coordination module is required to allocate computing resources.

[0036] Furthermore, there are technical issues such as how to properly divide the search space so that the communication between different evaluation processes is reduced (i.e., reducing the messages involved in weight sharing), how to provide guidance for the search of neural architectures to find better neural architectures, and how to determine whether the neural architecture search should stop.

[0037] The collaborative neural architecture search device and search method provided in the embodiments of the present application can effectively solve one or more of the above-mentioned problems by training and verifying the neural architecture in a distributed or discrete manner, realize effective evaluation of different neural architectures and facilitate their promotion and use.

[0038] Figure 1 A schematic diagram of a collaborative neural architecture search device provided in an embodiment of the present application. The device is a mathematical model search system for searching for a mathematical model with optimal weights / parameters for describing a data set. Figure 1 As shown, according to the functions performed, the system device can generally include three different nodes. In this article, for the sake of simplicity, they can be respectively referred to as "first node 110", "second node 120" and "third node 130". Each different node is used to run different processes including: initialization process, search space division process, parameter adjustment process, verification process and search process.

[0039] The "node" refers to a computing entity or functional module that is used to execute a specific process and realize the corresponding function. Figure 1 The rectangular box is used as an example to illustrate without limiting its specific implementation. It is similar to the technical term "node" in blockchain technology.

[0040] The first node 110 is used to provide a preset search space and data set. In other words, the first node 110 provides a search space and data set for neural architecture search, and distributes the settings of the data set and search space to other nodes.

[0041] The "search space" gives the types of neural architectures that can be searched or allowed to be generated and the performance evaluation methods of the neural architectures (for example, how to determine and test the accuracy of the neural architecture). The data set includes test data and training data. In some embodiments, the data set may also include verification data as needed in actual situations.

[0042] The second node 120 is used to generate a variational autoencoder and divide the search space so that the neural architecture can be combined together according to the structural similarity. The generated variational autoencoder can be broadcast to other nodes to realize the encoding and decoding of the neural architecture. In this article, "encoding" refers to the neural architecture as input, which is mapped to a continuously encoded vector space through a variational autoencoder. "Decoding" refers to a specific position in the vector space being remapped to a discontinuous neural architecture through a variational autoencoder.

[0043] The aforementioned vector space represents a low-dimensional space relative to the neural architecture input. The embedding position represents a specific vector in this continuous vector space. It is understandable that neural architectures with similar structures tend to be encoded in positions close to each other in the vector space, and vice versa.

[0044] Specifically, in the process of generating a variational autoencoder, the second node can first randomly generate several neural architectures according to the search space given by the first node. Then, the several randomly generated neural architectures are used as encoder training data sets to train and obtain a usable variational autoencoder. In addition, the trained variational autoencoder can be broadcast to other nodes for verification.

[0045] The verification process is the process of verifying whether a set of parameters has been adjusted for at least E epochs given the initial state of the model. The Merkle tree, each instance of the neural network model, and the gradients generated during the training process are used in the verification process. The verification process may include the following steps: First, the verification node will first verify the first and last pairs of hashes. Then, it randomly extracts a pair of hash values ​​generated during the training process. When using this verification procedure, there is no need to retrain the neural network. Instead, the verification node can verify the training process with a correct probability of 1-1 / ((E / 2-2)). Among them, E is the total number of rounds of training the neural network under the assumption that the attacker does not use computing resources for training and sends false information to the distributed network.

[0046] In this embodiment, the distribution of the encoding result of the variational autoencoder in the vector space can be a Gaussian distribution with a standard deviation of 1, so that the encoding results of similar neural architectures are combined together. Alternatively, in the case where the first node has a specified similarity measure, it is also possible to choose to use the distribution specified by the first node, such as a binomial distribution.

[0047] At least two third nodes 130 are provided, each of which is responsible for searching a portion of the search space. Different third nodes 130 cooperate with each other and can share data information with each other to help reduce the consumption of computing resources and time. In some embodiments, the data information includes: the performance of the neural architecture. Alternatively, the data information may also include the training results of the neural architecture.

[0048] For details, please continue to refer to Figure 1 The third node 130 includes at least one search subnode 131 and a working subnode 132. The search subnode 131 can select a candidate neural architecture. The working subnode 132 can train the candidate neural architecture and determine the performance evaluation result of the trained neural architecture in response to the request of the search subnode 131.

[0049] Specifically, the working subnode 132 can train the candidate neural architecture through the training data to obtain the corresponding neural model, and determine the performance (e.g., accuracy) of the neural model through the test data. Alternatively, the working subnode 132 can also obtain the training results and / or performance of the candidate neural architecture by collecting data information from other third nodes. Similarly, the training results of the working subnode 132 can also be obtained by self-training of the training data set, or by collecting data information from other third nodes.

[0050] In actual use, the third node 130 performs an optimization process of continuously updating the embedding position. It can update the embedding position according to the gradient direction provided by the performance trainer to obtain a neural architecture with better performance.

[0051] Specifically, you can choose to use the gradient ascent method to update the embedding position. In this article, "gradient ascent" refers to the process of gradually finding the embedding position with the largest gradient according to the direction of the gradient. The gradient ascent can be selected in any suitable type according to the needs of the actual situation, and is not limited to gradual ascent, as long as it can find the embedding position that maximizes the performance. Alternatively, the third node can also use other direction calculation methods.

[0052] In some embodiments, the step of updating the embedding position according to the gradient direction provided by the performance trainer specifically includes:

[0053] First, the initial embedding position is determined. Then, the gradient direction given by the performance predictor is calculated through the back-propagation algorithm. Finally, the updated result of the initial embedding position is obtained by gradient ascent. Among them, the updated result is the optimized embedding position with better performance than the initial embedding position.

[0054] Specifically, the performance predictor can be obtained by training with k sets of training data. Each set of training data contains a candidate neural architecture and corresponding performance. The aforementioned candidate neural architecture can be formed by a number of sampled embedding positions located in the surrounding area of ​​the initial embedding position.

[0055] It should be noted that the optimal neural architecture is only the current calculation result, not the global or final calculation result. The third node 130 can also repeat the position update and optimization process multiple times to obtain a better result. In other words, the aforementioned update step can be repeated multiple times.

[0056] In some embodiments, the search subnode 131 may encode the initial neural architecture through a variational autoencoder to obtain the aforementioned initial embedding position. The initial neural architecture, as an initialization result before the search, may be a neural architecture assigned by the aforementioned first node. Alternatively, the initial neural architecture may also be a neural architecture randomly generated by a third node.

[0057] For example, the search subnode 131 may use a unique identifier as a random seed and randomly generate it according to the search space given by the first node. The identifier may be assigned by the second node or automatically generated by the third node to distinguish different third nodes.

[0058] In a preferred embodiment, in order to search in a part of the search space where the optimal embedding position is more likely to appear, the third node can also directly jump to the first N preferred neural architectures as the search starting point, where N is a positive integer (e.g., 3, 4 or 5). In other words, the aforementioned initial embedding position is the encoding result of the variational autoencoder of the first N preferred neural architectures.

[0059] In this article, the preferred neural architecture is a neural architecture that is sorted according to performance based on the data information of all current third nodes. In other words, different candidate neural architectures have corresponding serial numbers according to the performance. (Neural architectures with better performance evaluation results have earlier serial numbers), thereby forming a preferred neural architecture sequence. Thus, the top N preferred neural architectures can be represented by the neural architecture ranked Nth according to performance among the search results of all current third nodes.

[0060] In other embodiments, in order to ensure the reliability and credibility of data information between different third nodes, the third node 130 may also include: a verification node 133, which is used to perform a verification process based on a Merkle tree. Of course, the verification process can be performed by an authorized node or by a competitive joint verification.

[0061] Specifically, Figure 2 A schematic diagram of the verification process provided in the embodiment of the present application, Figure 3 A schematic diagram of a Merkle tree provided in an embodiment of the present application. Figure 2 and 3As shown, during the verification process, the third node 130 does not necessarily need to retrain the entire network, and collective verification can also be performed by sampling a pair of adjacent hashes Hash n and Hash n+1.

[0062] Among them, during the training of candidate neural architectures by working nodes, the verification nodes calculate hashes and publish the trained candidate neural architectures (in the entire network), as well as information about calculating hashes for quick verification. Then, each verification node randomly selects a pair of verification hash sequences. The hash is then broadcast so that another verification node can calculate the Merkle root.

[0063] Based on the above-mentioned collaborative neural architecture search device, the embodiment of the present application also provides a collaborative neural architecture search method, which can be basically executed by the above-mentioned nodes to realize neural architecture search in a collaborative manner.

[0064] Figure 4 Flow chart of the collaborative neural architecture search method provided in the embodiment of the present application. Figure 4 As shown, the method may include the following steps:

[0065] S1. Determine the initial neural architecture g s .

[0066] The initial neural architecture can be generated based on a random seed (e.g., an adjacency matrix) or can be a specified neural architecture. The neural architecture can be considered as a graph model consisting of points and edges.

[0067] S2, train the initial neural architecture g s And map it to the vector space through the variational autoencoder to determine its embedding position z s .

[0068] Among them, the variational autoencoder is provided by the second node, which can encode the discrete neural architecture input into a continuous code in the vector space (neural architectures with similar structures and similar performance have embedding positions close to each other).

[0069] S3. Broadcast its data information to other nodes.

[0070] Among them, the data information may include: training results and performance evaluation results of the neural architecture.

[0071] S4. Receive or train k sets of training data to form predictor training data sets S and Z.

[0072] Each set of training data consists of a candidate neural architecture and its corresponding performance evaluation results. It can be specifically expressed by the following formula: S = {(g s ,p s),(g1,p1),(g2,p2)…(g k ,p k )}; Z = {(z s ,p s ),(z1,p1),(z2,p2)…(z k ,p k )}, p is the performance evaluation result of the neural architecture, and the subscript is the sequence number of the neural architecture. Several candidate neural architectures are formed by several sampled embedding positions located in the surrounding area of ​​the initial embedding position.

[0073] S5. Train the predictor training data set to obtain a performance predictor.

[0074] Specifically, the required performance predictor can be obtained by training through gradient descent and updating the weight of the performance predictor.

[0075] S6. Combined with the gradient of the current position, update the embedded position z by the following formula (1): s ':

[0076]

[0077] Where σ is the standard provided by the variational autoencoder, μ is the step size of the gradient ascent, and τ is a constant. The above updated embedding position z s ' is the embedding position with better performance, which can be re-decoded into a specific neural architecture through a variational autoencoder.

[0078] S7. Repeat steps S11 to S16 for L times or more.

[0079] Wherein, L is a positive integer, which can be set according to the actual needs (for example, it can be determined according to a preset upper limit of computing resources or a preset upper limit of computing time).

[0080] In some embodiments, please refer to Figure 4 After step S6, an optional jump step S8 may also be included to facilitate searching in a portion of the search space where a neural architecture with stronger performance is most likely to appear.

[0081] S8. Jump to the embedding position of the top N preferred neural architectures as the search starting point for searching.

[0082] Among them, the top N preferred neural architectures can be determined by the results of the previous search of this node or the search results of other nodes, indicating the Nth strongest neural architecture among the existing search results, ranked according to the performance evaluation results.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collaborative neural architecture search device, characterized in that: The system consists of a group of independent nodes that share a public ledger; The public ledger records the degree of adaptability of a series of mathematical models to the data set; The computing process running on the node includes: initialization process, search space division process, parameter adjustment process, verification process and search process; wherein the initialization process is a process for maintaining data supply, and in the initialization process, each node broadcasts the problem specification; The problem specification includes: the types of neural architectures allowed to be generated during the search process, the evaluation method, the termination criteria for parameter adjustment, the target dataset, the training dataset, and the test dataset; Wherein, the search space partitioning process is used to broadcast a fully trained variational autoencoder or a mathematical model that receives a random vector and outputs a graph; The variational autoencoder is used to map the neural architecture to the vector space so that the neural architectures with similar structures have adjacent embedding positions; At least two third nodes that cooperate with each other; Each of the third nodes is used to: determine the performance of multiple candidate neural architectures, train to obtain a performance predictor, and update the embedding position according to the gradient direction provided by the performance trainer; Among them, data information is shared between different third nodes; the data information includes: the performance of the candidate neural architecture.

2. The search device according to claim 1, characterized in that: The node running the initialization process is the first node, and the node running the search space partitioning process is the second node; Wherein, the first node is used to provide a preset search space and data set; the second node is used to generate the variational autoencoder and divide the search space; The variational autoencoder is obtained by training an encoder training data set and is broadcasted to the third node; The encoder training data set includes: a number of randomly generated neural architectures that satisfy the search space; The distribution of the neural architecture in the vector space is selected from: a Gaussian distribution with a standard deviation of 1 or a distribution specified by the first node.

3. The search device according to claim 1, characterized in that: The updating of the embedding position according to the gradient direction provided by the performance trainer specifically includes: Determine the initial embedding position; Calculating the gradient direction given by the performance predictor through a back-propagation algorithm; Obtaining an update result of the initial embedding position in a gradient ascent manner; The update result is: an optimized embedding position having a better performance than the initial embedding position.

4. The search device according to claim 3, characterized in that: The initial embedding position is the embedding position of the initial neural architecture in the vector space; The initial neural architecture is selected from: a neural architecture assigned to the first node or a randomly generated neural architecture; The randomly generated neural architecture is obtained by: An identifier is used as a random seed and is randomly generated according to the setting of the search space; the identifier is allocated by the second node or automatically generated by the third node.

5. The search device according to claim 3, characterized in that: The initial embedding position is the embedding position of the first N preferred neural architectures in the vector space; Wherein, N is a positive integer; the preferred neural architectures are sorted according to the performance evaluation results, and the neural architectures with better performance evaluation results have earlier serial numbers.

6. The search device according to claim 3, characterized in that: The training to obtain a performance predictor specifically includes: Obtain k groups of training data to form a predictor training data set; The performance predictor is obtained by training the predictor training data set; Each set of training data includes: a candidate neural architecture and corresponding performance; the candidate neural architecture is formed by sampling positions located in the peripheral area of ​​the initial embedding position.

7. The search device according to claim 1, characterized in that: The third node includes: Search child nodes to determine candidate neural architectures; A working subnode, for determining the performance of the candidate neural architecture, training a performance predictor, and updating an embedding position; The verification subnode is used to perform a Merkle tree-based verification process in an untrusted environment to verify the data information broadcast by the other third nodes.

8. The search device according to claim 7, characterized in that: The working sub-node is specifically used for: The performance of the candidate neural architecture is obtained by testing the data set; or the performance of the candidate neural architecture is obtained by collecting data information from other third nodes.

9. A collaborative neural architecture search method, characterized in that: include: S1. Determine and train the initial neural architecture; S2, mapping the initial neural architecture to an initial embedding position corresponding to the vector space through a variational autoencoder; S3, broadcasting the data information of the initial neural architecture; S4, obtaining k groups of training data to form a predictor training data set, where k is a positive integer; S5, training the performance predictor using the predictor training data set; S6. Based on the gradient direction provided by the performance predictor, updating the embedding position by gradient ascent; the performance of the updated embedding position is better than that of the initial embedding position; S7. Repeat steps S1 to S6 L times, where L is a positive integer.

10. The collaborative neural architecture search method according to claim 9, characterized in that: Also includes: S8, jump to the embedding position of the top N preferred neural architectures in the vector space for searching; Wherein, N is a positive integer, and the preferred neural architectures are sorted according to performance, and the neural architectures with better performance have earlier serial numbers.

Images