A neural network architecture search method and system based on polyhedron transformation representation

Abstract

The present invention provides a neural network architecture search method and system based on polyhedral transformation representation. The method comprises: obtaining the edit distance between any two neural network architectures in the neural network architecture, converting the edit distance metric space into a polyhedral structure; cascading multiple kernel functions and combining the results of the kernel function calculations; selecting one or more blocks as a basis, and finding the optimal neural network architecture by searching and evaluating different block structures; decomposing the multiple neural network architecture search tasks into multiple subtasks, and implementing parallel computing by communication between computing nodes. The system comprises: an edit distance calculation module, a calculation result combination module, an optimal architecture output module, and a task decomposition module. The polyhedral representation of the present invention achieves parallelism and scalability. By distributing the kernel function calculation operations across multiple faces, multiple kernel function calculations can be performed simultaneously, thereby improving computing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network architecture search, and in particular to a neural network architecture search method and system based on polyhedron transformation representation. Background Art

[0002] Neural architecture search is a cutting-edge research field aimed at automating the design of neural network models. Traditional neural network model design typically relies on manual experience and expertise, requiring significant time and effort through trial and error and fine-tuning. With advances in machine learning algorithms and search strategies, neural architecture search can automatically discover more efficient and optimized neural network architectures. Applications of Neural Architecture Search: Computer Vision: Neural architecture search has a wide range of applications in computer vision. It can be used for tasks such as image classification, object detection, and image segmentation. By automatically searching for the optimal network architecture, it improves model accuracy and generalization. Natural Language Processing: In natural language processing, neural architecture search can be used for tasks such as text classification, machine translation, and speech recognition. By searching for the optimal network architecture, it can enhance model performance in text processing and semantic understanding. Reinforcement Learning: Neural architecture search can also be applied to reinforcement learning, automatically designing neural network architectures suitable for specific environments and tasks, enhancing the learning and decision-making capabilities of intelligent agents in complex tasks. Automatically searching for the optimal network architecture can improve model performance, accelerate the model design process, and promote the development of artificial intelligence technology.

[0003] Prior art 1, application number: CN202210705263.7 discloses a visual analysis system and method for a neural network architecture space, the method comprising: receiving at least one neural network architecture; obtaining a graph edit distance between any two neural network architectures in the neural network architecture; obtaining a clustered hierarchical structure of the neural network architecture based on the graph edit distance, wherein each layer of the clustered hierarchical structure includes at least one neural network architecture category, and each category includes a neural network architecture; and visualizing at least one layer in the clustered hierarchical structure of the neural network architecture, wherein the visualization view provides a global overview of the neural network architecture and contextual information of any one of the neural network architectures; although it is beneficial for neural network architecture designers to design or neural network architecture search tools to search for neural network architectures with good performance on multiple data sets in the search space, and reduce the search space and computational cost of the neural network architecture search tool, the layer-based evolution guided by the edit distance limits the final prediction accuracy of the network. Changing only one layer at a time may have a relatively weak impact on the entire neural network architecture, so the evolution is often inefficient.

[0004] Prior art 2, application number: CN202211585844.8 discloses a network search method and device, electronic device, and storage medium. The network search method includes: obtaining multiple sampling networks from a sample data set; encoding the current sampling network among the multiple sampling networks to obtain a current structure code, and aggregating the current structure code with the historical structure code to obtain a model mean; obtaining an optimized structure code based on the optimization accuracy loss of the model mean; decoding the optimized structure code to obtain an optimized network structure, until the optimized network structure of all sampling networks is iteratively obtained based on the optimized network structure, determining a target network structure, and performing processing operations on the processing object based on the target network structure. Although it can improve the accuracy of network search, the evolutionary process requires calculating the traditional edit distance and comparing it with the historical architecture to find the optimal network. This calculation always has a very serious data dependency, resulting in the use of only sequential kernel functions and a lack of parallel opportunities. The sequential kernel function in network generation has therefore become a significant timing bottleneck.

[0005] Prior art three, application number: CN202010144350.0 discloses a parallelization method and device for neural network architecture search based on MPI, the method comprising: starting an MPI process according to the number of GPUs in the current multi-machine environment and arranging them in sequence; the started MPI process reads data from a specified position in the training set according to its own sequence number and performs gradient calculation; the GPU of each node performs gradient reduction calculation according to a hierarchical structure, and aggregates the calculation results to the first GPU in the GPU; the first GPU performs gradient full reduction calculation according to a ring structure; starting from the first GPU in each node, the gradient calculation results are broadcasted according to the hierarchical structure; and the weights and bias values ​​of the neural network are updated using the new gradient values. Although it can effectively accelerate the efficiency of neural network architecture search training and significantly reduce training time while ensuring the recognition rate of the neural network architecture search result model, thereby improving the efficiency of the automated deep learning process; however, the basic performance of the initial network based on the layer structure is relatively backward. Searching on a layer structure with poor basic performance results in suboptimal search results, which cannot provide the optimal search space and ultimately cannot find the best-performing neural network architecture.

[0006] Currently, the existing technologies one, two, and three calculate the traditional edit distance and compare it with the historical architecture, lacking parallel opportunities; the layer-based evolution guided by the edit distance limits the final prediction accuracy of the network; the initial network based on the layer structure has backward performance, and searching on a layer structure with poor basic performance results in suboptimal search results; therefore, the present invention provides a neural network architecture search method based on polyhedron transformation representation, which can abstract the sequential edit distance calculation into a polyhedron transformation representation, eliminate the calculation dependency at the new level of abstraction, and implement the calculation in parallel on the graphics processor, thereby improving the speed of running the search; block-based transformation is added for the new abstraction, thereby improving the prediction accuracy of the neural network model results. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention provides a neural network architecture search method based on polyhedron transformation representation, comprising the following steps:

[0008] receiving at least one neural network architecture; obtaining an edit distance between any two neural network architectures in the neural network architecture, and converting the metric space of the edit distance into a polyhedron structure, wherein each face of the polyhedron structure represents a kernel function operation;

[0009] In the order from simple to complex, multiple kernel functions are cascaded, and the output of each kernel function is used as the input of the next kernel function to form a multi-layer kernel function structure, combining the results of multiple kernel function calculations together;

[0010] Select one or more blocks from the block structure of the current neural network architecture as the basis, increase the number of layers of the selected blocks or change the parameters of the blocks, and find the optimal neural network architecture by searching and evaluating different block structures;

[0011] Decompose multiple neural network architecture search tasks into multiple subtasks, and different computing nodes execute different subtasks on the optimal neural network architecture. Use communication between computing nodes to achieve parallel computing; balance the load of different computing nodes to ensure that the computing tasks of each computing node are evenly distributed and executed.

[0012] Optionally, the process of obtaining the edit distance includes the following steps:

[0013] Obtain any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstract the first neural network architecture diagram and the second neural network architecture diagram;

[0014] Convert the hierarchical relationships in the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent the layers in the neural network and the edges represent the connection relationships between the layers;

[0015] In the metric space, a distance parameter is defined. The distance parameter is used to measure the distance between the two neural network architectures. The Euclidean distance function is selected as the distance function. The first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration.

[0016] In each iteration, the number of layers of nodes is compared, and the number of layers of |L| is selected as the distance calculation for the current iteration. During each iteration, the min(M,N) distance can be calculated in parallel, where M represents the number of layers of the first neural network architecture and N represents the number of layers of the second neural network architecture; |L| is the minimum number of layers to compare the two neural network architectures;

[0017] Until the distance calculation of all nodes is completed, the final distance result is output, which represents the distance between the two neural network architectures.

[0018] Optionally, the process of cascading multiple kernel functions includes the following steps:

[0019] Connect a new M-layer neural network architecture to two historical neural network architectures, where the two historical neural network architectures contain N layers and N' layers respectively; generate an M*N matrix of the original neural network comparison kernel, where each element in the matrix represents the connection strength between the M-th layer of the new neural network and the N-th layer of the historical neural network;

[0020] Add zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculations;

[0021] All newly generated neural networks are connected layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis.

[0022] Optionally, the process of splitting the M*(N+N') matrix includes the following steps:

[0023] Split the entire M*(N+N') matrix into multiple small matrices of matching sizes. The size of each small matrix is ​​the same as the size of the corresponding neural network graph, ensuring that the calculation task of each small matrix can be processed by one thread block;

[0024] On the parallel computing platform of the image processor, each small matrix is ​​assigned to a thread block for calculation. A thread block is a group of threads that execute in parallel, share memory and perform collaborative calculations.

[0025] In each thread block, parallel computing is used to calculate the small matrix; for each thread block, calculations are performed in the corresponding small matrix according to the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network;

[0026] Use the shared memory of the thread block to store the intermediate calculation results, merge the calculation results of each thread block, and obtain the calculation results of the entire M*(N+N') matrix;

[0027] All newly generated neural networks are connected layer by layer to generate a two-dimensional kernel for subsequent calculations and analysis.

[0028] Optionally, the process of searching for different block structures includes the following steps:

[0029] Add a selected block to extend the neural network architecture. After adding the new block, concatenate the new block with the existing block and adjust the weights, bias values, activation functions or regularization terms of the new block.

[0030] Based on the block transformation, the number of neurons in the selected block is increased and scaled. The evolutionary algorithm is used to search for the optimal network architecture and block parameters. According to the requirements, the objective function and constraints are defined to guide the search process.

[0031] After the conversion is completed, the new neural network architecture is evaluated and tested, and further optimized and adjusted based on the evaluation results.

[0032] Optionally, different computing nodes may execute different subtasks, including the following steps:

[0033] A master node without a GPU is used to generate asynchronous control logic. The master node is responsible for coordinating the allocation of logical tasks. Once the generation and training task logic is determined, the master node dispatches the task to an idle node through application software.

[0034] The central processing unit and artificial intelligence accelerator card are used on the slave nodes to perform independent tasks. The slave nodes are responsible for asynchronous network generation and training. Each slave node uses a graphics processor to generate and train the neural network architecture in parallel. At the same time, the slave node will train the current neural network and save the current model file through the shared file system.

[0035] Optionally, all nodes access shared data to select the best candidate neural network in the next iteration and optimize parameters; by accessing shared data, each node obtains the candidate neural networks and corresponding performance indicators generated by other nodes, selects the best candidate neural network, and updates parameters to optimize the model.

[0036] Optionally, the slave nodes read the historical candidate neural network structures in the shared file system respectively, and transform them into a new set of candidate neural networks on their respective nodes.

[0037] The present invention provides a neural network architecture search system based on polyhedron transformation representation, comprising:

[0038] An edit distance calculation module is responsible for receiving at least one neural network architecture; obtaining the edit distance between any two neural network architectures in the neural network architecture, and converting the metric space of the edit distance into a polyhedron structure, where each face of the polyhedron structure represents a kernel function operation;

[0039] The calculation result combination module is responsible for cascading multiple kernel functions in the order from simple to complex. The output of each kernel function is used as the input of the next kernel function to form a multi-layer kernel function structure, combining the results of multiple kernel functions together;

[0040] The optimal architecture output module is responsible for selecting one or more blocks from the block structure of the current neural network architecture as the basis, increasing the number of layers of the selected blocks or changing the parameters of the blocks, and finding the optimal neural network architecture by searching and evaluating different block structures;

[0041] The task decomposition module is responsible for decomposing multiple neural network architecture search tasks into multiple subtasks. Different computing nodes execute different subtasks on the optimal neural network architecture, and use communication between computing nodes to achieve parallel computing; balance the load of different computing nodes to ensure that the computing tasks of each computing node are evenly distributed and executed.

[0042] Optional, edit distance calculation module, including:

[0043] An abstraction processing unit is responsible for obtaining any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstracting the first neural network architecture diagram and the second neural network architecture diagram;

[0044] A relationship conversion unit is responsible for converting the hierarchical relationship between the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent the layers in the neural network and the edges represent the connection relationship between the layers;

[0045] The parallel traversal unit is responsible for defining a distance parameter in the metric space. The distance parameter is used to measure the distance between the two neural network architectures. The distance function selects the Euclidean distance function. The first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration.

[0046] The iterative processing unit is responsible for comparing the number of layers of nodes in each iteration and selecting the number of layers |L| as the distance calculation for the current iteration. During each iteration, it can parallelly calculate the min(M,N) distance, where M represents the number of layers of the first neural network architecture and N represents the number of layers of the second neural network architecture; |L| is the minimum number of layers to compare between the two neural network architectures;

[0047] The result output unit is responsible for completing the distance calculation of all nodes and outputting the final distance result, which represents the distance between the two neural network architectures.

[0048] The present invention introduces polyhedron representation to solve the problem that sequential calculation of kernel functions usually leads to performance bottlenecks. Polyhedron representation can achieve parallelism and scalability. By distributing the calculation operations of kernel functions on multiple faces, multiple kernel functions can be calculated simultaneously, thereby improving calculation efficiency; multi-dimensional kernel cascade is used to further optimize the search space and realize large-scale parallel calculation. By combining the calculation results of multiple kernel functions, the search space can be reduced and the calculation efficiency can be improved; at the same time, multi-dimensional kernel cascade can also accelerate the entire neural network architecture search process through parallel calculation; the purpose of introducing block-based transformation is to change the layer architecture of the network. Unlike traditional single-layer structure transformation, block-based transformation can more flexibly find the optimal network by changing a group of layer architectures at one time. The optimal neural network architecture can be found by searching and evaluating different block structures. In this way, the performance and efficiency of the neural network model can be further improved; expansion for large-scale clusters can make full use of the cluster's computing resources by allocating computing tasks to different computing nodes and using communication between computing nodes to achieve parallel computing, further improving the efficiency and scalability of the algorithm.

[0049] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 This is a flow chart of a neural network architecture search method based on polyhedron transformation representation in Example 1 of the present invention;

[0053] Figure 2 1 is a diagram of the process of obtaining the edit distance in Example 1 of the present invention;

[0054] Figure 3 Schematic diagram of a sequence kernel function based on edit distance in Example 2 of the present invention;

[0055] Figure 4 A schematic diagram of adding a parallel kernel function to the edit distance using the polyhedron representation method in Example 2 of the present invention;

[0056] Figure 5 In the embodiment of the present invention.

[0057] Figure 6 This is a schematic diagram of connecting one-dimensional kernel functions in series to maintain parallelism in Example 3 of the present invention;

[0058] Figure 7 Schematic diagram of large-scale parallel computing of two-dimensional kernel function splicing in Example 3 of the present invention;

[0059] Figure 8 4 is a diagram of the splitting process of the matrix M*(N+N') in Example 4 of the present invention;

[0060] Figure 9 A flowchart of assigning each small matrix to a thread block for calculation in embodiment 4 of the present invention;

[0061] Figure 10 A diagram showing a process of searching for different block structures in embodiment 5 of the present invention;

[0062] Figure 11 Schematic diagram of layer-based transformation in Example 5 of the present invention;

[0063] Figure 12 Schematic diagram of block-based transformation in embodiment 5 of the present invention;

[0064] Figure 13 This is a process diagram of different computing nodes executing different subtasks in Example 6 of the present invention;

[0065] Figure 14 This is a schematic diagram of decoupling task division and task execution in Example 6 of the present invention;

[0066] Figure 15 This is a block diagram of a neural network architecture search system based on polyhedron transformation representation in Example 7 of the present invention;

[0067] Figure 16 This is a block diagram of an edit distance calculation module in Example 8 of the present invention;

[0068] Figure 17This is a block diagram of the calculation result combination module in Example 9 of the present invention;

[0069] Figure 18 This is a block diagram of the optimal architecture output module in embodiment 10 of the present invention;

[0070] Figure 19 This is a block diagram of the task decomposition module in Example 11 of the present invention. DETAILED DESCRIPTION

[0071] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0072] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present application. The singular forms "a," "the," and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0073] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims. For those of ordinary skill in the art, the specific meanings of the above terms in this application will be understood according to specific circumstances.

[0074] Example 1: Figure 1 As shown, an embodiment of the present invention provides a neural network architecture search method based on polyhedron transformation representation, comprising the following steps:

[0075] S100: Receive at least one neural network architecture; obtain the edit distance between any two neural network architectures in the neural network architecture, and convert the metric space of the edit distance into a polyhedron structure, wherein each face of the polyhedron structure represents a kernel function operation;

[0076] S200: cascading multiple kernel functions in order from simple to complex, with the output of each kernel function serving as the input of the next kernel function, to form a multi-layer kernel function structure, combining the calculation results of multiple kernel functions;

[0077] S300: selecting one or more blocks from the block structure of the current neural network architecture as a basis, performing operations such as increasing the number of layers of the blocks or changing the parameters of the blocks on the selected blocks, and finding the optimal neural network architecture by searching and evaluating different block structures;

[0078] S400: Decomposing multiple neural network architecture search tasks into multiple subtasks, having different computing nodes execute different subtasks on the optimal neural network architecture, and utilizing communication between computing nodes to achieve parallel computing; balancing the loads of different computing nodes to ensure that computing tasks of each computing node are evenly distributed and executed;

[0079] The working principle and beneficial effects of the above technical solution are as follows: this embodiment first receives at least one neural network architecture; obtains the edit distance between any two neural network architectures in the neural network architecture, and converts the metric space of the edit distance into a polyhedron structure, wherein each face of the polyhedron structure represents a kernel function operation; secondly, multiple kernel functions are cascaded in order from simple to complex, and the output of each kernel function is used as the input of the next kernel function to form a multi-layer kernel function structure, and the results of multiple kernel function calculations are combined together; then one or more blocks are selected from the block structure of the current neural network architecture as a basis, and the selected blocks are subjected to operations such as increasing the number of layers of the blocks or changing the parameters of the blocks, and the optimal neural network architecture is found by searching and evaluating different block structures; finally, the multiple neural network architecture search tasks are decomposed into multiple subtasks, and different computing nodes execute different subtasks on the optimal neural network architecture, and parallel computing is realized by using communication between computing nodes; the load of different computing nodes is balanced to ensure that the computing tasks of each computing node are evenly distributed and executed; the above solution introduces polyhedron representation to solve The sequential calculation of kernel functions usually leads to performance bottlenecks. Polyhedral representation can achieve parallelism and scalability. By distributing the calculation operations of kernel functions on multiple faces, multiple kernel functions can be calculated simultaneously, thereby improving computational efficiency. Multi-dimensional kernel cascades are used to further optimize the search space and achieve large-scale parallel computing. By combining the calculation results of multiple kernel functions, the search space can be reduced and computational efficiency can be improved. At the same time, multi-dimensional kernel cascades can also accelerate the entire neural network architecture search process through parallel computing. The purpose of introducing block-based transformations is to change the layer architecture of the network. Unlike traditional single-layer structure transformations, block-based transformations can more flexibly find the optimal network by changing a group of layer architectures at one time. The optimal neural network architecture can be found by searching and evaluating different block structures. In this way, the performance and efficiency of the neural network model can be further improved. Expansion for large-scale clusters can make full use of the cluster's computing resources by allocating computing tasks to different computing nodes and using communication between computing nodes to achieve parallel computing, further improving the efficiency and scalability of the algorithm.

[0080] The embodiments of the present invention abstract sequential edit distance calculations into a representation of polyhedral transformations. At this new level of abstraction, computational dependencies are eliminated and parallelized on a graphics processor, improving search speed. Block-based transformations are added to the new abstraction, enhancing the predictive accuracy of neural network model results. Based on this abstract representation, optimizations are performed for search speed and result accuracy, achieving up to 10x acceleration for kernel functions, 3.92x acceleration for network generation, and a 20.03% performance gain in accuracy.

[0081] Example 2: Figure 2 As shown, based on Example 1, the process of obtaining the edit distance provided by the embodiment of the present invention includes the following steps:

[0082] S101: Obtain any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstract the first neural network architecture diagram and the second neural network architecture diagram;

[0083] S102: Converting hierarchical relationships in the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent layers in the neural network and the edges represent connection relationships between the layers;

[0084] S103: In the metric space, a distance parameter is defined, where the distance parameter is used to measure the distance between the two neural network architectures, and a Euclidean distance function is selected as the distance function; the first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration;

[0085] S104: In each iteration, compare the number of layers of the nodes and select the number of layers |L| as the distance calculation for the current iteration, and calculate the min(M,N) distance in parallel during each iteration, where M represents the number of layers of the first neural network architecture, N represents the number of layers of the second neural network architecture; |L| is the minimum number of layers for comparing the two neural network architectures;

[0086] S105: Repeat steps S103 and S104 until the distance calculation of all nodes is completed, and output the final distance result, which represents the distance between the two neural network architectures;

[0087] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, any two neural network architectures in the neural network architecture are first obtained, one of which is named the first neural network architecture diagram and the other is named the second neural network architecture diagram; the first neural network architecture diagram and the second neural network architecture diagram are abstracted into graphs; secondly, the hierarchical relationship in the first neural network architecture diagram and the second neural network architecture diagram is converted into nodes and edges in the graph, where the nodes represent the layers in the neural network and the edges represent the connection relationship between the layers; again, a distance parameter is defined in the metric space, and the distance parameter is used to measure the distance between the two neural network architectures, and the distance function selects the Euclidean distance function; the first neural network architecture diagram and the second neural network architecture diagram are traversed in parallel, and the distance between each node is calculated during the iteration; then, in each iteration, the number of layers of the nodes is compared, and the smaller number of layers is selected as the distance calculation for the current iteration, and the min(M,N) distance can be calculated in parallel during each iteration, where M represents the number of layers of the first neural network architecture and N represents the number of layers of the second neural network architecture ( Figure 3 and Figure 4 (where N represents an N-layer historical network, M represents an M-layer historical network, and → represents a dependency relationship); finally, the steps are repeated until the distance calculation for all nodes is completed, and the final distance result is output, representing the distance between the two neural network architectures. The above scheme can better understand and compare different neural network architectures. By abstracting the neural network architecture into a graph and calculating the distances between the graph nodes, the hierarchical relationships and connection structures between different architectures can be intuitively viewed. By comparing distances, the similarities and differences between different architectures can be determined, which can assist in the design and selection of neural network architectures. By comparing the distances between different architectures, architectures with high similarity to the target architecture can be found, providing reference and helpful for designing new neural network architectures or selecting appropriate pre-trained models. It can also be used for neural network architecture optimization and transfer learning. By comparing the distances between different architectures, similarities and differences between similar architectures can be identified, thereby guiding network parameter optimization and transfer learning strategies. By abstracting neural network architectures, measuring distances, and comparing them, the embodiments of the present invention provide a quantitative method to evaluate and compare the similarities and differences between different architectures, thereby assisting in the design, selection, and optimization of neural networks.

[0088] Example 3: Figure 5 As shown, based on Example 1, the process of cascading multiple kernel functions provided by the embodiment of the present invention includes the following steps:

[0089] S201: Connect a new M-layer neural network architecture with two historical neural network architectures, where the two historical neural network architectures include N layers and N' layers respectively; generate an original neural network comparison kernel of an M*N matrix, where each element in the matrix represents the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network;

[0090] S202: Add zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculation.

[0091] S203: Connect all newly generated neural networks layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis;

[0092] The working principle and beneficial effects of the above technical solution are as follows: This embodiment first connects a new M-layer neural network architecture with two historical neural network architectures, wherein the two historical neural network architectures respectively include N layers and N' layers; generates an original neural network comparison kernel of an M*N matrix, where each element in the matrix represents the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network; secondly, adds zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel execution of calculations ( Figure 6 and Figure 7 In the example, N represents an N-layer historical network, N' represents an N'-layer historical network, M represents an M-layer historical network, and M' represents an M'-layer historical network. → represents a dependency relationship. The 0 between adjacent N and N' is also zero pooling, and the 0 between adjacent M and M' is also zero pooling. Finally, all newly generated neural networks are connected layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis. The above scheme improves the expressive power of the neural network. By connecting neural networks at different levels, the features and information at different levels can be utilized to improve the learning and expressive power of the network. It improves the computational efficiency of the neural network: by adding zero padding and executing calculations in parallel, the computational speed can be accelerated, the computational efficiency of the neural network can be improved, and the model training and inference can be performed faster. It increases the flexibility and adaptability of the neural network: by connecting all newly generated neural networks layer by layer, the network structure can be flexibly adjusted according to specific tasks and needs, thereby improving the adaptability and generalization ability of the network. The new neural network architecture of this embodiment is of great significance in improving the expressive power, computational efficiency and adaptability of neural networks, and is expected to bring new breakthroughs to the development and application of neural networks.

[0093] Example 4: Figure 8 As shown, based on Example 3, the splitting process of the M*(N+N') matrix provided in this embodiment of the present invention includes the following steps:

[0094] S2021: Split the entire M*(N+N') matrix into multiple small matrices of matching sizes. The size of each small matrix is ​​the same as the size of the corresponding neural network graph, ensuring that the computation task of each small matrix can be processed by one thread block.

[0095] S2022: On the GPU parallel computing platform, each small matrix is ​​assigned to a thread block for computation. A thread block is a group of threads that execute in parallel, share memory, and perform collaborative computations.

[0096] S2023: In each thread block, use parallel computing to calculate the small matrix; for each thread block, calculate in the corresponding small matrix according to the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network;

[0097] S2024: Using the shared memory of the thread blocks to store the intermediate calculation results, merging the calculation results of each thread block to obtain the calculation result of the entire M*(N+N') matrix;

[0098] S2025: Repeat steps S2021 to S2024 to connect all newly generated neural networks layer by layer to generate a two-dimensional kernel for subsequent calculations and analysis;

[0099] The working principle and beneficial effects of the above technical solution are as follows: First, the embodiment splits the entire M*(N+N') matrix into multiple small matrices of matching sizes. The size of each small matrix is ​​the same as the size of the corresponding neural network graph, ensuring that the calculation task of each small matrix can be processed by a thread block; secondly, on the image processor parallel computing platform, each small matrix is ​​assigned to a thread block for calculation (such as Figure 9 As shown), a thread block is a group of threads that execute in parallel, share memory and collaborate on computing; then in each thread block, parallel computing is used to calculate the small matrix; for each thread block, calculations are performed in the corresponding small matrix according to the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network; the shared memory of the thread block is used again to store the intermediate calculation results, and the calculation results of each thread block are merged to obtain the calculation results of the entire M*(N+N') matrix; finally, steps S2021-S2024 are repeated to connect all the newly generated neural networks layer by layer to generate a two-dimensional kernel for subsequent calculations and analysis; the above scheme optimizes parallel computing and task allocation by splitting the matrix and assigning each small matrix to a thread block. By utilizing the characteristics of the parallel computing platform, the computing process of the neural network can be accelerated, and the computing efficiency and performance can be improved; at the same time, by using shared memory and optimized computing algorithms, data transmission and access delays can be reduced, and the computing speed can be further improved; it can be applied to computing tasks of various neural networks, improve the training and inference speed of neural networks, and improve the efficiency and performance of the entire system.

[0100] Example 5: Figure 10 As shown, based on Example 1, the process of searching for different block structures provided by the embodiment of the present invention includes the following steps:

[0101] S301: Add a selected block to expand the neural network architecture. After adding the new block, concatenate the new block with the existing block and adjust the weight, bias value, activation function or regularization term of the new block;

[0102] S302: Based on the block conversion, the number of neurons in the selected block is increased and scaled; an evolutionary algorithm is used to search for the optimal network architecture and block parameters, and the objective function and constraints are defined to guide the search process according to the requirements;

[0103] S303: After the conversion is completed, the new neural network architecture is evaluated and tested, and further optimized and adjusted based on the evaluation results;

[0104] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, a selected block is first added to expand the neural network architecture. After adding the new block, the new block is connected in series with the existing block ( Figure 11 is a schematic diagram of layer-based transformation), adjust the weights, bias values, activation functions and regularization terms of the new blocks ( Figure 12 ( is a block-based transformation diagram); secondly, based on the block-based transformation, the number of neurons in the selected block is increased and scaled; the evolutionary algorithm is used to search for the optimal network architecture and block parameters, and the objective function and constraints are defined according to the needs to guide the search process; finally, after the transformation is completed, the new neural network architecture is evaluated and tested, and further optimized and adjusted based on the evaluation results; the above scheme can increase the depth and complexity of the neural network by adding new blocks, provide more expressive power, and enable the neural network to learn more complex patterns and features, thereby improving the performance and generalization ability of the neural network; by connecting the new block in series with the existing block and adjusting the parameters of the new block, the transmission and fusion of information is promoted, the information flow of the neural network is improved, and the neural network can better utilize the information of the input data, thereby improving the performance of the neural network; by adjusting the number of neurons in the selected block, the neural network can be further optimized and adjusted. Row scaling can adjust the capacity and complexity of the block, so that the neural network can adapt to tasks and data sets of different sizes, and improve the flexibility and adaptability of the network; using the evolutionary algorithm to search for the optimal neural network architecture and block parameters can automatically optimize the network, and by defining the objective function and constraints, the search process can be guided to find the optimal neural network structure and parameter settings, thereby improving the performance of the neural network; by evaluating and testing the new neural network architecture, the performance and generalization ability of the neural network can be understood, and based on the evaluation results, the network can be further optimized and adjusted to obtain better performance; the embodiments of the present invention optimize the neural network architecture and improve the performance and adaptability of the neural network by adding new blocks, connection and parameter adjustment, block conversion and scaling, evolutionary algorithm search and other methods, so as to make the neural network better adapt to different tasks and data sets and achieve higher performance and effects.

[0105] Example 6: Figure 13 As shown, based on Example 1, the process of different computing nodes performing different subtasks provided by the embodiment of the present invention includes the following steps:

[0106] S401: Generate asynchronous control logic using a master node without a graphics processor. The master node is responsible for coordinating the allocation of logical tasks. Once the generation and training task logic is determined, the master node dispatches the task to an idle node through application software.

[0107] S402: Using the central processing unit and artificial intelligence accelerator card to execute independent tasks on the slave nodes, the slave nodes are responsible for asynchronously performing network generation and training. Each slave node uses a graphics processing unit to generate and train the neural network architecture in parallel. Simultaneously, the slave nodes train the current neural network and save the current model file through the shared file system.

[0108] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, a master node without a graphics processor is first used to generate asynchronous control logic, and the master node is responsible for coordinating the logical task allocation ( Figure 14 Schematic diagram for decoupling task division and task execution); once the generation and training task logic is determined, the master node dispatches the task to the idle node through the application software; secondly, the central processing unit and artificial intelligence accelerator card are used on the slave node to perform independent tasks, and the slave node is responsible for asynchronous execution of network generation and training. Each slave node uses the graphics processor to generate and train the neural network architecture in parallel; at the same time, the slave node will train the current neural network and save the current model file through the shared file system; all nodes in the above scheme can access the shared data to select the best candidate neural network in the next iteration and optimize the parameters; by accessing the shared data, each node can obtain the candidate neural network and corresponding performance indicators generated by other nodes. The node can select the best candidate neural network and update the parameters to optimize the model; many slave nodes read the historical candidate neural network structure in the shared file system respectively, and transform it on their respective nodes. A new set of candidate neural networks is generated; each node compares the generated candidate neural network with the historical structure to evaluate its performance and improvement potential; the generation and training process of the neural network is accelerated by distributed computing. By assigning tasks to multiple slave nodes and utilizing the parallel computing capabilities of the central processing unit and the artificial intelligence accelerator card, the training time can be greatly shortened; at the same time, by asynchronously executing network generation and training, computing resources can be better utilized and the efficiency of the system can be improved; in addition, by using a shared file system to save the current model file, the latest model can be accessed at any time for subsequent reasoning or other operations. The distributed computing solution can also achieve flexible task scheduling and resource management, thereby better meeting different application requirements; this embodiment can improve the efficiency of neural network generation and training, and can better utilize computing resources to achieve faster and more flexible task processing.

[0109] Example 7: Figure 15As shown, an embodiment of the present invention provides a neural network architecture search system based on polyhedron transformation representation, comprising:

[0110] An edit distance calculation module is responsible for receiving at least one neural network architecture; obtaining the edit distance between any two neural network architectures in the neural network architecture, and converting the metric space of the edit distance into a polyhedron structure, where each face of the polyhedron structure represents a kernel function operation;

[0111] The calculation result combination module is responsible for cascading multiple kernel functions in the order from simple to complex. The output of each kernel function is used as the input of the next kernel function to form a multi-layer kernel function structure, combining the results of multiple kernel functions together;

[0112] The optimal architecture output module is responsible for selecting one or more blocks from the block structure of the current neural network architecture as the basis, and performing operations such as increasing the number of layers of the selected blocks or changing the parameters of the blocks. By searching and evaluating different block structures, the optimal neural network architecture is found;

[0113] The task decomposition module is responsible for decomposing multiple neural network architecture search tasks into multiple subtasks. Different computing nodes execute different subtasks on the optimal neural network architecture, and use communication between computing nodes to achieve parallel computing; balance the load of different computing nodes to ensure that the computing tasks of each computing node are evenly distributed and executed;

[0114] The working principle and beneficial effects of the above technical solution are as follows: the edit distance calculation module of this embodiment receives at least one neural network architecture; obtains the edit distance between any two neural network architectures in the neural network architecture, and converts the metric space of the edit distance into a polyhedron structure, wherein each face of the polyhedron structure represents a kernel function operation; the calculation result combination module cascades multiple kernel functions in the order from simple to complex, and the output of each kernel function is used as the input of the next kernel function to form a multi-layer kernel function structure, and combines the results of multiple kernel function calculations together; the optimal architecture output module selects one or more blocks from the block structure of the current neural network architecture as the basis, and performs operations such as increasing the number of layers of the selected blocks or changing the parameters of the blocks, and finds the optimal neural network architecture by searching and evaluating different block structures; the task decomposition module decomposes multiple neural network architecture search tasks into multiple subtasks, and different computing nodes execute different subtasks on the optimal neural network architecture, and uses communication between computing nodes to achieve parallel computing; balances the load of different computing nodes to ensure that the computing tasks of each computing node are evenly distributed and executed; The above scheme introduces polyhedron representation to solve the problem that sequential calculation of kernel functions usually leads to performance bottlenecks. Polyhedron representation can achieve parallelism and scalability. By distributing the calculation operations of kernel functions on multiple faces, multiple kernel functions can be calculated simultaneously, thereby improving computational efficiency. Multi-dimensional kernel cascades are used to further optimize the search space and achieve large-scale parallel computing. By combining the calculation results of multiple kernel functions, the search space can be reduced and computational efficiency can be improved. At the same time, multi-dimensional kernel cascades can also accelerate the entire neural network architecture search process through parallel computing. The purpose of introducing block-based transformation is to change the layer architecture of the network. Unlike traditional single-layer structure transformation, block-based transformation can more flexibly find the optimal network by changing a group of layer architectures at one time. The optimal neural network architecture can be found by searching and evaluating different block structures. In this way, the performance and efficiency of the neural network model can be further improved. The expansion to large-scale clusters can make full use of the cluster's computing resources by allocating computing tasks to different computing nodes and using communication between computing nodes to achieve parallel computing, further improving the efficiency and scalability of the algorithm.

[0115] The embodiments of the present invention abstract sequential edit distance calculations into a representation of polyhedral transformations. At this new level of abstraction, computational dependencies are eliminated and parallelized on a graphics processor, improving search speed. Block-based transformations are added to the new abstraction, enhancing the predictive accuracy of neural network model results. Based on this abstract representation, optimizations are performed for search speed and result accuracy, achieving up to 10x acceleration for kernel functions, 3.92x acceleration for network generation, and a 20.03% performance gain in accuracy.

[0116] Example 8: Figure 16 As shown, based on Example 7, the edit distance calculation module provided by this embodiment of the present invention includes:

[0117] An abstraction processing unit is responsible for obtaining any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstracting the first neural network architecture diagram and the second neural network architecture diagram;

[0118] A relationship conversion unit is responsible for converting the hierarchical relationship between the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent the layers in the neural network and the edges represent the connection relationship between the layers;

[0119] The parallel traversal unit is responsible for defining a distance parameter in the metric space. The distance parameter is used to measure the distance between the two neural network architectures. The distance function selects the Euclidean distance function. The first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration.

[0120] The iterative processing unit is responsible for comparing the number of layers of nodes in each iteration and selecting the number of layers |L| as the distance calculation for the current iteration. During each iteration, it can parallelly calculate the min(M,N) distance, where M represents the number of layers of the first neural network architecture and N represents the number of layers of the second neural network architecture; |L| is the minimum number of layers to compare between the two neural network architectures;

[0121] The result output unit is responsible for completing the distance calculation of all nodes and outputting the final distance result, which represents the distance between the two neural network architectures;

[0122] The working principle and beneficial effects of the above technical solution are as follows: the abstract processing unit of this embodiment obtains any two neural network architectures in the neural network architecture, one of which is named the first neural network architecture diagram and the other is named the second neural network architecture diagram; the first neural network architecture diagram and the second neural network architecture diagram are abstracted; the relationship conversion unit converts the hierarchical relationship in the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, the nodes represent the layers in the neural network, and the edges represent the connection relationship between the layers; the parallel traversal unit defines a distance parameter in the metric space, the distance parameter is used to measure the distance between the two neural network architectures, and the distance function selects the Euclidean distance function; the first neural network architecture diagram and the second neural network architecture diagram are traversed in parallel During each iteration, the distance between each node is calculated. During each iteration, the iterative processing unit compares the number of layers of the nodes and selects the number of layers with |L| as the distance calculation for the current iteration. During each iteration, the min(M,N) distance is calculated in parallel, where M represents the number of layers in the first neural network architecture and N represents the number of layers in the second neural network architecture. |L| is the minimum number of layers required to compare the two neural network architectures. The result output unit completes the distance calculation for all nodes and outputs the final distance result, which represents the distance between the two neural network architectures. This scheme can better understand and compare different neural network architectures. By abstracting the neural network architecture into a graph and calculating the distance between the graph nodes, the hierarchical relationships and connectivity between different architectures can be intuitively visualized. By comparing distances, the similarities and differences between different architectures can be determined, which can assist in the design and selection of neural network architectures. By comparing the distances between different architectures, architectures with high similarity to the target architecture can be identified, providing reference and helpful for designing new neural network architectures or selecting appropriate pre-trained models. It can also be used for neural network architecture optimization and transfer learning. By comparing the distances between different architectures, we can identify commonalities and differences between similar architectures, thereby guiding network parameter optimization and transfer learning strategies. By abstracting, measuring, and comparing neural network architectures, the present invention provides a quantitative approach to assessing and comparing similarities and differences between different architectures, thereby assisting in the design, selection, and optimization of neural networks.

[0123] Example 9: Figure 17 As shown, based on Example 7, the calculation result combination module provided by the embodiment of the present invention includes:

[0124] The architecture connection unit is responsible for connecting a new M-layer neural network architecture with two historical neural network architectures, where the two historical neural network architectures contain N layers and N' layers respectively; generating an M*N matrix of the original neural network comparison kernel, where each element in the matrix represents the connection strength between the M-th layer of the new neural network and the N-th layer of the historical neural network;

[0125] The matrix generation unit is responsible for adding zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculation.

[0126] The kernel generation unit is responsible for connecting all newly generated neural networks layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis;

[0127] The working principle and beneficial effects of the above technical solution are as follows: the architecture connection unit of this embodiment connects a new M-layer neural network architecture with two historical neural network architectures, wherein the two historical neural network architectures respectively include N layers and N' layers; generates an original neural network comparison kernel of an M*N matrix, where each element in the matrix represents the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network; the matrix generation unit adds zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculation; the kernel generation unit connects all newly generated neural networks layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis; the above solution improves the expressive power of the neural network. By connecting neural networks at different levels, the features and information at different levels can be utilized to improve the learning and expressive power of the network; the computational efficiency of the neural network is improved: by adding zero padding and performing calculations in parallel, the calculation speed can be accelerated and the computational efficiency of the neural network can be improved, thereby performing model training and inference faster; the flexibility and adaptability of the neural network are increased: by connecting all newly generated neural networks layer by layer, the network structure can be flexibly adjusted according to specific tasks and needs, thereby improving the adaptability and generalization ability of the network. The new neural network architecture of this embodiment is of great significance in improving the expressive power, computational efficiency, and adaptability of neural networks, and is expected to bring new breakthroughs to the development and application of neural networks.

[0128] Example 10: Figure 18 As shown, based on Example 7, the optimal architecture output module provided by this embodiment of the present invention includes:

[0129] The layer transformer is responsible for adding a selected block to extend the neural network architecture. After adding the new block, it is concatenated with the existing blocks and the weights, bias values, activation functions, and regularization terms of the new block are adjusted.

[0130] The block transformation unit is responsible for block-based transformation and scaling the number of neurons in the selected block. It uses evolutionary algorithms to search for the optimal network architecture and block parameters, and defines objective functions and constraints to guide the search process according to requirements.

[0131] The evaluation unit is responsible for evaluating and testing the new neural network architecture after the conversion is completed, and further optimizing and adjusting it based on the evaluation results;

[0132] The working principle and beneficial effects of the above technical solution are as follows: the layer transformation unit of this embodiment adds a selected block to expand the neural network architecture. After adding the new block, the new block is connected in series with the existing block, and the weight, bias value, activation function or regularization term of the new block are adjusted; the block transformation unit scales the parameters of the number of neurons in the selected block based on the conversion of the block; the evolutionary algorithm is used to search for the optimal network architecture and block parameters, and the objective function and constraints are defined according to the needs to guide the search process; after the evaluation unit completes the conversion, the new neural network architecture is evaluated and tested, and further optimized and adjusted according to the evaluation results; the above solution can increase the depth and complexity of the neural network by adding new blocks, provide more expressive power, and enable the neural network to learn more complex patterns and features, thereby improving the performance and generalization ability of the neural network; by connecting the new block in series with the existing block and adjusting the parameters of the new block, the transmission and fusion of information is promoted, the information fluidity of the neural network is improved, and the neural network Better use is made of the information of input data, thereby improving the performance of the neural network; by scaling the parameters of the number of neurons in the selected block, the capacity and complexity of the block can be adjusted, so that the neural network can adapt to tasks and data sets of different sizes, thereby improving the flexibility and adaptability of the network; using an evolutionary algorithm to search for the optimal neural network architecture and block parameters, the network can be automatically optimized, and by defining the objective function and constraints, the search process can be guided to find the optimal neural network structure and parameter settings, thereby improving the performance of the neural network; by evaluating and testing the new neural network architecture, the performance and generalization ability of the neural network can be understood, and based on the evaluation results, the network can be further optimized and adjusted to obtain better performance; the embodiments of the present invention optimize the neural network architecture and improve the performance and adaptability of the neural network by adding new blocks, connection and parameter adjustment, block conversion and scaling, evolutionary algorithm search and other methods, so that the neural network can be better adapted to different tasks and data sets, achieving higher performance and effects.

[0133] Example 11: Figure 19 As shown, based on Example 7, the task decomposition module provided by this embodiment of the present invention includes:

[0134] The logic control unit is responsible for generating asynchronous control logic using a master node without a graphics processor. The master node is responsible for coordinating the allocation of logical tasks. Once the generation and training task logic is determined, the master node dispatches the task to an idle node through application software.

[0135] The node processing unit is responsible for executing independent tasks on the slave nodes using the central processing unit and artificial intelligence accelerator card. The slave nodes are responsible for asynchronously performing network generation and training. Each slave node uses a graphics processing unit to generate and train the neural network architecture in parallel. At the same time, the slave node will train the current neural network and save the current model file through the shared file system.

[0136] The working principle and beneficial effects of the above technical solution are as follows: the logic control unit of this embodiment uses a master node without a graphics processor to generate asynchronous control logic, and the master node is responsible for coordinating the allocation of logic tasks; once the generation and training task logic is determined, the master node dispatches the task to the idle node through the application software; the node processing unit uses the central processing unit and the artificial intelligence accelerator card on the slave node to perform independent tasks, and the slave node is responsible for asynchronous execution of network generation and training. Each slave node uses a graphics processor to generate and train the neural network architecture in parallel; at the same time, the slave node will train the current neural network and save the current model file through a shared file system; all nodes in the above solution can access shared data to select the best candidate neural network in the next iteration and optimize the parameters; by accessing shared data, each node can obtain the candidate neural networks and corresponding performance indicators generated by other nodes. The node can select the optimal candidate neural network and update the parameters to optimize the model; many slave nodes read the shared The historical candidate neural network structure in the file system is converted into a set of new candidate neural networks on their respective nodes; each node compares the generated candidate neural network with the historical structure to evaluate its performance and improvement potential; the generation and training process of the neural network is accelerated by distributed computing, and the training time can be greatly shortened by assigning tasks to multiple slave nodes and utilizing the parallel computing capabilities of the central processing unit and the artificial intelligence accelerator card; at the same time, by asynchronously executing network generation and training, computing resources can be better utilized and the efficiency of the system can be improved; in addition, by using a shared file system to save the current model file, the latest model can be accessed at any time for subsequent reasoning or other operations. The distributed computing solution can also realize flexible task scheduling and resource management, so as to better meet different application requirements; this embodiment can improve the efficiency of neural network generation and training, and can better utilize computing resources to achieve faster and more flexible task processing.

[0137] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A neural network architecture search method based on polyhedron transformation representation, characterized in that: The following steps are involved: receiving at least one neural network architecture, obtaining an edit distance between any two neural network architectures in the received architecture, and converting the metric space of the edit distance into a polyhedron structure, wherein each face of the polyhedron represents a kernel function operation; Cascade multiple kernel functions in order of increasing complexity, where the output of each kernel function serves as the input of the next kernel function, forming a multi-layer kernel function structure and combining the results calculated by multiple kernel functions; Select one or more blocks from the block structure of the current neural network architecture as the basis, increase the number of layers of the selected blocks or change the parameters of the blocks; search and evaluate different block structures to find the optimal neural network architecture; Decompose multiple neural network architecture search tasks into multiple subtasks, and different computing nodes execute different subtasks on the optimal neural network architecture; use communication between computing nodes to achieve parallel computing, balance the workload of different computing nodes, and ensure the uniform distribution and execution of computing tasks; The process of cascading multiple kernel functions includes the following steps: Connect a new M-layer neural network architecture to two historical neural network architectures, where the two historical neural network architectures contain N layers and N' layers respectively; generate an M*N matrix of the original neural network comparison kernel, where each element in the matrix represents the connection strength between the M-th layer of the new neural network and the N-th layer of the historical neural network; Add zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculations; Connect all newly generated neural networks layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis; The process of splitting the M*(N + N') matrix includes the following steps: Split the entire M*(N + N') matrix into multiple small matrices of matching sizes. The size of each small matrix is ​​the same as the size of the corresponding neural network graph, ensuring that the calculation task of each small matrix is ​​processed by one thread block; On the parallel computing platform of the image processor, each small matrix is ​​assigned to a thread block for calculation. A thread block is a group of threads that execute in parallel, share memory and perform collaborative calculations. In each thread block, parallel computing is used to calculate the small matrix; for each thread block, calculations are performed in the corresponding small matrix according to the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network; Use the shared memory of the thread block to store the intermediate calculation results, merge the calculation results of each thread block, and obtain the calculation results of the entire M*(N + N') matrix; All newly generated neural networks are connected layer by layer to generate a two-dimensional kernel for subsequent calculations and analysis.

2. The neural network architecture search method based on polyhedron transformation representation according to claim 1, characterized in that: The process of obtaining the edit distance includes the following steps: Obtain any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstract the first neural network architecture diagram and the second neural network architecture diagram; Convert the hierarchical relationships in the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent the layers in the neural network and the edges represent the connection relationships between the layers; In the metric space, a distance parameter is defined. The distance parameter is used to measure the distance between the two neural network architectures. The Euclidean distance function is selected as the distance function. The first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration. In each iteration, the number of layers of nodes is compared and |L| layers are selected as the distance calculation for the current iteration. During each iteration, the min(X, Y) distance can be calculated in parallel, where X represents the number of layers of the first neural network architecture and Y represents the number of layers of the second neural network architecture; |L| is the minimum number of layers of the two neural network architectures to be compared. Until the distance calculation of all nodes is completed, the final distance result is output, which represents the distance between the two neural network architectures.

3. The neural network architecture search method based on polyhedron transformation representation according to claim 1, characterized in that: The process of different computing nodes executing different subtasks includes the following steps: A master node without a GPU is used to generate asynchronous control logic. The master node is responsible for coordinating the allocation of logical tasks. Once the generation and training task logic is determined, the master node dispatches the task to an idle node through application software. The central processing unit and artificial intelligence accelerator card are used on the slave nodes to perform independent tasks. The slave nodes are responsible for asynchronous network generation and training. Each slave node uses a graphics processor to generate and train the neural network architecture in parallel. At the same time, the slave node will train the current neural network and save the current model file through the shared file system.

4. The neural network architecture search method based on polyhedron transformation representation according to claim 3, characterized in that: All nodes access shared data to select the best candidate neural network in the next iteration and optimize the parameters; by accessing shared data, each node obtains the candidate neural networks and corresponding performance indicators generated by other nodes, selects the best candidate neural network, and updates the parameters to optimize the model.

5. The neural network architecture search method based on polyhedron transformation representation according to claim 3, characterized in that: The slave nodes read the historical candidate neural network structures in the shared file system respectively, and transform them into a new set of candidate neural networks on their respective nodes.

6. A neural network architecture search system based on polyhedron transformation representation, characterized in that: Include: An edit distance calculation module is responsible for receiving at least one neural network architecture; obtaining the edit distance between any two neural network architectures in the neural network architecture, and converting the metric space of the edit distance into a polyhedron structure, where each face of the polyhedron structure represents a kernel function operation; The calculation result combination module is responsible for cascading multiple kernel functions in the order from simple to complex. The output of each kernel function is used as the input of the next kernel function, forming a multi-layer kernel function structure and combining the results of multiple kernel functions. The optimal architecture output module is responsible for selecting one or more blocks from the block structure of the current neural network architecture as the basis, increasing the number of layers of the selected blocks or changing the parameters of the blocks, and finding the optimal neural network architecture by searching and evaluating different block structures; The task decomposition module is responsible for decomposing multiple neural network architecture search tasks into multiple subtasks. Different computing nodes execute different subtasks on the optimal neural network architecture, and use communication between computing nodes to achieve parallel computing; balance the load of different computing nodes to ensure that the computing tasks of each computing node are evenly distributed and executed; The process of cascading multiple kernel functions includes: Connect a new M-layer neural network architecture to two historical neural network architectures, where the two historical neural network architectures contain N layers and N' layers respectively; generate an M*N matrix of the original neural network comparison kernel, where each element in the matrix represents the connection strength between the M-th layer of the new neural network and the N-th layer of the historical neural network; Add zero padding between the last layer of the current neural network and the first layer of the additional network to generate an M*(N+N') matrix for parallel calculations; Connect all newly generated neural networks layer by layer on the original neural network comparison kernel to generate a two-dimensional kernel for calculation and analysis; The splitting process of the M*(N + N') matrix includes: Split the entire M*(N + N') matrix into multiple small matrices of matching sizes. The size of each small matrix is ​​the same as the size of the corresponding neural network graph, ensuring that the calculation task of each small matrix is ​​processed by one thread block; On the parallel computing platform of the image processor, each small matrix is ​​assigned to a thread block for calculation. A thread block is a group of threads that execute in parallel, share memory and perform collaborative calculations. In each thread block, parallel computing is used to calculate the small matrix; for each thread block, calculations are performed in the corresponding small matrix according to the connection strength between the Mth layer of the new neural network and the Nth layer of the historical neural network; Use the shared memory of the thread block to store the intermediate calculation results, merge the calculation results of each thread block, and obtain the calculation results of the entire M*(N + N') matrix; All newly generated neural networks are connected layer by layer to generate a two-dimensional kernel for subsequent calculations and analysis.

7. The neural network architecture search system based on polyhedron transformation representation according to claim 6, characterized in that: Edit distance calculation module, including: An abstraction processing unit is responsible for obtaining any two neural network architectures in the neural network architecture, one of which is named a first neural network architecture diagram and the other is named a second neural network architecture diagram; abstracting the first neural network architecture diagram and the second neural network architecture diagram; A relationship conversion unit is responsible for converting the hierarchical relationship between the first neural network architecture diagram and the second neural network architecture diagram into nodes and edges in the diagram, where the nodes represent the layers in the neural network and the edges represent the connection relationship between the layers; The parallel traversal unit is responsible for defining a distance parameter in the metric space. The distance parameter is used to measure the distance between the two neural network architectures. The distance function selects the Euclidean distance function. The first neural network architecture graph and the second neural network architecture graph are traversed in parallel, and the distance between each node is calculated during the iteration. The iterative processing unit is responsible for comparing the number of layers of nodes in each iteration and selecting the number of layers |L| as the distance calculation for the current iteration. During each iteration, it can parallelly calculate the min(X, Y) distance, where X represents the number of layers of the first neural network architecture and Y represents the number of layers of the second neural network architecture; |L| is the minimum number of layers to compare between the two neural network architectures; The result output unit is responsible for completing the distance calculation of all nodes and outputting the final distance result, which represents the distance between the two neural network architectures.

Images