Neural network structure automatic search method and device for small sample fitting

Abstract

This invention provides an automatic search method and apparatus for neural network structures for few-sample fitting. The method includes: generating multiple candidate neural network structures in a search space using a controller; for each candidate neural network structure, performing a comprehensive performance evaluation based on a few-sample dataset to obtain its comprehensive performance index; selecting candidate neural network structures that meet preset requirements based on the comprehensive performance index of each candidate neural network structure, the comprehensive performance index including fitting accuracy and model complexity, and updating the controller parameters according to the performance index of the candidate neural networks that meet the preset requirements; and repeating the steps of generation, evaluation, selection, and updating based on the updated controller parameters until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting, which can then be used for few-sample fitting tasks. This invention effectively overcomes overfitting in few samples and significantly improves the model's generalization ability.

Description

Technical Field

[0001] This invention relates to the field of neural architecture search technology, and in particular to an automatic search method and apparatus for neural network structures for few-sample fitting. Background Technology

[0002] Currently, Neural Architecture Search (NAS) technology has achieved significant results in large-scale data tasks such as image classification, object detection, and data fitting. Mainstream methods include reinforcement learning-based methods (e.g., Neural optimizer search with reinforcement learning), evolutionary algorithm-based methods (e.g., Regularized Evolution for Image Classifier Architecture Search), and Differentiable Architecture Search (DARTS). These methods typically rely on large amounts of training data and computational resources, performing architecture evaluation and optimization on datasets with millions of samples, such as ImageNet.

[0003] In small-sample scenarios such as numerical regression, image classification, object detection, and scientific modeling, existing techniques often use neural networks as surrogate models to fit observed data, and then perform analysis and prediction based on the fitted data. However, existing neural architecture search NAS methods face the following challenges when directly applied to such tasks: High risk of overfitting: With small samples (usually tens to hundreds of points), complex networks are very prone to overfitting, resulting in poor generalization ability.

[0004] Therefore, an effective technical solution is urgently needed to solve the above-mentioned technical problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an automatic search method and apparatus for neural network structures for small sample fitting, which effectively overcomes small sample overfitting, significantly improves the model's generalization ability, and generates an optimal neural network structure with low fitting error and clear structure, greatly improving the success rate and simplicity of subsequent symbolic expression reconstruction.

[0006] In a first aspect, the present invention provides an automatic search method for neural network structures for small-sample fitting, the method comprising the following steps: The controller generates multiple candidate neural network structures in the search space; For each candidate neural network structure, a comprehensive performance evaluation is performed on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples contained in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the parameters of the controller are updated using the performance indicators of the candidate neural network structures that meet the preset requirements. Based on the updated parameters of the controller, the steps of generation, evaluation, screening, and updating are repeated until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0007] According to the present invention, an automatic search method for neural network structures for few-sample fitting is provided, wherein the comprehensive performance evaluation of the candidate neural network structures based on a few-sample dataset is performed to obtain a comprehensive performance index of the candidate neural network structures, including: Based on the small sample dataset, cross-validation is used to evaluate the fitting accuracy of the candidate neural network structure; Calculate the model complexity of the candidate neural network structure; Based on the fitting accuracy and model complexity of the candidate neural network structure, the comprehensive performance index of the candidate neural network structure is determined.

[0008] According to the present invention, an automatic search method for neural network structures for few-sample fitting is provided, wherein the fitting accuracy of the candidate neural network structures is evaluated using cross-validation based on the few-sample dataset, including: Based on the small sample dataset, the mean square error of the candidate neural network structure is calculated using k-fold cross-validation. The mean square error of the candidate neural network structure is determined as the fitting accuracy of the candidate neural network structure.

[0009] The present invention provides an automatic neural network structure search method for small sample fitting. The model complexity of the candidate neural network structure includes the number of parameters and the computational cost of the candidate neural network structure. The determination of the comprehensive performance index of the candidate neural network structure based on the fitting accuracy and model complexity of the candidate neural network structure includes: The comprehensive performance index of the candidate neural network structure is obtained by weighted summing of the fitting accuracy, the number of parameters, and the computational cost of the candidate neural network structure.

[0010] According to the present invention, an automatic search method for neural network structures for few-sample fitting is provided, wherein the method utilizes a controller to generate multiple candidate neural network structures in the search space, including: Initialize the network weight parameters of the controller: The controller uses a lightweight recurrent neural network or a lightweight transformer; Each candidate neural network structure to be generated is represented as a variable-length decision sequence. The controller predicts each element in each decision sequence step by step based on the network weight parameters initialized by the controller. The elements of the decision sequence include at least one of the following: the operation type selected by the current layer, the connection method between the current layer and the previous layer, and the total number of network layers. Based on the complete decision sequences generated by the controller prediction, they are mapped and assembled in the search space to obtain the candidate neural network structures; the search space includes various neural network operation types and various network connection topologies.

[0011] According to the present invention, an automatic neural network structure search method for few-sample fitting is provided, the method further includes: The optimal neural network structure is post-processed and compressed to obtain the final neural network model; the post-processing compression includes at least one of the following: channel pruning, layer pruning, and knowledge distillation.

[0012] Secondly, the present invention also provides an automatic neural network structure search device for small sample fitting, the device comprising the following modules: The generation module is used to generate multiple candidate neural network structures in the search space using the controller; An evaluation module is used to perform a comprehensive performance evaluation on each candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational load or number of parameters of the candidate neural network structure when processing data samples; An update module is used to select candidate neural network structures that meet preset requirements based on the fitting accuracy and model complexity of each candidate neural network structure, and update the parameters of the controller using the performance indicators of the candidate neural network structures that meet the preset requirements. An iteration and output module is used to repeatedly execute the steps of generation, evaluation, screening, and updating based on the updated parameters of the controller until a preset stopping condition is reached, thereby obtaining an optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0013] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic search method for neural network structures for few-sample fitting as described above.

[0014] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automatic search method for neural network structures for few-sample fitting as described above.

[0015] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the automatic search method for neural network structures oriented to few-sample fitting as described above.

[0016] The present invention provides an automatic search method and apparatus for neural network structures for few-sample fitting. First, a controller generates multiple candidate neural network structures in a search space, including various neural network operation types and network connection topologies. Then, for each candidate neural network structure, a comprehensive performance evaluation is performed based on a few-sample dataset to obtain a comprehensive performance index. The number of samples in the few-sample dataset is less than or equal to a preset threshold. The comprehensive performance index is determined based on fitting accuracy and model complexity. Fitting accuracy characterizes the fitting error of the candidate neural network structure to the data in the few-sample dataset. Model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples. Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the controller parameters are updated using the performance index of the candidate neural network structures that meet the preset requirements. Then, based on the updated controller parameters, the steps of generation, evaluation, selection, and updating are repeated until a preset stopping condition is reached, resulting in an optimal neural network structure for few-sample fitting. The optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0017] In this invention, the controller generates candidate neural network structures within a limited search space and performs a comprehensive performance evaluation of the candidate neural network structures based on a small sample dataset. The comprehensive performance indicators include fitting accuracy and model complexity. This forces the search process to balance fitting ability and model simplicity under small sample conditions, effectively overcomes small sample overfitting, significantly improves the model's generalization ability, and generates an optimal neural network structure with low fitting error and clear structure, which greatly improves the success rate and simplicity of small sample fitting. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the automatic search method for neural network structures for small sample fitting provided by the present invention.

[0020] Figure 2 This is a schematic diagram illustrating the principle of the automatic neural network structure search method for small sample fitting provided by the present invention.

[0021] Figure 3This is the second flowchart of the automatic search method for neural network structures for small sample fitting provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the automatic neural network structure search device for small sample fitting provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first node can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] To more clearly understand the various embodiments provided by the present invention, the technical content involved in the present invention will first be described as follows: Existing NAS technology has the following core shortcomings in small sample fitting scenarios (especially symbolic regression): (1) The search strategy was not designed for the characteristics of small samples, resulting in weak generalization ability of the searched structures; (2) The model complexity was not included in the joint optimization objective, and the resulting network is difficult to use for subsequent symbolic reduction; (3) It cannot automatically balance the relationship between accuracy, complexity and generalization, making it difficult to generate a "high-quality, low-complexity" surrogate model.

[0027] Therefore, there is an urgent need for an automatic neural network structure search method that is oriented towards small sample fitting and balances accuracy and complexity, so as to provide a reliable foundation for interpretable modeling tasks such as symbolic regression.

[0028] The following is combined with Figures 1 to 5The present invention describes an automatic search method and apparatus for neural network structures for small sample fitting.

[0029] Figure 1 This is one of the flowcharts illustrating the automatic neural network structure search method for few-sample fitting provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Use the controller to generate multiple candidate neural network structures in the search space.

[0030] It should be noted that the execution subject of the automatic neural network structure search method for few-shot fitting provided by the present invention can be an electronic device, or any other automatic neural network structure search system for few-shot fitting that can implement the automatic neural network structure search method for few-shot fitting.

[0031] The search space in this invention is a predefined "design range." It is "restricted" because in small-sample scenarios, there is no need to explore overly large or redundant structures, thus improving search efficiency and reducing the risk of overfitting. The search space includes various neural network operation types and various network connection topologies. The neural network operation types include at least one of the following: normalization operation, pooling operation, and activation function operation. The network connection topologies include at least one of chain connections, skip connections, multi-branch parallel connections, and intra-unit connections based on directed acyclic graphs.

[0032] For example, the search space is typically limited to: ① Operation set (neural network operation type): Only includes necessary lightweight operations, such as normalization, pooling, activation, etc. in different layers. Large and complex operations are excluded.

[0033] ② Topology pattern (network connection topology): limits the complexity of the structure. For example, only chain structures, structures with a small number of residual connections, or predefined cell structures are allowed.

[0034] This invention designs lightweight controllers, such as small Recurrent Neural Networks (RNNs) or Transformer models. "Lightweight" refers to their small parameter count and low computational cost, making them suitable for rapid iteration in small-sample scenarios. RNNs, due to their ability to remember previous inputs, are often used to process sequential data. In NAS, they can take a description of the network structure (a variable-length sequence) as input and progressively predict the probability of generating the next network component. Transformers, based on self-attention mechanisms, are better at capturing long-range dependencies and have higher training efficiency than RNNs. In recent NAS research, using Transformers as controllers can more effectively handle complex search spaces.

[0035] In practical applications, generating multiple candidate neural network structures within a constrained search space using a lightweight controller is a sequential decision-making process. This process specifically includes initializing parameters, predicting sequences layer by layer or stepwise (the controller views the neural network construction process as generating a variable-length string or token sequence), and assembling the candidate neural network structures. Each time the controller outputs a complete decision sequence (e.g., {Layer 1: Layer Normalization, connected to input} -> {Layer 2: Pooling, connected to Layer 1} -> {Layer 3: Stop}), it maps this sequence to the constrained search space, assembling it into a specific, trainable candidate neural network structure.

[0036] Finally, this step outputs one or more candidate neural network structures (sampled by the controller according to the current policy). These generated candidate structures are then trained and evaluated (e.g., using the previously mentioned k-fold cross-validation to calculate MSE and complexity). The decision sequences corresponding to the structures with good evaluation results are used to reward the controller (updating the controller's parameters), making the controller more inclined to generate this high-performance pattern when generating structures in the future.

[0037] In short, this process involves a lightweight controller, within a limited menu of options, meticulously "writing out" network structure designs, word by word, like writing an essay. This constrained and lightweight design ensures that the entire search process can still start quickly and remain uncontrolled even in scenarios with small sample sizes.

[0038] Step 102: For each candidate neural network structure, perform a comprehensive performance evaluation on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index of the candidate neural network structure; the number of samples in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; model complexity includes the amount of computation or number of parameters of the candidate neural network structure when processing data samples.

[0039] Specifically, a small sample dataset contains a number of samples less than or equal to a preset threshold, such as 500. Represented as Where n≤500, For the input of the i-th sample, e.g., the i-th sample, The output for the i-th sample is the result of the fitting, where n represents the total number of samples in the small sample dataset.

[0040] The comprehensive performance evaluation includes fitting accuracy evaluation and model complexity evaluation. Correspondingly, the comprehensive performance evaluation indicators include fitting accuracy indicators and model complexity indicators, such as the number of parameters (Params) and the amount of computation. The amount of computation is represented by the number of floating point operations (FLOPs).

[0041] In practical applications, after obtaining one or more candidate neural network structures through the search, a comprehensive performance evaluation is performed on each candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for each candidate neural network structure. For example, the following formula can be used to evaluate the comprehensive performance of candidate neural network structures: in, This represents a multi-objective evaluation function. Indicates the number of samples. This represents the root mean square error of the i-th subset. and These are adjustable weighting parameters used to control the balance between fitting accuracy and model complexity. Indicates the number of parameters. This indicates the computational cost. This function forces the search process to balance fitting ability and model simplicity with small sample sizes. The smaller the value of the function corresponding to the candidate neural network structure, the better the overall performance of the candidate neural network structure.

[0042] Step 103: Based on the fitting accuracy and model complexity of each candidate neural network structure, select candidate neural network structures that meet the preset requirements, and update the controller parameters using the performance indicators of the candidate neural network structures that meet the preset requirements.

[0043] Specifically, after obtaining the comprehensive performance index of each candidate neural network structure, the candidate neural network structure that meets the preset requirements is further selected based on the comprehensive performance index of each candidate neural network structure.

[0044] It's important to understand that the smaller the overall performance index of a candidate neural network structure, the better its overall performance. Therefore, by sorting the candidate neural network structures according to their overall performance index from smallest to largest and retaining the top m candidate neural network structures (where m is a preset positive integer), we can obtain candidate neural network structures that meet the preset requirements.

[0045] Furthermore, the decision sequences corresponding to the top m candidate neural network structures are used to reward the controller (updating the controller's parameters), thereby making the controller more inclined to generate this high-performance pattern when generating structures in the future. Updating the controller's parameters includes using the policy gradient method to update the controller parameters, increasing the probability that the controller will generate network structures with better overall performance metrics.

[0046] Step 104: Based on the updated controller parameters, repeat the steps of generation, evaluation, screening, and updating until the preset stopping condition is met to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0047] Specifically, after updating the controller parameters, the process iterates based on the updated controller parameters, repeating the steps of generation, evaluation, screening, and updating until a preset stopping condition is reached. The preset stopping condition is reaching a preset number of iterations or the controller strategy converges, ultimately obtaining the optimal neural network structure for small sample fitting.

[0048] For example, Figure 2 This is a schematic diagram illustrating the principle of the automatic neural network structure search method for few-sample fitting provided by the present invention, as shown below. Figure 2 As shown, the method includes: First, a reinforcement learning-based neural network architecture search is performed: the controller, implemented using a lightweight recurrent neural network (RNN) or transformer, makes sequential decisions within a constrained search space, progressively generating candidate neural network structures (lightweight neural network model structures). These candidate structures are then fed into the training neural network module for training on a small sample dataset. Specifically, in each iteration, the controller samples a neural network structure with probability p, then trains the sampled neural network to obtain a reward R (the reward R comes from the reward calculation module, and this reward signal is used for updating the controller's parameters). The policy gradient of probability p is calculated, and the controller is updated. Finally, the iteration terminates, and the generated candidate structures are fed into the "training neural network" module for training and evaluation.

[0049] Furthermore, after the controller generates candidate neural network structures, they are fed into the training module for training on a small sample dataset, i.e., high-precision data fitting of the network structure. The optimization objective evaluates the trained model from two dimensions: fitting accuracy (calculating the average MSE using cross-validation) and model complexity (calculating Params and FLOPs). The comprehensive loss function L is calculated, which includes fitting accuracy (weight parameter α) and model complexity (weight parameter β). The weighted fusion of accuracy and complexity yields the reward R (i.e., the reward R obtained from training the neural network), which is fed back to the controller for parameter updates (i.e., calculating the policy gradient of p and updating the controller). Through this closed-loop iteration, the optimal neural network structure that balances accuracy and complexity is finally searched, i.e., the optimal neural network model for data fitting.

[0050] The present invention provides an automatic search method and apparatus for neural network structures for few-sample fitting. First, a controller generates multiple candidate neural network structures in a search space, including various neural network operation types and network connection topologies. Then, for each candidate neural network structure, a comprehensive performance evaluation is performed based on a few-sample dataset to obtain a comprehensive performance index. The number of samples in the few-sample dataset is less than or equal to a preset threshold. The comprehensive performance index is determined based on fitting accuracy and model complexity. Fitting accuracy characterizes the fitting error of the candidate neural network structure to image features in the few-sample dataset. Model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing samples. Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the controller parameters are updated using the performance index of the candidate neural network structures that meet the preset requirements. Then, based on the updated controller parameters, the steps of generation, evaluation, selection, and updating are repeated until a preset stopping condition is reached, resulting in an optimal neural network structure for few-sample fitting. The optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0051] In this invention, the controller generates candidate neural network structures within a limited search space and performs a comprehensive performance evaluation of the candidate neural network structures based on a small sample dataset. The comprehensive performance indicators include fitting accuracy and model complexity. This forces the search process to balance fitting ability and model simplicity under small sample conditions, effectively overcomes small sample overfitting, significantly improves the model's generalization ability, and generates an optimal neural network structure with low fitting error and clear structure, which greatly improves the success rate and simplicity of small sample fitting.

[0052] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0053] According to the present invention, an automatic search method for neural network structures for small-sample fitting is provided, which performs a comprehensive performance evaluation of candidate neural network structures based on a small-sample dataset to obtain a comprehensive performance index of the candidate neural network structures, including: Based on a small sample dataset, cross-validation was used to evaluate the fitting accuracy of candidate neural network structures. Calculate the model complexity of candidate neural network structures; Based on the fitting accuracy and model complexity of the candidate neural network structure, the comprehensive performance index of the candidate neural network structure is determined.

[0054] Specifically, in some embodiments, step 102 performs a comprehensive performance evaluation of the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index of the candidate neural network structure, which is achieved through the following steps: First, calculate the fitting accuracy and model complexity of the candidate neural network structure. Model complexity is represented by the number of parameters and the amount of computation. The larger the number of parameters, the higher the model complexity; the larger the amount of computation, the higher the model complexity.

[0055] Then, the overall performance index of the candidate neural network structure is determined based on its fitting accuracy and model complexity. For example, a weighted sum of the fitting accuracy and model complexity of the candidate neural network structure can be used to obtain the overall performance index.

[0056] The method provided in this invention comprehensively considers the fitting accuracy and model complexity of the candidate neural network structure, and realizes a comprehensive evaluation of the model. The subsequently generated surrogate model has low fitting error, clear structure, and significantly reduces the number of output network parameters and computational cost, which facilitates subsequent symbolization.

[0057] The present invention provides an automatic search method for neural network structures for small-sample fitting, which evaluates the fitting accuracy of candidate neural network structures based on a small-sample dataset using cross-validation, including: Based on a small sample dataset, k-fold cross-validation is used to calculate the mean squared error of candidate neural network structures. The mean square error of the candidate neural network structure is determined as the fitting accuracy of the candidate neural network structure.

[0058] Specifically, in some embodiments, the fitting accuracy of the candidate neural network structure is evaluated through the following steps: Based on a small sample dataset, k-fold cross-validation is used to calculate the mean squared error (MSE) of the candidate neural network structure. Then, the mean squared error of the candidate neural network structure is determined as the fitting accuracy of the candidate neural network structure. The value of k is determined based on the number of samples, and k ≤ the number of samples, k = min(5, n), where n is the number of samples.

[0059] For example, the average MSE of a candidate neural network structure is calculated through the following steps: Step 1: Determine the value of k Principle: Since the sample size n is very small, the value of k should not be set too large (otherwise the training set is too small) or too small (otherwise the validation set is not representative).

[0060] Specific approach: Usually, k = min(5, n). For example, if the number of samples n = 50, then k = 5; if n = 500, k = 5 as well; if the number of samples is very small (e.g., n = 20), in order to ensure the size of the training set as much as possible, the leave-one-out method (i.e., k = n) may be used, but it is usually controlled within 50%.

[0061] Step 2: Divide the dataset The specific method is as follows: The small sample dataset D is randomly and uniformly divided into k disjoint subsets (i.e., folded), D = {D1, D2, ..., D...} k Each subset D i It contains approximately n / k samples.

[0062] Step 3: Iterative Training and Validation For the candidate neural network architecture that needs to be evaluated, execute k iterations of the loop: Loop: For i=1 to k: 1. Combine training sets: except for D i All k-1 subsets other than the training set are used as the training set (approximately n*(k-1) / k samples).

[0063] 2. Set the validation set: Set D i As a validation set.

[0064] 3. Training: Train the candidate neural network structure on the training set. Due to the small sample size, an early stopping mechanism is needed to prevent overfitting.

[0065] 4. Prediction and Calculation of MSE: After training is complete (or during early stopping), use the model to predict the validation set D. i Calculate the mean square error (MSE) of this verification. i .

[0066] Step 4: Calculate the mean square error The specific method is to take the arithmetic mean of the k MSE values ​​obtained from k iterations.

[0067] in, This represents the mean squared error of the candidate neural network architecture on the current small sample dataset. This represents the mean squared error corresponding to the i-th subset. This is the final fitting accuracy metric of the candidate neural network architecture on the current small sample dataset. It reflects the model's expected generalization error on unseen data, and its evaluation results are more stable and reliable compared to a single split of the training / validation set.

[0068] In this step, by repeatedly training k times (with only a different validation set each time), every piece of data in the small sample is fully utilized for both training and validation, thus ensuring the number of training samples while effectively testing the model's generalization ability.

[0069] The method provided in this invention calculates the mean square error of a candidate neural network structure and uses it as the final fitting accuracy index of the candidate neural network structure on the current small sample dataset. This index will serve as the basis for subsequently constructing a multi-objective evaluation function (accuracy part) to guide the controller in selecting structures with strong generalization ability.

[0070] According to the present invention, an automatic search method for neural network structures for small-sample fitting is provided. The model complexity of the candidate neural network structure includes the number of parameters and the computational cost of the candidate neural network structure. Based on the fitting accuracy and model complexity of the candidate neural network structure, the comprehensive performance indicators of the candidate neural network structure are determined, including: By weighting and summing the fitting accuracy, parameter count, and computational cost of the candidate neural network structure, a comprehensive performance index for the candidate neural network structure is obtained.

[0071] Specifically, in some embodiments, the model complexity of the candidate neural network structure includes the number of parameters (Params) and the computational cost, which is represented by the number of floating-point operations (FLOPs).

[0072] The overall performance index of candidate neural network structures is calculated using the following multi-objective evaluation function: By weighting and summing the fitting accuracy (mean squared error), the number of parameters, and the computational cost (FLOPs) of the candidate neural network structure, a comprehensive performance index for the candidate neural network structure is obtained. in, This represents a multi-objective evaluation function. Indicates the number of samples. This represents the root mean square error of the i-th subset. and These are adjustable weighting parameters used to control the balance between fitting accuracy and model complexity. Indicates the number of parameters. Indicates the computational cost.

[0073] The method provided in this invention constructs a multi-objective evaluation function that integrates fitting accuracy and model complexity. This multi-objective evaluation function forces the network structure search process to balance fitting ability and model simplicity under small sample conditions. Through cross-validation and complexity regularization, the generalization ability is significantly improved. The generated surrogate model has low fitting error and clear structure, greatly improving the success rate and simplicity of subsequent symbolic expression reconstruction.

[0074] According to the present invention, an automatic search method for neural network structures for small-sample fitting is provided, which utilizes a controller to generate multiple candidate neural network structures in the search space, including: Initialize the network weight parameters of the controller: The controller uses a lightweight recurrent neural network or a lightweight transformer; Each candidate neural network structure to be generated is represented as a variable-length decision sequence. The controller predicts each element in each decision sequence step by step based on the network weight parameters initialized by the controller. The elements of the decision sequence include at least one of the following: the operation type selected by the current layer, the connection method between the current layer and the previous layer, and the total number of network layers. Based on the complete decision sequences generated by the controller prediction, they are mapped and assembled in the search space to obtain candidate neural network structures; the search space includes various neural network operation types and various network connection topologies.

[0075] Specifically, in some embodiments, generating multiple candidate neural network structures in the search space using a controller is achieved through the following steps: Step 1: Initialization Initialize the network weight parameters of the controller. Usually, the initial parameters are randomized, and it faces an empty "network blueprint".

[0076] Step 2: Layer-by-layer / stepwise prediction (sequence generation) Each candidate neural network structure to be generated is represented as a variable-length decision sequence. The controller predicts each element in each decision sequence step by step based on the current network weight parameters of the controller. The elements of the decision sequence include at least one of the following: the operation type selected by the current layer, the connection method between the current layer and the previous layer, and the total number of network layers.

[0077] In practical applications, the controller treats the construction process of the neural network as generating a variable-length string or token sequence. It performs the following predictions sequentially: (1) Predict the operation type: In the restricted operation set, select an operation for the Nth layer to be added (e.g., layer normalization or pooling?).

[0078] (2) Predictive connection method: Determines where the input of this layer comes from. Does it only receive the output of the previous layer (chained), or does it need to receive the output of several previous layers and fuse them (skip connection)? The controller needs to output a connection vector indicating which previous layers it is connected to.

[0079] (3) Predicting when to stop: After generating each layer, the controller will also decide whether to continue adding layers. If a "stop" signal is output, the generation process of the current structure ends.

[0080] Step 3: Assemble candidate structures That is, based on the complete decision sequences generated by the controller's predictions, they are mapped and assembled in the search space to obtain various candidate neural network structures. For example, each time the controller outputs a complete decision sequence (e.g., {Layer 1: Layer Normalization, connected to input} -> {Layer 2: Pooling, connected to Layer 1} -> {Layer 3: Stop}), this sequence is mapped to a restricted search space and assembled into a specific, trainable candidate neural network structure.

[0081] The output consists of one or more candidate neural network structures (sampled by the controller according to the current policy). These generated candidate neural network structures are then trained and evaluated (e.g., using the previously mentioned k-fold cross-validation to calculate MSE and complexity). Structures with good evaluation results are used to reward the controller (updating its parameters), making the controller more inclined to generate such high-performance patterns in the future.

[0082] The method provided in this invention utilizes a lightweight controller to search for various network structure design schemes within a limited menu of options. This restricted and lightweight design ensures that the entire search process can start quickly and remain uncontrolled even in small sample scenarios.

[0083] According to the present invention, an automatic neural network structure search method for small sample fitting is provided, the method further includes: The optimal neural network structure is post-processed and compressed to obtain the final neural network model; the post-processing compression includes at least one of the following: channel pruning, layer pruning, and knowledge distillation.

[0084] Specifically, in some embodiments, the method further includes post-processing compression of the optimal neural network structure obtained through the search, further compressing the model to obtain a simpler structure, ensuring that the final output is a minimalist but high-fidelity proxy network. ,in, Indicates the model input, Indicates model parameters.

[0085] Post-processing compression operations include at least one of channel pruning, layer pruning, and knowledge distillation. Channel pruning or layer pruning is used to remove redundant connections or channels. Specifically, channel pruning is a fine-grained pruning method, mainly reducing width, with minimal impact on accuracy, and is easily restored after fine-tuning. Layer pruning is a coarse-grained pruning method, mainly reducing depth, which is more hardware-acceleration friendly (more regular structure), but may introduce larger accuracy fluctuations.

[0086] The operational principle and steps of channel pruning are as follows: The core idea is to eliminate unimportant input or output channels in convolutional layers (i.e., reduce the number of filters), thereby directly reducing the number of parameters and computational cost.

[0087] The specific operating steps include the following steps 1-3: 1. Evaluating Channel Importance: After training the model, evaluate each channel in each convolutional layer. Common evaluation criteria include: ① Weight Magnitude: Calculate the L1 or L2 norm of the filter weights corresponding to the channel. The smaller the norm, the less the channel contributes to subsequent feature extraction, and the more likely it is to be pruned. ② Batch Normalization (BN) Layer Scaling Factor: If the network has batch normalization layers, their scaling factor γ can be used as an indicator. During training, sparse regularization is applied to γ, and during pruning, channels with γ coefficients close to 0 are directly removed.

[0088] 2. Sorting and Pruning: Sort the importance scores of all channels within a layer, set a global or local pruning rate (e.g., prune 20%), and remove the less important channels with lower rankings.

[0089] 3. Reconstruction and fine-tuning: The accuracy of the model usually decreases after pruning, so it is necessary to perform short-term fine-tuning on the training set to allow the remaining channels to relearn in order to restore accuracy.

[0090] The operational principle and steps of layered pruning are as follows: The core idea is to directly delete an entire network layer (such as a residual block or convolutional layer) to significantly simplify the network depth.

[0091] The specific operating steps include the following steps 1-3: 1. Assessing Layer Redundancy: Common methods for determining whether a layer is redundant include: Feature map similarity: Calculates the cosine similarity or mutual information between the feature maps of the input and output of a certain layer. If the input and output are very similar (e.g., after an identity mapping), it means that the layer does not contribute much to information transformation and can be removed.

[0092] Contribution assessment: On the validation set, skip a certain layer and observe the change in the loss function. If removing that layer results in a small increase in loss, it indicates that the layer is of low importance.

[0093] 2. Inter-layer connection reconstruction: After removing the selected layer, the previous layer needs to be directly connected to the next layer (if it is a residual network, attention should be paid to the reconstruction of shortcut connections).

[0094] 3. Global fine-tuning: Layer pruning significantly alters the network structure, and removing it usually requires a longer training time to restore performance.

[0095] The knowledge distillation operation includes performing knowledge distillation on the pruned network, using the optimal neural network structure as the teacher network, and distilling the pruned student network to restore or maintain the fitting accuracy.

[0096] For example, Figure 3 This is the second flowchart of the automatic neural network structure search method for few-sample fitting provided by the present invention, as shown below. Figure 3 As shown, the method includes: After fitting data to the candidate neural network (iterically until the optimization conditions are met) to obtain the optimal neural network, the optimal neural network is further pruned to obtain the pruned simplified network.

[0097] The method provided in this embodiment of the invention further performs post-processing compression on the optimal neural network structure obtained by the search to obtain the final neural network model, ensuring that the final output is a minimalist but high-fidelity proxy network.

[0098] Alternatively, the present invention may be extended or replaced as follows: Evaluation function variant: Params / FLOPs can be replaced with expression complexity predictions (e.g., estimating the number of operators in subsequent symbolic expressions using a surrogate model); Search algorithm replacement: Bayesian optimization and genetic algorithms can be used to replace reinforcement learning controllers; Online incremental search: dynamically adjusts the network structure as new data arrives, suitable for streaming scientific data modeling.

[0099] The automatic neural network structure search device for few-shot fitting provided by the present invention will be described below. The automatic neural network structure search device for few-shot fitting described below can be referred to in correspondence with the automatic neural network structure search method for few-shot fitting described above.

[0100] Figure 4 This is a schematic diagram of the automatic neural network structure search device for small sample fitting provided by the present invention, as shown below. Figure 4 As shown, the automatic neural network structure search device 400 for small sample fitting includes the following modules: The generation module 410 is used to generate multiple candidate neural network structures in the search space using the controller; Evaluation module 420 is used to perform a comprehensive performance evaluation on each candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational load or number of parameters of the candidate neural network structure when processing data samples; The update module 430 is used to select candidate neural network structures that meet preset requirements based on the fitting accuracy and model complexity of each candidate neural network structure, and update the parameters of the controller using the performance indicators of the candidate neural network structures that meet the preset requirements. The iteration and output module 440 is used to repeatedly execute the steps of generation, evaluation, screening and updating based on the updated parameters of the controller until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification and object detection.

[0101] The present invention provides an automatic search method and apparatus for neural network structures for few-sample fitting. A generation module 410 is used to generate multiple candidate neural network structures in a search space using a controller. The search space includes various neural network operation types and various network connection topologies. An evaluation module 420 is used to perform a comprehensive performance evaluation on each candidate neural network structure based on a few-sample dataset to obtain a comprehensive performance index for the candidate neural network structure. The number of samples in the few-sample dataset is less than or equal to a preset threshold. The comprehensive performance index is determined based on fitting accuracy and model complexity. Fitting accuracy is used to characterize the fitting error of the candidate neural network structure to image features in the few-sample dataset. Model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; update module 430 selects candidate neural network structures that meet preset requirements based on the fitting accuracy and model complexity of each candidate neural network structure, and updates the parameters of the controller using the performance indicators of the candidate neural network structures that meet the preset requirements; iteration and output module 440 is used to repeatedly execute the steps of generation, evaluation, screening and updating based on the updated controller parameters until a preset stopping condition is reached to obtain the optimal neural network structure for small sample fitting. The optimal neural network structure is used for small sample fitting tasks, which include at least one of numerical regression, image classification and object detection.

[0102] In this invention, the controller generates candidate neural network structures within a limited search space and performs a comprehensive performance evaluation of the candidate neural network structures based on a small sample dataset. The comprehensive performance indicators include fitting accuracy and model complexity. This forces the search process to balance fitting ability and model simplicity under small sample conditions, effectively overcomes small sample overfitting, significantly improves the model's generalization ability, and generates an optimal neural network structure with low fitting error and clear structure, which greatly improves the success rate and simplicity of small sample fitting.

[0103] According to the present invention, an automatic neural network structure search device 400 for small sample fitting is provided, wherein the evaluation module 420 is specifically used for: Based on the small sample dataset, cross-validation is used to evaluate the fitting accuracy of the candidate neural network structure; Calculate the model complexity of the candidate neural network structure; Based on the fitting accuracy and model complexity of the candidate neural network structure, the comprehensive performance index of the candidate neural network structure is determined.

[0104] According to the present invention, an automatic neural network structure search device 400 for small sample fitting is provided, wherein the evaluation module 420 is further configured to: Based on the small sample dataset, the mean square error of the candidate neural network structure is calculated using k-fold cross-validation. The mean square error of the candidate neural network structure is determined as the fitting accuracy of the candidate neural network structure.

[0105] According to the present invention, an automatic neural network structure search device 400 for small sample fitting is provided, wherein the model complexity of the candidate neural network structure includes the number of parameters of the candidate neural network structure and the computational cost of the candidate neural network structure; The evaluation module 420 is also used for: The comprehensive performance index of the candidate neural network structure is obtained by weighted summing of the fitting accuracy, the number of parameters, and the computational cost of the candidate neural network structure.

[0106] According to the present invention, an automatic neural network structure search device 400 for small sample fitting is provided, wherein the generation module 410 is specifically used for: Initialize the network weight parameters of the controller: The controller uses a lightweight recurrent neural network or a lightweight transformer; Each candidate neural network structure to be generated is represented as a variable-length decision sequence. The controller predicts each element in each decision sequence step by step based on the network weight parameters initialized by the controller. The elements of the decision sequence include at least one of the following: the operation type selected by the current layer, the connection method between the current layer and the previous layer, and the total number of network layers. Based on the complete decision sequences generated by the controller prediction, they are mapped and assembled in the search space to obtain the candidate neural network structures; the search space includes various neural network operation types and various network connection topologies.

[0107] According to the present invention, an automatic neural network structure search device 400 for small sample fitting is provided, the device further includes a post-processing compression module; The post-processing compression module is used for: The optimal neural network structure is post-processed and compressed to obtain the final neural network model; the post-processing compression includes at least one of the following: channel pruning, layer pruning, and knowledge distillation.

[0108] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an automatic neural network structure search method for few-sample fitting, the method including: The controller generates multiple candidate neural network structures in the search space; For each candidate neural network structure, a comprehensive performance evaluation is performed on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples contained in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the parameters of the controller are updated using the performance indicators of the candidate neural network structures that meet the preset requirements. Based on the updated parameters of the controller, the steps of generation, evaluation, screening, and updating are repeated until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0109] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing the automatic search method for neural network structures for small-sample fitting provided by the above methods, the method comprising: The controller generates multiple candidate neural network structures in the search space; For each candidate neural network structure, a comprehensive performance evaluation is performed on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples contained in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the parameters of the controller are updated using the performance indicators of the candidate neural network structures that meet the preset requirements. Based on the updated parameters of the controller, the steps of generation, evaluation, screening, and updating are repeated until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0111] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an automatic search method for neural network structures for few-sample fitting provided by the methods described above, the method comprising: The controller generates multiple candidate neural network structures in the search space; For each candidate neural network structure, a comprehensive performance evaluation is performed on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples contained in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the parameters of the controller are updated using the performance indicators of the candidate neural network structures that meet the preset requirements. Based on the updated parameters of the controller, the steps of generation, evaluation, screening, and updating are repeated until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic neural network structure search method for small sample fitting, characterized in that, include: The controller generates multiple candidate neural network structures in the search space; For each candidate neural network structure, a comprehensive performance evaluation is performed on the candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples contained in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational cost or number of parameters of the candidate neural network structure when processing data samples; Based on the fitting accuracy and model complexity of each candidate neural network structure, candidate neural network structures that meet preset requirements are selected, and the parameters of the controller are updated using the performance indicators of the candidate neural network structures that meet the preset requirements. Based on the updated parameters of the controller, the steps of generation, evaluation, screening, and updating are repeated until a preset stopping condition is reached to obtain the optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

2. The automatic neural network structure search method for small sample fitting according to claim 1, characterized in that, The comprehensive performance evaluation of the candidate neural network structure based on a small sample dataset yields a comprehensive performance index for the candidate neural network structure, including: Based on the small sample dataset, cross-validation is used to evaluate the fitting accuracy of the candidate neural network structure; Calculate the model complexity of the candidate neural network structure; Based on the fitting accuracy and model complexity of the candidate neural network structure, the comprehensive performance index of the candidate neural network structure is determined.

3. The automatic neural network structure search method for small sample fitting according to claim 2, characterized in that, The step of evaluating the fitting accuracy of the candidate neural network structure using cross-validation based on the small sample dataset includes: Based on the small sample dataset, the mean square error of the candidate neural network structure is calculated using k-fold cross-validation. The mean square error of the candidate neural network structure is determined as the fitting accuracy of the candidate neural network structure.

4. The automatic search method for neural network structures for small-sample fitting according to claim 2, characterized in that, The model complexity of the candidate neural network structure includes the number of parameters and the computational cost of the candidate neural network structure. The determination of the comprehensive performance index of the candidate neural network structure based on the fitting accuracy and model complexity of the candidate neural network structure includes: The comprehensive performance index of the candidate neural network structure is obtained by weighted summing of the fitting accuracy, the number of parameters, and the computational cost of the candidate neural network structure.

5. The automatic search method for neural network structures for small-sample fitting according to any one of claims 1-4, characterized in that, The method of generating multiple candidate neural network structures in the search space using a controller includes: Initialize the network weight parameters of the controller: The controller uses a lightweight recurrent neural network or a lightweight transformer; Each candidate neural network structure to be generated is represented as a variable-length decision sequence. The controller predicts each element in each decision sequence step by step based on the network weight parameters initialized by the controller. The elements of the decision sequence include at least one of the following: the operation type selected by the current layer, the connection method between the current layer and the previous layer, and the total number of network layers. Based on the complete decision sequences generated by the controller prediction, they are mapped and assembled in the search space to obtain the candidate neural network structures; the search space includes various neural network operation types and various network connection topologies.

6. The automatic search method for neural network structures for small-sample fitting according to any one of claims 1-4, characterized in that, The method further includes: The optimal neural network structure is post-processed and compressed to obtain the final neural network model; the post-processing compression includes at least one of the following: channel pruning, layer pruning, and knowledge distillation.

7. An automatic neural network structure search device for small sample fitting, characterized in that, include: The generation module is used to generate multiple candidate neural network structures in the search space using the controller; An evaluation module is used to perform a comprehensive performance evaluation on each candidate neural network structure based on a small sample dataset to obtain a comprehensive performance index for the candidate neural network structure; the number of samples in the small sample dataset is less than or equal to a preset threshold; the comprehensive performance index is determined based on fitting accuracy and model complexity; the fitting accuracy is used to characterize the fitting error of the candidate neural network structure to the data in the small sample dataset; the model complexity includes the computational load or number of parameters of the candidate neural network structure when processing data samples; An update module is used to select candidate neural network structures that meet preset requirements based on the fitting accuracy and model complexity of each candidate neural network structure, and update the parameters of the controller using the performance indicators of the candidate neural network structures that meet the preset requirements. An iteration and output module is used to repeatedly execute the steps of generation, evaluation, screening, and updating based on the updated parameters of the controller until a preset stopping condition is reached, thereby obtaining an optimal neural network structure for few-sample fitting; the optimal neural network structure is used for few-sample fitting tasks, which include at least one of numerical regression, image classification, and object detection.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the automatic search method for neural network structures for few-sample fitting as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic search method for neural network structures for few-sample fitting as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the automatic search method for neural network structures for few-sample fitting as described in any one of claims 1 to 6.

Images