Neural network control method and system and storage medium

Abstract

The invention relates to the technical field of deep learning, and discloses a neural network control method and system and a storage medium, and the method comprises the steps: obtaining input data, feature dimensions, a sample number and label distribution; performing data preprocessing according to the feature dimension and the label distribution to obtain updated input data; training a model based on the updated input data and an initialized model hyper-parameter to obtain an initialized model; updating a model hyper-parameter according to the performance index of the initialized model and a preset performance threshold to obtain an updated hyper-parameter; performing a single reasoning task through the initialized model, recording single reasoning time, and compressing the model to obtain a compressed model; and according to the compression model and the updated hyper-parameters, reconfiguring the model to obtain a final neural network model, and performing binary coding output on weight features and structural features of the neural network model. The method can improve the neural network training efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a neural network control method, system, and storage medium. Background Art

[0002] Currently, in the design and implementation of neural networks, the trade-off between model performance and computational efficiency is a key challenge. With the rapid development of deep learning technology, neural networks with deeper and broader architectures are continuously explored to improve prediction accuracy. However, this trend of pursuing high performance also brings significant computational resource consumption problems. Therefore, in the design stage, the impact of model complexity on hardware resources must be carefully considered, including but not limited to CPU / GPU utilization, memory occupancy, and the time required for training and inference.

[0003] In an existing technology, the model architecture is simplified, and techniques such as depthwise separable convolution and residual connection are used to simplify the model architecture, reduce the computational burden, while maintaining or improving prediction accuracy. Further, various strategies are explored for further optimization. Further, model compression techniques such as pruning and quantization are used to make the model more compact and efficient by reducing the number of parameters and lowering the weight precision; knowledge distillation enables a small student model to learn from a large teacher model and quickly obtain similar generalization capabilities. Adaptive Inference Time (AIT) allows the model to dynamically adjust the computational path according to the complexity of the input data, while Neural Architecture Search (NAS) automatically searches for a network structure that is both efficient and accurate. Specific hardware optimizations customize the model for different computational platforms to maximize the utilization of hardware resources, while effective data augmentation and preprocessing can improve the generalization ability and learning efficiency of the model.

[0004] Although the above methods alleviate the contradiction between performance and efficiency to a certain extent, they also face some limitations. For problems in certain specific fields, existing lightweight networks cannot provide sufficient expressiveness, resulting in a decrease in accuracy and low training efficiency of the neural network. Summary of the Invention

[0005] The present invention provides a neural network control method, system, and storage medium to improve the training efficiency of the neural network.

[0006] In a first aspect, to solve the above technical problems, the present invention provides a neural network control method, including:

[0007] Obtain input data, and determine the feature dimension, number of samples, and label distribution of the input data;

[0008] Perform data preprocessing according to the feature dimension and the label distribution to obtain updated input data;

[0009] Initialize the model hyperparameters according to the updated feature dimensions of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model;

[0010] Update the model hyperparameters according to the performance metrics of the initialized model and a preset performance threshold to obtain updated hyperparameters;

[0011] Perform a single inference task through the initialized model, record the single inference time, and count the throughput of the model. When the inference time exceeds the preset time limit or the throughput is lower than the preset throughput lower limit, compress the model to obtain a compressed model;

[0012] Reconfigure the model according to the compressed model and the updated hyperparameters to obtain a final neural network model, and perform binary encoding on the weight features and structural features of the neural network model for output.

[0013] In an alternative embodiment, the data preprocessing according to the feature dimensions and the label distribution to obtain updated input data includes:

[0014] When the feature dimension is greater than the number of samples, perform dimensionality reduction on the input data to obtain dimensionality-reduced input data;

[0015] Segment the dimensionality-reduced input data according to a preset class number threshold to obtain majority class data and minority class data;

[0016] Perform interpolation processing on the minority class data according to the following formula to obtain updated minority class data:

[0017] x new = x li + λ × (x lj - x li )

[0018] where x new is the updated minority class data, x li is the i-th data in the minority class data, λ is the interpolation feature vector, and x lj is the j-th data in the minority class data;

[0019] Perform data deduplication on the majority class data to obtain updated majority class data;

[0020] The updated input data includes the updated majority class data and the updated minority class data.

[0021] In an alternative embodiment, initializing the model hyperparameters according to the updated feature dimensions of the updated input data and training the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model includes:

[0022] wherein the model hyperparameters include a learning rate, the number of hidden layers, and the number of nodes in each layer;

[0023] Obtaining a pre-stored list of initial hyperparameters and using the model hyperparameters corresponding to the updated feature dimensions as the initialized model hyperparameters;

[0024] Training the model based on the updated input data and the initialized model hyperparameters, and when it is detected that the loss function of the model meets the conditions or the number of training times reaches the preset upper limit of the number of training times, completing the training of the model to obtain an initialized model.

[0025] In an alternative embodiment, updating the model hyperparameters according to the performance metrics of the initialized model and a preset performance threshold to obtain updated hyperparameters includes:

[0026] The performance metrics include accuracy;

[0027] When the accuracy is less than the preset accuracy threshold, increasing the number of hidden layers and the number of neurons in each layer;

[0028] When the accuracy is greater than the preset accuracy threshold, keeping the model hyperparameters unchanged;

[0029] The updated hyperparameters are the model hyperparameters after adjustment.

[0030] In an alternative embodiment, performing a single inference task through the initialized model, recording the single inference time, and statistically calculating the throughput of the model. When the inference time exceeds the preset time upper limit or the throughput is lower than the preset throughput lower limit, compressing the model to obtain a compressed model includes:

[0031] Obtaining test set data;

[0032] Inputting the test set data into the initialized model, starting a single inference task, and recording the time from the input data to the output result as the single inference time;

[0033] Statistically calculating the number of inference times within a preset test time and calculating the throughput of the model, where the throughput is the number of inference times completed per unit time;

[0034] When the single inference time exceeds the preset time upper limit, it is determined that the inference speed does not meet the standard;

[0035] When the throughput is lower than the lower limit of the preset throughput, it is determined that the throughput capacity does not meet the standard;

[0036] When the inference speed does not meet the standard or the throughput capacity does not meet the standard, redundant neurons and neuron connections are removed from the initialized model to obtain a compressed model.

[0037] In an alternative embodiment, the reconfiguring the model according to the compressed model and the updated hyperparameters to obtain the final neural network model includes:

[0038] Changing the optimizer of the compressed model to an updated optimizer;

[0039] Updating the number of hidden layers of the compressed model to be consistent with the updated hyperparameters;

[0040] Wherein, the updated hyperparameters include an updated optimizer and the number of hidden layers.

[0041] In an alternative embodiment, when the feature dimension is greater than the number of samples, performing dimensionality reduction processing on the input data to obtain dimensionality-reduced input data, including:

[0042] Normalizing the input data to obtain normalized input data;

[0043] Performing singular value decomposition on the normalized input data to obtain the principal component vectors and principal component eigenvalues of the normalized input data;

[0044] Performing maximum value screening according to the principal component vectors, principal component eigenvalues and a preset number of principal components to construct a projection vector;

[0045] Multiplying the projection vector and the normalized input data in matrix form to obtain dimensionality-reduced input data.

[0046] In a second aspect, the present invention provides a neural network control system, including:

[0047] A data acquisition module, configured to acquire input data and determine the feature dimension, the number of samples and the label distribution of the input data;

[0048] A data processing module, configured to perform data preprocessing according to the feature dimension and the label distribution to obtain updated input data;

[0049] A model initialization module, configured to initialize model hyperparameters according to the updated feature dimension of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model;

[0050] A parameter update module, configured to update model hyperparameters according to the performance metrics of the initialization model and a preset performance threshold, so as to obtain updated hyperparameters;

[0051] A model compression module, configured to perform a single inference task through the initialization model, record the single inference time, and count the throughput of the model. When the inference time exceeds a preset time upper limit or the throughput is lower than a preset throughput lower limit, the model is compressed to obtain a compressed model;

[0052] A model output module, configured to reconfigure the model according to the compressed model and the updated hyperparameters to obtain a final neural network model, and output binary codes for the weight features and structural features of the neural network model.

[0053] In a third aspect, the present invention further provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the neural network control method described in any one of the above is implemented.

[0054] In a fourth aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the neural network control method described in any one of the above.

[0055] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a neural network control method, system, and storage medium. The method includes obtaining input data, feature dimensions, the number of samples, and label distributions; performing data preprocessing according to the feature dimensions and the label distributions to obtain updated input data; training a model based on the updated input data and initialization model hyperparameters to obtain an initialization model; updating model hyperparameters according to the performance metrics of the initialization model and a preset performance threshold to obtain updated hyperparameters; performing a single inference task through the initialization model, recording the single inference time, and compressing the model to obtain a compressed model; reconfiguring the model according to the compressed model and the updated hyperparameters to obtain a final neural network model, and outputting binary codes for the weight features and structural features of the neural network model. This method can improve the training efficiency of the neural network.

[0056] Specifically, this method adjusts the hyperparameters of the model, such as key settings like the learning rate, the number of hidden layers, or the number of nodes in each layer, according to the new feature dimensions carried by the updated input data, and trains the model based on these adjusted hyperparameters to finally obtain a preliminarily trained initialization model.

[0057] First, select the model hyperparameters that match the updated feature dimension from the pre-stored initial hyperparameter list as the initialization values. The initialization values are based on the previously accumulated experience or experimental results, ensuring that the selected hyperparameters can better adapt to the new feature dimension and providing a reasonable starting point for subsequent training. This process reflects a dynamic adjustment ability for the model architecture and training strategy, which helps improve the adaptability of the model to different tasks and datasets.

[0058] This method not only takes into account the effectiveness and efficiency of model training but also improves the model convergence speed and final performance by reasonably initializing the hyperparameters. At the same time, since the adjustment is based on the updated feature dimension of the input data, it can enhance the generalization ability and adaptability of the model to new data types to a certain extent, thus maintaining good prediction performance in the face of a constantly changing data environment and improving the training efficiency of the neural network.

[0059] Furthermore, this method proposes two key performance indicators, inference time and throughput. When the model fails to reach the set threshold in terms of inference speed or throughput capacity, measures are taken to compress the model to improve its efficiency.

[0060] After obtaining the test set data, perform a single inference task by inputting this data into the initialized model, and record the time taken for the entire process from input to output, which is the single inference time. Subsequently, count the number of inferences that can be completed within the preset test time to calculate the throughput of the model, that is, the number of inferences completed per unit time. These two indicators are used to measure the real-time performance and processing ability of the model.

[0061] If the single inference time exceeds the preset time limit, it is considered that the inference speed of the model does not meet the standard; if the number of inferences completed per unit time (throughput) is lower than the preset lower limit, it is considered that the throughput capacity of the model does not meet the standard. Once the model fails to meet the standard in any of the above indicators, remove the redundant neurons and neuron connections from the initialized model to obtain a more streamlined and efficient compressed model.

[0062] This optimization method not only helps reduce the computational complexity of the model, improve the inference speed and throughput, but also reduces the demand for hardware resources, enabling the model to run better in resource-constrained environments. In addition, by removing redundant structures, the overfitting phenomenon can be alleviated and the generalization ability of the model can be improved. It should be noted that model compression needs to be carried out on the premise of minimizing the loss of model accuracy to ensure that the optimized model can still maintain high prediction accuracy. Description of the Drawings

[0063] Figure 1 is a schematic flowchart of a neural network control method provided by the first embodiment of the present invention;

[0064] Figure 2 It is a schematic diagram of the structure of a neural network control system provided by the second embodiment of the present invention. Specific implementation manners

[0065] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0066] Currently, in the design and implementation of neural networks, the trade-off between model performance and computational efficiency is a key challenge. With the rapid development of deep learning technology, neural networks with deeper layers and broader architectures are continuously explored to improve prediction accuracy. However, this trend of pursuing high performance also brings significant computational resource consumption problems. Therefore, in the design stage, the impact of model complexity on hardware resources must be carefully considered, including but not limited to CPU / GPU utilization, memory occupancy, and the time required for training and inference.

[0067] In an existing technology, the model architecture is simplified, and techniques such as depthwise separable convolution and residual connection are adopted to simplify the model architecture, reduce the computational burden, while maintaining or improving the prediction accuracy. Further, various strategies are explored for further optimization. Further, model compression techniques such as pruning and quantization are adopted. By reducing the number of parameters and lowering the weight precision, the model becomes more compact and efficient; knowledge distillation enables a small student model to learn from a large teacher model and quickly obtain similar generalization capabilities. Adaptive inference time (AIT) allows the model to dynamically adjust the computational path according to the complexity of the input data, while neural architecture search (NAS) automatically searches for a network structure that is both efficient and accurate. Specific hardware optimizations customize the model for different computing platforms to maximize the utilization of hardware resources. At the same time, effective data augmentation and preprocessing can improve the generalization ability and learning efficiency of the model.

[0068] Although the above methods alleviate the contradiction between performance and efficiency to a certain extent, they also face some limitations. For problems in certain specific fields, existing lightweight networks cannot provide sufficient expressiveness, resulting in a decrease in accuracy and low training efficiency of neural networks.

[0069] To solve the above problems, referring to Figure 1 , the first embodiment of the present invention provides a neural network control method, including the following steps:

[0070] S11. Obtain the input data and determine the feature dimension, number of samples, and label distribution of the input data;

[0071] S12. Perform data preprocessing based on the feature dimension and the label distribution to obtain updated input data;

[0072] S13. Initialize the model hyperparameters according to the updated feature dimension of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model;

[0073] S14. Update the model hyperparameters according to the performance metrics of the initialized model and a preset performance threshold to obtain updated hyperparameters;

[0074] S15. Perform a single inference task through the initialized model, record the single inference time, and count the throughput of the model. When the inference time exceeds the preset time limit or the throughput is lower than the preset throughput lower limit, compress the model to obtain a compressed model;

[0075] S16. Reconfigure the model according to the compressed model and the updated hyperparameters to obtain a final neural network model, and perform binary encoding on the weight features and structural features of the neural network model for output.

[0076] In step S11, obtain the input data and determine the feature dimension, number of samples, and label distribution of the input data.

[0077] In one implementation, directly read the input data from the database. The feature dimension refers to the number of attributes or variables included in each sample. For structured data, it is the number of columns in the table. The number of samples refers to how many independent observations or instances there are in the entire dataset. This refers to the number of rows in the dataset (for tabular data). The label distribution refers to the amount of data included in each type of data, which is obtained through statistical methods.

[0078] In step S12, perform data preprocessing based on the feature dimension and the label distribution to obtain updated input data.

[0079] In one implementation, when the feature dimension is greater than the number of samples, perform dimensionality reduction on the input data to obtain dimensionality-reduced input data;

[0080] Segment the dimensionality-reduced input data according to a preset category quantity threshold to obtain majority category data and minority category data;

[0081] Interpolate the minority category data according to the following formula to obtain updated minority category data:

[0082] xnew = x li + λ × (x lj - x li )

[0083] where x new is the updated minority class data, x li is the i-th data in the minority class data, λ is the interpolation feature vector, and x lj is the j-th data in the minority class data;

[0084] Deduplicate the data according to the majority class data to obtain the updated majority class data;

[0085] The updated input data includes the updated majority class data and the updated minority class data.

[0086] In one implementation, standardize the input data to obtain the standardized input data; perform singular value decomposition on the standardized input data to obtain the principal component vectors and principal component eigenvalues of the standardized input data; perform maximum value screening according to the principal component vectors, principal component eigenvalues, and a preset number of principal components to construct a projection vector; multiply the projection vector and the standardized input data in matrix form to obtain the dimensionality-reduced input data.

[0087] In one implementation, standardization is performed using the following formula:

[0088]

[0089] where x' is the standardized data, x is the original data, μ represents the mean of the feature, and σ represents the standard deviation of the feature.

[0090] In one formula method, the singular value decomposition formula is as follows:

[0091] A = UΣV T

[0092] where A is the matrix of the standardized input data, U represents the left singular vector, Σ is the singular value matrix, and V T represents the transpose of V, and V represents the right singular vector;

[0093] Solve the equation to obtain the singular value matrix, which is the principal component eigenvalue mentioned in this method.

[0094] In one implementation, the deduplication operation uses mean deduplication. For two data points with an Euclidean distance less than a preset distance threshold, their average value is used as the updated data, reducing the number of data points from two to one.

[0095] In step S13, initialize the model hyperparameters according to the updated feature dimensions of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model.

[0096] In one implementation, the model hyperparameters include a learning rate, the number of hidden layers, and the number of nodes in each layer.

[0097] Obtain a pre-stored list of initial hyperparameters, and use the model hyperparameters corresponding to the updated feature dimensions as the initialized model hyperparameters.

[0098] Train the model based on the updated input data and the initialized model hyperparameters. When it is detected that the loss function of the model meets the conditions or the number of training times reaches the preset upper limit of the number of training times, complete the training of the model to obtain an initialized model.

[0099] It should be noted that for obtaining the pre-stored list of initial hyperparameters, this is a predefined configuration file or table that contains the model hyperparameter values recommended for datasets with different feature dimensions. Such a list can help quickly start experiments and reduce unnecessary parameter tuning time. Model hyperparameters refer to those parameters that cannot be automatically learned during the training process but need to be set by the user before the start of training. Hyperparameters include, but are not limited to, the learning rate, the number of hidden layers, or the number of nodes in each layer, etc. For example, for 8-dimensional input data, set the learning rate to 0.001, the number of hidden layers to 2, the number of nodes in each layer to [64, 32], the batch size to 32, and the number of training epochs to 100. For higher-dimensional input data, there will be a larger number of hidden layers, the number of nodes in each layer, the batch size, and the number of training epochs, as well as a smaller learning rate.

[0100] In step S14, update the model hyperparameters according to the performance metrics of the initialized model and the preset performance threshold to obtain updated hyperparameters.

[0101] In one implementation, the performance metrics include accuracy; when the accuracy is less than the preset accuracy threshold, increase the number of hidden layers and the number of neurons in each layer; when the accuracy is greater than the preset accuracy threshold, keep the model hyperparameters unchanged; the updated hyperparameters are the adjusted model hyperparameters.

[0102] It is worth noting that in addition to accuracy, other evaluation criteria can be considered, such as Precision, Recall, F1-score, etc., to comprehensively evaluate the quality of the model. When the accuracy is less than the preset accuracy threshold, it means that the model has an underfitting problem, that is, it fails to fully capture the patterns in the training data. At this time, the model complexity can be increased in the following ways: increasing the number of hidden layers, because more hidden layers can enable the neural network to learn more complex feature representations and thus better fit the training data. Or increasing the number of neurons in each layer. However, too many neurons will lead to overfitting, that is, the model is too focused on memorizing the training samples rather than generalizing to unseen data.

[0103] In step S15, a single inference task is performed through the initialized model, the single inference time is recorded, and the throughput of the model is statistically calculated. When the inference time exceeds the preset time limit or the throughput is lower than the preset throughput lower limit, the model is compressed to obtain a compressed model.

[0104] In one implementation, test set data is obtained;

[0105] The test set data is input into the initialized model to start a single inference task, and the time from inputting the data to obtaining the output result is recorded as the single inference time;

[0106] The number of inferences within a preset test time is statistically calculated, and the throughput of the model is calculated. Among them, the throughput is the number of inferences completed per unit time;

[0107] When the single inference time exceeds the preset time limit, it is determined that the inference speed does not meet the standard;

[0108] When the throughput is lower than the preset throughput lower limit, it is determined that the throughput capacity does not meet the standard;

[0109] When the inference speed does not meet the standard or the throughput capacity does not meet the standard, redundant neurons and neuron connections are removed from the initialized model to obtain a compressed model.

[0110] It is worth noting that the inference speed refers to the time required for the model to process a single input sample, which is directly related to whether the model can meet the requirements of real-time or near-real-time in actual applications. For example, in scenarios such as autonomous driving and intelligent manufacturing, there are extremely high requirements for the inference speed of the model. If the inference time exceeds the preset time limit, it is determined that the inference speed does not meet the standard, which indicates that the model cannot respond to new input data in a timely manner, thus affecting the user experience or system performance. The throughput is the maximum number of input instances that the model can process per unit time. Different from the latency involved in processing a single instance, in order to achieve the maximum throughput, multiple instances need to be processed in parallel.

[0111] In step S16, according to the compression model and the updated hyperparameters, the model is reconfigured to obtain the final neural network model, and the weight features and structural features of the neural network model are output in binary encoding.

[0112] In one implementation, the optimizer of the compression model is changed to an updated optimizer; the number of hidden layers of the compression model is updated to be consistent with the updated hyperparameters; wherein, the updated hyperparameters include the updated optimizer and the number of hidden layers.

[0113] It should be noted that after updating the optimizer, it is necessary to apply it to the compression model. This includes setting hyperparameters such as a new learning rate, momentum, and weight decay.

[0114] In one implementation, after completing all the above adjustments, the next step is to export the final version of the neural network model as a binary format file. First, it is necessary to record the topological structure information of the entire network, including but not limited to the input / output layer sizes, types of activation functions, convolutional kernel sizes, etc. This part of the content can be implemented by defining a model class or serializing a framework-specific data structure. Then, the trained weight parameters also need to be saved. Since the weights are represented in floating-point form, in order to improve storage efficiency and transmission speed, quantization technology needs to be used to convert them into a lower-precision data type (such as INT8). Then, they are extracted through an extraction method and written into the file in binary form. Finally, in order to facilitate subsequent loading and deployment, it is selected to use Protocol Buffers, JSON, or other serialization libraries for encoding. This can not only ensure cross-platform compatibility but also simplify the parsing logic.

[0115] The present invention discloses a neural network control method, system, and storage medium. This technical solution mainly aims at the problem of improving the training efficiency of neural networks and provides an optimized solution to achieve more efficient model training and deployment. The method of the present invention covers a series of steps from data preprocessing to model adjustment and optimization, ensuring that the model can effectively improve the calculation efficiency while ensuring prediction accuracy and adapting to different types of input data.

[0116] First, the present invention focuses on obtaining information about the original input data, its feature dimensions, number of samples, and label distribution, and preprocessing the data based on this information to generate an updated input data set. In particular, in the case where the feature dimensions exceed the number of samples, dimensionality reduction processing is adopted to ensure the effectiveness of subsequent training; for data sets with class imbalance, interpolation processing and data deduplication and other means are used to improve the representation of minority classes, ensuring a more balanced relationship between classes during the training process.

[0117] Next, initialize the model hyperparameters according to the updated feature dimensions, including but not limited to selecting an appropriate learning rate, the number of hidden layers, or the number of nodes in each layer. This stage utilizes the optimal configuration in the pre-stored experimental results as a starting point, thereby improving the model convergence speed and enhancing its generalization ability. After completing the preliminary training, the present invention further evaluates the model performance (such as accuracy) and adjusts the hyperparameters according to the evaluation results to achieve the best performance. If the model fails to meet the preset inference speed or throughput criteria, a model compression operation will be performed to remove redundant neurons and connections, thereby obtaining a more streamlined and efficient version.

[0118] Finally, the present invention reconfigures the compressed model, including updating the optimizer of the model to a new type and adjusting the number of hidden layers to be consistent with the latest hyperparameters. This process also includes encoding and outputting the weight features and structural features of the model, so that the neural network model optimized through a series of steps can not only improve the computational efficiency while maintaining the prediction accuracy, but also better adapt to various input data and maintain good generalization performance. The whole process reflects a dynamic adjustment ability of the model architecture and training strategy, which helps to improve the adaptability of the model to various tasks and datasets and ensure excellent prediction effects when facing a changing data environment.

[0119] In summary, the neural network control method, system, and storage medium proposed by the present invention solve problems such as high computational resource consumption and long training time faced in current deep learning practices through a series of technical measures, providing new ideas and technical support for future research and development. In summary, the neural network control method, system, and storage medium provided by the present invention achieve a comprehensive optimization of the neural network training process through a series of technical measures without sacrificing the model accuracy, thereby solving problems such as high computational resource consumption and long training time faced in current deep learning practices and improving the training efficiency of the neural network model.

[0120] Referring to Figure 2 , the second embodiment of the present invention provides a neural network control system, including:

[0121] A data acquisition module, configured to acquire input data and determine the feature dimension, sample quantity, and label distribution of the input data;

[0122] A data processing module, configured to perform data preprocessing according to the feature dimension and the label distribution to obtain updated input data;

[0123] A model initialization module, which is used to initialize model hyperparameters according to the updated feature dimension of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model;

[0124] A parameter update module, which is used to update model hyperparameters according to the performance index of the initialized model and a preset performance threshold to obtain updated hyperparameters;

[0125] A model compression module, which is used to perform a single inference task through the initialized model, record the single inference time, and count the throughput of the model. When the inference time exceeds the preset time limit or the throughput is lower than the preset throughput lower limit, the model is compressed to obtain a compressed model;

[0126] A model output module, which is used to reconfigure the model according to the compressed model and the updated hyperparameters to obtain a final neural network model, and perform binary encoding on and output the weight features and structural features of the neural network model.

[0127] Preferably, the data acquisition module is used for:

[0128] Obtain input data, and determine the feature dimension, sample quantity, and label distribution of the input data.

[0129] Preferably, the data processing module is used for:

[0130] Perform data preprocessing according to the feature dimension and the label distribution to obtain updated input data, including:

[0131] When the feature dimension is greater than the sample quantity, perform dimensionality reduction processing on the input data to obtain dimensionality-reduced input data, including:

[0132] Normalize the input data to obtain normalized input data;

[0133] Perform singular value decomposition on the normalized input data to obtain the principal component vectors and principal component eigenvalues of the normalized input data;

[0134] Perform maximum value screening according to the principal component vectors, principal component eigenvalues, and a preset number of principal components to construct a projection vector;

[0135] Multiply the projection vector and the normalized input data in matrix form to obtain dimensionality-reduced input data;

[0136] Divide the dimensionality-reduced input data according to a preset class quantity threshold to obtain majority class data and minority class data;

[0137] Interpolate the minority class data according to the following formula to obtain updated minority class data:

[0138] x new = x li + λ × (x lj - x li )

[0139] where x new is the updated minority class data, x li is the i-th data in the minority class data, λ is the interpolation feature vector, and x lj is the j-th data in the minority class data;

[0140] Deduplicate the majority class data to obtain updated majority class data;

[0141] The updated input data includes the updated majority class data and the updated minority class data.

[0142] Preferably, the model initialization module is used for:

[0143] Initialize the model hyperparameters according to the updated feature dimension of the updated input data, and train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model, including:

[0144] where the model hyperparameters include the learning rate, the number of hidden layers, and the number of nodes in each layer;

[0145] Obtain a pre-stored list of initial hyperparameters, and use the model hyperparameters corresponding to the updated feature dimension as the initialized model hyperparameters;

[0146] Train the model based on the updated input data and the initialized model hyperparameters. When it is detected that the loss function of the model meets the conditions or the number of training times reaches the preset upper limit of the number of training times, complete the training of the model to obtain an initialized model.

[0147] Preferably, the parameter update module is used for:

[0148] Update the model hyperparameters according to the performance index of the initialized model and a preset performance threshold to obtain updated hyperparameters, including:

[0149] The performance index includes the accuracy rate;

[0150] When the accuracy rate is less than the preset accuracy threshold, increase the number of hidden layers and the number of neurons in each layer;

[0151] When the accuracy rate is greater than the preset accuracy threshold, keep the model hyperparameters unchanged;

[0152] The updated hyperparameters are the hyperparameters of the model after adjustment.

[0153] Preferably, the model compression module is configured to:

[0154] Perform a single inference task through the initialized model, record the single inference time, and count the throughput of the model. When the inference time exceeds the preset time limit or the throughput is lower than the preset throughput lower limit, compress the model to obtain a compressed model, including:

[0155] Obtain test set data;

[0156] Input the test set data into the initialized model, start a single inference task, and record the time from input data to output result as the single inference time;

[0157] Count the number of inferences within a preset test time, and calculate the throughput of the model. The throughput is the number of inferences completed per unit time;

[0158] When the single inference time exceeds the preset time limit, it is determined that the inference speed does not meet the standard;

[0159] When the throughput is lower than the preset throughput lower limit, it is determined that the throughput capacity does not meet the standard;

[0160] When the inference speed does not meet the standard or the throughput capacity does not meet the standard, remove redundant neurons and neuron connections from the initialized model to obtain a compressed model.

[0161] Preferably, the model output module is configured to:

[0162] Reconfigure the model according to the compressed model and the updated hyperparameters to obtain the final neural network model, and perform binary encoding and output on the weight features and structural features of the neural network model, including:

[0163] Change the optimizer of the compressed model to an updated optimizer;

[0164] Update the number of hidden layers of the compressed model to be consistent with the updated hyperparameters;

[0165] Among them, the updated hyperparameters include an updated optimizer and the number of hidden layers.

[0166] It should be noted that a neural network control system provided by an embodiment of the present invention is used to execute all process steps of a neural network control method in the above embodiment, and the working principles and beneficial effects of the two correspond one by one, so details are not described herein again.

[0167] An embodiment of the present invention also provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a data acquisition program. When the processor executes the computer program, the steps in the above-mentioned embodiments of each neural network control method are implemented, such as Figure 1 step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, such as a data acquisition module.

[0168] Exemplarily, the computer program may be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device.

[0169] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device. It may include more or fewer components than the above, or combine some components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0170] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device and connects various parts of the entire electronic device through various interfaces and lines.

[0171] The memory can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and by invoking the data stored in the memory, the processor can implement various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0172] Among them, if the modules / units integrated in the electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0173] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0174] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. In particular, for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A neural network control method, characterized in that: include: Obtain input data, and determine the feature dimension, sample quantity, and label distribution of the input data; Perform data preprocessing according to the feature dimension and the label distribution to obtain updated input data; Initializing model hyperparameters according to the updated feature dimensions of the updated input data, and training the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model; The model hyperparameters are updated according to the performance indicators of the initialization model and the preset performance thresholds to obtain updated hyperparameters; Perform a single inference task through the initialization model, record the single inference time, and count the throughput of the model. When the inference time exceeds the preset time upper limit or the throughput is lower than the preset throughput lower limit, compress the model to obtain a compressed model; According to the compression model and the updated hyperparameters, the model is reconfigured to obtain a final neural network model, and the weight features and structural features of the neural network model are binary-encoded and output.

2. The neural network control method according to claim 1, characterized in that: The performing data preprocessing according to the feature dimension and the label distribution to obtain updated input data includes: When the feature dimension is greater than the sample quantity, performing dimension reduction processing on the input data to obtain dimension-reduced input data; Segmenting the reduced-dimensionality input data according to a preset category quantity threshold to obtain majority category data and minority category data; Interpolation processing is performed according to the minority category data through the following formula to obtain updated minority category data: x new =x li +λ×(x lj -x li ) Among them, x new To update the minority class data, x li is the i-th data in the minority category data, λ is the interpolation feature vector, x lj The j-number data is the minority category data; Deduplication of data is performed according to the majority category data to obtain updated majority category data; The updated input data includes the updated majority class data and the updated minority class data.

3. The neural network control method according to claim 1, characterized in that: Initializing the model hyperparameters according to the updated feature dimensions of the updated input data, and training the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model, include: The model hyperparameters include learning rate, number of hidden layers and number of nodes per layer; Obtain a pre-stored initial hyperparameter list, and use the model hyperparameters corresponding to the updated feature dimensions as the initialization model hyperparameters; The model is trained based on the updated input data and the initialized model hyperparameters. When it is detected that the loss function of the model meets the conditions or the number of training times reaches a preset upper limit of the number of training times, the model training is completed to obtain the initialized model.

4. The neural network control method according to claim 1, characterized in that: The updating of the model hyperparameters according to the performance indicators of the initialization model and the preset performance threshold to obtain updated hyperparameters includes: The performance indicators include accuracy; When the accuracy is less than a preset accuracy threshold, increasing the number of hidden layers and the number of neurons in each layer; When the accuracy is greater than a preset accuracy threshold, keeping the model hyperparameters unchanged; The updated hyperparameters are the adjusted model hyperparameters.

5. The neural network control method according to claim 1, characterized in that: The initialization model is used to perform a single reasoning task, record the single reasoning time, and count the throughput of the model. When the reasoning time exceeds the preset time upper limit or the throughput is lower than the preset throughput lower limit, the model is compressed to obtain a compressed model, including: Get the test set data; Input the test set data into the initialization model, start a single reasoning task, and record the time from inputting data to outputting results as the single reasoning time; Count the number of inferences within a preset test time and calculate the throughput of the model, where the throughput is the number of inferences completed per unit time; When the single inference time exceeds the preset time limit, it is determined that the inference speed does not meet the standard; When the throughput is lower than the preset throughput lower limit, it is determined that the throughput capacity does not meet the standard; When the inference speed does not meet the standard or the throughput capacity does not meet the standard, redundant neurons and neuron connections are removed from the initialization model to obtain a compressed model.

6. The neural network control method according to claim 1, characterized in that: The reconfiguring of the model according to the compression model and the updating hyperparameters to obtain a final neural network model comprises: Changing the optimizer of the compression model to an update optimizer; Updating the number of hidden layers of the compression model to be consistent with the updated hyperparameter; The updating of the hyperparameters includes updating the optimizer and the number of hidden layers.

7. The neural network control method according to claim 2, characterized in that: When the feature dimension is greater than the sample quantity, the input data is subjected to dimensionality reduction processing to obtain dimensionality-reduced input data, including: Standardizing the input data to obtain standardized input data; Performing singular value decomposition on the standardized input data to obtain a principal component vector and a principal component eigenvalue of the standardized input data; Perform maximum value screening according to the principal component vector, the principal component eigenvalue and the preset number of principal components to construct a projection vector; The projection vector and the standardized input data are matrix multiplied to obtain dimension-reduced input data.

8. A neural network control system, characterized in that: include: A data acquisition module is used to acquire input data and determine the feature dimension, sample quantity and label distribution of the input data; A data processing module, used for performing data preprocessing according to the feature dimension and the label distribution to obtain updated input data; A model initialization module, used to initialize model hyperparameters according to the updated feature dimensions of the updated input data, and to train the model based on the updated input data and the initialized model hyperparameters to obtain an initialized model; A parameter updating module, used to update the model hyperparameters according to the performance indicators of the initialization model and a preset performance threshold to obtain updated hyperparameters; A model compression module is used to perform a single reasoning task through the initialization model, record the single reasoning time, and count the throughput of the model. When the reasoning time exceeds the preset time upper limit or the throughput is lower than the preset throughput lower limit, the model is compressed to obtain a compressed model; The model output module is used to reconfigure the model according to the compression model and the updated hyperparameters to obtain the final neural network model, and output the weight features and structural features of the neural network model in binary encoding.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the neural network control method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the neural network control method according to any one of claims 1 to 7.

Images