A neural network weight initialization and training method for digital recognition

Abstract

The application discloses a neural network weight initialization and training method for digital identification, comprising the following steps: dividing a training set and a test set according to a digital image dataset; calculating information entropy gain; performing normalization processing on the calculated information entropy gain; initializing the weight of a neural network according to the normalized information entropy gain; adding a self-defined learning rate callback function, gradually increasing the learning rate value by calculating the proportion of the learning rate in the training period, and selecting and saving the learning rate value and the neural network model weight parameter in the state of the highest model accuracy after the termination of all iteration times; and taking the trained model as a digital image identification model to realize digital image identification through the digital image identification model. The application combines feature selection and information gain technology with a deep learning model to solve the digital image identification problem and improve the performance of the digital image identification model, thereby improving the accuracy and identification speed of the digital image identification.

Description

Technical Field

[0001] This invention belongs to the field of digital image recognition and relates to deep learning technology, specifically to a method for initializing and training neural network weights for digital recognition. Background Technology

[0002] Digital image recognition, as a core field of computer vision, plays a vital role in numerous areas such as security monitoring, medical diagnosis, and autonomous driving. With technological advancements, the effective recognition and processing of image data has become crucial, which is precisely where feature selection, information gain, and deep learning—modern machine learning techniques—are key applications.

[0003] Feature selection, information gain, and deep learning are key techniques in modern machine learning. Feature selection aims to pick the most relevant and useful features from a multitude of options to improve model performance and reduce overfitting. Information gain, as a feature selection method, evaluates the contribution of a feature by comparing its information entropy with the information entropy of the feature combined with its label.

[0004] Deep learning is a method that simulates the workings of neurons in the human brain, achieving learning and prediction tasks by constructing deep neural networks. Introducing feature selection and information gain into deep learning means merging the concepts of traditional feature selection with the capabilities of deep learning to more effectively learn and utilize key features, improving model performance and interpretability. This cross-application brings innovative solutions to complex data tasks, highlighting the crucial role of feature selection in optimizing deep learning models.

[0005] Despite the significant progress made in digital image recognition models, such as traditional CNNs, GANs, and RNNs, a series of challenges and problems still need to be addressed. For example, for complex models with a large number of parameters, the training and inference processes often require a lot of computing resources, or the model may perform well on training data, but its performance degrades when it encounters new and unseen data, i.e., overfitting.

[0006] Therefore, a new technical solution is needed to solve this problem. Summary of the Invention

[0007] Purpose of the invention: In order to overcome the shortcomings of the existing technology, a method for initializing and training neural network weights for digital recognition is provided. This method combines feature selection and information gain techniques with deep learning models to solve the problem of digital image recognition and improve the performance of digital image recognition models, thereby increasing the accuracy and speed of digital image recognition.

[0008] Technical Solution: To achieve the above objectives, this invention provides a method for initializing and training neural network weights for digit recognition, comprising the following steps:

[0009] S1: Divide the digital image dataset into training and test sets;

[0010] S2: Import the training set into the deep neural network for training and calculate the information entropy gain;

[0011] S3: Normalize the calculated information entropy gain;

[0012] S4: Initialize the weights of the neural network based on the normalized information entropy gain;

[0013] S5: Add a custom learning rate callback function. During the training cycle, the learning rate value is gradually increased by calculating the proportion of the learning rate. After all iterations are terminated, the learning rate value and neural network model weight parameters with the highest model accuracy are selected and saved.

[0014] S6: Use the model trained in step S5 as a digital image recognition model to achieve digital image recognition.

[0015] Furthermore, in step S1, the MINIST handwritten digit image dataset is used to divide the training set and the test set.

[0016] Furthermore, the information entropy gain in step S2 is calculated as follows: the features and output labels are treated as two random variables, the label data is preprocessed by one-hot encoding, the Scikit-Learn tool library is called, and its internal mutual_info_classif() function is used to calculate the information entropy gain between each feature and the output.

[0017] Furthermore, the formula for calculating the information entropy gain sg is as follows:

[0018]

[0019] Where X represents the feature, Y represents the output label, P(x,y) represents the probability of them occurring simultaneously, P(x) and P(y) represent the probability of them occurring individually, and the calculated information entropy gain is stored in the scores variable.

[0020] Furthermore, the normalization process in step S3 is performed as follows: the information entropy gain is normalized using the MinMaxScaler tool, and the feature_range parameter in MinMaxScaler is adjusted so that the distribution range of the normalized information entropy gain value is (0.001, 0.1).

[0021] Furthermore, in step S4, the method for initializing the weights of the neural network is as follows: during the initialization process of the first hidden layer of the neural network, the normalized information entropy gain value is used as the initial value of the weights of the first hidden layer neurons connected to the input features.

[0022] Furthermore, the specific expression of using the normalized information entropy gain value as the initial value of the weight of the first hidden layer neuron connected to the input feature in step S4 is as follows:

[0023] The weights ω from the input layer to the hidden layer 11 Related to the first-layer input information t, and since this invention uses information entropy gain instead of random initialization of the input information, it can be initialized to information entropy gain sg, i.e.

[0024] ω 11 ~ω 1n =sg1

[0025] Where n is the total number of neurons in the hidden layer, sg1 is the information entropy gain of the connection between all neurons in the first hidden layer and the first neuron in the input layer, and t is the input information of the input layer. Since this invention is not limited to the first neuron in the input layer, therefore...

[0026] ω in =t i =sg i

[0027] Where i represents the total number of neurons in the input layer.

[0028] Furthermore, step S5 is specifically expressed as follows:

[0029] Used when obtaining the learning rate (lr) for each iteration.

[0030]

[0031] The learning rate lr in the current training cycle is multiplied by the scaling constant to obtain the learning rate lr used in the next training cycle. Here, start_lr represents the initial learning rate, end_lr represents the ending learning rate, and N represents the total number of training cycles.

[0032] Furthermore, in step S5, the trained model is tested and evaluated using a test set. The processed MINIST test set is input into the original model and the trained model respectively, and quantitative comparison curves of learning rate, weight convergence speed and correlation coefficient are generated on the same coordinate axis to evaluate the model improvement effect.

[0033] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0034] 1. Integration of Feature Selection and Information Gain: By using information entropy gain to initialize the weights of the neural network, features with larger information entropy gains will receive larger weight values ​​in the initial state, so as to better capture important features in the early learning stage of the model. This can increase the model accuracy and shorten the learning time while reducing the interference of redundant features on the model.

[0035] 2. Create a learning rate callback function: By incrementing the learning rate, it is easier to find a suitable learning rate range in the early stages of model training, avoiding choosing an excessively large learning rate at the beginning, which may cause the model to fail to converge or become unstable.

[0036] 3. Compared to traditional deep learning methods, the innovation of this invention lies in applying feature selection and information gain to deep learning, combining the advantages of feature optimization, weight initialization, and incremental learning rate. This can effectively improve the performance of digit recognition models, increase training speed and recognition accuracy, and provide a novel solution for improving the speed and accuracy of digital image recognition and classification. Attached Figure Description

[0037] Figure 1 This is an overall flowchart of the method of the present invention;

[0038] Figure 2 This is a flowchart of the dataset preprocessing in this invention;

[0039] Figure 3 A schematic diagram showing the neural network structure initialized with the weights of the input layer and the first hidden layer.

[0040] Figure 4 This is a comparison chart showing the convergence speed of the weights in the present invention model and the traditional model before and after the improvement.

[0041] Figure 5 This is a comparison chart of the correlation coefficient between weights and information entropy gain before and after the improvement in this invention;

[0042] Figure 6 This is a comparison chart showing the effects of the present invention and existing learning rate changing methods. Detailed Implementation

[0043] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0044] like Figure 1 As shown, the present invention provides a method for initializing and training neural network weights for digit recognition, comprising the following steps:

[0045] S1: Use the MINIST handwritten digit image dataset to divide the training and test sets;

[0046] S2: Import the training set into the deep neural network for training and calculate the information entropy gain:

[0047] Features and output labels are treated as two random variables. One-hot encoding preprocessing is performed on the label data. The Scikit-Learn library is called, and its internal `mutual_info_classif()` function is used to calculate the information entropy gain between each feature and the output. The formula for calculating the information entropy gain is:

[0048]

[0049] Where X represents the feature, Y represents the output label, P(x,y) represents the probability of them occurring simultaneously, P(x) and P(y) represent the probability of them occurring individually, and the calculated information entropy gain is stored in the scores variable;

[0050] S3: Normalize the calculated information entropy gain:

[0051] like Figure 2 As shown, the MinMaxScaler tool is used to normalize the information entropy gain. The feature_range parameter in MinMaxScaler is adjusted so that the distribution range of the normalized information entropy gain value is (0.001, 0.1).

[0052] S4: Initialize the weights of the neural network based on the normalized information entropy gain;

[0053] During the initialization of the first hidden layer of the neural network, the normalized information entropy gain value is used as the initial value for the weights of the first hidden layer neurons connected to the input features, as specifically expressed below:

[0054] like Figure 3 As shown, the weights ω from the input layer to the hidden layer 11 Related to the first-layer input information t, and since this invention uses information entropy gain instead of random initialization of the input information, it can be initialized to information entropy gain sg, i.e.

[0055] ω 11 ~ω 1n =sg1

[0056] Where n is the total number of neurons in the hidden layer, sg1 is the information entropy gain of the connection between all neurons in the first hidden layer and the first neuron in the input layer, and t is the input information of the input layer. Since this invention is not limited to the first neuron in the input layer, therefore...

[0057] ω in =t i =sg i

[0058] Where i is the total number of neurons in the input layer;

[0059] S5: Add a custom learning rate callback function. During the training cycle, the learning rate value is gradually increased by calculating the percentage of the learning rate. After all iterations are completed, the learning rate value and neural network model weight parameters at the state with the highest model accuracy are selected and saved.

[0060] Used when obtaining the learning rate (lr) for each iteration.

[0061]

[0062] The learning rate Ir in the current training cycle is multiplied by the scaling constant to obtain the learning rate Ir used in the next training cycle. Here, start_lr represents the starting learning rate, end_lr represents the ending learning rate, and N represents the total number of training cycles.

[0063] The trained model is tested and evaluated using a test set. The processed MINIST test set is input into the original model and the trained model respectively, and quantitative comparison curves of learning rate, weight convergence speed and correlation coefficient are generated on the same coordinate axis to evaluate the model improvement effect. The model that meets the test effect is taken as the trained model.

[0064] In this invention, the model trained in step S5 is used as a digital image recognition model, and digital image recognition is achieved through the digital image recognition model.

[0065] Based on the above, it is evident that features with higher information entropy gains in the method of this invention will receive larger weight values ​​in the initial state, enabling better capture of important features in the early learning stages of the model. By calculating the learning rate ratio and gradually increasing the learning rate, a suitable learning rate range can be found more quickly in the early stages of model training. Data processing shows that initializing weights with information entropy gains in conjunction with a learning rate callback function can shorten the training time of the neural network, improve the accuracy of digital image recognition, and accelerate the recognition speed of digital images.

[0066] To verify the effectiveness and practical results of the present invention, this embodiment applies and tests the above-mentioned present invention solution in the following ways:

[0067] The MNIST handwritten digit dataset was used, with each image having a pixel size of 28*28. The training set contained 60,000 samples, and the test set contained 10,000 samples, covering handwritten digits from 0 to 9.

[0068] In this embodiment, the overall solution is implemented using PyCharm CE software, and the final experimental data obtained includes the following:

[0069] like Figure 4 As shown in the figure, the curve of the correlation coefficient between the information entropy gain and the weights of the model using random initial values ​​before the improvement gradually increases with the increase of the number of iterations, while after the improvement, as shown in the figure... Figure 5 As shown, the curve of the correlation coefficient between the information entropy gain and the weights of the model using information entropy gain instead of random initial values ​​is the highest at the beginning of the iteration, and gradually decreases as the number of iterations increases due to the influence of weight updates during training.

[0070] like Figure 6 As shown, the learning rate increment method is used as follows: Figure 6 As shown in (A), when training the model, compared to... Figure 6 The learning rate decrease method shown in (B) and the learning rate decrease method shown in (B) are as follows: Figure 6 Compared to the learning rate setting method shown in (C), this method achieves higher model accuracy, faster model recognition speed, and the accuracy does not decrease with increasing iterations. In summary, experimental results demonstrate that the improved digital image recognition model of this invention can effectively improve the performance of digital recognition models, increase training speed and recognition accuracy, and play a significant role in enhancing the performance of digital image recognition.

[0071] This embodiment also provides a digital image recognition system based on information entropy gain. The system includes a network interface, a memory, and a processor. The network interface is used to receive and send signals during the process of sending and receiving information with other external network elements. The memory is used to store computer program instructions that can run on the processor. The processor is used to execute the steps of the consensus method described above when running the computer program instructions.

[0072] This embodiment also provides a computer storage medium storing a computer program that, when executed by a processor, can implement the methods described above. The computer-readable medium can be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media include non-volatile memory circuitry (e.g., flash memory circuitry, erasable programmable read-only memory circuitry, or masked read-only memory circuitry), volatile memory circuitry (e.g., static random access memory circuitry or dynamic random access memory circuitry), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drive), and optical storage media (e.g., CD, DVD, or Blu-ray disc). The computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer program may also include or depend on stored data. The computer program may include a basic input / output system (BIOS) for interacting with the hardware of a dedicated computer, device drivers for interacting with specific devices of the dedicated computer, one or more operating systems, user applications, background services, background applications, etc.

Claims

1. A method for initializing and training neural network weights for digit recognition, characterized in that, Includes the following steps: S1: Divide the digital image dataset into training and test sets; S2: Import the training set into the deep neural network for training and calculate the information entropy gain; S3: Normalize the calculated information entropy gain; S4: Initialize the weights of the neural network based on the normalized information entropy gain; S5: Add a custom learning rate callback function. During the training cycle, the learning rate value is gradually increased by calculating the proportion of the learning rate. After all iterations are terminated, the learning rate value and neural network model weight parameters with the highest model accuracy are selected and saved. S6: Use the model trained in step S5 as a digital image recognition model, and realize digital image recognition through the digital image recognition model; The information entropy gain in step S2 is calculated as follows: the features and output labels are treated as two random variables, the label data is preprocessed by one-hot encoding, the Scikit-Learn tool library is called, and its internal mutual_info_classif() function is used to calculate the information entropy gain between each feature and the output. In step S4, the weights of the neural network are initialized as follows: during the initialization of the first hidden layer of the neural network, the normalized information entropy gain value is used as the initial value of the weights of the first hidden layer neurons connected to the input features. In step S4, the normalized information entropy gain value is used as the initial value for the weights of the first hidden layer neuron connected to the input features. The specific expression of this is as follows: Weights from input layer to hidden layer Since it is related to the first-layer input information t, and the information entropy gain is used instead of the random initialization of the input information, it can be initialized as the information entropy gain. ,Right now = ； Where n is the total number of neurons in the hidden layer. Let be the information entropy gain of the connections between all neurons in the first hidden layer and the first neuron in the input layer, and let t be the input information of the input layer. ； Where i represents the total number of neurons in the input layer.

2. The method for initializing and training neural network weights for digit recognition according to claim 1, characterized in that, In step S1, the MINIST handwritten digit image dataset is used to divide the training set and the test set.

3. The method for initializing and training neural network weights for digit recognition according to claim 1, characterized in that, The formula for calculating the information entropy gain is as follows: ； Where X represents the feature and Y represents the output label. This indicates the probability that they occur simultaneously. These represent the probabilities of each occurring individually, and the calculated information entropy gain is stored in the scores variable.

4. The method for initializing and training neural network weights for digit recognition according to claim 1, characterized in that, The normalization process in step S3 is as follows: the information entropy gain is normalized using the MinMaxScaler tool, and the feature_range parameter in MinMaxScaler is adjusted so that the distribution range of the normalized information entropy gain value is (0.001, 0.1).

5. The method for initializing and training neural network weights for digit recognition according to claim 1, characterized in that, The specific expression of step S5 is as follows: Obtain the learning rate for each iteration. When using ； The learning rate in the current training cycle Multiply by the scaling constant to obtain the learning rate to be used in the next training cycle. Where start_lr represents the initial learning rate, end_lr represents the final learning rate, and N represents the total number of training cycles.

6. The method for initializing and training neural network weights for digit recognition according to claim 1, characterized in that, In step S5, the trained model is tested and evaluated using a test set. The processed MINIST test set is input into the original model and the trained model respectively, and quantitative comparison curves of learning rate, weight convergence speed and correlation coefficient are generated on the same coordinate axis to evaluate the model improvement effect.

Images