Image recognition method based on convolutional neural network

Abstract

The invention discloses an image recognition method based on a convolutional neural network, and belongs to the technical field of deep learning, and the method comprises the steps: 1, obtaining an image data set; 2, preprocessing an image data set; 3, dividing a data set; 4, a training process; step 5, a verification process; step 6, a test process; 7, data visualization is carried out; according to the method, a Dropout layer is added behind a linear layer, so that the number of middle features is reduced, redundancy is reduced, namely orthogonality among all the features of each layer is increased, and an over-fitting phenomenon is well prevented; meanwhile, 0.5 is selected as the Dropout rate, so that the number of randomly generated networks in the Dropout process is maximum, the diversity of features is ensured, and the generalization ability of the model is improved; according to the method, a machine learning algorithm model is adopted, through image recognition, rough distinguishing and recognition of dialects and different dialect contents in different regions in China are achieved, and the problem of gradient disappearance is solved; and meanwhile, the data is normalized, so that the network has a better convergence speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and particularly relates to an image recognition method based on a convolutional neural network. Background Art

[0002] The elderly suffering from diseases related to language functions such as nasopharyngeal carcinoma and aphasia are facing many life obstacles, including language communication barriers. The most direct impact is the inability to clearly express their own thoughts and needs, nor can they fully understand the speech of others, resulting in difficulties in daily communication; the decline in daily living ability, the loss of language function may indirectly affect the daily living ability of the elderly, such as activities that require language communication like shopping and seeing a doctor; the increase in safety hazards, due to communication barriers, the elderly may not be able to express help in time when encountering emergencies, increasing the safety risks, etc. Therefore, it is particularly important to understand the daily expression content of the above-mentioned elderly. Additionally, considering that most of the elderly can only speak their native dialects, it is of great significance to identify the dialects and their content.

[0003] Convolutional Neural Networks (abbreviated as CNN) have significant advantages in the field of image recognition: Convolutional neural networks have powerful feature extraction capabilities. Through the alternating stacking of convolutional layers and pooling layers, convolutional neural networks can automatically learn multi-level feature expressions in images; convolutional neural networks have good spatial structure information processing capabilities. The convolution operation effectively retains the spatial structure information in the image by means of local connection and weight sharing, thereby improving the accuracy of image recognition; convolutional neural networks have strong generalization capabilities. By training on large-scale datasets, convolutional neural networks can learn rich image feature representations and pattern recognition rules to achieve fast and accurate recognition. These advantages make convolutional neural networks one of the preferred technologies in the field of image recognition and have achieved remarkable results in practical applications. With the continuous development of deep learning technology, the model structure and optimization methods of convolutional neural networks are also constantly improved and perfected. This has continuously improved the performance of convolutional neural networks in the field of image recognition and provided strong support for various practical application scenarios. Summary of the Invention

[0004] Aiming at the problems in the prior art such as the daily life of the elderly with language barriers, the present invention provides an image recognition method based on a convolutional neural network. This method adopts a machine learning algorithm model, and through the recognition of images, it realizes the rough distinction and recognition of different dialects and different dialect contents in different regions of China, providing ideas for solving the daily life problems of the elderly with language barriers.

[0005] The present invention is achieved through the following technical solutions:

[0006] An image recognition method based on a convolutional neural network specifically includes the following steps:

[0007] Step 1: Obtain an image dataset;

[0008] Place a pressure sensor at the vocal cords and surrounding muscles of the tester, use an oscilloscope to measure the voltage curve of the pressure sensor changing with time when the tester speaks different dialect types or speech contents, and then use drawing software to obtain the image of the above voltage curve as the input image of the convolutional neural network;

[0009] Step 2: Preprocess the image dataset;

[0010] Step 3: Divide the dataset;

[0011] Step 4: Training process;

[0012] Input the images in the training set into the training function, and call the convolutional neural network model for training. Define the accuracy function, select the optimizer, loss function, and learning rate. During the training process, continuously update the parameters using backpropagation. The output of the model is the class label corresponding to the input image, which is used as the predicted value. At the same time, extract the original label of each input image as the true value. Calculate the accuracy corresponding to each epoch according to the accuracy function, calculate the corresponding loss value according to the loss function, and save them to the lists respectively;

[0013] Step 5: Validation process;

[0014] Input the images in the validation set into the training function, and call the convolutional neural network model mentioned above. Repeat the training process in Step 4 to obtain the accuracy of each epoch. Compare the accuracy of each epoch in the process with the accuracy of the previous epoch, and select the optimal model parameters for saving;

[0015] Step 6: Testing process;

[0016] Call the optimal model parameters saved in the validation process in Step 5, input the images in the test set into the training function, and call the convolutional neural network model for training to output the recognition result, thus completing the recognition of dialects and language contents through images based on the convolutional neural network; Calculate the accuracy and loss value according to the accuracy function and loss function;

[0017] Step 7: Data visualization;

[0018] Record and save the accuracy and loss values corresponding to each epoch during the training process, and draw a line chart with the epoch as the horizontal axis, the accuracy as the left vertical axis, and the loss value as the right vertical axis; save the accuracy during the test set testing process, as well as the recognition results and true values corresponding to each image, draw a confusion matrix, and based on the accuracies of each part of the confusion matrix, it is considered that this model can achieve the recognition of different dialect types and different speech contents.

[0019] Further, in step one, the accuracy function is the ratio of the number of correctly recognized ones in the same category to the total number, and the loss function is selected according to the task for comparison. Different tasks may be different, so a specific functional formula cannot be given here.

[0020] Further, in step two, the input images in step one are subjected to data augmentation processing such as rotation, flipping, translation, scaling, brightness adjustment, and contrast adjustment, the data set is expanded, and saved.

[0021] Further, in step three, all the images obtained after preprocessing in step two are divided into a training set, a validation set, and a test set according to the ratio of 8:1:1.

[0022] Further, the convolutional neural network model includes 4 convolutional layers, 4 pooling layers, 4 BN layers, 3 linear layers, and 2 Dropout layers. The activation function used is the ReLU function. Among them, the convolutional kernel size of the two-dimensional convolutional layer is 3*3, and the convolutional kernel moving step size is 1; the size of the pooling layer for downsampling is 2*2, and the step size is 2; the Dropout rate in the Dropout layer is 0.5.

[0023] Further, the convolutional layer is used to extract features from the input data through convolutional operations; due to its parameter sharing mechanism, it can reduce the number of model parameters, reduce the computational cost; maintain the feature position information of the input data, which is beneficial to subsequent recognition and classification;

[0024] The pooling layer is used to reduce the spatial size of the image after feature extraction, reduce the computational amount; extract representative features within the region, and reduce redundant information;

[0025] The BN layer is used to normalize the input data, which can effectively avoid the problems of gradient explosion or gradient disappearance, and increase the stability of the model;

[0026] The linear layer is used to expand the feature map obtained by the last convolutional layer into a one-dimensional vector, providing input for classification and recognition;

[0027] The Dropout layer is used to remove some neurons according to the set Dropout rate and corresponding probability, which can prevent overfitting.

[0028] Compared with the prior art, the advantages of the present invention are as follows:

[0029] (1) For an image recognition method based on a convolutional neural network of the present invention, batch normalization is performed after each convolutional operation to normalize the input distribution to a 0, 1 distribution, enabling the activation function to function better and solving the problem of gradient disappearance; at the same time, since the data is normalized, the network has a better convergence speed;

[0030] (2) A Dropout layer is added after the linear layer to reduce the number of intermediate features, thereby reducing redundancy, that is, increasing the orthogonality between each feature in each layer, and providing good prevention against overfitting; at the same time, 0.5 is selected as the Dropout rate to maximize the number of randomly generated networks during the Dropout process, ensuring the diversity of features and improving the generalization ability of the model;

[0031] (3) The ReLU activation function is introduced. It has a small computational amount, can enhance the sparsity of the network, alleviate the overfitting problem, and maintain information integrity, so the network performance can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts do not necessarily draw according to the actual scale.

[0033] Figure 1 : Training Loss graph for different types of dialect recognition implemented in Embodiment 1 of the present invention;

[0034] Figure 2 : Training accuracy graph for different types of dialect recognition implemented in Embodiment 1 of the present invention;

[0035] Figure 3 : Confusion matrix of the test set results for different types of dialect recognition implemented in Embodiment 1 of the present invention

[0036] Figure 4 : Training Loss graph for the recognition of different contents of the same dialect implemented in Embodiment 2 of the present invention;

[0037] Figure 5 : Training accuracy graph for the recognition of different contents of the same dialect implemented in Embodiment 2 of the present invention;

[0038] Figure 6 : Confusion matrix of the test set results for the recognition of different contents of the same dialect implemented in Embodiment 2 of the present invention;

[0039] Figure 7 This is a schematic flow chart of an image recognition method based on a convolutional neural network according to the present invention. Specific embodiments

[0040] To clearly and completely describe the technical solutions and their specific working processes of the present invention, in combination with the accompanying drawings of the specification, the specific embodiments of the present invention are as follows:

[0041] Embodiment 1

[0042] The pressure sensor used in this embodiment is a wearable pressure sensor prepared based on Ti2CTx MXene; the dialects involved include dialect types in different regions of seven provinces in China, namely Jilin, Shandong, Jiangsu, Anhui, Shaanxi, Sichuan, and Zhejiang, and the language contents involved include six groups of common expressions in daily life such as eating and drinking water.

[0043] This embodiment provides an image recognition method based on a convolutional neural network for distinguishing and recognizing dialects in different regions of China with the same content, specifically including the following steps:

[0044] Step 1: Obtain an image data set;

[0045] Place the pressure sensor at the vocal cords and surrounding muscles of the tester, use an oscilloscope to measure the voltage curve of the pressure sensor changing with time when the tester speaks different dialect types or voice contents, and then use Origin drawing software to obtain the image of the voltage curve as the input image of the convolutional neural network, and save it in the corresponding folder in.jpg format. The folder is named after the dialect type or voice content;

[0046] Step 2: Preprocess the image data set;

[0047] S201: To adapt to the parameters of the model, use a programming language to uniformly adjust the size of the input image to 1*64*64, that is, a single channel (grayscale image) with both the width and height of the image being 64;

[0048] S202: The input image passes through the first convolutional layer, the input channel is 1, and the output channel is 256; then it passes through the first BN layer and the first pooling layer in sequence, and the output channels are both 256; at this time, the image size is 31*31, that is, the input is 1*64*64, and the output is 256*31*31;

[0049] S203: The output in step S202 passes through the second convolutional layer, the input channel is 256, and the output channel is 128; then it passes through the second BN layer and the second pooling layer in sequence, and the output channels are both 128; at this time, the image size is 14*14, that is, the input is 256*31*31, and the output is 128*14*14;

[0050] In S204, the output in step S203 passes through the third convolutional layer with 128 input channels and 128 output channels; then it passes through the third BN layer and the third pooling layer successively, both with 128 output channels; at this time, the image size is 6*6, that is, the input is 128*14*14 and the output is 128*6*6;

[0051] In S205, the output in step S204 passes through the fourth convolutional layer with 128 input channels and 64 output channels; then it passes through the fourth BN layer and the fourth pooling layer successively, both with 64 output channels; at this time, the image size is 2*2, that is, the input is 128*6*6 and the output is 64*2*2;

[0052] In S206, the output in step S205 passes through the Rearragne function (array dimension conversion function) to flatten 64*2*2 into 1*256;

[0053] In S207, the output in step S206 passes through three linear layers successively, and a Dropout layer is set between every two linear layers. Among them: the input and output channels of the first linear layer are 256 and 512 respectively, the input and output channels of the second linear layer are 512 and 256 respectively, and the input and output channels of the third linear layer are 256 and n (n is the number of labels to be recognized); this step is a classification process. After the operation of this step, all input images are output as a label file according to the image features extracted during the convolution process, and the label file has the same name as the initial folder name;

[0054] Step 3: Dataset division: Divide all images into a training set, a validation set, and a test set according to the ratio of 8:1:1;

[0055] Step 4: Training process: Input the data into the defined training function and call the convolutional neural network model of the present invention (at this time, the output of the last linear layer is 7), define the accuracy function (that is, the proportion of correctly recognized images in each dialect category), select Adam as the optimizer, CrossEntryLoss as the loss function, and 0.005 as the learning rate, and compare the accuracy of each epoch during the training process, and save the model parameters of the one with the best effect;

[0056] Step 5: Validation process: Input the images in the validation set into the defined training function and call the convolutional neural network model of the present invention, repeat the training process in step (4), obtain the accuracy of each epoch, compare the accuracy of each epoch in the process with the accuracy of the previous epoch, and save the model parameters of the one with the best effect;

[0057] Step 6: Testing process: Call the model parameters with the best effect saved during the verification process in step (5), input the images in the test set into the defined training function, and call the convolutional neural network model of the present invention to output the recognition results, thereby completing the recognition of dialects and language content through images based on the convolutional neural network; calculate the accuracy rate and loss value according to the accuracy rate function and the loss function;

[0058] Step 7: Data visualization: Record and save the accuracy rate and loss value corresponding to each epoch during the training process, use the epoch as the horizontal axis, and the accuracy rate and loss value as the vertical axis to draw a line graph; save the accuracy rate during the testing process of the test set, as well as the recognition results and true values corresponding to each image, and draw a confusion matrix.

[0059] Example 2:

[0060] This example provides a method for recognizing dialects and language content through images based on a convolutional neural network, and uses this method to distinguish and recognize different contents of dialects in the same region. The specific steps are as follows:

[0061] (1) Obtain an image dataset: According to the test data of the vocal cord vibration of volunteers speaking the same dialect with different contents by a pressure sensor, draw the images corresponding to 6 different contents and save them to the folders corresponding to each content expression;

[0062] (2) Data augmentation: Expand the dataset by performing data augmentation methods such as rotating, flipping, translating, scaling, adjusting brightness, and adjusting contrast on each image.

[0063] (3) Dataset division: Divide all the images into a training set, a validation set, and a test set according to the ratio of 8:1:1;

[0064] (4) Training process: Input the data into the defined training function and call the convolutional neural network model of the present invention (at this time, the output of the last linear layer is 6), define the accuracy rate function (that is, the proportion of correctly recognized images in each content category), select Adam as the optimizer, CrossEntryLoss as the loss function, and 0.001 as the learning rate (learning rate), and compare the accuracy rate of each epoch during the training process, and select the one with the best effect to save the model parameters;

[0065] As Figure 1 shown, the abscissa is the number of epochs passed, and the ordinate is the loss value corresponding to each epoch during the training process; after 8 epochs, the Loss value has started to converge, and after 11 epochs, the Loss value has almost converged to a stable state.

[0066] As Figure 2 shown, the abscissa is the number of epochs passed, and the ordinate is the accuracy corresponding to each epoch during the training process; after 6 epochs, the accuracy has reached over 95%, and after 11 epochs, the accuracy is basically stable at over 99%; this indicates that the convolutional neural network described in this embodiment has a very fast convergence speed and a high accuracy.

[0067] As Figure 4 shown, the abscissa is the number of epochs passed, and the ordinate is the loss value corresponding to each epoch during the training process; after 3 epochs, the Loss value has started to converge, and after 6 epochs, the Loss value has almost converged to stability.

[0068] As Figure 5 shown, the abscissa is the number of epochs passed, and the ordinate is the accuracy corresponding to each epoch during the training process; after 9 epochs, the accuracy has reached over 95%, and after 11 epochs, it is basically stable at over 99%. This indicates that the convolutional neural network described in the present invention has a strong generalization ability.

[0069] (5) Verification process: Feed the images in the verification set into the defined training function and call the convolutional neural network model described in the present invention, repeat the training process in step (4), obtain the accuracy of each epoch, compare the accuracy of each epoch in the process with the accuracy of the previous epoch, and select the best-performing model parameters for saving;

[0070] (6) Testing process: Call the best-performing model parameters saved in the verification process in step (5), feed the images in the test set into the defined training function and call the convolutional neural network model described in the present invention, output the recognition results, thereby completing the recognition of dialects and language content through images based on the convolutional neural network; calculate the accuracy and loss value according to the accuracy function and the loss function;

[0071] As Figure 3 shown, the abscissa is the serial number of the dialect category recognized by the model, and the ordinate is the serial number of the true dialect category corresponding to the image. The recognition accuracy is in the intersection grid. The number in the grid where both the abscissa and ordinate are 0 is 0.96, corresponding to 96% of the number of correctly recognized image data of the dialect type represented by 0; the number in the grid where the ordinate is 0 and the abscissa is 1 is 0.02, corresponding to 2% of the number of image data of the dialect type represented by 0 that the model recognizes as the dialect type represented by 1; the recognition accuracy for each dialect is over 95%, where 0 - 6 represent 7 different dialects in different regions of China to be recognized. Figure 3It shows that the convolutional neural network and the image recognition method described in this embodiment have well completed the recognition task of different regional dialects.

[0072] As Figure 6 shown, the abscissa is the serial number of the content category recognized by the model, the ordinate is the serial number of the true content corresponding to the image, and the recognition accuracy is in the cross grid. The number in the grid where both the abscissa and ordinate are 0 is 0.99, corresponding to 99% of the number of correctly recognized image data of the content represented by 0; the number in the grid where the ordinate is 0 and the abscissa is 4 is 0.01, corresponding to 1% of the number of image data represented by 0 that the model recognizes as the content represented by 4; the recognition accuracy for each type of content is above 94%, where 0 - 5 represent 6 different language contents to be recognized. Figure 6 It shows that the convolutional neural network and the image recognition method described in the present invention have well completed the recognition task of different contents of the same dialect.

[0073] (7) Data visualization: Record and save the accuracy and loss values corresponding to each epoch during the training process, and draw a line chart with the epoch as the horizontal axis and the accuracy and loss values as the vertical axis; save the accuracy during the test process of the test set, as well as the recognition results and true values corresponding to each image, and draw a confusion matrix.

[0074] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solutions of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0075] In addition, it should be noted that, in the case of no contradiction, the various specific technical features described in the above specific embodiments can be combined in any suitable way. To avoid unnecessary repetition, the present invention will not separately describe various possible combination methods.

[0076] Furthermore, any combination can be made between various different embodiments of the present invention, as long as it does not violate the idea of the present invention, and it should also be regarded as the content disclosed by the present invention.

Claims

1. An image recognition method based on convolutional neural network, characterized in that: The specific steps include: Step 1: Obtain image dataset; The pressure sensor is placed on the tester's vocal cords and surrounding muscles, and an oscilloscope is used to measure the voltage curve of the pressure sensor that changes over time when the tester speaks different dialect types or speech content. Then, a drawing software is used to obtain an image of the above voltage curve as an input image of the convolutional neural network; Step 2: Image dataset preprocessing; Step 3: Dataset division; Step 4: Training process; Input the images in the training set into the training function, call the convolutional neural network model for training, define the accuracy function, select the optimizer, loss function and learning rate, and continuously update the parameters using back propagation during the training process. The output of the model is the category label corresponding to the input image, which is used as the predicted value. At the same time, the original label of each input image is extracted as the true value. The accuracy corresponding to each epoch is calculated according to the accuracy function, and the corresponding loss value is calculated according to the loss function, and they are saved in the list respectively; Step 5: Verification process; Send the images in the validation set to the training function, call the convolutional neural network model, repeat the training process in step 4, get the accuracy of each epoch, compare the accuracy of each epoch with the accuracy of the previous epoch, and select the model parameters with the best effect to save; Step 6: Testing process; Call the model parameters with the best effect saved in the verification process of step 5, send the images in the test set to the training function, call the convolutional neural network model for training, and output the recognition results, so as to complete the recognition of dialects and language content through images based on the convolutional neural network; calculate the accuracy and loss values ​​according to the accuracy function and loss function; Step 7: Data visualization; Record and save the accuracy and loss value corresponding to each epoch in the training process, and draw a line graph with epoch as the horizontal axis, accuracy as the left vertical axis, and loss value as the right vertical axis; save the accuracy of the test set during the test and the recognition result and true value corresponding to each image, draw a confusion matrix, and according to the accuracy of the confusion matrix, it is believed that this model can realize the recognition of different dialect types and different speech contents.

2. The image recognition method based on convolutional neural network according to claim 1, characterized in that: In step 1, the accuracy function is the ratio of the number of correctly identified items to the total number in the same category, and the loss function is selected based on the task comparison, which may be different for different tasks, so the specific function formula cannot be given here.

3. The image recognition method based on convolutional neural network according to claim 1, characterized in that: In step 2, the input image of step 1 is rotated, flipped, translated, scaled, brightness adjusted, contrast adjusted, data enhancement processed, the data set is expanded, and saved.

4. The image recognition method based on convolutional neural network according to claim 1, characterized in that: In step 3, all images obtained after preprocessing in step 2 are divided into training set, validation set and test set in a ratio of 8:1:

1.

5. The image recognition method based on convolutional neural network according to claim 1, characterized in that: The convolutional neural network model includes 4 convolutional layers, 4 pooling layers, 4 BN layers, 3 linear layers and 2 Dropout layers. The activation function used is the ReLU function. The convolution kernel size of the two-dimensional convolutional layer is 3*3, and the convolution kernel moving step is 1; the size of the pooling layer for downsampling is 2*2, and the step size is 2; the Dropout rate in the Dropout layer is 0.

5.

6. The image recognition method based on convolutional neural network according to claim 5, characterized in that: The convolution layer is used to extract features from input data through convolution operations; due to its parameter sharing mechanism, the amount of model parameters can be reduced, reducing the computational cost; and maintaining the feature position information of the input data is conducive to subsequent recognition and classification; The pooling layer is used to reduce the spatial size of the image after feature extraction, reduce the amount of calculation, extract representative features in the area, and reduce redundant information; The BN layer is used to normalize the input data, which can effectively avoid the problem of gradient explosion or gradient disappearance and increase the stability of the model; The linear layer is used to expand the feature map obtained by the last convolutional layer into a one-dimensional vector to provide input for classification and recognition; The Dropout layer is used to remove some neurons according to the corresponding probability based on the set Dropout rate, which can prevent overfitting.

Images