College student facial expression recognition method based on improved DenseNet

Abstract

The invention relates to the technical field of facial expression recognition, in particular to a college student facial expression recognition method based on improved DenseNet, and the method comprises the steps: collecting a college student facial expression image; preprocessing the facial expression image to obtain a processed first image set; marking facial expression types of the images in the first image set to obtain a marked second image set; 4, obtaining a training set image, training the improved DenseNet network based on the training set image and the image in the second image set, and constructing a college student facial expression image recognition model; and identifying and analyzing the facial expressions of the college students captured in real time through the final facial expression image identification model of the college students to obtain an identification and analysis result of the facial expressions of the college students. According to the method, the depth features related to the facial expressions can be extracted, and the accuracy of facial expression analysis is improved, so that powerful support is provided for related applications.

Description

Technical Field

[0001] The present invention relates to the technical field of facial expression recognition, and in particular to a method for recognizing college students' facial expressions based on an improved DenseNet. Background Art

[0002] In daily life, people perceive other people's emotions through facial expressions, and then respond with corresponding communication content and methods to achieve normal communication. With the popularization of the Internet, computers can obtain facial expression images more and more quickly and efficiently, making it a feasible solution to obtain emotional information by analyzing facial expressions. At present, facial expression recognition has become an important tool for college students' smart education and is widely used in college students' psychological, emotional, and learning educational activities. However, there is currently no public facial expression database for college students. At the same time, the existing common deep learning models such as ResNet are not accurate enough and real-time enough for college students' facial expression recognition. Summary of the invention

[0003] In view of this, the purpose of the present invention is to propose a method for recognizing college students' facial expressions based on an improved DenseNet, so as to solve the problem that the existing common deep learning models such as ResNet are not accurate enough and real-time enough for recognizing college students' facial expressions.

[0004] Based on the above purpose, the present invention provides a method for recognizing college students' facial expressions based on an improved DenseNet, comprising: step 1: collecting college students' facial expression images;

[0005] Step 2: preprocessing the facial expression image to obtain a first processed image set;

[0006] Step 3: annotating the facial expression types of the images in the first image set to obtain a second annotated image set;

[0007] Step 4: Obtain training set images, train the improved DenseNet network based on the training set images and the images in the second image set, and construct a college student facial expression image recognition model, wherein the improved DenseNet network includes a dense block and a transition layer, and the dense block includes BN+ReLU+1x1 Conv+BN+ReLU+3x3Conv and Gate factors arranged in sequence, and the Gate factor is x i+1 =Gate(x i +x0), x i Represents the i-th layer in Dense Block;

[0008] Step 5: Obtain the test set images, and evaluate the constructed college student facial expression image recognition model based on the test set images. If the evaluation fails, return to step 5. If the evaluation passes, stop the training of the college student facial expression image recognition model to obtain the final college student facial expression image recognition model.

[0009] Step 6: Use the final college student facial expression image recognition model to identify and analyze the facial expressions of college students captured in real time to obtain the recognition and analysis results of the facial expressions of college students.

[0010] Optionally, after step 3, the method further includes: demonstrating the practicality of the second image set, wherein the demonstration of the practicality includes:

[0011] Data scale justification: Justify the number of facial expression image samples required;

[0012] Annotation quality verification: Use cross-validation to verify the quality of annotation results;

[0013] Data balance demonstration: Analyze the number and distribution of facial expression images of each category in the second image set. For categories with a small number, use data augmentation or synthesis methods to expand the number of images to ensure balanced image quality in the database.

[0014] Optionally, collecting the facial expression images of college students includes collecting the facial expression images of college students through classroom cameras and Internet downloading.

[0015] Optionally, the preprocessing of the facial expression image includes:

[0016] Face detection: Use convolutional neural network (CNN) to detect the face area in the collected image;

[0017] Image alignment: Use the facial key point detection algorithm to determine the positions of key parts such as eyes, nose, and mouth, and perform expression image alignment;

[0018] Lighting normalization: Use histogram equalization and gamma correction to adjust the brightness of the expression image;

[0019] Noise removal and smoothing: Use median filtering and Gaussian filtering to remove noise and detail texture in the image;

[0020] Contrast enhancement: Enhance the contrast of the image through Laplace sharpening and adaptive histogram equalization;

[0021] Color space conversion: Convert an image from RGB color space to YUV color space or HSV color space.

[0022] Optionally, the labeling of facial expression types for images in the first image set includes: obtaining images that are initially manually labeled by psychology experts for a portion of facial expression images, using the images as a data set for training an automatic labeling algorithm, obtaining an optimized automatic labeling algorithm, and using the optimized automatic labeling algorithm to label the remaining facial expression images.

[0023] Optionally, the facial expression types include 8 types: neutral, happy, surprised, sad, angry, disgusted, fearful, and contemptuous.

[0024] When the method is used to identify the facial expressions of college students, firstly, the facial expression images of college students are collected, and then the facial expression images are preprocessed to obtain a first image set after processing, and the images in the first image set are annotated with facial expression types to obtain a second annotated image set, and then the training set images are obtained, and based on the training set images and the images in the second image set, an improved DenseNet network is trained to construct a college student facial expression image recognition model, wherein the improved DenseNet network includes a dense block and a transition layer, and the dense block includes BN+ReLU+1x1Conv+BN+ReLU+3x3 Conv and Gate factors arranged in sequence, and the Gate factor is x i+1 =Gate(x i +x0), x i Represents the i-th layer in DenseBlock. The DenseNet network structure is set around two core components: Dense Block and Transition Layer. Figure 2, the layers in the existing Dense Block are arranged in the order of BN (batch normalization) + ReLU (rectified linear unit) + 1x1 Conv (convolution) + BN + ReLU + 3x3Conv. The present invention adds a Gate factor to the layers in the existing DenseBlock, and the Gate factor comprehensively considers the output feature maps of all the previous layers, and dynamically selects important features for output according to the characteristics of the task requirements and the input data, which helps the network to more effectively utilize the input information and improve the expression ability of the model. Through dense connection, according to the Gate factor, the output of each layer is obtained, and then connected to the following layer, the depth features related to the facial expression are extracted, and then the test set image is obtained, and the constructed college student facial expression image recognition model is evaluated based on the test set image, when the evaluation fails, return to step five, when the evaluation passes, stop the training of the college student facial expression image recognition model, obtain the final college student facial expression image recognition model, and recognize and analyze the real-time captured college student facial expression through the final college student facial expression image recognition model to obtain the recognition analysis result of the college student facial expression. The final college student facial expression image recognition model can be deployed in practical application scenarios, such as online education, mental health assessment, etc. By capturing college students' facial expressions in real time, the model can identify and analyze them, thus providing strong support for related applications.

[0025] From the above, it can be seen that this method trains the improved DenseNet network, can extract deep features related to facial expressions, improve the accuracy of facial expression analysis, and thus provide strong support for related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a technical roadmap of the method for recognizing college students' facial expressions based on improved DenseNet in the embodiment;

[0028] Figure 2 It is a schematic diagram of a 5-layer dense block of the DenseNet network involved in the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0030] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0031] like Figure 1 As shown, a method for college students' facial expression recognition based on improved DenseNet includes:

[0032] Step 1: Collect college students’ facial expression images;

[0033] Step 2: preprocessing the facial expression image to obtain a first processed image set;

[0034] Step 3: annotating the images in the first image set with facial expression types to obtain an annotated second image set, wherein the facial expression types include eight types: neutral, happy, surprised, sad, angry, disgusted, fearful, and contemptuous;

[0035] Step 4: Obtain training set images, train the improved DenseNet network based on the training set images and the images in the second image set, and construct a college student facial expression image recognition model, wherein the improved DenseNet network includes a dense block and a transition layer, and the dense block includes BN+ReLU+1x1 Conv+BN+ReLU+3x3Conv and Gate factors arranged in sequence, and the Gate factor is x i+1 =Gate(x i +x0), x i Represents the i-th layer in Dense Block;

[0036] Step 5: Obtain the test set images, and evaluate the constructed college student facial expression image recognition model based on the test set images. If the evaluation fails, return to step 5. If the evaluation passes, stop the training of the college student facial expression image recognition model to obtain the final college student facial expression image recognition model.

[0037] Step 6: Use the final college student facial expression image recognition model to identify and analyze the facial expressions of college students captured in real time to obtain the recognition and analysis results of the facial expressions of college students.

[0038] When the method is used to identify the facial expressions of college students, firstly, the facial expression images of college students are collected, and then the facial expression images are preprocessed to obtain a first image set after processing, and the images in the first image set are annotated with facial expression types to obtain a second annotated image set, and then the training set images are obtained, and based on the training set images and the images in the second image set, an improved DenseNet network is trained to construct a college student facial expression image recognition model, wherein the improved DenseNet network includes a dense block and a transition layer, and the dense block includes BN+ReLU+1x1Conv+BN+ReLU+3x3 Conv and Gate factors arranged in sequence, and the Gate factor is x i+1 =Gate(x i +x0), x i Represents the i-th layer in DenseBlock. The DenseNet network structure is set around two core components: Dense Block and Transition Layer. Figure 2 , the layers in the existing Dense Block are arranged in the order of BN (batch normalization) + ReLU (rectified linear unit) + 1x1 Conv (convolution) + BN + ReLU + 3x3Conv. The present invention adds a Gate factor to the layers in the existing DenseBlock, and the Gate factor comprehensively considers the output feature maps of all the previous layers, and dynamically selects important features for output according to the characteristics of the task requirements and the input data, which helps the network to more effectively utilize the input information and improve the expression ability of the model. Through dense connection, according to the Gate factor, the output of each layer is obtained, and then connected to the following layer, the depth features related to the facial expression are extracted, and then the test set image is obtained, and the constructed college student facial expression image recognition model is evaluated based on the test set image, when the evaluation fails, return to step five, when the evaluation passes, stop the training of the college student facial expression image recognition model, obtain the final college student facial expression image recognition model, and recognize and analyze the real-time captured college student facial expression through the final college student facial expression image recognition model to obtain the recognition analysis result of the college student facial expression. The final college student facial expression image recognition model can be deployed in practical application scenarios, such as online education, mental health assessment, etc. By capturing college students' facial expressions in real time, the model can identify and analyze them, thus providing strong support for related applications.

[0039] From the above, it can be seen that this method trains the improved DenseNet network, can extract deep features related to facial expressions, improve the accuracy of facial expression analysis, and thus provide strong support for related applications.

[0040] In some embodiments, the step 3 further includes: demonstrating the practicality of the second image set, wherein the demonstration of the practicality includes:

[0041] Data scale justification: According to the research purpose and needs, justify the number of facial expression image samples required; consider samples of college students of different genders, ages, and cultural backgrounds to ensure the diversity and generalization ability of the database; analyze the distribution of different facial expression categories to ensure the balance of data between categories. For rare or difficult to collect expression categories, you can explore the use of synthesis or enhancement technology to expand the data set.

[0042] Labeling quality demonstration: formulate facial expression labeling standards, including labeling granularity, label definition and classification basis, to ensure the consistency and accuracy of the labeling process; provide professional training for personnel involved in labeling, including the teaching of psychological knowledge and labeling skills. Improve the professional quality and labeling quality of labelers through practical exercises and case analysis; use cross-validation, expert review and other methods to evaluate the quality of labeling results. Re-label or correct images with inconsistent or incorrect labels to ensure the accuracy and reliability of the labeling results.

[0043] Data balance demonstration: Analyze the number and distribution of facial expression images of each category in the second image set to ensure data balance between categories. For categories with a small number, data enhancement or synthesis technology can be used to expand the data set; ensure that the image quality in the database is balanced to avoid deviations in subsequent analysis results due to differences in image quality. For images of poor quality, appropriate preprocessing or screening can be performed.

[0044] In some embodiments, collecting the facial expression images of college students includes collecting the facial expression images of college students through classroom cameras and Internet downloading.

[0045] In some embodiments, the preprocessing of the facial expression image includes:

[0046] Face detection: Use convolutional neural network (CNN) to detect the face area in the collected image.

[0047] Image alignment: The facial key point detection algorithm is used to determine the positions of key parts such as eyes, nose, and mouth, and expression images are aligned to eliminate the impact of posture changes.

[0048] Lighting normalization: The brightness of the expression image is adjusted using histogram equalization and gamma correction methods so that images under different lighting conditions have similar brightness distribution.

[0049] Noise removal and smoothing: Use median filtering and Gaussian filtering methods to remove noise and detail textures in the image to make the image smoother.

[0050] Contrast enhancement: Enhance the contrast of the image through Laplace sharpening and adaptive histogram equalization to make facial expressions more prominent.

[0051] Color space conversion: Convert the image from RGB color space to YUV color space or HSV color space, etc., which is more suitable for face recognition, reducing the impact of lighting and color changes on recognition performance, etc.

[0052] In some embodiments, the labeling of facial expression types for images in the first image set includes: obtaining images that are initially manually labeled by a psychology expert for a portion of facial expression images as a data set for training an automatic labeling algorithm, obtaining an optimized automatic labeling algorithm, using the optimized automatic labeling algorithm to label the remaining facial expression images, and having the psychology expert verify and correct the results of the automatic labeling; reusing the verified and corrected labeled data to train the automatic labeling algorithm, and performing iterative optimization until the expected effect is achieved.

[0053] It should be understood by those skilled in the art that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Within the scope of the present invention, the technical features in the above embodiments or in different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the above aspects of the present invention, which are not provided in detail for the sake of simplicity.

[0054] The present invention is intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for college students' facial expression recognition based on improved DenseNet, characterized in that: include: Step 1: Collect facial expression images of college students; Step 2: preprocessing the facial expression image to obtain a first processed image set; Step 3: annotating the facial expression types of the images in the first image set to obtain a second annotated image set; Step 4: Obtain training set images, train the improved DenseNet network based on the training set images and the images in the second image set, and construct a college student facial expression image recognition model, wherein the improved DenseNet network includes a dense block and a transition layer, and the dense block includes BN+ReLU+1x1 Conv+BN+ReLU+3x3Conv and Gate factors arranged in sequence, and the Gate factor is x i+1 =Gate(x i +x0), x i Represents the i-th layer in Dense Block; Step 5: Obtain the test set images, and evaluate the constructed college student facial expression image recognition model based on the test set images. If the evaluation fails, return to step 5. If the evaluation passes, stop the training of the college student facial expression image recognition model to obtain the final college student facial expression image recognition model. Step 6: Use the final college student facial expression image recognition model to identify and analyze the facial expressions of college students captured in real time to obtain the recognition and analysis results of the facial expressions of college students.

2. According to claim 1, a method for recognizing college students' facial expressions based on improved DenseNet is characterized in that: After step 3, the method further includes: demonstrating the practicality of the second image set, wherein the practicality demonstration includes: Data scale justification: Justify the number of facial expression image samples required; Annotation quality verification: Use cross-validation to verify the quality of annotation results; Data balance demonstration: Analyze the number and distribution of facial expression images of each category in the second image set. For categories with a small number, use data augmentation or synthesis methods to expand the number of images to ensure balanced image quality in the database.

3. A method for recognizing college students' facial expressions based on improved DenseNet according to claim 1, characterized in that: The method of collecting the facial expression images of college students includes collecting the facial expression images of college students in school by means of classroom cameras and Internet downloading.

4. A method for recognizing college students' facial expressions based on improved DenseNet according to claim 1, characterized in that: The preprocessing of the facial expression image comprises: Face detection: Use convolutional neural network (CNN) to detect the face area in the collected image; Image alignment: Use the facial key point detection algorithm to determine the positions of key parts such as eyes, nose, and mouth, and perform expression image alignment; Lighting normalization: Use histogram equalization and gamma correction to adjust the brightness of the expression image; Noise removal and smoothing: Use median filtering and Gaussian filtering to remove noise and detail texture in the image; Contrast enhancement: Enhance the contrast of the image through Laplace sharpening and adaptive histogram equalization; Color space conversion: Convert an image from RGB color space to YUV color space or HSV color space.

5. A method for recognizing college students' facial expressions based on improved DenseNet according to claim 1, characterized in that: The facial expression type labeling of the images in the first image set includes: obtaining images that are initially manually labeled by psychology experts for a portion of facial expression images as a data set for training an automatic labeling algorithm, obtaining an optimized automatic labeling algorithm, and using the optimized automatic labeling algorithm to label the remaining facial expression images.

6. A method for recognizing college students' facial expressions based on improved DenseNet according to claim 1, characterized in that: The facial expression types include 8 types: neutral, happy, surprised, sad, angry, disgusted, fearful, and contemptuous.

Images