Facial expression recognition method and device, computer device and readable storage medium

Abstract

This application provides a facial expression recognition method and apparatus, a computer device, and a readable storage medium, relating to the field of computer vision technology. This application leverages the significant performance of Gabor filters in facial expression recognition by applying Gabor filters to the original convolutional operations of a traditional ResNet network model. This reduces the overall memory consumption of the model and enhances its ability to capture significant visual features. Furthermore, by introducing channel transformation and channel enhancement mechanisms into the traditional ResNet network model, information exchange and feature complementarity between adjacent channels are achieved, further reducing computational cost and complexity while improving the ability to capture subtle facial features. This enables the corresponding Gabor residual network visual model to achieve highly computationally efficient and effective facial expression recognition.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a facial expression recognition method and apparatus, a computer device, and a readable storage medium. Background Technology

[0002] With the continuous development of science and technology, computer vision technology is being used more and more widely in fields such as video surveillance, healthcare, and social security. It is commonly used to achieve computer vision recognition functions such as face recognition, pedestrian recognition, vehicle recognition, and object recognition. Among these, facial expressions are one of the most powerful, natural, and universal ways for humans to express their emotional state and intentions. Therefore, facial expression recognition (FER) is an important research direction in computer vision technology today.

[0003] Currently, mainstream industry methods typically utilize facial images directly to model the correlation between images and emotions using either handcrafted features or deep network methods. This ensures the final model can achieve facial expression recognition. To improve computational efficiency, efficient handcrafted features are often extracted from facial images using techniques such as Local Binary Pattern (LBP), Local Neighborhood Encoding Pattern (LNEP), and Histogram of Oriented Gradient (HOG). Alternatively, lightweight models like MobileNet and ShuffleNet are used for modeling. However, it's worth noting that due to the limited capabilities of handcrafted features and lightweight networks in feature learning, models obtained using these methods often exhibit poor performance in terms of accuracy and robustness in facial expression recognition. Therefore, balancing computational efficiency and recognition performance in facial expression recognition remains a crucial and pressing issue in computer vision technology. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a facial expression recognition method and apparatus, computer device and readable storage medium, which can utilize the significant performance of Gabor filters in the facial expression recognition process by applying Gabor filters to the original convolution operation of traditional ResNet (Deep Residual Networks) network models, reduce the overall memory consumption of the model and enhance the model's ability to capture significant visual features. At the same time, by introducing channel transformation and channel enhancement mechanisms into the traditional ResNet network model, information exchange and feature complementarity between adjacent channels are achieved, thereby further reducing the model's computational consumption and complexity, and improving the ability to capture subtle facial features. Thus, a highly computationally efficient and well-performing facial expression recognition effect can be achieved using the corresponding Gabor residual network visual model.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, this application provides a facial expression recognition method applied to a computer device, wherein the computer device is loaded with a target Gabor residual network visual model, the target Gabor residual network visual model including a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer, and the method includes:

[0007] A face image to be identified is acquired, and the Gabor filter convolutional layer is called to extract facial features from the face image to be identified, thereby obtaining a corresponding facial feature map set, wherein the facial feature map set includes the initial facial feature map of the face image to be identified in different Gabor directional channels;

[0008] The channel conversion enhancement layer is invoked to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, wherein the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor direction channels;

[0009] The expression prediction layer is invoked to perform facial emotion classification prediction on the facial key feature map to obtain the facial emotion recognition result of the face image to be identified.

[0010] In an optional implementation, the Gabor filter convolutional layer is implemented using a Gabor convolutional kernel with multiple Gabor directional channels, wherein the Gabor convolutional kernel is obtained by merging a self-learning convolutional kernel with a Gabor filter having multiple Gabor filtering directions. The step of calling the Gabor filter convolutional layer to extract facial features from the face image to be identified and obtaining the corresponding facial feature map includes:

[0011] Based on the total number of Gabor directional channels in the Gabor filter convolutional layer, an image copying operation is performed on the face image to be identified to obtain multiple target face images with the same image content, wherein the total number of the multiple target face images is consistent with the total number of Gabor directional channels;

[0012] The Gabor convolution kernels included in the Gabor filter convolution layer are invoked to perform image convolution operations with the multiple target face images in the Gabor filter directions corresponding to different Gabor direction channels, so as to obtain the facial feature map set.

[0013] In an optional implementation, the channel transformation enhancement layer includes a first Gabor residual module, a channel transformation module, a channel enhancement module, and a second Gabor residual module connected in series. Both the first Gabor residual module and the second Gabor residual module are obtained by connecting the residuals of two Gabor convolutional kernels. The step of calling the channel transformation enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map to obtain the corresponding facial key feature map includes:

[0014] The facial feature map set is input into the first Gabor residual module for depth residual operation to obtain the output feature map set output by the first Gabor residual module;

[0015] The channel conversion module is invoked to perform adjacent channel feature information exchange processing on the output feature map set, resulting in the exchanged feature map set output by the channel conversion module;

[0016] The channel enhancement module is invoked to perform complementary aggregation processing on adjacent channel features of the exchanged feature set, resulting in the aggregated feature set output by the channel enhancement module;

[0017] The aggregated feature map is input into the second Gabor residual module for depth residual operation to obtain the facial key feature map output by the second Gabor residual module.

[0018] In an optional implementation, the channel conversion module includes a channel left shift block, a channel right shift block, and a non-channel shift block. The step of calling the channel conversion module to perform adjacent channel feature information exchange processing on the output feature map set to obtain the exchanged feature map set output by the channel conversion module includes:

[0019] The output feature map set is partitioned in the spatial dimension to obtain initial feature map subsets that match the channel left shift block, the channel right shift block, and the non-channel shift block, respectively.

[0020] The channel left shift block, the channel right shift block, and the non-channel shift block are invoked to perform channel feature shifting on the matched initial feature map subset through depthwise convolution operation based on their own convolution kernel weights, so as to obtain the intermediate feature map subsets output by the channel left shift block, the channel right shift block, and the non-channel shift block respectively;

[0021] The exchanged feature map set is obtained by concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level.

[0022] In an optional implementation, the channel enhancement module includes feature enhancement blocks and non-feature enhancement blocks. The step of calling the channel enhancement module to perform adjacent channel feature complement aggregation processing on the exchanged feature map set to obtain the aggregated feature map set output by the channel enhancement module includes:

[0023] The exchanged feature map set is partitioned in the spatial dimension to obtain initial feature map subsets that match the feature enhancement block and the non-feature enhancement block, respectively.

[0024] The feature enhancement block and the non-feature enhancement block are invoked to perform channel feature enhancement on the matched initial feature map subsets through depthwise convolution operations based on their own convolution kernel weights, so as to obtain the intermediate feature map subsets output by the feature enhancement block and the non-feature enhancement block respectively;

[0025] The aggregated feature map set is obtained by concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level.

[0026] In an optional implementation, the target Gabor residual network visual model further includes a first Gabor residual intermediate layer, which is positioned between the Gabor filter convolutional layer and the channel conversion enhancement layer. The facial feature map output by the Gabor filter convolutional layer is processed by the first Gabor residual intermediate layer and then input into the channel conversion enhancement layer for adjacent channel feature exchange and aggregation processing.

[0027] The first Gabor residual intermediate layer includes at least one Gabor residual intermediate module connected in series. Each Gabor residual intermediate module is obtained by connecting two Gabor residual modules in series, and each Gabor residual module is obtained by connecting two Gabor convolution kernel residuals.

[0028] In an optional implementation, the target Gabor residual network visual model further includes a second Gabor residual intermediate layer, which is positioned between the channel transformation enhancement layer and the expression prediction layer. The facial key feature map output by the channel transformation enhancement layer is processed by the second Gabor residual intermediate layer and then input into the expression prediction layer for facial emotion classification prediction.

[0029] The second Gabor residual intermediate layer is obtained by connecting two Gabor residual modules in series, and each Gabor residual module is obtained by connecting two Gabor convolution kernel residuals.

[0030] Secondly, this application provides a facial expression recognition device applied to a computer device, wherein the computer device is loaded with a target Gabor residual network visual model, the target Gabor residual network visual model including a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer, and the device includes:

[0031] The facial feature extraction module is used to acquire the face image to be identified and call the Gabor filter convolutional layer to extract facial features from the face image to be identified, thereby obtaining a corresponding facial feature map set, wherein the facial feature map set includes the initial facial feature map of the face image to be identified in different Gabor directional channels.

[0032] The feature exchange and aggregation module is used to call the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, wherein the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor directional channels.

[0033] The facial emotion prediction module is used to call the expression prediction layer to perform facial emotion classification prediction on the facial key feature map, and obtain the facial emotion recognition result of the face image to be identified.

[0034] Thirdly, this application provides a computer device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to drive a target Gabor residual network visual model stored in the memory to implement the facial expression recognition method described in any of the foregoing embodiments, wherein the target Gabor residual network visual model includes a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer.

[0035] Fourthly, this application provides a readable storage medium storing a computer program thereon. When the computer program is executed by a computer device, it drives a target Gabor residual network visual model to implement the facial expression recognition method described in any of the foregoing embodiments. The target Gabor residual network visual model includes a Gabor filter convolutional layer, a channel conversion enhancement layer, and an expression prediction layer.

[0036] In this case, the beneficial effects of the embodiments of this application may include the following:

[0037] After acquiring the face image to be recognized, this application calls the Gabor filtering convolutional layer of the target Gabor residual network visual model to extract facial features from the face image, obtaining the corresponding facial feature map set. Then, it calls the channel transformation enhancement layer of the target Gabor residual network visual model to perform adjacent channel feature exchange and aggregation processing on the facial feature map set, obtaining the corresponding key facial feature map set. Next, it uses the expression prediction layer of the target Gabor residual network visual model to perform facial emotion classification prediction on the key facial feature map set, obtaining the facial emotion recognition result of the face image to be recognized. This allows the application of Gabor filters to the original convolutional operations of the traditional ResNet network model, leveraging the Gabor filters in facial expression... Significant performance improvements in facial expression recognition, reduced overall model memory consumption, and enhanced model capture of salient visual features. Furthermore, by introducing channel transformation and channel enhancement mechanisms into the traditional ResNet network model, information exchange and feature complementarity between adjacent channels are promoted, further reducing model computational cost and complexity, and improving the capture of subtle facial features. This allows the corresponding Gabor residual network visual model to maintain high computational efficiency in facial expression recognition, and to capture key facial features from the face image with less computational resources and memory cost, ensuring that the final facial expression recognition result exhibits excellent performance in both accuracy and robustness.

[0038] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A schematic diagram of the composition of a computer device provided in the embodiments of this application;

[0041] Figure 2 One of the schematic diagrams of the model architecture of the target Gabor residual network visual model provided in the embodiments of this application;

[0042] Figure 3 A schematic diagram illustrating the composition of a single Gabor residual module provided in an embodiment of this application;

[0043] Figure 4 The second schematic diagram of the model architecture of the target Gabor residual network visual model provided in the embodiments of this application;

[0044] Figure 5 The third schematic diagram of the model architecture of the target Gabor residual network visual model provided in the embodiments of this application;

[0045] Figure 6 A flowchart illustrating the facial expression recognition method provided in this application embodiment;

[0046] Figure 7 for Figure 6 A flowchart illustrating the sub-steps included in step 220;

[0047] Figure 8 This is a schematic diagram illustrating the composition of the facial expression recognition device provided in the embodiments of this application.

[0048] Icons: 10-Computer equipment; 11-Memory; 12-Processor; 13-Communication unit; 100-Facial expression recognition device; 110-Facial feature extraction module; 120-Feature exchange and aggregation module; 130-Facial emotion prediction module. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0051] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0052] In the description of this application, it should be understood that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0053] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0054] Please refer to Figure 1 , Figure 1This is a schematic diagram of the composition of the computer device 10 provided in this application embodiment. In this application embodiment, the computer device 10 may be equipped with a target Gabor residual network visual model, and the target Gabor residual network visual model is used to achieve a facial expression recognition effect with low computational resource consumption, high computational efficiency and good recognition performance in the face image to be recognized; wherein, the computer device 10 may be, but is not limited to, a smartphone, robot, laptop, personal computer, server, etc. Specifically, Gabor filters can be applied to the original convolutional operations of a traditional ResNet network model to leverage the significant performance of Gabor filters in facial expression recognition, reduce overall model memory consumption, and enhance the model's ability to capture significant visual features. Simultaneously, channel transformation and channel enhancement mechanisms are introduced into the traditional ResNet network model. The channel transformation mechanism promotes information exchange between adjacent channels, while the channel enhancement mechanism captures and aggregates complementary features between adjacent channels, achieving feature complementarity and enhancing latent features in facial images. Through the synergy of these two mechanisms, model computational cost and complexity are further reduced, while the ability to capture subtle facial features is improved. This allows the resulting target Gabor residual network visual model to maintain high computational efficiency during facial expression recognition and capture key facial feature information in facial images, achieving excellent model performance with minimal computational resources and memory costs.

[0055] In this embodiment, the computer device 10 may include a memory 11, a processor 12, and a communication unit 13. The memory 11, the processor 12, and the communication unit 13 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0056] In this embodiment, the memory 11 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 11 is used to store computer programs, and the processor 12 can execute the computer programs accordingly after receiving execution instructions; the memory 11 can also be used to store the aforementioned target Gabor residual network visual model.

[0057] In this embodiment, the processor 12 can be an integrated circuit chip with signal processing capabilities. The processor 12 can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0058] In this embodiment, the communication unit 13 is used to establish a communication connection between the computer device 10 and other electronic devices via a network, and to send and receive data through the network, wherein the network includes wired communication networks and wireless communication networks. For example, the computer device 10 can obtain images of faces to be identified uploaded by other electronic devices through the communication unit 13, wherein the electronic devices may be, but are not limited to, surveillance equipment, cameras, etc.

[0059] In this embodiment, the computer device 10 may further include a facial expression recognition device 100. The facial expression recognition device 100 may include at least one software functional module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system of the computer device 10. The processor 12 can be used to execute executable modules stored in the memory 11, such as the software functional modules and computer programs included in the facial expression recognition device 100. The computer device 10 can drive the aforementioned target Gabor residual network visual model to perform facial expression recognition on the face image to be recognized through the facial expression recognition device 100, and maintain the model computation efficiency of the target Gabor residual network visual model at a high level during the facial expression recognition process. Furthermore, it can capture key facial feature information (including significant facial visual features and subtle facial features) from the face image to be recognized using relatively few computational resources and memory costs, thereby ensuring that the final output facial expression recognition result exhibits excellent performance in terms of accuracy and robustness.

[0060] Understandable, Figure 1 The block diagram shown is only a schematic diagram of one configuration of the computer device 10. The computer device 10 may also include components such as... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0061] Regarding the aforementioned target Gabor residual network visual model, in the first embodiment of this application, the target Gabor residual network visual model can adopt... Figure 2 The model architecture shown is implemented in such a case where the target Gabor residual network visual model may include a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer. The feature input of the Gabor filter convolutional layer is used to receive the face image to be recognized. The feature output of the Gabor filter convolutional layer can be directly connected to the feature input of the channel transformation enhancement layer. The feature output of the channel transformation enhancement layer can be directly connected to the feature input of the expression prediction layer. The result output of the expression prediction layer outputs the facial expression recognition result (i.e., facial emotion recognition result) corresponding to the face image to be recognized.

[0062] In this embodiment, the Gabor filter convolutional layer is implemented using a Gabor convolutional kernel with multiple Gabor direction channels. The Gabor convolutional kernel is obtained by merging the self-learning convolutional kernel of the traditional ResNet network model with a Gabor filter with multiple Gabor filtering directions. Each Gabor direction channel corresponds to a Gabor filtering direction. By replacing the original convolutional kernel in the traditional ResNet network model with a Gabor convolutional kernel, the significant performance of the Gabor filter in the facial expression recognition process is utilized. Compared with the original convolutional kernel, the robustness of the model to scale changes and filtering direction changes in the process of capturing facial features is enhanced. At the same time, since the Gabor filter has the function of predefined scale and filtering direction, the number of model parameters can be significantly reduced compared with the traditional ResNet network model, without the need to build a large number of residual modules as in the traditional ResNet network model.

[0063] In this embodiment, the channel conversion enhancement layer is used to perform adjacent channel feature information exchange processing and adjacent channel feature complementary aggregation processing on the received input features to achieve the adjacent channel feature exchange and aggregation effect of the received input features. The channel conversion enhancement layer may include a first Gabor residual module, a channel conversion module, a channel enhancement module, and a second Gabor residual module connected in series. The feature input end of the first Gabor residual module serves as the feature input end of the channel conversion enhancement layer, and the feature output end of the second Gabor residual module serves as the feature output end of the channel conversion enhancement layer. The channel conversion module is designed based on a channel conversion mechanism, and the channel enhancement module is designed based on a channel enhancement mechanism. The channel conversion module can utilize depthwise convolution operations and the movement of a portion of the channels along the spatial dimension to promote information exchange between adjacent channels of the input features; the channel enhancement module can utilize depthwise convolution operations to capture complementary features between adjacent channels of the input features for feature aggregation to enhance the latent features in the input features; for either the first Gabor residual module or the second Gabor residual module, a specific approach can be adopted. Figure 3 The two Gabor convolution kernels shown are obtained by residual concatenation, so that... Figure 2 The target Gabor residual network visual model shown can effectively increase the receptive field of the model by eliminating the need to build a max pooling layer compared to the traditional ResNet network model.

[0064] In this embodiment, the expression prediction layer may include interconnected fully connected layers and a classification network layer. The feature input of the fully connected layer serves as the feature input of the expression prediction layer, and the output of the classification network layer serves as the output of the expression prediction layer. The fully connected layer may be implemented using a linear function, and the classification network layer may be implemented using a softmax function. The fully connected layer linearly converts the input features into relevant features that the classification network layer can recognize. The classification network layer classifies the received facial features according to preset facial emotion types (e.g., anger, excitement, liking, surprise, pain, fear, sadness, disgust, etc.), obtaining a facial emotion recognition result (i.e., a facial expression recognition result) that includes the probability of expression under different facial emotion types for the corresponding face image to be recognized.

[0065] In a second embodiment of this application, the target Gabor residual network visual model can be adopted. Figure 4 The model architecture shown is implemented, at which point the target Gabor residual network visual model and Figure 2 Compared to the model architecture shown, it may also include a first Gabor residual intermediate layer. The first Gabor residual intermediate layer is positioned between the Gabor filter convolutional layer and the channel conversion enhancement layer, so that the Gabor filter convolutional layer is connected to the channel conversion enhancement layer through the first Gabor residual intermediate layer. The first Gabor residual intermediate layer is used to perform multiple depth residual operations on the output features of the Gabor filter convolutional layer, so that the output features processed by the first Gabor residual intermediate layer can reveal as many facial feature details as possible.

[0066] In this embodiment, the first Gabor residual intermediate layer may include at least one Gabor residual intermediate module connected in series, wherein each Gabor residual intermediate module is obtained by connecting two Gabor residual modules in series, and each Gabor residual module is obtained by connecting two Gabor convolution kernel residuals. In one implementation of this embodiment, the first Gabor residual intermediate layer may be obtained by connecting two Gabor residual intermediate modules in series, so as to effectively maintain the balance between model feature extraction accuracy and model computation efficiency, and at the same time, it can also make Figure 4 The target Gabor residual network visual model shown can effectively increase the receptive field of the model by eliminating the need to build a max pooling layer compared to the traditional ResNet network model.

[0067] In a third embodiment of this application, the target Gabor residual network visual model can be adopted. Figure 5 The model architecture shown is implemented, at which point the target Gabor residual network visual model and Figure 4 Compared to the model architecture shown, it may also include a second Gabor residual intermediate layer. The second Gabor residual intermediate layer is positioned between the channel transformation enhancement layer and the expression prediction layer, so that the channel transformation enhancement layer is connected to the expression prediction layer through the second Gabor residual intermediate layer. The second Gabor residual intermediate layer is used to perform multiple deep residual operations on the output features of the channel transformation enhancement layer, so that the output features processed by the second Gabor residual intermediate layer can reveal as many facial feature details as possible.

[0068] In this embodiment, the second Gabor residual intermediate layer is obtained by directly connecting two Gabor residual modules in series, wherein each Gabor residual module can be directly obtained by connecting two Gabor convolution kernel residuals.

[0069] Understandable, Figure 2 , 4 The model architecture diagram shown in Figure 5 is only a schematic diagram of the architecture composition of the target Gabor residual network visual model. The target Gabor residual network visual model may also include... Figure 2 , 4 Or more or fewer module components as shown in 5, or having the same Figure 2 , 4 Or, as shown in 5, different configurations.

[0070] In this application, to ensure that the computer device 10 can drive the target Gabor residual network visual model to perform facial expression recognition on the face image to be recognized, and to maintain the model computation efficiency of the target Gabor residual network visual model at a high level during the facial expression recognition process, and to capture key facial feature information from the face image to be recognized with less computational resources and memory cost, thereby ensuring that the final output facial expression recognition result exhibits excellent performance in terms of accuracy and robustness, this application provides a facial expression recognition method to achieve the aforementioned objectives. The facial expression recognition method provided in this application will be described in detail below.

[0071] Please refer to Figure 6 , Figure 6 This is a schematic flowchart of the facial expression recognition method provided in this application embodiment. In this application embodiment, the facial expression recognition method is applied to a computer device 10 equipped with a target Gabor residual network visual model, and the facial expression recognition method may include steps S210 to S230.

[0072] Step S210: Obtain the face image to be recognized, and call the Gabor filter convolutional layer to extract facial features from the face image to obtain the corresponding facial feature map.

[0073] In this embodiment, since the Gabor filter convolutional layer is implemented using a Gabor convolutional kernel with multiple Gabor directional channels, the Gabor filter convolutional layer actually involves multiple Gabor directional channels. The resulting facial feature map set includes initial facial feature maps of the face image to be recognized in different Gabor directional channels. To maintain consistency in the number of channels during forward convolution, the number of channels in the self-learning convolutional kernels involved in the Gabor convolutional kernels can be set to the total number of Gabor directional channels supported by the Gabor filter. Since the output of the Gabor convolutional kernel has one more filtering direction dimension than the traditional self-learning convolutional kernel, the output channel size of each Gabor convolutional kernel can be set to "a multiple of the total number of Gabor directional channels of the traditional self-learning convolutional kernel" (i.e., C). GC-out =C out / N, where C GC-ou t is used to represent the output channel size of the Gabor convolution kernel, C out The output channel size of a traditional self-learning convolution kernel is represented by N, where N represents the total number of Gabor direction channels, effectively reducing the number of model parameters. In one embodiment of this example, the total number of Gabor direction channels involved in a single Gabor convolution kernel can be 4, and the corresponding Gabor filtering direction can be represented as... The scale involved in a single Gabor convolution kernel can be 3.

[0074] Optionally, in one embodiment of this example, the step "calling the Gabor filter convolutional layer to extract facial features from the face image to be recognized, and obtaining the corresponding facial feature map set" in step S210 above may include:

[0075] Based on the total number of Gabor directional channels in the Gabor filter convolutional layer, an image copying operation is performed on the face image to be identified to obtain multiple target face images with the same image content, wherein the total number of the multiple target face images is consistent with the total number of Gabor directional channels;

[0076] The Gabor convolution kernels included in the Gabor filter convolution layer are invoked to perform image convolution operations with the multiple target face images in the Gabor filter directions corresponding to different Gabor direction channels, so as to obtain the facial feature map set.

[0077] Specifically, for the i-th Gabor convolution kernel included in the Gabor filter convolution layer, the image convolution operation of the i-th Gabor convolution kernel can be represented by the following formula:

[0078] in,

[0079] in, F is used to represent the output feature map of the i-th Gabor convolution kernel in the j-th Gabor filtering direction. i (n) Used to represent the nth input feature map corresponding to the i-th Gabor convolution kernel. Used to represent the i-th Gabor convolution kernel in the u-th Gabor filtering direction U u The above involves the actual convolution kernel content at scale v, C i G(U) is used to represent the traditional self-learning convolution kernel corresponding to the i-th Gabor convolution kernel. u (v) represents the filter parameters of the Gabor filter corresponding to the i-th Gabor convolution kernel, involving scale v in the u-th Gabor filtering direction. The symbol '·' represents the convolution operation, '·' represents the element-wise multiplication operation, and 'N' represents the total number of Gabor direction channels.

[0080] Therefore, based on the image convolution operation expression of the Gabor convolution kernel described above, this application can extract a set of facial feature maps representing significant facial visual features from the face image to be identified by calling the Gabor convolution kernel included in the Gabor filter convolution layer.

[0081] Step S220: Call the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set.

[0082] In this embodiment, the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor directional channels; if the target Gabor residual network visual model adopts Figure 2The model architecture shown is implemented such that the Gabor filtering convolutional layer in the target Gabor residual network visual model directly transmits its output facial feature map set to the channel transformation enhancement layer. This allows the channel transformation enhancement layer to use the facial feature map set as its own input feature for adjacent channel feature exchange and aggregation, thereby enhancing its ability to capture subtle facial features in the face image to be recognized. This ensures that the facial key feature map set output by the channel transformation enhancement layer can effectively represent the key facial feature information in the face image to be recognized, thus improving the accuracy of facial expression recognition. If the target Gabor residual network visual model adopts... Figure 4 or Figure 5 If the model architecture shown is implemented, the Gabor filter convolutional layer in the target Gabor residual network visual model will transmit its output facial feature map set to the channel conversion enhancement layer after processing by the first Gabor residual intermediate layer for adjacent channel feature exchange and aggregation processing.

[0083] Alternatively, please refer to Figure 7 , Figure 7 yes Figure 6 The flowchart of step 220 includes the sub-steps. In the embodiments of this application, step 220 may include sub-steps S221 to S224, which, through the cooperation of channel conversion mechanism and channel enhancement mechanism, reduce model computational consumption and model complexity, improve the performance of capturing subtle facial features, and enable the target Gabor residual network visual model to capture key facial feature information from the face image to be recognized during the facial expression recognition process.

[0084] Sub-step S221: Input the facial feature map set into the first Gabor residual module to perform depth residual operation, and obtain the output feature map set output by the first Gabor residual module.

[0085] The output feature set can reveal more facial feature details compared to the facial feature set.

[0086] Sub-step S222: Call the channel conversion module to perform adjacent channel feature information exchange processing on the output feature map set, and obtain the exchanged feature map set output by the channel conversion module.

[0087] In this embodiment, the channel conversion module may include a channel left shift block, a channel right shift block, and a non-channel movement block. The channel left shift block is used to move adjacent channel features to the left, the channel right shift block is used to move adjacent channel features to the right, and the non-channel movement block is used to maintain the initial channel unchanged. At this time, sub-step S222 may include sub-steps A to C:

[0088] Sub-step A involves partitioning the output feature map set of the first Gabor residual module in the spatial dimension to obtain initial feature map subsets that match the channel left shift block, the channel right shift block, and the non-channel shift block, respectively.

[0089] The spatial dimension distribution range of the initial feature map subset matching the right-shifting channel block can be represented by [i*h*w / σ:(i+1)*h*w / σ], the spatial dimension distribution range of the initial feature map subset matching the left-shifting channel block can be represented by [(i+1)*h*w / σ:(i+2)*h*w / σ], and the spatial dimension distribution range of the initial feature map subset matching the non-channel shifting block can be represented by [:i*h*w / σ∪(i+2)*h*w / σ:], where i represents the starting division position of the output feature map set in the spatial dimension when it is used as the input feature of the channel conversion module, h represents the feature map height, w represents the feature map width, and σ represents the reduction factor of the feature map width of the divided input feature.

[0090] Sub-step B involves calling the channel left shift block, the channel right shift block, and the non-channel shift block to perform channel feature shifting on the matched initial feature map subsets through depthwise convolution operations based on their own convolution kernel weights, thereby obtaining the intermediate feature map subsets output by the channel left shift block, the channel right shift block, and the non-channel shift block, respectively.

[0091] In this configuration, the convolutional kernel weights of the channel left-shifting block, the channel right-shifting block, and the non-channel moving block are fixed. The convolutional kernel weights of the channel left-shifting block are represented by [0,0,1], the channel right-shifting block by [1,0,0], and the non-channel moving block by [0,1,0]. Therefore, each of the channel left-shifting block, the channel right-shifting block, and the non-channel moving block can perform a depthwise convolution operation on the matched initial feature map subset based on its own convolutional kernel weights. This shifts adjacent channel features within the corresponding initial feature map subset to the left, right, or keeps them unchanged, resulting in the intermediate feature map subsets output by each of the channel left-shifting block, the channel right-shifting block, and the non-channel moving block.

[0092] Sub-step C involves concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level to obtain the exchanged feature map set.

[0093] Specifically, based on the relative positional relationship of the initial feature map subsets corresponding to the multiple intermediate feature map subsets in the spatial dimension, the feature map splicing operation of the multiple intermediate feature map subsets can be completed in the spatial dimension to obtain the exchanged feature map set.

[0094] Therefore, by executing the above sub-steps A to C, this application ensures that the channel conversion module can facilitate information exchange between adjacent channels with minimal computing resources, so that the corresponding exchange feature set can obtain information representation from the adjacent channels of the input feature, while retaining some of the original spatial information of the input feature.

[0095] Sub-step S223: Call the channel enhancement module to perform complementary aggregation processing on adjacent channel features of the exchanged feature set, and obtain the aggregated feature set output by the channel enhancement module.

[0096] In this embodiment, the channel enhancement module may include feature enhancement blocks and non-feature enhancement blocks. The feature enhancement blocks are used to aggregate adjacent channel features in the input features for complementarity to enhance salient representation, thereby enhancing the latent features in the input features. The non-feature enhancement blocks are used to retain some original features in the input features unchanged. At this time, sub-step S223 may include sub-steps D to F:

[0097] Sub-step D involves partitioning the exchange feature map set output by the channel conversion module into feature maps at the spatial dimension level to obtain initial feature map subsets that match the feature enhancement block and the non-feature enhancement block, respectively.

[0098] The spatial dimension distribution range of the initial feature map subset matched with the feature enhancement block can be represented by [i*h*w / σ:(i+2)*h*w / σ], and the spatial dimension distribution range of the initial feature map subset matched with the non-feature enhancement block can be represented by [:i*h*w / σ∪(i+2)*h*w / σ:], where i represents the starting division position of the exchanged feature map set in the spatial dimension when it is used as the input feature of the channel enhancement module, h represents the feature map height, w represents the feature map width, and σ represents the feature map width reduction factor of the divided input feature.

[0099] Sub-step E involves calling the feature enhancement block and the non-feature enhancement block to perform channel feature enhancement on the matched initial feature map subsets through depthwise convolution operations based on their own convolution kernel weights, thereby obtaining the intermediate feature map subsets output by the feature enhancement block and the non-feature enhancement block respectively.

[0100] In this embodiment, the convolutional kernel weights of the non-feature enhancement blocks are fixed and can be represented by [0,1,0]. The convolutional kernel weights of the feature enhancement blocks can be dynamically updated during model training to achieve selective feature enhancement by extracting local features. For the feature enhancement blocks, the convolutional kernel size controls the size of the model's receptive field. Using different initial convolutional kernel weights during model training will cause varying degrees of fluctuation in the performance of the corresponding Gabor residual network visual model. In one implementation of this embodiment, the initial convolutional kernel weights of the feature enhancement blocks during model training can be represented by [1,1,1].

[0101] Therefore, the feature enhancement block and the non-feature enhancement block can each perform a depthwise convolution operation on the matched initial feature map subset based on their own convolution kernel weights to aggregate or maintain the adjacent channel features within the corresponding initial feature map subset, so as to obtain the intermediate feature map subsets output by the feature enhancement block and the non-feature enhancement block respectively.

[0102] Sub-step F involves concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level to obtain the aggregated feature map set.

[0103] Specifically, based on the relative positional relationship of the initial feature map subsets corresponding to the multiple intermediate feature map subsets in the spatial dimension, the feature map splicing operation of the multiple intermediate feature map subsets can be completed in the spatial dimension to obtain the aggregated feature map set.

[0104] Therefore, by executing the above sub-steps D to F, this application ensures that the channel enhancement module can achieve the feature complementarity aggregation effect of adjacent channels with minimal computing resources, so that the corresponding aggregated feature set can enhance the latent features in the input features while retaining some of the original feature information of the input features.

[0105] Sub-step S224: Input the aggregated feature map set into the second Gabor residual module for depth residual operation to obtain the facial key feature map set output by the second Gabor residual module.

[0106] Therefore, by executing the above sub-steps S221 to S224, through the cooperation of the channel conversion mechanism and the channel enhancement mechanism, this application can reduce the computational cost and complexity of the model, improve the performance of capturing subtle facial features, and enable the target Gabor residual network visual model to capture key facial feature information from the face image to be recognized during the facial expression recognition process.

[0107] Step S230: Call the expression prediction layer to perform facial emotion classification prediction on the facial key feature map to obtain the facial emotion recognition result of the face image to be identified.

[0108] In this embodiment, if the target Gabor residual network visual model adopts Figure 2 or Figure 4 If the model architecture shown is implemented, the channel transformation enhancement layer in the target Gabor residual network visual model will directly transmit its output facial key feature set to the expression prediction layer, enabling the expression prediction layer to achieve a good facial expression recognition effect based on the acquired facial key feature information, thus improving the accuracy of facial expression recognition; if the target Gabor residual network visual model adopts Figure 5 If the model architecture shown is implemented, the channel transformation enhancement layer in the target Gabor residual network visual model will transmit its output facial key feature map to the expression prediction layer after processing by the second Gabor residual intermediate layer for facial emotion classification and prediction.

[0109] Therefore, by executing the above steps S210 to S230, this application can drive the target Gabor residual network visual model to perform facial expression recognition on the face image to be recognized, and maintain the model computation efficiency of the target Gabor residual network visual model in the facial expression recognition process at a high level. It can also capture key facial feature information from the face image to be recognized with less computational resources and memory cost, so as to ensure that the final output facial expression recognition result can show excellent performance in terms of accuracy and robustness.

[0110] In this application, to ensure that the computer device 10 can effectively execute the aforementioned facial expression recognition method, the aforementioned function is achieved by dividing the facial expression recognition device 100 stored in the computer device 10 into functional modules. The specific composition of the facial expression recognition device 100 applied to the aforementioned computer device 10 provided in this application will be described below.

[0111] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the composition of the facial expression recognition device 100 provided in this application embodiment. In this application embodiment, the facial expression recognition device 100 is applied to a computer device 10 equipped with a target Gabor residual network visual model, wherein the facial expression recognition device 100 may include a facial feature extraction module 110, a feature exchange and aggregation module 120, and a facial emotion prediction module 130.

[0112] The facial feature extraction module 110 is used to acquire the face image to be recognized and call the Gabor filter convolution layer to extract facial features from the face image to be recognized, thereby obtaining the corresponding facial feature map set, wherein the facial feature map set includes the initial facial feature map of the face image to be recognized in different Gabor directional channels.

[0113] The feature exchange and aggregation module 120 is used to call the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, wherein the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor direction channels.

[0114] The facial emotion prediction module 130 is used to call the expression prediction layer to perform facial emotion classification prediction on the facial key feature map and obtain the facial emotion recognition result of the face image to be recognized.

[0115] It should be noted that the facial expression recognition device 100 provided in this embodiment has the same basic principle and technical effect as the aforementioned facial expression recognition method. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the above description of the facial expression recognition method.

[0116] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0117] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the various functions provided in this application are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned readable storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0118] In summary, in the facial expression recognition method, apparatus, computer device, and readable storage medium provided in this application embodiment, after acquiring the face image to be recognized, this application calls the Gabor filter convolutional layer of the target Gabor residual network visual model to extract facial features from the face image to be recognized, obtaining the corresponding facial feature map set. Then, it calls the channel transformation enhancement layer of the target Gabor residual network visual model to perform adjacent channel feature exchange and aggregation processing on the facial feature map set, obtaining the corresponding facial key feature map set. Next, it uses the expression prediction layer of the target Gabor residual network visual model to perform facial emotion classification prediction on the facial key feature map set, obtaining the facial emotion recognition result of the face image to be recognized. This allows for the application of Gabor filters to the original traditional ResNet network model. In the convolution operation, the significant performance of Gabor filters in facial expression recognition is utilized to reduce the overall memory consumption of the model and enhance its ability to capture significant visual features. At the same time, by introducing channel transformation and channel enhancement mechanisms into the traditional ResNet network model, information exchange and feature complementarity between adjacent channels are promoted to further reduce the computational cost and complexity of the model and improve the ability to capture subtle facial features. This allows the computational efficiency of the corresponding Gabor residual network visual model in facial expression recognition to remain at a high level, and it can capture key facial feature information from the face image to be recognized with less computational resources and memory cost, so as to ensure that the final output facial expression recognition result can show excellent performance in terms of accuracy and robustness.

[0119] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A facial expression recognition method, characterized in that, The method is applied to computer devices, wherein the computer devices are loaded with a target Gabor residual network visual model. The target Gabor residual network visual model includes a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer. The channel transformation enhancement layer includes a first Gabor residual module, a channel transformation module, a channel enhancement module, and a second Gabor residual module connected in series. Both the first Gabor residual module and the second Gabor residual module are obtained by connecting the residuals of two Gabor convolutional kernels. The method includes: A face image to be identified is acquired, and the Gabor filter convolutional layer is called to extract facial features from the face image to be identified, thereby obtaining a corresponding facial feature map set, wherein the facial feature map set includes the initial facial feature map of the face image to be identified in different Gabor directional channels; The channel conversion enhancement layer is invoked to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, wherein the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor direction channels; The expression prediction layer is invoked to perform facial emotion classification prediction on the facial key feature map to obtain the facial emotion recognition result of the face image to be identified; The step of calling the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set includes: The facial feature map set is input into the first Gabor residual module for depth residual operation to obtain the output feature map set output by the first Gabor residual module; The channel conversion module is invoked to perform adjacent channel feature information exchange processing on the output feature map set, resulting in the exchanged feature map set output by the channel conversion module; The channel enhancement module is invoked to perform complementary aggregation processing on adjacent channel features of the exchanged feature set, resulting in the aggregated feature set output by the channel enhancement module; The aggregated feature map is input into the second Gabor residual module for depth residual operation to obtain the facial key feature map output by the second Gabor residual module.

2. The method according to claim 1, characterized in that, The Gabor filter convolutional layer is implemented using a Gabor convolutional kernel with multiple Gabor directional channels. The Gabor convolutional kernel is obtained by merging a self-learning convolutional kernel with a Gabor filter having multiple Gabor filtering directions. The step of calling the Gabor filter convolutional layer to extract facial features from the face image to be recognized and obtaining the corresponding facial feature map includes: Based on the total number of Gabor directional channels in the Gabor filter convolutional layer, an image copying operation is performed on the face image to be identified to obtain multiple target face images with the same image content, wherein the total number of the multiple target face images is consistent with the total number of Gabor directional channels; The Gabor convolution kernels included in the Gabor filter convolution layer are invoked to perform image convolution operations with the multiple target face images in the Gabor filter directions corresponding to different Gabor direction channels, so as to obtain the facial feature map set.

3. The method according to claim 1, characterized in that, The channel conversion module includes a channel left shift block, a channel right shift block, and a non-channel shift block. The step of calling the channel conversion module to perform adjacent channel feature information exchange processing on the output feature map set to obtain the exchanged feature map set output by the channel conversion module includes: The output feature map set is partitioned in the spatial dimension to obtain initial feature map subsets that match the channel left shift block, the channel right shift block, and the non-channel shift block, respectively. The channel left shift block, the channel right shift block, and the non-channel shift block are invoked to perform channel feature shifting on the matched initial feature map subset through depthwise convolution operation based on their own convolution kernel weights, so as to obtain the intermediate feature map subsets output by the channel left shift block, the channel right shift block, and the non-channel shift block respectively; The exchanged feature map set is obtained by concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level.

4. The method according to claim 1, characterized in that, The channel enhancement module includes feature enhancement blocks and non-feature enhancement blocks. The step of calling the channel enhancement module to perform adjacent channel feature complement aggregation processing on the exchanged feature map set to obtain the aggregated feature map set output by the channel enhancement module includes: The exchanged feature map set is partitioned in the spatial dimension to obtain initial feature map subsets that match the feature enhancement block and the non-feature enhancement block, respectively. The feature enhancement block and the non-feature enhancement block are invoked to perform channel feature enhancement on the matched initial feature map subsets through depthwise convolution operations based on their own convolution kernel weights, so as to obtain the intermediate feature map subsets output by the feature enhancement block and the non-feature enhancement block respectively; The aggregated feature map set is obtained by concatenating the feature maps of multiple intermediate feature map subsets at the spatial dimension level.

5. The method according to any one of claims 1-4, characterized in that, The target Gabor residual network visual model also includes a first Gabor residual intermediate layer, which is located between the Gabor filter convolutional layer and the channel conversion enhancement layer. The facial feature map output by the Gabor filter convolutional layer is processed by the first Gabor residual intermediate layer and then input into the channel conversion enhancement layer for adjacent channel feature exchange and aggregation processing. The first Gabor residual intermediate layer includes at least one Gabor residual intermediate module connected in series. Each Gabor residual intermediate module is obtained by connecting two Gabor residual modules in series, and each Gabor residual module is obtained by connecting two Gabor convolution kernel residuals.

6. The method according to claim 5, characterized in that, The target Gabor residual network visual model also includes a second Gabor residual intermediate layer, which is located between the channel transformation enhancement layer and the expression prediction layer. The facial key feature map output by the channel transformation enhancement layer is processed by the second Gabor residual intermediate layer and then input into the expression prediction layer for facial emotion classification and prediction. The second Gabor residual intermediate layer is obtained by connecting two Gabor residual modules in series, and each Gabor residual module is obtained by connecting two Gabor convolution kernel residuals.

7. A facial expression recognition device, characterized in that, An application is made in a computer device, wherein the computer device is loaded with a target Gabor residual network visual model, the target Gabor residual network visual model including a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer, the channel transformation enhancement layer including a first Gabor residual module, a channel transformation module, a channel enhancement module, and a second Gabor residual module connected in series, wherein both the first Gabor residual module and the second Gabor residual module are obtained by connecting the residuals of two Gabor convolutional kernels, the device comprising: The facial feature extraction module is used to acquire the face image to be identified and call the Gabor filter convolutional layer to extract facial features from the face image to be identified, thereby obtaining a corresponding facial feature map set, wherein the facial feature map set includes the initial facial feature map of the face image to be identified in different Gabor directional channels. The feature exchange and aggregation module is used to call the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, wherein the facial key feature map set includes facial key feature maps of the face image to be identified in different Gabor directional channels. The facial emotion prediction module is used to call the expression prediction layer to perform facial emotion classification prediction on the facial key feature map to obtain the facial emotion recognition result of the face image to be identified. The feature exchange and aggregation module calls the channel conversion enhancement layer to perform adjacent channel feature exchange and aggregation processing on the facial feature map set to obtain the corresponding facial key feature map set, including the following methods: The facial feature map set is input into the first Gabor residual module for depth residual operation to obtain the output feature map set output by the first Gabor residual module; The channel conversion module is invoked to perform adjacent channel feature information exchange processing on the output feature map set, resulting in the exchanged feature map set output by the channel conversion module; The channel enhancement module is invoked to perform complementary aggregation processing on adjacent channel features of the exchanged feature set, resulting in the aggregated feature set output by the channel enhancement module; The aggregated feature map is input into the second Gabor residual module for depth residual operation to obtain the facial key feature map output by the second Gabor residual module.

8. A computer device, characterized in that, The method includes a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to drive a target Gabor residual network visual model stored in the memory to implement the facial expression recognition method according to any one of claims 1-6, wherein the target Gabor residual network visual model includes a Gabor filter convolutional layer, a channel transformation enhancement layer, and an expression prediction layer.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a computer device, it drives the target Gabor residual network visual model to implement the facial expression recognition method according to any one of claims 1-6, wherein the target Gabor residual network visual model includes a Gabor filter convolutional layer, a channel conversion enhancement layer, and an expression prediction layer.

Images