Facial expression recognition method based on visual invariant features
By combining the viewpoint-invariant contrastive loss function, the supervised contrastive loss function, and the Barlow Twins loss function, a facial expression recognition model is constructed, which solves the problem of poor facial expression recognition accuracy when shot from different angles, and achieves high accuracy and improved generalization ability.
Patent Information
- Application Number
- CN202510906332.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-10-21
AI Technical Summary
Existing facial expression recognition technology has poor recognition accuracy when shooting at different angles and cannot effectively recognize facial expressions, especially in non-frontal views.
A method combining viewpoint-invariant contrastive loss function, supervised contrastive loss function and Barlow Twins loss function is adopted to construct a facial expression recognition model through viewpoint-invariant representation learning, expression representation learning and facial feature learning, thereby improving recognition accuracy and generalization ability.
It enables facial expression recognition from any viewpoint, significantly improving the accuracy of facial expression recognition and the model's generalization ability.
Smart Images
Figure CN120823633A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a facial expression recognition method based on visually invariant features. Background Art
[0002] Facial expressions are a crucial form of nonverbal communication and play a significant role in human interaction. In particular, in nonverbal communication, human emotion recognition often relies on facial expressions. If computers can perceive and understand users' intentions through facial expressions, the system can then provide suggestions and advice based on the perceived situation to assist the user. Consequently, facial expression recognition has garnered widespread attention in the fields of computer vision and artificial intelligence.
[0003] However, despite significant progress in facial expression recognition, which has been widely used in areas such as emotion perception and personal emotion management, in real-world settings, faces are likely captured from a variety of angles, rather than always from a clear, standardized frontal view. Consequently, most existing methods perform poorly when trained with images that are not frontal, and are unable to effectively recognize facial expressions. Consequently, their recognition accuracy needs to be further improved. Summary of the Invention
[0004] The purpose of the present invention is to provide a facial expression recognition method based on visually invariant features, which can effectively recognize facial expressions from any viewing angle and significantly improve the recognition accuracy of facial expressions.
[0005] The technical solution of the present invention is:
[0006] In a first aspect, the present application provides a facial expression recognition method based on visually invariant features, which comprises the following steps:
[0007] S1, obtaining facial images of the subject to be collected with different expressions under different viewing angles to construct a multi-view facial expression dataset;
[0008] S2, randomly extract a batch of images from the multi-view facial expression dataset through the data sampler, and apply two different image enhancement methods to enhance the randomly extracted images and then divide them into two groups;
[0009] S3. Input the two divided groups of images into the encoder for encoding to obtain the embedded representations and intermediate representations of the two groups of images respectively, and input the embedded representations and intermediate representations of the two groups of images into the shallow neural network respectively to obtain two final projected embedding vectors and two intermediate projected embedding vectors;
[0010] S4. Performing perspective-invariant representation learning on one of the final projected embedding vectors through a perspective-invariant contrast loss function to obtain a perspective-invariant representation, and performing expression representation learning on the other final projected embedding vector through a supervised contrast loss function based on the perspective-invariant representation to obtain a representation mapped with a corresponding expression category label;
[0011] S5. Based on the representation of the corresponding expression category label, the facial feature learning of the intermediate projection embedding vector is performed using the Barlow Twins loss function to obtain a compact representation that is both distinctive and recognizable.
[0012] S6. The supervised contrast loss function, the view-invariant contrast loss function, and the Barlow Twins loss are integrated to calculate the total loss function. A facial expression recognition model is constructed based on the compact representation and the total loss function. The facial image to be recognized is input into the facial expression recognition model to obtain the facial expression recognition result.
[0013] Furthermore, in step S4, before performing expression representation learning on one of the final projection embedding vectors through the perspective-invariant contrast loss function, the method also includes: clustering images of different perspectives showing a specific expression of the same object in the two divided groups of images through a self-supervised contrast loss function.
[0014] Furthermore, the calculation formula of the above self-supervised contrast loss function includes:
[0015]
[0016] Where, L self is the loss function of self-supervised contrastive learning, N is the image size, i is the i-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z k(i) For z i The corresponding final projected embedding vector, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] It is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0.
[0017] Furthermore, in step S4, the calculation formula of the above-mentioned view-invariant contrast loss function includes:
[0018]
[0019] Where, L view is the perspective invariant contrast loss function, N is the image size, N φ(i)is the correlation value of the image perspective invariant index φ(i), i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, φ(i) is the image perspective invariant index corresponding to i, φ(j) is the image perspective invariant index corresponding to j, and z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; i≠j Is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 φ(i)＝φ(j) is an indicator function that returns 1 when φ(i) equals φ(j), and 0 otherwise.
[0020] Furthermore, in step S4, the calculation formula of the above-mentioned supervised contrast loss function is:
[0021]
[0022] Where, L sup is the supervised contrast loss function, N is the image size, N yi is the number of image category attribute labels, yi is the image category attribute label, yj is the label used for comparison and exponential function judgment, i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1[k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; 1 i≠j Is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 yi＝yj is an indicator function that returns 1 when yi equals yj, otherwise it returns 0.
[0023] Furthermore, in step S5, the calculation formula of the Barlow Twins loss function includes:
[0024]
[0025] Where C ij is the cross-correlation matrix of the two intermediate projected embedding vectors, N is the image size, b is the batch index, A represents the A layer, B represents the B layer, is the value of the A layer and b batch in the i-th vector dimension index, is the value of the j-th vector dimension index in the B layer b batch, i is the i-th vector dimension index, j-th vector dimension index, α is the weight coefficient, L BT is the Barlow Twins loss function, C ii is the cross-correlation matrix of the two intermediate projected embedding vectors when i=j.
[0026] Furthermore, in step S6, the calculation formula of the above total loss function includes:
[0027]
[0028] Where, L ViewFX is the total loss function, L sup is the supervised contrast loss function, γ and β are the coefficients of the multi-view loss term and BT loss term respectively, L View is the view-invariant contrast loss function, is the loss function used to train the two middle layers of the encoder.
[0029] In a second aspect, the present application provides an electronic device, comprising:
[0030] a memory for storing one or more programs;
[0031] processor;
[0032] When the one or more programs are executed by the processor, a facial expression recognition method based on visually invariant features as described in any one of the first aspects is implemented.
[0033] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a facial expression recognition method based on visually invariant features as described in any one of the first aspects above.
[0034] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:
[0035] (1) The present invention provides a facial expression recognition method based on visual invariant features, which adopts a view-invariant contrast loss function to learn the view-invariant representation of facial expressions, adopts a supervised contrast loss function to perform expression representation learning of supervised expression category information, and adopts a Barlow Twins loss function to perform facial feature learning on the intermediate projection embedding vector to eliminate subtle differences between expressions to make the representation more compact. Combining three different loss functions, the method can effectively capture subtle changes in facial expressions, so that the model significantly improves the recognition accuracy of facial expressions without increasing the learning cost, and enables it to actually classify facial expressions regardless of the input observation angle during the reasoning process, thereby enhancing the model's ability to learn features within each video frame and improving the ability to capture dynamic relationships between different frames;
[0036] (2) The present invention adopts a view-invariant contrast loss function to learn the view-invariant representation of facial expressions, which enables effective recognition of expressions from any viewpoint and significantly improves the generalization ability of the facial expression recognition model. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.
[0038] Figure 1 A diagram showing the steps of a facial expression recognition method based on visually invariant features according to the present invention;
[0039] Figure 2 The figure is a schematic structural block diagram of an electronic device according to an embodiment of the present invention.
[0040] Icon: 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION
[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.
[0042] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without making any creative efforts shall fall within the scope of protection of the present application.
[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.
[0044] It should be noted that, in this document, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus. In the absence of further limitations, the elements defined by the phrase "comprises..." do not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the elements.
[0045] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features thereof may be combined with each other.
[0046] Example 1
[0047] See also Figure 1 , Figure 1 The figure shows a step diagram of a facial expression recognition method based on visual invariant features provided in an embodiment of the present application.
[0048] In a first aspect, the present application provides a facial expression recognition method based on visually invariant features, which comprises the following steps:
[0049] S1, obtaining facial images of the subject to be collected with different expressions under different viewing angles to construct a multi-view facial expression dataset;
[0050] S2, randomly extract a batch of images from the multi-view facial expression dataset through the data sampler, and apply two different image enhancement methods to enhance the randomly extracted images and then divide them into two groups;
[0051] S3. Input the two divided groups of images into the encoder for encoding to obtain the embedded representations and intermediate representations of the two groups of images respectively, and input the embedded representations and intermediate representations of the two groups of images into the shallow neural network respectively to obtain two final projected embedding vectors and two intermediate projected embedding vectors;
[0052] S4. Performing perspective-invariant representation learning on one of the final projected embedding vectors through a perspective-invariant contrast loss function to obtain a perspective-invariant representation, and performing expression representation learning on the other final projected embedding vector through a supervised contrast loss function based on the perspective-invariant representation to obtain a representation mapped with a corresponding expression category label;
[0053] S5. Based on the representation of the corresponding expression category label, the facial feature learning of the intermediate projection embedding vector is performed using the Barlow Twins loss function to obtain a compact representation that is both distinctive and recognizable.
[0054] S6. The supervised contrast loss function, the view-invariant contrast loss function, and the Barlow Twins loss are integrated to calculate the total loss function. A facial expression recognition model is constructed based on the compact representation and the total loss function. The facial image to be recognized is input into the facial expression recognition model to obtain the facial expression recognition result.
[0055] As a preferred embodiment, in step S4, before performing expression representation learning on one of the final projection embedding vectors through the perspective-invariant contrast loss function, it also includes: clustering images of different perspectives of a specific expression displayed by the same object in the two divided groups of images through a self-supervised contrast loss function.
[0056] As a preferred embodiment, the calculation formula of the self-supervised contrast loss function includes:
[0057]
[0058] Where, L self is the loss function of self-supervised contrastive learning, N is the image size, i is the i-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z k(i) For z i The corresponding final projected embedding vector, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] It is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0.
[0059] It should be noted that for each image, two enhancement operations are applied to generate two enhanced versions. These enhanced images are first passed through an encoder to obtain their embedded representations; then, the output of the encoder is passed through a shallow neural network to obtain the final projected embedding vector z i; When processing a batch of images of size N, we set i∈{1,…,2N} as the index of a random enhanced image, and k(i) represents the index of the corresponding second enhanced image, from which we can customize the above self-supervised contrast loss function
[0060] As a preferred embodiment, in step S4, the calculation formula of the perspective invariant contrast loss function includes:
[0061]
[0062] Where, L view is the perspective invariant contrast loss function, N is the image size, N φ(i) is the correlation value of the image perspective invariant index φ(i), i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, φ(i) is the image perspective invariant index corresponding to i, φ(j) is the image perspective invariant index corresponding to j, and z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; i≠j Is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 φ(i)＝φ(j) is an indicator function that returns 1 when φ(i) equals φ(j), and 0 otherwise.
[0063] It should be noted that the present invention aims to learn perspective-invariant features of expressions, that is, to generate similar representations of input images regardless of their perspective; this actually means creating an embedding space in which different perspectives of the same image are close to each other. To achieve this goal, the present invention proposes a perspective-invariant contrast loss function, which brings the representations of different perspectives of an image closer to each other and distinguishes these representations from the representations of other images; in an embodiment of the present invention, instead of only considering the enhanced version of the same image as a positive sample (i.e., instance discrimination), the definition of a positive sample is extended to include all perspectives and enhanced versions of the input image; therefore, all other images are considered negative samples, and this modification leads to the above perspective-invariant contrast loss function.
[0064] As a preferred implementation, in step S4, the calculation formula of the supervised contrast loss function is:
[0065]
[0066] Where, L sup is the supervised contrast loss function, N is the image size, Nyi is the number of image category attribute labels, yi is the image category attribute label, yj is the label used for comparison and exponential function judgment, i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1[k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; 1 i≠j Is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 yi＝yj is an indicator function that returns 1 when yi equals yj, otherwise it returns 0.
[0067] It should be noted that after successfully learning the perspective-invariant representations, the next step is to establish a mapping relationship between these representations and expression category labels. To this end, the present invention introduces a supervised contrast loss function, which can accurately map the learned representations to the corresponding expression labels.
[0068] As a preferred embodiment, in step S5, the calculation formula of the Barlow Twins loss function includes:
[0069]
[0070] Where C ij is the cross-correlation matrix of the two intermediate projected embedding vectors, N is the image size, b is the batch index, A represents the A layer, B represents the B layer, is the value of the A layer and b batch in the i-th vector dimension index, is the value of the j-th vector dimension index in the B layer b batch, i is the i-th vector dimension index, j-th vector dimension index, α is the weight coefficient, L BT is the Barlow Twins loss function, C ii is the cross-correlation matrix of the two intermediate projected embedding vectors when i=j.
[0071] It should be noted that facial expressions are usually distinguished by subtle differences between the learned representations. Therefore, facial expression recognition solutions are expected to learn more discriminative and recognizable representations. We introduced the Barlow Twins loss function in the encoder. This can help generate compact representations by measuring the correlation between the embedding vectors of the enhanced image and forcing them to be as close to the unit matrix as possible; the first term in the formula is called the "invariance" term, which aims to make the diagonal elements of the cross-correlation matrix equal to 1, focusing on representations that are invariant to image distortion / enhancement. The second term is called "redundancy reduction", which is committed to setting the off-diagonal elements to 0, thereby effectively eliminating the correlation between different vector components in the representation; where α is the coefficient of the second term, and its value needs to be manually set in the range of 0 to 1.
[0072] As a preferred embodiment, in step S6, the calculation formula of the total loss function includes:
[0073]
[0074] Where, L ViewFX is the total loss function, L sup is the supervised contrast loss function, γ and β are the coefficients of the multi-view loss term and BT loss term respectively, L View is the view-invariant contrast loss function, is the loss function used to train the two middle layers of the encoder.
[0075] Example 2
[0076] See also Figure 2 , Figure 2 A schematic structural block diagram of an electronic device provided in an embodiment of the present application.
[0077] An electronic device includes a memory 101, a processor 102, and a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to perform various functional applications and data processing. The communication interface 103 can be used to communicate signaling or data with other node devices.
[0078] Among them, the memory 101 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0079] The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0080] It is understood that the structure shown in the figure is for illustration only, and a method for facial expression recognition based on visually invariant features may include more or fewer components than shown in the figure, or have a different configuration than shown in the figure. Each component shown in the figure may be implemented using hardware, software, or a combination thereof.
[0081] In the embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The embodiments described above are merely illustrative. For example, the flowcharts or block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the methods and computer program products according to multiple embodiments of the application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.
[0082] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0083] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0084] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.
[0085] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.
Claims
1. A facial expression recognition method based on visually invariant features, characterized in that: The following steps are involved: S1, obtaining facial images of the subject to be collected with different expressions under different viewing angles to construct a multi-view facial expression dataset; S2, randomly extract a batch of images from the multi-view facial expression dataset through the data sampler, and apply two different image enhancement methods to enhance the randomly extracted images and then divide them into two groups; S3. Input the two divided groups of images into the encoder for encoding to obtain the embedded representations and intermediate representations of the two groups of images respectively, and input the embedded representations and intermediate representations of the two groups of images into the shallow neural network respectively to obtain two final projected embedding vectors and two intermediate projected embedding vectors; S4. Performing perspective-invariant representation learning on one of the final projected embedding vectors through a perspective-invariant contrast loss function to obtain a perspective-invariant representation, and performing expression representation learning on the other final projected embedding vector through a supervised contrast loss function based on the perspective-invariant representation to obtain a representation mapped with a corresponding expression category label; S5. Based on the representation of the corresponding expression category label, the Barlow Twins loss function is used to learn facial features of the intermediate projection embedding vector to obtain a compact representation that is both distinctive and recognizable. S6. The supervised contrast loss function, the view-invariant contrast loss function, and the Barlow Twins loss are integrated to calculate the total loss function. A facial expression recognition model is constructed based on the compact representation and the total loss function. The facial expression recognition result is obtained by inputting the facial image to be recognized into the facial expression recognition model.
2. A facial expression recognition method based on visually invariant features as claimed in claim 1, characterized in that, In step S4, before performing expression representation learning on one of the final projected embedding vectors through the perspective-invariant contrast loss function, the method also includes: clustering images of different perspectives of a specific expression displayed by the same object in the two divided groups of images through a self-supervised contrast loss function.
3. A facial expression recognition method based on visually invariant features as claimed in claim 2, characterized in that, The calculation formula of the self-supervised contrast loss function includes: Where, L self is the loss function of self-supervised contrastive learning, N is the image size, i is the i-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z k(i) For z i The corresponding final projected embedding vector, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] It is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0.
4. A facial expression recognition method based on visually invariant features as claimed in claim 1, characterized in that, In step S4, the calculation formula of the perspective invariant contrast loss function includes: Where, L view is the perspective invariant contrast loss function, N is the image size, N φ(i) is the correlation value of the image perspective invariant index φ(i), i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, φ(i) is the image perspective invariant index corresponding to i, φ(j) is the image perspective invariant index corresponding to j, and z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1 [k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; i≠j Is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 φ(i)＝φ(j) is an indicator function that returns 1 when φ(i) equals φ(j), and 0 otherwise.
5. A facial expression recognition method based on visually invariant features as claimed in claim 1, characterized in that, In step S4, the calculation formula of the supervised contrast loss function is: Where, L sup is the supervised contrast loss function, N is the image size, N yi is the number of image category attribute labels, yi is the image category attribute label, yj is the label used for comparison and exponential function judgment, i is the i-th vector dimension index, j-th vector dimension index, k is the index of the second enhanced image corresponding to i, z i is one of the final projected embedding vectors, z j is the image feature vector representation, τ is the parameter, z k is another final projection embedding vector; 1[k≠i] is an indicator function that returns 1 when k is not equal to i, otherwise it returns 0; 1 i≠j is an indicator function that returns 1 when i is not equal to j, otherwise it returns 0; 1 yi＝yj is an indicator function that returns 1 when yi equals yj, otherwise it returns 0.
6. A facial expression recognition method based on visually invariant features as claimed in claim 1, characterized in that, In step S5, the calculation formula of the Barlow Twins loss function is include: Where C ij is the cross-correlation matrix of the two intermediate projected embedding vectors, N is the image size, b is the batch index, A represents the A layer, B represents the B layer, is the value of the A layer and b batch in the i-th vector dimension index, is the value of the j-th vector dimension index in the B layer b batch, i is the i-th vector dimension index, j-th vector dimension index, α is the weight coefficient, L BT is the BarlowTwins loss function, C ii is the cross-correlation matrix of the two intermediate projected embedding vectors when i=j.
7. A facial expression recognition method based on visually invariant features as claimed in claim 1, characterized in that, In step S6, the calculation formula of the total loss function includes: Where, L ViewFX is the total loss function, L sup is the supervised contrast loss function, γ and β are the coefficients of the multi-view loss term and BT loss term respectively, L View is the view-invariant contrast loss function, is the loss function used to train the two middle layers of the encoder.
8. An electronic device, characterized in that: include: a memory for storing one or more programs; processor; When the one or more programs are executed by the processor, a facial expression recognition method based on visual invariant features as described in any one of claims 1 to 7 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements a facial expression recognition method based on visual invariant features as described in any one of claims 1 to 7.