Hierarchical data augmentation of expression data, emotion recognition methods and systems therefrom

Abstract

This application belongs to the field of data augmentation for training data, specifically disclosing hierarchical data augmentation of facial expression data, and an emotion recognition method and system based thereon. This application introduces a pseudo-age recognition model to achieve explicit perception of implicit features in the dataset. It performs fine-grained segmentation of the facial image dataset based on age and emotion category, and further employs a differential dynamic adjustment augmentation strategy for samples from subgroups with different levels of sample sparsity. This method overcomes the limitations of traditional data balancing methods that rely solely on simple sampling or uniform augmentation based on a single label. It achieves hierarchical and differentiated data balancing processing across multiple dimensions, allowing the augmentation process to adaptively focus on data-scarce groups (such as negative emotion samples from the elderly). This effectively improves the consistency and fairness of emotion recognition models across different ages and emotion categories, providing higher reliability and generalization capabilities for applications related to elderly emotion monitoring and emotional companionship.

Description

Technical Field

[0001] This application belongs to the field of data augmentation of training data, and more specifically, relates to hierarchical data augmentation of facial expression data, and emotion recognition methods and systems based thereon. Background Technology

[0002] Existing facial expression recognition systems mostly rely on publicly available datasets for training, which generally suffer from uneven sample distribution. Specifically, the proportion of elderly samples is significantly low, there is insufficient sample size for negative emotions, and there are large differences in sample size between different groups (e.g., age, gender), leading to performance imbalances across different age groups and emotion categories. This is particularly problematic in scenarios involving monitoring the emotions of the elderly, where models often fail to accurately recognize facial expressions and exhibit low accuracy in recognizing negative emotions, impacting the system's reliability and fairness.

[0003] Existing technologies mainly employ simple resampling or uniform data augmentation strategies, which are difficult to optimize for the characteristic differences of different subgroups and lack explicit perception of the multidimensional feature distribution within the data. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide hierarchical data augmentation of facial expression data, and a method and system for emotion recognition based thereon, aiming to solve the problem of uneven distribution of dataset samples in dimensions such as age and emotion category in facial expression recognition tasks.

[0005] To address the aforementioned problems, according to at least one aspect of this application, a data augmentation system and a hierarchical data augmentation method for facial expression data are provided, comprising: providing an age recognition model generated based on a first face image dataset in a memory, the first face image dataset having age labels; receiving a second face image dataset on a processor communicating with the memory, the second face image dataset having emotion category labels; using the age recognition model to predict the age of each sample in the second face image dataset to obtain pseudo-age labels corresponding to the samples; determining the age range of each sample in the second face image dataset based on the pseudo-age labels; dividing the second face image dataset by combining the age range and emotion category to obtain several subgroups, all samples in the same subgroup having the same emotion category and age range; calculating the proportion of each subgroup; determining the corresponding sample scarcity level based on the proportion of each subgroup; performing data augmentation operations on very few and a few subgroups, the intensity of the data augmentation increasing with the scarcity level; performing data reduction operations on excessive subgroups; and performing no data augmentation or data reduction operations on normal subgroups, so that each subgroup tends to be balanced in both age and emotion category dimensions; and all processed samples constituting a data-augmented training set.

[0006] According to at least one aspect of this application, an emotion recognition method based on hierarchical data augmentation is provided, comprising: providing an emotion recognition model generated based on a training set augmented by the method described above in a memory; and receiving a face image to be recognized on a processor communicating with the memory, inputting it into the emotion recognition model to obtain an emotion recognition result.

[0007] According to at least one aspect of this application, a hierarchical data augmentation-based emotion recognition system is provided, comprising: a memory containing an emotion recognition model generated based on a training set augmented by the above method; and a processor communicating with the memory, the processor being configured to receive a face image to be recognized, input it into the emotion recognition model, and obtain an emotion recognition result.

[0008] In summary, this application introduces a pseudo-age recognition model to achieve explicit perception of implicit features in the dataset. It combines multi-dimensional labels such as age and emotion category to perform fine-grained segmentation of the face image dataset. Furthermore, it employs a differential dynamic adjustment enhancement strategy for samples from subgroups with different levels of sample sparsity. This method overcomes the limitations of traditional data balancing methods that rely solely on simple sampling or uniform enhancement based on a single label (such as emotion category). It enables hierarchical and differentiated data balancing processing across multiple dimensions, allowing the enhancement process to adaptively focus on data-scarce groups (such as elderly negative emotion samples). This effectively improves the consistency and fairness of emotion recognition models across different ages and emotion categories, providing higher reliability and generalization capabilities for applications related to elderly emotion monitoring and emotional companionship. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a layered data augmentation-based emotion recognition system according to some embodiments of this application.

[0010] Figure 2 This is a flowchart illustrating a layered data augmentation method for facial expression data according to some embodiments of this application.

[0011] Figure 3 This is a schematic diagram of the "age × emotion" distribution matrix constructed by some embodiments of this application. Detailed Implementation

[0012] Before explaining at least one embodiment of this application in detail, it should be understood that the application of this application is not necessarily limited to the details of the construction and arrangement of the components and / or methods shown in the following description and / or drawings and / or embodiments. This application may have other embodiments, or may be practiced or implemented in various ways.

[0013] Those skilled in the art will understand that various aspects of this application can be embodied as a system, method, or computer program product. Therefore, various aspects of this application can take the form of a purely hardware embodiment, a purely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware (collectively referred to herein as a "circuit," "module," or "system"). Furthermore, various aspects of this application can take the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0014] Any combination of one or more computer-readable media may be used. A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. A computer-readable storage medium can be, for example (but not limited to), an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), memory sticks, floppy disks, mechanical encoding devices (e.g., punched cards or raised structures in recesses where instructions are recorded), and any suitable combination of the above. The computer-readable storage medium used herein should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0015] Computer program code includes computer-readable program instructions embodied on a computer-readable medium that can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0016] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cabling, fiber optic cables, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0017] Computer-readable program instructions for performing the operations of this application may be written in any combination of one or more programming languages, such as assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and traditional procedural programming languages ​​(such as the "C" programming language or similar programming languages).

[0018] Computer-readable program instructions may execute entirely on the user's computer, or partially on the user's computer as a standalone software package, or partially on the user's computer, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected to the Internet via an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can be personalized by utilizing state information from the computer-readable program instructions to execute the computer-readable program instructions to implement various aspects of this application.

[0019] This document describes various aspects of the present application with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block in the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0020] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may differ from the order shown in the figures. For example, two consecutively shown blocks may actually execute substantially simultaneously, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0021] refer to Figure 1This is a schematic diagram of a layered data augmentation-based emotion recognition system according to some embodiments of this application. The emotion recognition system 100 (e.g., a computer, server, computing node, computing node cluster, etc.) may include an input / output (I / O) interface 102, a processor 104 for performing processes such as process 200, and a memory 106 for storing code (program storage) and / or data.

[0022] I / O interface 102 may include one or more network interfaces for connecting to one or more wired and / or wireless networks, such as a local area network (LAN), wireless local area network (WLAN), wide area network (WAN), metropolitan area network (MAN), cellular network, Internet, etc. I / O interface 102 may also include one or more local interfaces, such as a universal serial bus (USB), serial port, radio frequency (RF) interface, etc., for connecting to one or more local resources, such as attachable storage devices, other computing devices, etc. Through I / O interface 102, emotion recognition system 100 can receive images of faces to be recognized.

[0023] Processor 104 (homogeneous or heterogeneous) may include one or more processing nodes for parallel processing, such as a cluster and / or one or more multi-core processors. Processor 104 may execute one or more software modules, such as processes, scripts, applications, agents, utilities, tools, operating systems (OS), etc., each software module containing multiple program instructions stored in a non-transitory medium (e.g., memory 106) and executed by one or more processors (e.g., processor 104). Optionally, processor 104 may also integrate, utilize, and / or facilitate the integration and / or use of one or more hardware elements (modules) in the edge node, such as circuits, components, integrated circuits (ICs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), graphics processing units (GPUs), artificial intelligence (AI) accelerators, etc. Therefore, processor 104 may execute one or more functional modules implemented by one or more software modules, one or more hardware modules, and / or combinations thereof. For example, processor 104 can execute a hierarchical data augmentation module for facial expression data to execute process 200; it can also execute a trainer for the emotion recognition model to train the emotion recognition model; and it can also execute an emotion recognition module to input the face image to be recognized into the emotion recognition model to obtain the emotion recognition result.

[0024] The memory 106 for storing data and / or program code may include one or more non-transitory storage devices, which may be persistent non-volatile devices such as hard disks, solid-state drives (SSDs), disks, flash memory arrays, etc., or volatile devices such as random access memory (RAM), cache, etc. The memory 106 may also include one or more local and / or remote network storage resources, such as storage servers, network attached storage (NAS), network drives, etc., which can be accessed through one or more networks via I / O interface 102.

[0025] refer to Figure 2 This document illustrates a flowchart of a hierarchical data augmentation method for facial expression data according to some embodiments of this application, where all processed samples constitute the augmented training set. Exemplary process 200 can be performed to achieve balanced processing of multidimensional subgroups, thereby improving the performance balance and generalization ability of the emotion recognition model across different ages and emotion categories.

[0026] Step 202: Pseudo-age identification and label generation.

[0027] This application trains an age recognition model using a first face image dataset, which includes age labels. After training, the model is applied to a second face image dataset to predict the age of each image in the second face image dataset, resulting in pseudo-age labels. The second face image dataset includes emotion category labels but does not label the age.

[0028] In one example, the first face image dataset is the publicly available age dataset UTKFace. It's a large-scale face image dataset commonly used for age estimation, gender, and ethnicity classification tasks, containing over 20,000 face images covering a wide age range from 0 to 116 years old. Each image is labeled with age, gender, and ethnicity information. In another example, the first face image dataset is IMDB-WIKI. This dataset is also one of the most widely used large-scale face image resources in current age estimation research, containing over 500,000 age-labeled facial images of actors. Its annotation information covers a wide age range, from children to the elderly.

[0029] The Second Face Image Dataset is a relatively balanced dataset of facial expression images across emotion categories and age groups, with particularly richness in samples of elderly individuals and those expressing negative emotions. Each image is labeled with machine-predicted age and emotion category information, specifically divided into five age groups (Children C: 0–17 years old, Young Adults Y: 18–39 years old, Middle-aged M: 40–64 years old, Elderly O: 65–84 years old, and Very Elderly SO: 85 years old and above) and seven basic emotion categories (anger, disgust, fear, happiness, sadness, surprise, and neutral). The Second Face Image Dataset can be a small expression dataset like RAF-DB or a large dataset like AffectNet.

[0030] The age recognition model, provided by memory 106, can be any neural network model, such as ResNet50, VGG16, or EfficientNet. Trained on a large-scale first face image dataset, the model achieves good generalization ability, thus generating reliable age labels for a second face image dataset. In one example, the age recognition model uses a ResNet50 architecture, predicting age via ordered regression. Due to its deeper network layers and residual connection mechanism, it can extract richer, more hierarchical facial features while maintaining high training stability, making the model more likely to converge on large-scale age datasets and achieving higher age prediction accuracy.

[0031] To improve the reliability of the prediction results, this application further includes: after obtaining the pseudo-age label, performing a moving average or confidence filtering on the prediction results. In one example, a moving average is applied to the prediction results to reduce the impact of short-term fluctuations or noise. In another example, confidence filtering is applied to the prediction results to remove low-confidence samples.

[0032] Step 204: Multidimensional partitioning and statistical analysis of the dataset.

[0033] Based on pseudo-age labels, the second face image dataset is divided into several age groups. Combined with several basic emotion categories, the samples in the second face image dataset are divided into multiple subgroups by multi-dimensional feature combination, with each subgroup corresponding to an emotion and age layer.

[0034] In one example, based on pseudo-age labels, the samples are divided into five age groups: C (Child): 0–17 years old; Y (Young): 18–39 years old; M (Middle-aged): 40–64 years old; O (Old): 65–84 years old; SO (SuperOld): 85 years old and above. Combining the seven existing emotion category labels (Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral) from the second face image dataset, the samples in the second face image dataset are further divided into 35 subgroups, such as Y-Sad, O-Happy, etc.

[0035] If the second face image dataset contains gender or other attributes, it can be expanded into a three-dimensional combination (e.g., age × emotion × gender) to form a more granular multidimensional subgroup structure.

[0036] For each age-emotion combination subgroup in the dataset, the sample size is counted. And calculate its proportion relative to the entire sample:

[0037] Among them, subscript Indicates age group, Indicates the emotion category.

[0038] In one embodiment, the sample size of each subgroup is counted to construct a distribution matrix of "age × emotion", such as... Figure 3 As shown, the rows and columns of the distribution matrix correspond to the emotion category and age group, respectively. The matrix elements include the number of each subgroup and its proportion in the total data, which are used to characterize the internal structure of the dataset and the degree of sample imbalance.

[0039] At the same time, calculate the average proportion of all subgroups. This serves as a benchmark for grading and enhancement in subsequent steps.

[0040] Step 206: Hierarchical data augmentation mechanism based on subgroup proportion.

[0041] (1) Subgroup proportion classification Based on the differences in the proportion of subgroups, the system divides subgroups into four categories: very small subgroup, minority subgroup, normal subgroup, and excessive subgroup. This classification reflects the distributional bias of the dataset in the age and emotion dimensions, providing a basis for subsequent reinforcement decision-making.

[0042] The thresholds for each level can be adaptively adjusted based on the dataset size and the number of categories to ensure the overall distribution balance. In one example, if... If the sample is deemed extremely scarce, then the subgroup is classified as a very small subgroup; if If the sample size is deemed too small, then this subgroup is classified as a minority subgroup; if If the sample distribution is deemed reasonable, then this subgroup is classified as a normal subgroup; if If the sample size is deemed too large, then the subgroup is classified as an excessive number of subgroups.

[0043] (2) Enhancement and Reduction Strategies Different data augmentation or reduction schemes are implemented for different levels of subgroups. Data augmentation is performed on very few and a small number of subgroups, with the intensity of data augmentation increasing with scarcity. Data reduction is performed on excessive subgroups, and no data augmentation or reduction is performed on normal subgroups. By implementing different data augmentation or reduction schemes for different levels of subgroups, the number of each subgroup reaches the target multiple, and the groups tend to be balanced in terms of age and emotional category.

[0044] The data augmentation operation is performed on a very small number of subgroups and a minority subgroup, with the intensity of the data augmentation increasing with the scarcity level, specifically as follows: For a very small number of subgroups, a deep augmentation operation is performed; for a minority subgroup, if the ratio of the average proportion of all subgroups to the proportion of the subgroup is less than 1.5, a low-intensity augmentation operation is performed; if the ratio of the average proportion of all subgroups to the proportion of the subgroup is within a certain range... If the ratio of the average percentage of the entire subpopulation to the percentage of the subpopulation exceeds 2.5, perform a high-intensity enhancement operation.

[0045] The low-intensity enhancement operations include random rotation, horizontal flipping, slight translation, and noise perturbation, and the order of the four operations is not required.

[0046] The medium-intensity enhancement operation adds color jitter and random occlusion to the low-intensity enhancement operation. The order of the six operations is not required.

[0047] The high-intensity enhancement operation adds affine transformation and style transfer to the medium-intensity enhancement operation, and the order of the eight operations is not required.

[0048] The depth enhancement operation involves directly generating entirely new samples using a generative model. This generative model can be GAN or Diffusion. When generating new samples, it does not require feeding a dataset; it directly generates images based on a prompt. For example, the prompt might be: "Generate an image of a face with a sad expression, indicating an age between 65 and 84."

[0049] (3) Multidimensional consistency constraints and iterative correction After enhancement, the sample distribution of each subgroup is recalculated to obtain the new proportions. And calculate the distribution deviation coefficient:

[0050] This application also includes: after each round of enhancement or screening operation, performing a deviation judgment, if the distribution deviation coefficient... If the percentage exceeds a set threshold (e.g., 10%), it is considered a large deviation. For subgroups with large deviations, secondary data augmentation or reduction operations are performed. Through multiple iterations, the proportion of each subgroup is eventually ensured to meet the target. This mechanism achieves global balance in the age and emotion dimensions of the augmented dataset. It ensures that the augmented data achieves the best possible balance across both dimensions, avoiding secondary bias caused by over-correction of a single dimension.

[0051] (4) Enhance tracking and metadata recording To ensure the traceability and quality control of the data augmentation process, each data augmentation operation records information such as sample ID, subgroup category, augmentation type, parameter settings, and generation time, forming a data augmentation log table for subsequent verification and model evaluation.

[0052] Step 208: Equalization dataset generation and emotion recognition model training.

[0053] The enhanced and filtered samples were recombined to form a balanced training set. The new dataset is not only balanced in terms of emotion categories, but also structurally balanced in terms of age.

[0054] A new dataset is used to train an emotion recognition model. After training, the model receives a face image to be recognized and inputs it into the emotion recognition model to obtain the emotion recognition result.

[0055] The emotion recognition model is provided by memory 106 and can be ResNet50, VGG16 or EfficientNet.

[0056] This application uses the same emotion recognition model and trains and tests it on datasets before and after enhancement. The results show that the method proposed in this application can significantly reduce the performance bias of the emotion recognition model on the elderly group and the negative emotion category, and improve recognition consistency and fairness.

[0057] It should be understood that expressions such as “comprising” and “may include” used in this application indicate the existence of the disclosed functions, operations, or constituent elements, and do not limit one or more additional functions, operations, and constituent elements. In this application, terms such as “comprising” and / or “having” are to be interpreted as indicating a particular characteristic, number, operation, constituent element, component, or combination thereof, but not to exclude the existence or possibility of adding one or more other characteristics, numbers, operations, constituent elements, components, or combinations thereof.

[0058] Furthermore, in this application, the expression "and / or" includes any and all combinations of the associated listed words. For example, the expression "A and / or B" may include A, may include B, or may include both A and B.

[0059] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a detachable connection or a non-detachable connection; it can be a direct connection or an indirect connection through an intermediate medium. "Fixed connection" refers to a connection where the relative positional relationship remains unchanged after connection. "Rotary connection" refers to a connection where the components can rotate relative to each other after connection. "Sliding connection" refers to a connection where the components can slide relative to each other after connection. The directional terms mentioned in the embodiments of this application, such as "top," "bottom," "inner," "outer," "left," and "right," are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of this application, and are not intended to indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hierarchical data augmentation method for facial expression data, characterized in that, include: An age recognition model based on a first face image dataset with age annotations is provided in the memory; On a processor communicating with memory, a second face image dataset is received, which is labeled with emotion categories. An age recognition model is used to predict the age of each sample in the second face image dataset, and pseudo-age labels are obtained for each sample. The age range of each sample in the second face image dataset is determined based on the pseudo-age labels. The second face image dataset is divided into several subgroups by combining the age range and emotion category. All samples in the same subgroup have the same emotion category and age range. The proportion of each subgroup is calculated. Based on the proportion of each subgroup, the corresponding sample scarcity level is determined. For very few subgroups and a small number of subgroups, data augmentation operations are performed. The intensity of the data augmentation increases with the scarcity level. For too many subgroups, data reduction operations are performed. For normal subgroups, no data augmentation or data reduction operations are performed, so that each subgroup tends to be balanced in the two dimensions of age and emotion category. All processed samples constitute the data-augmented training set. The determination of the corresponding sample scarcity level based on the proportion of each subgroup is as follows: If the ratio of the average proportion of all subgroups to the proportion of the subgroup is greater than a first proportion, the subgroup is classified as a very small subgroup; if the ratio of the average proportion of all subgroups to the proportion of the subgroup is less than or equal to the first proportion but greater than the second proportion, the subgroup is classified as a minority subgroup; if the ratio of the average proportion of all subgroups to the proportion of the subgroup is less than or equal to the second proportion but greater than the third proportion, the subgroup is classified as a normal subgroup; if the ratio of the average proportion of all subgroups to the proportion of the subgroup is less than or equal to the third proportion, the subgroup is classified as an excessive subgroup; wherein, the first proportion > the second proportion > 1 > the third proportion.

2. The hierarchical data augmentation method as described in claim 1, characterized in that, The age recognition model is ResNet50, which predicts age using an ordinal regression approach.

3. The hierarchical data augmentation method as described in claim 1, characterized in that, The data augmentation operation is performed on a very small number of subgroups and a minority subgroup, with the intensity of the data augmentation increasing with the scarcity level, specifically as follows: For a very small number of subgroups, a deep augmentation operation is performed; for a minority subgroup, if the ratio of the average proportion of all subgroups to the proportion of the subgroup is less than 1.5, a low-intensity augmentation operation is performed; if the ratio of the average proportion of all subgroups to the proportion of the subgroup is within a certain range... If the ratio of the average percentage of the entire subpopulation to the percentage of the subpopulation exceeds 2.5, perform a high-intensity enhancement operation.

4. The hierarchical data augmentation method as described in claim 3, characterized in that, The low-intensity enhancement operations include random rotation, horizontal flipping, slight translation, and noise perturbation; the order of these four operations is not important. The medium-intensity enhancement operations, based on the low-intensity enhancement operations, add color jitter and random occlusion; the order of these six operations is not important. The high-intensity enhancement operations, based on the medium-intensity enhancement operations, add affine transformation and style transfer; the order of these eight operations is not important. The deep enhancement operation directly generates entirely new samples using a generative model.

5. The hierarchical data augmentation method as described in claim 1, characterized in that, The data reduction operation is downsampling, slight perturbation, or sample weighting.

6. The hierarchical data augmentation method according to any one of claims 1 to 5, characterized in that, After a round of data augmentation and reduction, the proportion of each subgroup is recalculated, and the distribution deviation of each subgroup is further calculated. If the distribution deviation of a certain subgroup exceeds the set threshold, a second data augmentation or reduction operation is performed on it. Through multiple rounds of iteration, all subgroups tend to be balanced in the two dimensions of age and emotion category.

7. The hierarchical data augmentation method as described in claim 6, characterized in that, The formula for calculating the distribution bias of a subpopulation is as follows: in, This represents the distribution bias of the subgroup. For the recalculated proportion of the subgroup, The subscript represents the average proportion of all subgroups. Indicates age group, Indicates the emotion category.

8. A sentiment recognition method based on hierarchical data augmentation, characterized in that, include: Provided in memory an emotion recognition model generated from a training set augmented by any one of the methods of claims 1 to 7; On a processor that communicates with the memory, the image of the face to be recognized is received, and then input into the emotion recognition model to obtain the emotion recognition result.

9. An emotion recognition system based on hierarchical data augmentation, characterized in that, include: A memory containing an emotion recognition model, which is generated based on a training set augmented by the method of any one of claims 1 to 7; A processor that communicates with memory is configured to receive an image of a face to be recognized, input it into an emotion recognition model, and obtain an emotion recognition result.

Images