Facial expression recognition model training method and device, electronic device, and storage medium

Abstract

The application provides a kind of expression recognition model training method and device, electronic equipment and storage medium, belong to artificial intelligence technical field.The method comprises: obtaining the sample expression data of sample object, sample expression data includes sample face image labeled with original label, the feature extraction of sample face image is carried out by depth feature extraction network, and sample expression feature vector is obtained;Sample expression feature vector is classified and handled according to original label, and label expression feature vector is obtained;Graph generation network is used to carry out graph generation processing to label expression feature vector, and sample expression graph structure is obtained;According to sample expression graph structure, neural network model is trained, and prediction expression category feature is obtained;The model parameters of neural network model are optimized and handled by loss function and prediction expression category feature, and expression recognition model is obtained.The application can improve the training effect of model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method and apparatus for an expression recognition model, an electronic device, and a storage medium. Background Technology

[0002] The training of neural network models relied upon by current facial expression recognition methods often requires a large amount of labeled data. However, in real-world scenarios, the cost of obtaining a large amount of high-quality labeled data is high, resulting in a shortage of labeled data for model training and affecting the training effect. Therefore, how to improve the training effect of the model has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this application is to provide a training method and apparatus, electronic device and storage medium for an expression recognition model, which aims to improve the training effect of the model.

[0004] To achieve the above objectives, a first aspect of this application proposes a training method for an expression recognition model, the training method comprising:

[0005] Obtain sample facial expression data of the sample object, wherein the sample facial expression data includes sample face images labeled with original labels, the original labels being used to characterize the facial expression category of the sample object;

[0006] The sample face image is used to extract features through a preset deep feature extraction network to obtain the sample expression feature vector;

[0007] The sample facial expression feature vector is classified according to the original label to obtain the label facial expression feature vector;

[0008] The labeled expression feature vector is processed by a preset graph generation network to obtain sample graph nodes and sample edge relationships.

[0009] Based on the relationships between the sample graph nodes and the sample edges, construct the sample expression graph structure;

[0010] The preset neural network model is trained based on the sample facial expression map structure to obtain predicted facial expression category features;

[0011] The model parameters of the neural network model are optimized by using a preset loss function and the predicted expression category features to obtain an expression recognition model.

[0012] In some embodiments, the step of performing graph generation processing on the labeled expression feature vector using a preset graph generation network to obtain sample graph nodes and sample edge relationships includes:

[0013] The distance between every two of the labeled facial feature vectors is calculated using the graph generation network to obtain the sample distance value;

[0014] The graph generation network is used to perform node-based processing on each of the label expression feature vectors to obtain the sample graph nodes.

[0015] The sample edge relationship is obtained based on the sample distance value and the sample graph node.

[0016] In some embodiments, the deep feature extraction network includes convolutional layers and normalization layers. The step of extracting features from the sample face image using the preset deep feature extraction network to obtain the sample expression feature vector includes:

[0017] The sample face image is convolved by the convolutional layer to obtain an initial expression feature map;

[0018] The initial facial expression feature map is processed by the normalization layer to obtain the sample facial expression feature vector.

[0019] In some embodiments, the neural network model includes an input layer, a regularization layer, and a prediction layer. The step of training the preset neural network model based on the sample expression map structure to obtain predicted expression category features includes:

[0020] The input layer is used to extract features from the sample facial expression map structure to obtain an initial facial expression map representation vector.

[0021] The initial facial expression representation vector is regularized using the regularization layer to obtain the sample facial expression representation vector.

[0022] The predicted expression category features are obtained by performing feature prediction on the representation vector of the sample expression image through the prediction layer.

[0023] In some embodiments, the loss function includes a first function and a second function, and the step of optimizing the model parameters of the neural network model using a preset loss function and the predicted expression category features to obtain an expression recognition model includes:

[0024] The noise loss value is obtained by calculating the noise loss between the predicted expression category features and the expression category labels of the labeled expression feature vector using the first function;

[0025] The feature loss value is obtained by calculating the feature loss of the predicted expression category features using the second function;

[0026] The feature loss value and the noise loss value are weighted according to preset weight parameters to obtain the target loss value;

[0027] Backpropagation is performed based on the target loss value to optimize the model parameters of the neural network model, thereby obtaining the expression recognition model.

[0028] To achieve the above objectives, a second aspect of this application proposes an expression recognition method, the method comprising:

[0029] Acquire the target facial expression data to be identified; wherein, the target facial expression data includes the target face image of the target object;

[0030] The target face image is subjected to feature extraction by a preset deep feature extraction network to obtain a target face feature vector; wherein the deep feature extraction network is the deep feature extraction network described in the first aspect;

[0031] The target facial feature vector is input into the expression recognition model to perform expression recognition and obtain the target expression category. The target expression category is used to characterize the expression features of the target facial image data. The expression recognition model is trained according to the training method described in any one of the first aspects.

[0032] To achieve the above objectives, a third aspect of this application provides a training apparatus for an expression recognition model, the training apparatus comprising:

[0033] The sample data acquisition module is used to acquire sample expression data of sample objects, wherein the sample expression data includes sample face images labeled with original labels, and the original labels are used to characterize the expression category of the sample object;

[0034] The first feature extraction module is used to extract features from the facial expression data of the sample face image through a preset deep feature extraction network to obtain the sample expression feature vector.

[0035] The classification module is used to classify the sample expression feature vector according to the original label to obtain the label expression feature vector;

[0036] The graph generation module is used to perform graph generation processing on the expression feature vector of the label expression feature vector sample through a preset graph generation network to obtain sample graph nodes and sample edge relationships.

[0037] The graph construction module is used to construct a sample expression graph structure based on the relationship between the sample graph nodes and the sample edges;

[0038] The model training module is used to train a preset neural network model based on the sample expression map structure to obtain predicted expression category features;

[0039] The parameter optimization module is used to optimize the model parameters of the neural network model using a preset loss function and the predicted expression category features to obtain an expression recognition model.

[0040] To achieve the above objectives, a fourth aspect of this application provides an expression recognition device, the device comprising:

[0041] The target data acquisition module is used to acquire target facial expression data to be identified; wherein, the target facial expression data includes a target face image of the target object;

[0042] The second feature extraction module is used to extract features from the target face image through a preset deep feature extraction network to obtain a target face feature vector; wherein, the deep feature extraction network is the deep feature extraction network described in the third aspect;

[0043] An expression recognition module is used to input the target face feature vector into an expression recognition model to perform expression recognition and obtain a target expression category. The target expression category is used to characterize the expression features of the target face image data. The expression recognition model is trained according to the training device described in the third aspect.

[0044] To achieve the above objectives, a fifth aspect of the present application provides an electronic device, the electronic device including a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing communication between the processor and the memory, wherein the program, when executed by the processor, implements the method described in the first aspect or the method described in the second aspect.

[0045] To achieve the above objectives, a sixth aspect of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the method described in the first aspect or the method described in the second aspect.

[0046] This application proposes a training method for an expression recognition model, an expression recognition method, a training device for an expression recognition model, an expression recognition device, an electronic device, and a storage medium. It acquires sample expression data of sample objects, including sample face images labeled with original labels. The original labels represent the expression category of the sample object. A pre-defined deep feature extraction network is used to extract features from the sample face images to obtain sample expression feature vectors. This effectively extracts the required expression feature information, removes irrelevant information, and improves the quantity and quality of training data. Furthermore, the sample expression feature vectors are classified according to the original labels to obtain labeled expression feature vectors. This effectively reduces intra-class variability among training data and improves the correlation of labeled expression feature vectors belonging to the same expression category. Furthermore, a pre-defined graph generation network is used to process the labeled facial expression feature vectors to generate a graph, resulting in sample graph nodes and edge relationships. Based on these nodes and edges, a sample facial expression graph structure is constructed, which facilitates the acquisition of training data in graph form. This allows the pre-defined neural network model to be trained using the sample facial expression graph structure, enabling the model to focus on learning and classifying facial expression features and obtaining predicted facial expression category features. Finally, the model parameters of the neural network model are optimized using a pre-defined loss function and the predicted facial expression category features to obtain a facial expression recognition model. This process allows for easy model optimization, improves training performance, and results in a facial expression recognition model with high recognition accuracy. Attached Figure Description

[0047] Figure 1 This is a flowchart of the training method for the facial expression recognition model provided in the embodiments of this application;

[0048] Figure 2 yes Figure 1 The flowchart of step S102 in the document;

[0049] Figure 3 yes Figure 1 The flowchart of step S104 in the process;

[0050] Figure 4 yes Figure 1 The flowchart of step S106 in the process;

[0051] Figure 5 yes Figure 1 The flowchart of step S107 in the process;

[0052] Figure 6 This is a flowchart of the facial expression recognition method provided in the embodiments of this application;

[0053] Figure 7 This is a schematic diagram of the structure of the training device for the facial expression recognition model provided in the embodiments of this application;

[0054] Figure 8 This is a schematic diagram of the structure of the facial expression recognition device provided in the embodiments of this application;

[0055] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0059] First, let's analyze some of the terms used in this application:

[0060] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0061] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0062] Information Extraction (NER) is a text processing technique that extracts factual information such as entities, relationships, and events from natural language text and outputs it as structured data. Information extraction is a technique for extracting specific information from text data. Text data is composed of specific units, such as sentences, paragraphs, and chapters. Text information is composed of smaller, specific units, such as characters, words, phrases, sentences, paragraphs, or combinations of these units. Extracting noun phrases, names of people, and place names from text data is an example of text information extraction. Of course, text information extraction techniques can extract information of various types.

[0063] Computed Tomography (CT): It uses precisely collimated X-ray beams, gamma rays, ultrasound, etc., together with highly sensitive detectors to scan a part of the human body one section after another. It has the characteristics of fast scanning time and clear images, and can be used to examine a variety of diseases. Depending on the type of radiation used, it can be divided into X-ray CT (X-CT) and gamma-ray CT (γ-CT), etc.

[0064] Magnetic Resonance Imaging (MRI) is a physical phenomenon widely used as an analytical tool in physics, chemistry, and biology. Its application in clinical medical testing only began in 1973. To avoid confusion with radiographic imaging in nuclear medicine, it is called Magnetic Resonance Imaging (MRI). MRI works by applying radiofrequency pulses of a specific frequency to the human body in a static magnetic field, exciting hydrogen protons within the body and causing them to resonate. After the pulse stops, the protons relax, generating MR signals. These MR signals are then processed through receiving, spatial encoding, and image reconstruction.

[0065] Graph Neural Networks (GNNs) are a general term for algorithms that use neural networks to learn graph-structured data, extract and discover features and patterns in graph-structured data, and meet the needs of graph learning tasks such as clustering, classification, prediction, segmentation, and generation.

[0066] Encoder: Transforms an input sequence into a fixed-length vector.

[0067] Decoding: This involves transforming a previously generated fixed vector into an output sequence; the input sequence can be text, speech, image, or video; the output sequence can be text or image.

[0068] Convolutional Neural Networks (CNNs) are a class of deep feedforward neural networks that incorporate convolutional computations and are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform shift-invariant classification of input information according to their hierarchical structure; therefore, they are also known as Shift-Invariant Artificial Neural Networks (SIANNs). The input layers of CNNs can process multidimensional data. Commonly, the input layer of a one-dimensional CNN receives one-dimensional or two-dimensional arrays, where the one-dimensional array is typically a time or spectral sample; the two-dimensional array may contain multiple channels; the input layer of a two-dimensional CNN receives two-dimensional or three-dimensional arrays; and the input layer of a three-dimensional CNN receives four-dimensional arrays. The hidden layers of CNNs commonly include three types of architectures: convolutional layers, pooling layers, and fully connected layers. More modern algorithms may use more complex architectures such as Inception modules and residual blocks. In common architectures, convolutional layers and pooling layers are unique to convolutional neural networks. Convolutional kernels in convolutional layers contain weights, while pooling layers do not; therefore, pooling layers may not be considered independent layers in the literature. Taking LeNet-5 as an example, the order of the three common architectures in the hidden layers is usually: input - convolutional layer - pooling layer - fully connected layer - output.

[0069] Normalization is a method of simplifying calculations by transforming dimensional expressions into dimensionless expressions, i.e., scalar expressions. This method is frequently used in various calculations.

[0070] Regularization (RRegularization): In linear algebra theory, ill-posed problems are typically defined by a set of linear algebraic equations, and this set of equations often originates from ill-posed inverse problems with large condition numbers. Large condition numbers mean that rounding errors or other errors can severely affect the results.

[0071] Undirected graph: A graph whose edges have no direction is called an undirected graph.

[0072] Softmax function: The Softmax function is a normalization exponential function that can "compress" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ(z), such that each element is in the range (0,1) and the sum of all elements is 1. This function is often used in multi-class classification problems.

[0073] The training of neural network models relied upon by current facial expression recognition methods often requires a large amount of labeled data. However, in real-world scenarios, the cost of obtaining a large amount of high-quality labeled data is high, resulting in a shortage of labeled data for model training and affecting the training effect. Therefore, how to improve the training effect of the model has become an urgent technical problem to be solved.

[0074] Based on this, embodiments of this application provide a training method for an expression recognition model, an expression recognition method, a training device for an expression recognition model, an expression recognition device, an electronic device, and a storage medium, aiming to improve the training effect of the model.

[0075] The training method, apparatus, electronic device, and storage medium for the facial expression recognition model provided in this application are specifically described through the following embodiments. First, the training method for the facial expression recognition model in this application embodiment is described.

[0076] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0077] Foundational artificial intelligence technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0078] The training method for an expression recognition model provided in this application relates to the field of artificial intelligence technology. This training method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the training method for the expression recognition model, but is not limited to the above forms.

[0079] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0080] Figure 1 This is an optional flowchart of the training method for the facial expression recognition model provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0081] Step S101: Obtain sample expression data of the sample object, wherein the sample expression data includes sample face images labeled with original labels, and the original labels are used to characterize the expression category of the sample object;

[0082] Step S102: Extract features from the sample face image using a preset deep feature extraction network to obtain the sample expression feature vector;

[0083] Step S103: Classify the sample expression feature vectors according to the original labels to obtain the label expression feature vectors;

[0084] Step S104: The label expression feature vector is processed by a preset graph generation network to obtain sample graph nodes and sample edge relationships.

[0085] Step S105: Construct the sample expression graph structure based on the relationship between sample graph nodes and sample edges;

[0086] Step S106: Train the preset neural network model based on the sample expression map structure to obtain the predicted expression category features;

[0087] Step S107: Optimize the model parameters of the neural network model using a preset loss function and predicted expression category features to obtain an expression recognition model.

[0088] Steps S101 to S107 of this embodiment involve acquiring sample facial expression data of the sample objects. This data includes sample face images labeled with original tags, which characterize the expression category of the sample objects. A preset deep feature extraction network is used to extract features from the sample face images to obtain sample facial expression feature vectors. This effectively extracts the required facial expression features, removes irrelevant information, and improves the quantity and quality of the training data. Classifying the sample facial expression feature vectors according to the original tags to obtain labeled facial expression feature vectors effectively reduces intra-class variability among training data and improves the correlation of labeled facial expression feature vectors belonging to the same expression category. The labeled facial expression feature vectors are processed by a pre-defined graph generation network to obtain sample graph nodes and sample edge relationships. Based on the sample graph nodes and sample edge relationships, a sample facial expression graph structure is constructed, which can easily obtain training data in graph structure form. The pre-defined neural network model is then trained based on the sample facial expression graph structure, making the neural network model focus on learning and classifying facial expression features to obtain predicted facial expression category features. Finally, the model parameters of the neural network model are optimized through a pre-defined loss function and the predicted facial expression category features to obtain an facial expression recognition model. This allows for easy optimization of the model, improves the model's training effect, and enables the facial expression recognition model to have high recognition accuracy.

[0089] In step S101 of some embodiments, a web crawler can be written, and after setting up the data source, data can be crawled in a targeted manner to obtain sample expression data of the sample object. Sample expression data of the sample object can also be obtained through other methods, and is not limited to these. The sample expression data includes sample face images labeled with original tags. These sample face images can be three-dimensional or two-dimensional images, without limitation. The sample face images can be formed through methods such as magnetic resonance imaging. The original tags are used to characterize the expression category of the sample object.

[0090] In some specific embodiments, sample face images labeled with original tags can be obtained from publicly available facial expression datasets, such as CK+, RAF-DB, AffectNet, etc. Alternatively, actual human facial expression images can be collected and labeled according to actual business needs. The original tags include laughing, smiling, fear, fright, apprehension, anger, disgust, sadness, frustration, etc. By labeling and classifying the collected sample face images, sample facial expression data of different people can be easily obtained.

[0091] Please see Figure 2 In some embodiments, the deep feature extraction network includes convolutional layers and normalization layers, and step S102 may include, but is not limited to, steps S201 to S202:

[0092] Step S201: Perform convolution processing on the sample face image through a convolutional layer to obtain the initial expression feature map;

[0093] Step S202: The initial facial expression feature map is processed by graph normalization through a normalization layer to obtain the sample facial expression feature vector.

[0094] In step S201 of some embodiments, the deep feature extraction network can be constructed based on a unidirectional multi-layer deep convolutional network. The deep feature extraction network includes convolutional layers and normalization layers. The convolutional layers can perform convolution processing on the sample face image to extract the expression image features in the sample face image and obtain an initial expression feature map.

[0095] In step S202 of some embodiments, the initial facial expression feature map is normalized by a normalization layer. The normalization process may include first centering the initial facial expression feature map by removing the mean to achieve image standardization, and then performing maximum and minimum value normalization on the standardized initial facial expression feature map to obtain the sample facial expression feature vector.

[0096] In some other embodiments, the deep feature extraction network described above can also be built based on an encoder-decoder structure, which extracts facial expression feature information from sample face images through encoding and decoding processes to obtain sample facial expression feature vectors.

[0097] Through the above steps S201 to S202, feature extraction of sample face images can be performed relatively easily, irrelevant information in sample expression images can be eliminated, useful expression feature information can be recovered, and data can be simplified to the maximum extent, thereby improving the reliability of expression recognition.

[0098] In step S103 of some embodiments, multiple expression category labels are first set according to the label content of the original labels. Then, based on the label content of the original labels, the sample expression feature vector corresponding to each sample face image is classified and assigned to the corresponding expression category label. The sample expression feature vector is then labeled to obtain the label expression feature vector. For example, when setting expression categories based on the label content of the original labels, "laughing" and "smiling" are both classified as the "happy" expression category label, and "crying" and "frustrated" are classified as the "sad" expression category label. This method allows for the convenient setting of multiple expression category labels according to actual business needs, and the classification of the corresponding sample expression feature vectors to the corresponding expression categories based on the original labels of the sample expression images. This effectively reduces the differences between sample expression feature vectors within the same expression category and improves the applicability of the expression recognition model.

[0099] Please see Figure 3 In some embodiments, step S104 may include, but is not limited to, steps S301 to S303:

[0100] Step S301: Calculate the distance between every two label expression feature vectors using a graph generation network to obtain the sample distance value;

[0101] Step S302: The feature vector of each label expression is processed into nodes by a graph generation network to obtain sample graph nodes;

[0102] Step S303: Obtain the sample edge relationship based on the sample distance value and sample graph nodes.

[0103] In step S301 of some embodiments, when calculating the distance value between every two labeled expression feature vectors through the graph generation network, the Euclidean distance or Manhattan distance between two labeled expression feature vectors can be calculated, and the Euclidean distance or Manhattan distance can be used as the sample distance value between the two labeled expression feature vectors.

[0104] In step S302 of some embodiments, each label expression feature vector is processed into nodes by a graph generation network, and the label expression feature vector is transformed into a graph node form to obtain sample graph nodes. The position of each sample graph node can be determined according to the coordinate value of the corresponding pixel in the sample expression image of the label expression feature vector, that is, the coordinate value of the corresponding pixel in the sample expression image is used as the coordinate of the sample graph node.

[0105] In step S303 of some embodiments, every two sample graph nodes are connected by line segments, and the connected line segments are numerically marked according to the sample distance value, thereby obtaining the sample edge relationship between sample graph nodes.

[0106] In step S105 of some embodiments, the relationships between sample graph nodes and sample edges are laid out in a preset graph structure template to construct a sample expression graph structure. This sample expression graph structure can be represented as an undirected graph. The graph structure template can be set according to actual business needs without restriction. By converting the label expression feature vector into a graph structure, the uncertainty between sample data of different categories can be effectively reduced, the data quality of the training data input to the model can be improved, and thus the robustness of the model can be improved during the model training process.

[0107] Please see Figure 4 In some embodiments, step S106 may include, but is not limited to, steps S401 to S403:

[0108] Step S401: Extract features from the sample facial expression map structure through the input layer to obtain the initial facial expression map representation vector;

[0109] Step S402: The initial facial expression representation vector is regularized using a regularization layer to obtain the sample facial expression representation vector.

[0110] Step S403: The feature prediction of the sample expression image representation vector is performed by the prediction layer to obtain the predicted expression category features.

[0111] In step S401 of some embodiments, the neural network model can be a graph neural network model, which includes an input layer, a regularization layer, and a prediction layer. The input layer extracts features from the sample expression map structure to obtain expression feature information from the sample expression map structure, thus obtaining an initial expression map representation vector.

[0112] In step S402 of some embodiments, when the initial facial expression representation vector is regularized by the regularization layer, the initial facial expression representation vector H can be regularized according to a preset regularization formula to obtain the sample facial expression representation vector. The specific process is shown in formula (1):

[0113]

[0114] Where A is the sample facial expression image structure, H is the degree matrix of A, H is the initial facial expression feature vector, H1 is the sample facial expression feature vector, and W is the trainable parameter of the neural network model, the specific value of which can be set according to the requirements.

[0115] In step S403 of some embodiments, the feature probability of the sample expression image representation vector is calculated by the classification function of the prediction layer to obtain the classification probability value. The classification function can be the softmax function. The softmax function creates a probability distribution of the sample expression image representation vector on each preset expression category label. According to the probability distribution, the classification probability value of the sample expression image representation vector belonging to each preset expression category label can be intuitively reflected. The expression category label with the largest classification probability value is selected as the predicted expression category feature.

[0116] Please see Figure 5 In some embodiments, the loss function includes a first function and a second function, and step S107 may include, but is not limited to, steps S501 to S504:

[0117] Step S501: Calculate the noise loss between the predicted expression category features and the expression category labels of the label expression feature vector using the first function to obtain the noise loss value;

[0118] Step S502: Calculate the feature loss of the predicted expression category features using the second function to obtain the feature loss value;

[0119] Step S503: The feature loss value and noise loss value are weighted and calculated according to the preset weight parameters to obtain the target loss value;

[0120] Step S504: Perform backpropagation based on the target loss value to optimize the model parameters of the neural network model and obtain the expression recognition model.

[0121] In step S501 of some embodiments, in order to reduce the impact of abnormal samples on model training, when calculating the noise loss between the predicted expression category features and the expression category labels of the label expression feature vector through the first function, the cross-entropy loss function is used as the first function for calculating the noise loss, wherein the first function can be expressed as shown in formula (2):

[0122]

[0123] Among them, L rce Here, y represents the noise loss value, i is the index of the sample facial expression image, and y represents the noise loss value. i It is the prediction result, that is, the probability value corresponding to the predicted expression category feature, y' i It is the label value of the expression category label in the label expression feature vector.

[0124] In step S502 of some embodiments, in order to reduce the impact of abnormal samples on model training, when calculating the feature loss of the predicted expression category features through the second function, the traditional cross-entropy loss function is used as the second function for feature loss calculation. The second function can be expressed as shown in formula (3):

[0125]

[0126] Where M represents the number of emoji category labels, y ic Let y be a sign function (0 or 1). If the true expression category of sample expression vector i matches the expression category label, then y ic Select 1, otherwise y ic Take 0, p ic It is the prediction result, that is, the probability value corresponding to the predicted expression category feature, and N is the number of sample expression images.

[0127] In step S503 of some embodiments, the preset weight parameters can be set according to actual business needs. For example, the weight parameter of the feature loss value is α, and the weight parameter of the noise loss value is β, where α+β=1. Then, the process of weighting the feature loss value and the noise loss value according to the preset weight parameters to obtain the target loss value LOSS can be represented as shown in formula (4):

[0128] LOSS = αL ce +βL rce Formula (4)

[0129] In step S504 of some embodiments, backpropagation is performed based on the target loss value, and the internal parameters (i.e., loss parameters) of the neural network model are updated by optimizing the target loss value to obtain the expression recognition model. It is understood that the backpropagation principle can be the conventional backpropagation principle, and the embodiments of this application are not limited thereto.

[0130] The training method for the facial expression recognition model in this application involves acquiring sample facial expression data of sample objects. This data includes sample face images labeled with original tags, which characterize the facial expression category of the sample object. A preset deep feature extraction network is used to extract features from the sample face images to obtain sample facial expression feature vectors. This effectively extracts the required facial expression features, removes irrelevant information, and improves the quantity and quality of the training data. Furthermore, the sample facial expression feature vectors are classified according to the original tags to obtain labeled facial expression feature vectors. This effectively reduces intra-class variability among training data and improves the correlation between labeled facial expression feature vectors belonging to the same facial expression category. Furthermore, a pre-defined graph generation network is used to process the labeled facial expression feature vectors to generate a graph, resulting in sample graph nodes and edge relationships. Based on these nodes and edges, a sample facial expression graph structure is constructed, which facilitates the acquisition of training data in graph form. This allows the pre-defined neural network model to be trained using the sample facial expression graph structure, enabling the model to focus on learning and classifying facial expression features and obtaining predicted facial expression category features. Finally, the model parameters of the neural network model are optimized using a pre-defined loss function and the predicted facial expression category features to obtain a facial expression recognition model. This process allows for easy model optimization, improves training performance, and results in a facial expression recognition model with high recognition accuracy.

[0131] Please see Figure 6 This is an optional flowchart of the facial expression recognition method provided in the embodiments of this application. Figure 6 The method may include, but is not limited to, steps S601 to S603.

[0132] Step S601: Obtain the target facial expression data to be identified; wherein, the target facial expression data includes the target face image of the target object;

[0133] Step S602: Extract features from the target face image using a preset deep feature extraction network to obtain the target face feature vector; wherein, the deep feature extraction network is the deep feature extraction network of the first aspect embodiment;

[0134] Step S603: Input the target face feature vector into the expression recognition model to perform expression recognition and obtain the target expression category. The target expression category is used to characterize the expression features of the target face image data. The expression recognition model is trained according to the training method in the first aspect.

[0135] In step S601 of some embodiments, a web crawler can be written, and after setting up the data source, data can be crawled in a targeted manner to obtain the target expression data to be identified. Other methods can also be used to obtain the target expression data, and are not limited to these. The target expression data includes a target face image of the target object, which can be a three-dimensional image or a two-dimensional image, without limitation.

[0136] In step S602 of some embodiments, the deep feature extraction network can be constructed based on a unidirectional multi-layer deep convolutional network. The deep feature extraction network includes convolutional layers and normalization layers. The convolutional layers perform convolution processing on the target face image to extract expression image features from the target face image, obtaining a target face feature map. The normalization layers perform normalization processing on the target face feature map. This normalization process may include first centering the target face feature map by removing the mean to achieve image standardization, and then performing maximum and minimum value normalization on the standardized target face feature map to obtain the target face feature vector.

[0137] In step S603 of some embodiments, the target face feature vector is input into the expression recognition model, the expression recognition model extracts the target face feature vector to obtain the target expression feature vector, and then the classification function in the expression recognition model is used to calculate the classification probability of the target expression feature vector on each preset expression category label to obtain the expression classification probability value of the target expression feature vector on each preset expression category label. The expression category label with the largest expression classification probability value is taken as the target expression category, and the expression feature of the target face image data is represented by the target expression category.

[0138] The facial expression recognition method of this application embodiment acquires target facial expression data to be recognized; wherein, the target facial expression data includes a target face image of the target object; features are extracted from the target face image using a preset deep feature extraction network to obtain a target face feature vector, which can effectively extract the facial expression feature information that meets the requirements and remove irrelevant information. Finally, the target face feature vector is input into an facial expression recognition model for facial expression recognition. The facial expression recognition model extracts and processes important facial expression information from the target face feature vector to obtain a target facial expression category. This target facial expression category can clearly represent the facial expression features of the target face image data, improving the accuracy and efficiency of facial expression recognition.

[0139] Please see Figure 7 This application also provides a training device for an expression recognition model, which can implement the above-mentioned training method for the expression recognition model. The device includes:

[0140] The sample data acquisition module 701 is used to acquire sample expression data of sample objects, wherein the sample expression data includes sample face images labeled with original labels, and the original labels are used to characterize the expression category of the sample objects;

[0141] The first feature extraction module 702 is used to extract features from the sample facial image sample expression data through a preset deep feature extraction network to obtain the sample expression feature vector;

[0142] The classification module 703 is used to classify the sample expression feature vectors according to the original labels to obtain the label expression feature vectors;

[0143] The graph generation module 704 is used to perform graph generation processing on the expression feature vector of the label expression feature vector sample through a preset graph generation network to obtain sample graph nodes and sample edge relationships.

[0144] Graph construction module 705 is used to construct the sample expression graph structure based on the relationship between sample graph nodes and sample edges;

[0145] The model training module 706 is used to train a preset neural network model based on the structure of the sample expression map to obtain the predicted expression category features;

[0146] The parameter optimization module 707 is used to optimize the model parameters of the neural network model through a preset loss function and predicted expression category features to obtain an expression recognition model.

[0147] In some embodiments, the deep feature extraction network includes convolutional layers and normalization layers, and the first feature extraction module 702 includes:

[0148] Convolutional units are used to perform convolution processing on sample face images through convolutional layers to obtain initial expression feature maps;

[0149] The normalization unit is used to perform graph normalization on the initial facial expression feature map through the normalization layer to obtain the sample facial expression feature vector.

[0150] In some embodiments, the graph generation module 704 includes:

[0151] The distance calculation unit is used to calculate the distance between every two labeled facial feature vectors through the graph generation network to obtain the sample distance value;

[0152] The node-based unit is used to process each label expression feature vector into nodes through the graph generation network to obtain sample graph nodes.

[0153] The relation building unit is used to obtain the sample edge relations based on the sample distance values ​​and sample graph nodes.

[0154] In some embodiments, the neural network model includes an input layer, a regularization layer, and a prediction layer, and the model training module 706 includes:

[0155] The feature extraction unit is used to extract features from the sample facial expression map structure through the input layer to obtain the initial facial expression map representation vector;

[0156] The regularization unit is used to regularize the initial facial expression representation vector through the regularization layer to obtain the sample facial expression representation vector.

[0157] The feature prediction unit is used to predict the features of the sample expression map representation vector through the prediction layer to obtain the predicted expression category features.

[0158] In some embodiments, the loss function includes a first function and a second function, and the parameter optimization module 707 includes:

[0159] The noise loss calculation unit is used to calculate the noise loss between the predicted expression category features and the expression category labels of the label expression feature vector through the first function, and obtain the noise loss value.

[0160] The feature loss calculation unit is used to calculate the feature loss of the predicted expression category features through the second function to obtain the feature loss value;

[0161] The weighted calculation unit is used to perform weighted calculation on the feature loss value and the noise loss value according to the preset weight parameters to obtain the target loss value;

[0162] The optimization unit is used to backpropagate based on the target loss value to optimize the model parameters of the neural network model and obtain the expression recognition model.

[0163] The specific implementation of the training device for the facial expression recognition model is basically the same as the specific implementation of the training method for the facial expression recognition model described above, and will not be repeated here.

[0164] Please see Figure 8 This application also provides an expression recognition device that can implement the above-described expression recognition method. The device includes:

[0165] The target data acquisition module 801 is used to acquire the target facial expression data to be identified; wherein, the target facial expression data includes the target face image of the target object;

[0166] The second feature extraction module 802 is used to extract features from the target face image through a preset deep feature extraction network to obtain the target face feature vector; wherein, the deep feature extraction network is based on the aforementioned deep feature extraction network;

[0167] The expression recognition module 803 is used to input the target face feature vector into the expression recognition model to perform expression recognition and obtain the target expression category. The target expression category is used to characterize the expression features of the target face image data. The expression recognition model is trained according to the training device mentioned above.

[0168] The specific implementation of this facial expression recognition device is basically the same as the specific implementation of the facial expression recognition method described above, and will not be repeated here.

[0169] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the training method or expression recognition method of the aforementioned expression recognition model. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0170] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0171] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0172] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the training method or expression recognition method of the expression recognition model in the embodiments of this application.

[0173] The input / output interface 903 is used to implement information input and output;

[0174] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0175] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0176] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0177] This application embodiment also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described training method or expression recognition method for the expression recognition model.

[0178] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0179] The present application provides a training method for an expression recognition model, an expression recognition method, an expression recognition model training device, an expression recognition device, an electronic device, and a storage medium. These methods acquire sample expression data of sample objects, including sample face images labeled with original tags. The original tags characterize the expression category of the sample object. A preset deep feature extraction network is used to extract features from the sample face images to obtain sample expression feature vectors. This effectively extracts the required expression feature information, removes irrelevant information, and improves the quantity and quality of training data. Furthermore, the sample expression feature vectors are classified according to the original tags to obtain labeled expression feature vectors. This effectively reduces intra-class variability among training data and improves the correlation between labeled expression feature vectors belonging to the same expression category. Furthermore, a pre-defined graph generation network is used to process the labeled facial expression feature vectors to generate a graph, resulting in sample graph nodes and edge relationships. Based on these nodes and edges, a sample facial expression graph structure is constructed, which facilitates the acquisition of training data in graph form. This allows the pre-defined neural network model to be trained using the sample facial expression graph structure, enabling the model to focus on learning and classifying facial expression features and obtaining predicted facial expression category features. Finally, the model parameters of the neural network model are optimized using a pre-defined loss function and the predicted facial expression category features to obtain a facial expression recognition model. This process allows for easy model optimization, improves training performance, and results in a facial expression recognition model with high recognition accuracy.

[0180] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0181] It will be understood by those skilled in the art that Figure 1-6 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0184] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0185] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0187] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A training method for an facial expression recognition model, characterized in that, The training method includes: Obtain sample facial expression data of the sample object, wherein the sample facial expression data includes sample face images labeled with original labels, the original labels being used to characterize the facial expression category of the sample object; The sample face image is used to extract features through a preset deep feature extraction network to obtain the sample expression feature vector; The sample facial expression feature vector is classified according to the original label to obtain the label facial expression feature vector; The labeled expression feature vector is processed by a preset graph generation network to obtain sample graph nodes and sample edge relationships. Based on the relationships between the sample graph nodes and the sample edges, construct the sample expression graph structure; The preset neural network model is trained based on the sample facial expression map structure to obtain predicted facial expression category features; The model parameters of the neural network model are optimized by using a preset loss function and the predicted expression category features to obtain an expression recognition model.

2. The training method according to claim 1, characterized in that, The step of performing graph generation processing on the labeled expression feature vector using a preset graph generation network to obtain sample graph nodes and sample edge relationships includes: The distance between every two of the labeled facial feature vectors is calculated using the graph generation network to obtain the sample distance value; The graph generation network is used to perform node-based processing on each of the label expression feature vectors to obtain the sample graph nodes. The sample edge relationship is obtained based on the sample distance value and the sample graph node.

3. The training method according to claim 1, characterized in that, The deep feature extraction network includes convolutional layers and normalization layers. The step of extracting features from the sample face image using the preset deep feature extraction network to obtain the sample expression feature vector includes: The sample face image is convolved by the convolutional layer to obtain an initial expression feature map; The initial facial expression feature map is processed by the normalization layer to obtain the sample facial expression feature vector.

4. The training method according to claim 1, characterized in that, The neural network model includes an input layer, a regularization layer, and a prediction layer. The step of training the preset neural network model based on the sample expression map structure to obtain the predicted expression category features includes: The input layer is used to extract features from the sample facial expression map structure to obtain an initial facial expression map representation vector. The initial facial expression representation vector is regularized using the regularization layer to obtain the sample facial expression representation vector. The predicted expression category features are obtained by performing feature prediction on the representation vector of the sample expression image through the prediction layer.

5. The training method according to any one of claims 1 to 4, characterized in that, The loss function includes a first function and a second function. The step of optimizing the model parameters of the neural network model using the preset loss function and the predicted expression category features to obtain the expression recognition model includes: The noise loss value is obtained by calculating the noise loss between the predicted expression category features and the expression category labels of the labeled expression feature vector using the first function; The feature loss value is obtained by calculating the feature loss of the predicted expression category features using the second function; The feature loss value and the noise loss value are weighted according to preset weight parameters to obtain the target loss value; Backpropagation is performed based on the target loss value to optimize the model parameters of the neural network model, thereby obtaining the expression recognition model.

6. A facial expression recognition method, characterized in that, The facial expression recognition method includes: Acquire the target facial expression data to be identified; wherein, the target facial expression data includes the target face image of the target object; The target face image is subjected to feature extraction by a preset deep feature extraction network to obtain a target face feature vector; wherein the deep feature extraction network is the deep feature extraction network according to any one of claims 1 to 5; The target facial feature vector is input into the expression recognition model to perform expression recognition and obtain the target expression category. The target expression category is used to characterize the expression features of the target facial image data. The expression recognition model is trained according to the training method of any one of claims 1 to 5.

7. A training device for an expression recognition model, characterized in that, The training device includes: The sample data acquisition module is used to acquire sample expression data of sample objects, wherein the sample expression data includes sample face images labeled with original labels, and the original labels are used to characterize the expression category of the sample object; The first feature extraction module is used to extract features from the facial expression data of the sample face image through a preset deep feature extraction network to obtain the sample expression feature vector. The classification module is used to classify the sample expression feature vector according to the original label to obtain the label expression feature vector; The graph generation module is used to perform graph generation processing on the expression feature vector of the label expression feature vector sample through a preset graph generation network to obtain sample graph nodes and sample edge relationships. The graph construction module is used to construct a sample expression graph structure based on the relationship between the sample graph nodes and the sample edges; The model training module is used to train a preset neural network model based on the sample expression map structure to obtain predicted expression category features; The parameter optimization module is used to optimize the model parameters of the neural network model using a preset loss function and the predicted expression category features to obtain an expression recognition model.

8. An expression recognition device, characterized in that, The facial expression recognition device includes: The target data acquisition module is used to acquire target facial expression data to be identified; wherein, the target facial expression data includes a target face image of the target object; The second feature extraction module is used to extract features from the target face image through a preset deep feature extraction network to obtain a target face feature vector; wherein, the deep feature extraction network is the deep feature extraction network according to claim 7; An expression recognition module is used to input the target face feature vector into an expression recognition model to perform expression recognition and obtain a target expression category. The target expression category is used to characterize the expression features of the target face image data. The expression recognition model is trained by the training device according to claim 7.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for establishing communication between the processor and the memory. When the program is executed by the processor, it implements the steps of the training method as described in any one of claims 1 to 5 or the steps of the facial expression recognition method as described in claim 6.

10. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the training method as described in any one of claims 1 to 5 or the steps of the facial expression recognition method as described in claim 6.

Images