A micro-expression recognition method based on domain adaptation

Through domain adaptation-based methods and adversarial networks, ordinary expression samples are used to assist micro-expression recognition and extract strong features, the problem of insufficient robustness of the micro-expression recognition model is solved, the recognition accuracy and robustness are improved, and the simplicity is achieved Efficient training process.

CN114998973BActive Publication Date: 2024-11-05NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210767992.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-30
Publication Date
2024-11-05
Estimated Expiration
2042-06-30

AI Technical Summary

Technical Problem

In the prior art, the micro-expression recognition model is difficult to train a robust deep learning model due to the small training sample size. Due to the short duration, small change amplitude and weak intensity, the recognition accuracy and robustness are insufficient.

Method used

Using domain adaptive micro-expression recognition method, ordinary expression samples assist in micro-expression recognition, and by constructing an adversarial network containing feature learning modules, classifiers and domain discriminators, the feature distribution of ordinary expressions and micro-expression samples through adversarial learning The difference is minimized, expression features with strong discrimination are extracted, and attention units are used to focus on areas with large muscle movements.

Benefits of technology

It improves the accuracy and robustness of micro-expression recognition, solves the problem of insufficient training samples, realizes end-to-end simple and efficient training, and enhances the model's discrimination ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114998973B_ABST
    Figure CN114998973B_ABST
Patent Text Reader

Abstract

The present invention discloses a micro-expression recognition method based on domain adaptation, which comprises the following steps: taking the ordinary expression domain as the source domain and the micro-expression domain as the target domain, pre-processing the ordinary expression and micro-expression images, and establishing the source domain and target domain sample sets; constructing an adversarial network including a feature learning module, a classifier and a domain discriminator; in each batch of training of the adversarial network, training the feature learning module, the classifier and the domain discriminator with source domain samples, and training the feature learning module and the domain discriminator with target domain samples, minimizing the feature distribution difference between the source domain and the target domain samples through adversarial learning, and realizing domain adaptation; constructing a micro-expression recognition model using the trained feature learning module and classifier, and performing micro-expression recognition on the input image. The present invention utilizes existing ordinary expression samples to assist micro-expression recognition, so as to solve the problem of insufficient number of micro-expression training samples and improve the accuracy and robustness of micro-expression recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a micro-expression recognition method based on domain adaptation, and belongs to the field of micro-expression recognition and artificial intelligence. Background Art

[0002] In the process of daily interpersonal communication, changes in human facial expressions can express a variety of emotional information and emotional states, which are closely related to human psychological states, intentions and purposes. "Micro-expression" is a spontaneous expression that lasts only 0.04 to 0.20 seconds. Unlike general facial expressions, micro-expressions are a stress response controlled by the autonomic nervous system. They are often revealed when people try to hide their true emotions. They express a person's true inner emotions that they try to suppress or hide, which can neither be disguised nor suppressed. Therefore, micro-expressions can be used as important clues for testing lies and behavioral analysis, and have potential application prospects in the fields of lie detection, clinical diagnosis of psychological diseases such as depression and schizophrenia, and investigative interrogation.

[0003] Since its discovery, micro-expressions have attracted the attention of a large number of researchers. With the development of deep learning technology, deep neural networks have been applied in tasks such as expression recognition. Researchers have also tried to use deep neural networks for micro-expression recognition. China's patent application "A method for micro-expression recognition based on 3D convolutional neural network" (patent application number CN201610954555.9, publication number CN106570474A) extracts grayscale channel feature maps, horizontal gradient channel feature maps, vertical gradient channel feature maps, horizontal optical flow channel feature maps, and vertical optical flow channel feature maps for each frame image in the micro-expression video sequence to obtain a feature map group corresponding to the micro-expression video sequence to be identified, and then inputs it into the 3D convolutional neural network for further feature extraction and classification. This method processes every frame image of the micro-expression video sequence, and the amount of calculation is extremely large. In addition, due to the lack of sufficient training samples, it is easy to have the problem of model overfitting, which affects the robustness and generalization performance of the model.

[0004] Compared with ordinary expressions, micro-expressions are very short in duration, small in amplitude, and weak in intensity. It is difficult to collect and annotate micro-expression samples, and obtaining a large number of samples with category labels requires a huge annotation cost. Because the category labels of micro-expressions not only need to be completed by well-trained psychologists with rich professional experience, but also take time and effort. Therefore, it is impossible to train a robust deep learning model directly using the existing small-scale micro-expression sample library. Summary of the invention

[0005] Purpose of the invention: In view of the problem that the scale of micro-expression training samples is small and it is difficult to train a robust deep learning model, the present invention proposes a micro-expression recognition method based on domain adaptation. The method uses existing ordinary expression samples to assist in micro-expression recognition, which can solve the problem of insufficient number of micro-expression training samples and improve the accuracy and robustness of micro-expression recognition. In order to extract highly discriminative expression features, the present invention proposes an attention unit, which enables the feature learning module to focus on areas with larger muscle movement amplitudes during the feature extraction process, while suppressing the adverse effects of irrelevant areas on expression recognition.

[0006] Technical solution: To achieve the above-mentioned purpose, the present invention adopts the following technical solution: a micro-expression recognition method based on domain adaptation, comprising the following steps:

[0007] Step 1: Take the normal expression domain as the source domain and the micro-expression domain as the target domain, pre-process the normal expression and micro-expression images, and establish the source domain and target domain sample sets;

[0008] Step 2: Build an adversarial network including feature learning module, classifier and domain discriminator;

[0009] Step 3: In each batch of adversarial network training, source domain samples are used to train the feature learning module, classifier, and domain discriminator, and target domain samples are used to train the feature learning module and domain discriminator. Through adversarial learning, the difference in feature distribution between source domain and target domain samples is minimized to achieve domain adaptation.

[0010] Step 4: Use the trained feature learning module and classifier to build a micro-expression recognition model and perform micro-expression recognition on the input image.

[0011] Furthermore, the adversarial network described in step 2 is mainly composed of a feature learning module, a domain discriminator and a classifier;

[0012] The feature learning module is composed of a feature extraction unit, a feature division unit, an average pooling layer, an attention unit and a feature fusion layer; the feature extraction unit is composed of o convolution modules, each of which contains two convolution layers and a pooling layer, where o is selected from 3, 4, and 5; the feature division unit is used to convert the feature tensor output by the feature extraction unit into Each channel feature map in the h dimension is divided into three equal parts to obtain the feature tensor corresponding to the eye area in the face image. Feature tensor of the nose region and the feature tensor of the mouth area Where c, h and w represent the number of channels, height and width of the feature tensor G respectively; the average pooling layer is used to reduce the feature map of each channel in the feature tensor G, L1, L2 and L3 into an eigenvalue, and obtain the feature vector F respectively.G , F1, F2 and F3; the attention unit consists of a fully connected layer and a softmax layer, which is used to calculate the contribution of feature vectors F1, F2 and F3 to the classification result, and weight F1, F2 and F3 to obtain feature vectors F L1 、F L2 and F L3 ; The feature fusion layer is used to transform the feature vector F G With F L1 、F L2 and F L3 To splice;

[0013] The domain discriminator, which consists of two fully connected layers and a softmax layer, is used to discriminate whether the sample input into the feature learning module is a source domain sample or a target domain sample;

[0014] The classifier, which consists of three fully connected layers and one softmax layer, is used to classify expressions of samples input into the feature learning module.

[0015] Furthermore, the specific steps of the attention unit weighting the feature vectors F1, F2 and F3 are:

[0016] First, the feature vectors F1, F2, and F3 are input into the fully connected layer of the attention unit in sequence, and the feature vectors F1 and F3 are obtained respectively. ′ 、F2 ′ and F3 ′ , U is the number of expression categories;

[0017] Then, according to the eigenvector F i ′ , calculate the eigenvector F i The weight Z i :

[0018]

[0019] in is the eigenvector F i ′ The eigenvalue of the θth dimension in , θ is the expression category label of the input sample, θ∈{1,..,U}, i=1,2,3;

[0020] Finally, the eigenvectors F1, F2, and F3 are multiplied by the corresponding weights Z1, Z2, and Z3 to obtain the eigenvectors F L1 、F L2 and F L3 .

[0021] Furthermore, the step (3) comprises the following sub-steps:

[0022] (3.1) Use source domain samples to train feature learning modules, domain discriminators, and classifiers;

[0023] Input the sample into the feature learning module to obtain the feature vector The feature vector f s Input to the classifier, and use the gradient descent algorithm to minimize the cross entropy loss of the classifier, so that the classifier can accurately classify the expression of the sample;

[0024] The feature vector f s Input to the domain discriminator to obtain the probability P that the sample comes from the source domain s (f s ), according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L d The direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized, which makes the training goal of the feature learning module opposite to that of the domain discriminator, forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module can extract the shared features of the source domain and the target domain samples.

[0025] (3.2) Use target domain samples to train feature learning module and domain discriminator;

[0026] Input the sample into the feature learning module to obtain the feature vector The feature vector f t Input to the domain discriminator to obtain the probability P that the sample comes from the target domain t (f t ), and according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L d The direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized, which makes the training goal of the feature learning module opposite to that of the domain discriminator, forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module can extract the shared features of the source domain and the target domain samples.

[0027] The loss function L of the domain discriminator in steps (3.1) and (3.2) is d is the cross entropy loss function, defined as:

[0028] L d=-dlog(P(f))-(1-d)log(1-P(f))

[0029] In the above formula, f is the feature vector input to the domain discriminator, P(f) is the probability value output by the domain discriminator, d is the binary domain label, and when the sample is a source domain sample, d = 1, f∈f s , P(f)∈P s (f s ), when the sample is a target domain sample, d=0, f∈f t , P(f)∈P t (f t );

[0030] (3.3) In each batch of training of the adversarial network, steps (3.1) and (3.2) are repeated. Through V batches of iterative training, the difference in feature distribution between source domain samples and target samples is minimized to achieve domain adaptation, where V is selected from the values ​​of 50, 100, and 150.

[0031] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0032] (1) The present invention discloses a micro-expression recognition method based on domain adaptation. The method utilizes existing common expression samples to assist in the recognition of micro-expressions, which can solve the problem of insufficient number of micro-expression training samples and improve the accuracy and robustness of micro-expression recognition.

[0033] (2) The present invention proposes an adversarial network including a feature learning module, a classifier and a domain discriminator. In the training of each batch of the adversarial network, the present invention uses ordinary expression samples to train the feature learning module, the classifier and the domain discriminator, and uses micro-expression samples to train the feature learning module and the domain discriminator. Through adversarial learning, the difference in feature distribution between ordinary expression and micro-expression samples is minimized, so that the knowledge learned from ordinary expression samples can be used to assist micro-expression recognition, thereby improving the accuracy and robustness of micro-expression recognition.

[0034] (3) The attention unit proposed in the present invention enables the feature learning module to focus on the areas with larger muscle movement amplitudes in the face image, namely the eyes, nose and mouth areas, during the feature extraction process, while suppressing the adverse effects of irrelevant areas on expression recognition, so that the feature learning module can extract highly discriminative expression features and improve the accuracy of micro-expression recognition.

[0035] (4) The micro-expression recognition method based on domain adaptation proposed in the present invention can achieve end-to-end training without the need for cumbersome multi-stage training, which is simpler and more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of a micro-expression recognition method based on domain adaptation designed by the present invention;

[0037] Figure 2 Schematic diagram of using source domain samples to train feature learning modules, classifiers, and domain discriminators;

[0038] Figure 3 It is a schematic diagram of training the feature learning module and domain discriminator with target domain samples;

[0039] Figure 4 It is a micro-expression recognition model based on domain adaptation designed by the present invention. DETAILED DESCRIPTION

[0040] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0041] The present invention designs a micro-expression recognition method based on domain adaptation, which specifically includes the following steps in practical application:

[0042] Step 1: Take the normal expression domain as the source domain and the micro-expression domain as the target domain, pre-process the normal expression and micro-expression images, and establish the source domain and target domain sample sets.

[0043] Firstly, the CASME II database is used as the source of micro-expression images to extract the peak frames in each facial image sequence, and the CK+ database is used as the source of normal expression images to extract the peak frame images in each facial image sequence;

[0044] Then, the extracted peak frame images are preprocessed, wherein the preprocessing process includes: face detection, face alignment and face region cropping for each image, and the processed images are resized and fixed to a size of 192×192;

[0045] Finally, the preprocessed micro-expression peak frame images are motion amplified using the Euler motion amplification algorithm with a magnification factor of 5, 6, 7, 8, and 9 to amplify slight changes in the face and expand the sample size to 5 times the original size.

[0046] Step 2: Build an adversarial network consisting of a feature learning module, a classifier, and a domain discriminator.

[0047] The feature learning module is composed of a feature extraction unit, a feature division unit, an average pooling layer, an attention unit and a feature fusion layer; the feature extraction unit is composed of o convolution modules, each of which contains two convolution layers and a pooling layer, where o is selected from 3, 4, and 5; the feature division unit is used to convert the feature tensor output by the feature extraction unit into Each channel feature map in the h dimension is divided into three equal parts to obtain the feature tensor corresponding to the eye area in the face image. Feature tensor of the nose region and the feature tensor of the mouth area Where c, h and w represent the number of channels, height and width of the feature tensor G respectively; the average pooling layer is used to reduce the feature map of each channel in the feature tensor G, L1, L2 and L3 into an eigenvalue, and obtain the feature vector F respectively. G , F1, F2 and F3; the attention unit consists of a fully connected layer and a softmax layer, which is used to calculate the contribution of feature vectors F1, F2 and F3 to the classification result, and weight F1, F2 and F3 to obtain feature vectors F L1 、F L2 and F L3 ; The feature fusion layer is used to transform the feature vector F G With F L1 、F L2 and F L3 To splice;

[0048] The domain discriminator, which consists of two fully connected layers and a softmax layer, is used to discriminate whether the sample input into the feature learning module is a source domain sample or a target domain sample;

[0049] The classifier, which consists of three fully connected layers and one softmax layer, is used to classify expressions of samples input into the feature learning module.

[0050] Furthermore, the specific steps of the attention unit weighting the feature vectors F1, F2 and F3 are:

[0051] First, the feature vectors F1, F2, and F3 are input into the fully connected layer of the attention unit in sequence to obtain the feature vectors F1′, F2′, and F3′, respectively. U is the number of expression categories;

[0052] Then, according to the eigenvector F i ′, calculate the eigenvector F i The weight Z i :

[0053]

[0054] in is the eigenvector F i ′, where θ is the expression category label of the input sample, θ∈{1,..,U}, i=1,2,3;

[0055] Finally, the eigenvectors F1, F2, and F3 are multiplied by the corresponding weights Z1, Z2, and Z3 to obtain the eigenvectors F L1 、F L2 and F L3 .

[0056] Step 3: In each batch of training of the adversarial network, the source domain samples are used to train the feature learning module, classifier and domain discriminator, and the target domain samples are used to train the feature learning module and domain discriminator. Through adversarial learning, the difference in feature distribution between source domain and target domain samples is minimized to achieve domain adaptation. The specific steps include the following:

[0057] (3.1) Use source domain samples to train feature learning modules, domain discriminators, and classifiers;

[0058] Input the sample into the feature learning module to obtain the feature vector The feature vector f s Input to the classifier, and use the gradient descent algorithm to minimize the cross entropy loss of the classifier, so that the classifier can accurately classify the expression of the sample;

[0059] The feature vector f s Input to the domain discriminator to obtain the probability P that the sample comes from the source domain s (f s ), according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L d The direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized, which makes the training goal of the feature learning module opposite to that of the domain discriminator, forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module can extract the shared features of the source domain and the target domain samples.

[0060] (3.2) Use target domain samples to train feature learning module and domain discriminator;

[0061] Input the sample into the feature learning module to obtain the feature vector The feature vector f t Input to the domain discriminator to obtain the probability P that the sample comes from the target domain t (f t ), and according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L dThe direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized, which makes the training goal of the feature learning module opposite to that of the domain discriminator, forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module can extract the shared features of the source domain and the target domain samples.

[0062] The loss function L of the domain discriminator in steps (3.1) and (3.2) is d is the cross entropy loss function, defined as:

[0063] L d =-dlog(P(f))-(1-d)log(1-P(f))

[0064] In the above formula, f is the feature vector input to the domain discriminator, P(f) is the probability value output by the domain discriminator, d is the binary domain label, and when the sample is a source domain sample, d = 1, f∈f s , P(f)∈P s (f s ), when the sample is a target domain sample, d=0, f∈f t , P(f)∈P t (f t );

[0065] (3.3) In each batch of training of the adversarial network, steps (3.1) and (3.2) are repeated. Through 50 batches of iterative training, the difference in feature distribution between source domain samples and target samples is minimized to achieve domain adaptation.

[0066] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Some changes that may be made to certain parts thereof by technicians in this technical field all reflect the principles of the present invention and fall within the protection scope of the present invention.

Claims

1. A micro-expression recognition method based on domain adaptation, It is characterized in that The method comprises the following steps: S1: Take the normal expression domain as the source domain and the micro-expression domain as the target domain, pre-process the normal expression and micro-expression images, and establish the source domain and target domain sample sets; S2: Build an adversarial network including feature learning module, classifier and domain discriminator; S3: In each batch of adversarial network training, source domain samples are used to train the feature learning module, classifier, and domain discriminator, and target domain samples are used to train the feature learning module and domain discriminator. Through adversarial learning, the difference in feature distribution between source domain and target domain samples is minimized to achieve domain adaptation. S4: Use the trained feature learning module and classifier to build a micro-expression recognition model and perform micro-expression recognition on the input image; The step S3 comprises the following sub-steps: (3.1) Use source domain samples to train feature learning modules, domain discriminators, and classifiers; Input the source domain sample into the feature learning module to obtain the feature vector The feature vector f s Input to the classifier, and use the gradient descent algorithm to minimize the cross entropy loss of the classifier, so that the classifier can classify the expression of the sample; The feature vector f s Input to the domain discriminator to obtain the probability P that the sample comes from the source domain s (f s ), according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L d The direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized so that the training objective of the feature learning module is opposite to that of the domain discriminator, thus forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module extracts shared features of the source domain and target domain samples. (3.2) Use target domain samples to train feature learning module and domain discriminator; Input the target domain sample into the feature learning module to obtain the feature vector The feature vector f t Input to the domain discriminator to obtain the probability P that the sample comes from the target domain t (f t ), according to the loss function L of the domain discriminator d Perform gradient calculation and use the gradient descent algorithm to minimize the weight parameters of the domain discriminator L d The direction of the loss is optimized; before the gradient is back-propagated to the feature learning module, the gradient reversal layer is used to reverse the gradient so that the weight parameters of the feature learning module are maximized. d The direction of the loss is optimized so that the training goal of the feature learning module is opposite to that of the domain discriminator, forming a confrontation. Through the confrontation between the feature learning module and the domain discriminator, the feature learning module can extract the shared features of the source domain and the target domain samples. Among them, the loss function L of the domain discriminator in steps (3.1) and (3.2) is d is the cross entropy loss function, defined as: L d =-dlog(P(f))-(1-d)log(1-P(f)) Among them, f is the feature vector input to the domain discriminator, P(f) is the probability value output by the domain discriminator, d is the binary classification domain label, when the sample is a source domain sample, d = 1, f∈f s , P(f)∈P s (f s ), when the sample is a target domain sample, d=0, f∈f t , P(f)∈P t (f t ); (3.3) In each batch of training of the adversarial network, steps (3.1) and (3.2) are repeated. Through V batches of iterative training, the difference in feature distribution between source domain samples and target samples is minimized to achieve domain adaptation.

2. The micro-expression recognition method based on domain adaptation according to claim 1, It is characterized in that The adversarial network described in step S2 is composed of a feature learning module, a domain discriminator and a classifier; The feature learning module is composed of a feature extraction unit, a feature division unit, an average pooling layer, an attention unit and a feature fusion layer; the feature extraction unit is composed of o convolution modules, each of which contains two convolution layers and a pooling layer; the feature division unit is used to convert the feature tensor output by the feature extraction unit Each channel feature map in the h dimension is divided into three equal parts to obtain the feature tensor corresponding to the eye area in the face image. Feature tensor of the nose region Feature tensor of the mouth region Among them, c, h and w represent the number of channels, height and width of the feature tensor G respectively; the average pooling layer is used to transform the feature tensors G, L 1 , L 2 and L 3 The feature map of each channel in is reduced to an eigenvalue, and the eigenvectors F are obtained respectively. G 、F 1 、F 2 and F 3 ; The attention unit consists of a fully connected layer and a softmax layer to calculate the feature vector F 1 、F 2 and F 3 The contribution to the classification results, F 1 、F 2 and F 3 After weighting, we can get the eigenvectors F L1 、F L2 and F L3 ; The feature fusion layer is used to transform the feature vector F G With F L1 、F L2 and F L3 To splice; The domain discriminator, which consists of two fully connected layers and a softmax layer, is used to discriminate whether the sample input into the feature learning module is a source domain sample or a target domain sample; The classifier, which consists of three fully connected layers and one softmax layer, is used to classify expressions of samples input into the feature learning module.

3. The micro-expression recognition method based on domain adaptation according to claim 2, It is characterized in that in, The value of o is selected from 3, 4, and 5.

4. The micro-expression recognition method based on domain adaptation according to claim 2, It is characterized in that The attention unit is concerned with the feature vector F 1 、F 2 and F 3 The specific steps for weighting are: First, the eigenvector F 1 、F 2 and F 3 Input into the fully connected layer of the attention unit in sequence, and obtain the feature vector F 1 ′、F 2 ′ and F 3 ′, i=1, 2, 3, U is the number of expression categories; according to the feature vector F i ′, calculate the eigenvector F i The weight Z i : in, is the eigenvector F i ′, where θ is the expression category label of the input sample, θ∈{1, .., U}, i=1, 2, 3; The feature vector F 1 、F 2 and F 3 And the corresponding weight Z 1 , Z 2 and Z 3 Multiply them to get the eigenvector F L1 、F L2 and F L3 .

5. The micro-expression recognition method based on domain adaptation according to claim 1, It is characterized in that The V is selected from the values ​​of 50, 100, and 150.