A humanoid robot facial friendliness evaluation and design method

Abstract

The present invention discloses a humanoid robot face friendliness assessment and design method. The method comprises: (1) simulating and generating a humanoid robot face based on different facial features; (2) collecting EEG signals and eye movement signals generated by participants in the friendliness assessment experiment and recording the participants' preference feedback; (3) combining the participants' preference feedback, performing a preference assessment on the collected EEG signals and eye movement signals, and obtaining a preference ranking based on the EEG signals and eye movement signals; (4) obtaining a facial feature preference ranking based on the preference ranking results of the EEG signals and eye movement signals, and then selecting friendly facial features to form a humanoid robot face. Based on the method of the present invention, a humanoid robot face that is preferred by users can be designed, so that users feel comfortable and do not feel resistance during human-machine interaction, thereby obtaining a good user experience and promoting the popularization of humanoid robots.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence, brain-computer interface, product design, human-computer interaction and emotional computing, and involves the emotional computing problem of human-computer interaction. Specifically, it is a method for evaluating and designing the friendliness of a humanoid robot face through EEG and eye movement signals. Background Art

[0002] With the development of robotics and artificial intelligence, humanoid robots have become increasingly popular in entertainment, service, and healthcare fields, and are a hot area of intelligent robotics research. However, this development has also been accompanied by some challenges. When interacting with a humanoid robot, the robot's facial contours significantly impact the user experience. If the robot's face is highly realistic, exceedingly close to a human face, this can increase user resistance. Recent research suggests that the more human-like a humanoid robot is, the better. Humanoid robots may even affect human decision-making. Conversely, if the face's realism is low, the user experience can be compromised, creating a false impression and negatively impacting the emotional aspects of the human-robot interaction.

[0003] However, current research on product appearance design is mostly based on a single modality. Few studies have utilized EEG and eye movement signals for modality fusion to detect preference, and even fewer have focused on the assessment and design of friendly faces in humanoid robots. Most existing EEG-based product appearance design research extracts EEG signals from the frontal lobe and calculates the power spectral density of different bands as features to classify preferences and non-preferences.

[0004] According to a search of prior art, Yisi Liu, Fan Li, Lin Hei Tang, Zirui Lan, Jian Cui, Olga Sourina, Chun-Hsien Chen, et al. wrote an article titled "Detection of Humanoid Robot Design Preferences Using EEG and Eye Tracker" at the 2019 International Conference on Cyberworlds (CW), 2019, pp. 219-224. The study pointed out that head and facial features are prominent features in humanoid robot design, and robots that appear more attractive are given more attention. However, the study did not focus on the combination of detailed facial features of humanoid robots, nor did it simultaneously collect EEG and eye movement signals to evaluate and design preferences for the humanoid robot's face.

[0005] In summary, EEG and eye movement signals have achieved initial success in product design. However, current research lacks specific research on humanoid robots, specifically the assessment and design of facial appearance preferences. As humanoid robot technology rapidly develops and evolves, the issue of humanoid robot facial design cannot be ignored. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a humanoid robot face friendliness assessment and design method; based on the method of the present invention, a humanoid robot face that the user prefers can be designed, so that the user will feel comfortable and not resist during human-computer interaction, thereby obtaining a good user experience and promoting humanoid robots to the public.

[0007] The present invention provides a humanoid robot face friendliness assessment and design method, comprising the following steps:

[0008] (1) Generate several humanoid robot faces based on different facial features;

[0009] (2) Collect the EEG signals and eye movement signals generated by the testers in response to the humanoid robot's face during the friendliness evaluation experiment, and record the testers' preference feedback to verify the accuracy of the preferences derived from physiological signals;

[0010] (3) Combined with the testers' preference feedback, the collected EEG signals are evaluated for preference using the frontal asymmetry theory, valence, selection index, and power spectral density index. For the collected eye movement signals, the eye movement features are extracted based on gaze, saccade, and pupil information to evaluate the preference, and the humanoid robot face preference ranking based on EEG signals and eye movement signals is obtained;

[0011] (4) The facial feature preference ranking results of the humanoid robot face are obtained based on the EEG signals and eye movement signals, and then the friendly facial features are selected to form a humanoid robot face.

[0012] In the present invention, in step (1), facial features include facial features, age, gender, hair, head shape and facial muscles. In the embodiment, MakeHuman software is used to simulate and generate a human face.

[0013] In the present invention, in step (2), the friendliness evaluation experimental process includes drift detection, resting state and humanoid robot facial stimulation presentation; drift detection is used to recalibrate pupil position, and the resting state lasts for 2 seconds to capture the EEG baseline; EEG and eye movement signals are collected during the 2-second presentation of the humanoid robot facial stimulation picture; after the stimulation picture disappears, the participant enters the preference feedback stage.

[0014] In the present invention, in step (3), the collected EEG signals and eye movement signals are preprocessed before preference assessment. The EEG signal preprocessing includes downsampling, filtering with a bandpass filter, dividing the original EEG signal into different time windows according to the time when the picture stimulus appears, removing artifacts from the EEG signal, normalizing the signal, and finally deleting the time window with a voltage amplitude greater than 100 μV.

[0015] In the present invention, in step (3), preference evaluation is performed using frontal asymmetry theory, valence, choice index, and power spectral density index, as follows:

[0016] Frontal asymmetry of preference refers to the fact that the left hemisphere of the brain is more activated for preference, while the right hemisphere is more activated for non-preference; valence is an indicator of frontal asymmetry, calculated by the formula: (right hemisphere activation - left hemisphere activation) / (right hemisphere activation + left hemisphere activation); the selection index is positively correlated with preference; the alpha power spectral density of the forehead will increase when preference is present.

[0017] In the present invention, in step (4), the friendly facial features are features within a certain range.

[0018] In an embodiment of the present invention, in step (4), the age range of the humanoid robot face formed by combining the friendly facial features is 18 to 27 years old, the gender is male or female, and the hair color is black, brown or light red. In the embodiment, the percentage of the facial features relative to the initial face in the simulation software MakeHuman (the percentage of the parameters is all 0), the percentage of the vertical movement, movement depth, horizontal movement, vertical scaling, horizontal scaling, and scaling depth of the eye size is 30% to 70%, the percentage of the vertical movement, movement depth, horizontal movement, vertical scaling, horizontal scaling, and scaling depth of the nose size is 30% to 70%, the percentage of the vertical movement, movement depth, horizontal movement, vertical scaling, horizontal scaling, and scaling depth of the mouth size is 30% to 70%, and the percentage of the vertical movement, movement depth, horizontal movement, vertical scaling, horizontal scaling, and scaling depth of the ear size is 30% to 70%.

[0019] In the present invention, what is designed is a simulated 2D picture of a humanoid robot face.

[0020] The friendliness mentioned in the present invention is to make the user feel comfortable or emotionally connected to the robot's face when interacting with the humanoid robot, without any resistance, thereby obtaining a better user experience;

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention utilizes software to simulate different humanoid robot faces, can arbitrarily adjust facial features, conduct preference evaluation through EEG and eye movement signals, gradually converge on the humanoid robot facial features preferred by the user, and ultimately obtain a humanoid robot face that makes good use of preferences.

[0023] Based on the method of the present invention, a humanoid robot face that is preferred by the user can be designed, so that the user will feel comfortable and not resist during human-machine interaction, thereby obtaining a good user experience and promoting humanoid robots to be popular among the public. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Four 2D images of humanoid robot faces generated to simulate different facial features.

[0025] Figure 2 This is the process of collecting EEG and eye movement signals. DETAILED DESCRIPTION

[0026] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0027] Example 1

[0028] This embodiment provides a humanoid robot face friendliness assessment and design method, which includes simulating and generating a humanoid robot face, collecting EEG and eye movement signals, performing a preference assessment on the collected physiological signals, and finally, based on the friendliness assessment, selecting friendly facial features and combining them into a humanoid robot face.

[0029] The MakeHuman software is used to simulate and generate several humanoid robot faces. The facial features of the humanoid robot should be adjustable. Specifically, the purpose of simulating different humanoid robot faces can be achieved by adjusting the gender, head shape, eyes, hair, facial muscles, nose, mouth, ears and other facial features. Figure 1 shown.

[0030] Figure 2This is the signal acquisition process. During the experiment, participants' EEG and eye movement signals were collected simultaneously. The experiment consisted of four parts, totaling 160 trials. Each trial was divided into four sections: drift detection, resting state, presentation of humanoid robot face stimuli, and preference feedback. Drift detection was achieved by gazing at a dot on the screen. The dot disappeared when the eye tracker detected the subject's pupil position. During the resting state, the subject remained still, collecting a baseline EEG signal. Drift detection was used to recalibrate pupil position at the beginning of each trial. The resting state lasted for 2 seconds to capture the EEG baseline. The stimulus image was presented for 2 seconds, during which both EEG and eye movement signals were collected. After the stimulus image disappeared, participants entered the preference feedback phase, in which they were required to rate the stimulus according to a prompt on the computer screen: 1 for like and 0 for dislike. When the stimulus image appeared and disappeared, a level signal was transmitted from the eye movement data acquisition device to the EEG data acquisition device, enabling simultaneous recording of EEG and eye movement signals.

[0031] The collected EEG and eye movement signals first underwent signal preprocessing. For EEG signals, they were downsampled to 128 Hz and then filtered through a 4–45 Hz bandpass filter to eliminate noise and power frequency interference. The raw EEG signals were divided into time windows based on the time of image stimulus presentation. Each time window contained data from -0.2 to 1.0 seconds, with -0.2 to 0 seconds serving as baseline data. Artifact removal was performed on the EEG signals using independent component analysis and normalization using the average reference method. Finally, time windows with voltage amplitudes greater than 100 μV were removed. For eye movement signals, fixation, saccade, and pupil information were extracted from the region of interest based on the characteristic eye movement patterns. The eye movement data were then preprocessed to remove trials with abnormal data. After signal preprocessing, preference evaluation was performed on EEG signals using frontal asymmetry theory, valence, selectivity index, and power spectral density. Specifically, frontal asymmetry of preference refers to higher activation of the left hemisphere for preferred and higher activation of the right hemisphere for non-preferred actions. Valence is an indicator of frontal asymmetry, calculated as (right hemisphere activation - left hemisphere activation) / (right hemisphere activation + left hemisphere activation). The selectivity index is positively correlated with preference, with an increase in the alpha power spectral density of the forehead when preference is present. For eye movement signals, 17 features were extracted from gaze, saccade, and pupil information, as shown in Table 1. Preference evaluation was performed on these 17 eye movement features. The preference evaluation results of the EEG and eye movement signals were combined to determine the preference ranking of the initially simulated humanoid robot faces. Specifically, a facial feature that was preferred in both the EEG and eye movement evaluations was selected as the facial feature of the friendly humanoid robot.

[0032] Table 1 Classification of eye movement features

[0033]

[0034]

[0035] After preference evaluation, the preference rankings of different facial features were obtained. After combining and fine-tuning the optimal facial features, the humanoid robot face preferred by the user was finally obtained. The details of its facial features are as follows: age is 20 years old; the vertical movement percentage of eye size is 56%; the movement depth percentage of eye size is 47%; the horizontal movement percentage of eye size is 50%; the vertical scaling percentage of eye size is 55%; the horizontal scaling percentage of eye size is 50%; the scaling depth percentage of eye size is 53%; the vertical movement percentage of nose size is 50%; the movement depth percentage of nose size is 49%; the horizontal movement percentage of nose size is 53%; the vertical scaling percentage of nose size is 55%; the nose size The horizontal scaling percentage of the face is 52%; the scaling depth percentage of the nose size is 53%; the vertical movement percentage of the mouth size is 60%; the moving depth percentage of the mouth size is 57%; the horizontal movement percentage of the mouth size is 55%; the vertical scaling percentage of the mouth size is 50%; the horizontal scaling percentage of the mouth size is 55%; the scaling depth percentage of the mouth size is 50%; the vertical movement percentage of the ear size is 57%; the moving depth percentage of the ear size is 45%; the horizontal movement percentage of the ear size is 55%; the vertical scaling percentage of the ear size is 56%; the horizontal scaling percentage of the ear size is 50%; the scaling depth percentage of the ear size is 50%; the gender is male; and the hair color is black. The percentages of the facial features are all relative to the initial face in the simulation software (not relative to the face simulated at the beginning of the experiment).

[0036] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A humanoid robot facial friendliness assessment and design method, characterized in that: The following steps are involved: (1) Generate several humanoid robot faces based on different facial features; (2) Collect the EEG signals and eye movement signals generated by the testers in response to the humanoid robot's face during the friendliness evaluation experiment, and record the testers' preference feedback to verify the accuracy of the preferences derived from physiological signals; (3) Combined with the tester's preference feedback, the collected EEG signals are evaluated based on the frontal asymmetry theory, valence, selection index and power spectral density index. For the collected eye movement signals, the gaze, saccade and pupil information are extracted. Eye movement features are used to evaluate preferences and obtain humanoid robot face preference rankings based on EEG signals and eye movement signals; (4) Based on the humanoid robot face preference ranking results of EEG signals and eye movement signals, a facial feature preference ranking is obtained, and then friendly facial features are selected and combined into a humanoid robot face; wherein: In step (2), the friendliness evaluation experiment process includes drift detection, resting state, and humanoid robot face stimulus presentation. Drift detection is used to recalibrate pupil position, and the resting state lasts for 2 seconds to capture the EEG baseline. EEG and eye movement signals are collected during the 2-second presentation of the humanoid robot face stimulus picture. After the stimulus picture disappears, the participants enter the preference feedback phase. In step (4), a facial feature that has a preference in both the EEG and eye movement evaluation results is taken as a friendly facial feature and combined into a humanoid robot face; the friendly facial feature is a feature within a certain range; the age range of the humanoid robot face that is combined with the friendly facial features is 18 to 27 years old, the gender is male or female, and the hair color is black, brown or light red. Compared with the simulation generation of several humanoid robot faces based on different facial features in step (1), the eye size, nose size, mouth size, and ear size are independently movable and scalable.

2. The humanoid robot face friendliness assessment and design method according to claim 1, characterized in that: In step (1), facial features include facial features, age, gender, hair, head shape and facial muscles.

3. The humanoid robot face friendliness assessment and design method according to claim 1, characterized in that: In step (3), the collected EEG signals and eye movement signals are preprocessed before preference assessment. The EEG signal preprocessing includes downsampling, filtering with a bandpass filter, dividing the original EEG signal into different time windows according to the time when the picture stimulus appears, removing artifacts from the EEG signal, normalizing it, and finally deleting the time window with a voltage amplitude greater than 100 mV.

4. The humanoid robot face friendliness assessment and design method according to claim 1, characterized in that: In step (3), preference evaluation was performed using the frontal asymmetry theory, valence, choice index, and power spectral density index, as follows: Frontal asymmetry of preference refers to the fact that the left hemisphere of the brain is more activated for preferred options, while the right hemisphere is more activated for non-preferred options. Valence is an indicator of frontal asymmetry, calculated using the formula: (right hemisphere activation - left hemisphere activation) / (right hemisphere activation + left hemisphere activation). The selectivity index is positively correlated with preference. The alpha power spectral density of the forehead increases when preference is present.

5. The humanoid robot face friendliness assessment and design method according to claim 1, characterized in that: In step (4), a simulated 2D image of the humanoid robot's face is designed.

Images