A humanoid robot emotion design method and system based on neural network model

Abstract

The present invention discloses a method and system for designing humanoid robot emotions based on a neural network model. The method comprises: generating speech emotion evaluation word samples and combined emotion evaluation word samples; conducting research and experiments on the speech emotion evaluation word samples and combined emotion evaluation word samples to extract characteristic elements; conducting orthogonal experiments to design a scheme for generating appearance and speech combination elements; using a feedforward neural network and a back propagation algorithm to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and speech combination elements; and optimizing the neural network model using a multi-objective optimization algorithm. The system comprises a sample collection module, a characteristic element extraction module, a scheme design module, a neural network model construction module and a model optimization module. The present invention can optimize the appearance and speech combination of a humanoid robot by constructing a comprehensive relationship model.

Description

Technical Field

[0001] The present invention relates to the technical field of humanoid robots, and in particular to a method and system for designing humanoid robot emotions based on a neural network model. Background Art

[0002] In the study of human-computer interaction, robot appearance and voice design as key factors have an important impact on user experience. The appearance of a robot is one of the important factors in human-computer interface design, which affects the user's first impression and emotional response. The development of automatic speech recognition technology enables robots to communicate effectively with users through voice, thereby improving the naturalness and convenience of human-computer interaction. However, research on the psychological impact of voice in the process of robot-user interaction is still relatively insufficient.

[0003] The appearance and voice of robots are both key media for conveying information. The robot's body movements, expressions, etc. can convey a lot of information in the interaction, but the emotions conveyed by the voice are also worthy of attention. In the process of human-computer interaction, the visual appearance of the robot plays an important role in the perception process. Users usually invest a lot of time and resources to ensure the visual aesthetics of the robot, ensure that it can bring users a good visual experience, and promote clear communication. However, the research on the user's perceptual needs brought about by the combined design of the robot's appearance and voice is still insufficient. The mismatch of the combination of appearance and voice may cause confusion effects for users during the interaction process, affecting the acceptability and effectiveness of human-computer interaction.

[0004] The main shortcomings of the current study include:

[0005] (1) In the existing technology, more attention is paid to the single appearance or voice design, while there is less research on the combination and comprehensive impact of the two. There is a lack of systematic research on the impact of the combination of appearance and voice on the emotional needs of users, especially on how to optimize the appearance and voice design elements to meet the emotional needs of users.

[0006] (2) Existing technologies fail to comprehensively optimize humanoid robots at the design element level and often fail to integrate these elements into a system for comprehensive optimization, which makes it difficult to obtain the optimal design solution in practical applications.

[0007] (3) Although there are existing technologies for measuring user subjective evaluations, there is a lack of methods for building systematic relationship models based on these data. How to convert user subjective evaluation data into specific design optimization guidance has not yet been fully addressed. Summary of the invention

[0008] In view of this, the purpose of an embodiment of the present invention is to provide a humanoid robot emotion design method and system based on a neural network model, which can optimize the design of the appearance and voice combination of the humanoid robot by constructing a comprehensive relationship model. It not only focuses on the independent optimization of appearance and voice, but also explores the best combination of the two, thereby improving the overall experience of human-computer interaction.

[0009] A method for designing humanoid robot emotions based on a neural network model, comprising:

[0010] Generate a synthetic speech sample of a humanoid robot, collect and analyze the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and determine a speech emotion evaluation word sample.

[0011] Generate appearance and voice combination video samples, collect and analyze the words of user's subjective emotional response induced by the appearance and voice combination video samples, and determine the combination emotional evaluation word samples.

[0012] The speech emotion evaluation word samples and the combined emotion evaluation word samples are studied and experimented to extract characteristic elements.

[0013] Orthogonal experiments are carried out on the characteristic elements to design a solution for generating the combination of appearance and voice elements.

[0014] A feedforward neural network and back-propagation algorithm are used to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the combined elements of appearance and voice.

[0015] A multi-objective optimization algorithm is used to optimize the neural network model.

[0016] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the generation of the synthetic speech sample of the humanoid robot, the collection and analysis of the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and the determination of the speech emotion evaluation word samples include:

[0017] Select n synthetic speech samples covering different speech features, where the speech features include at least one of speech speed, pitch and emotional expression, and generate the synthetic speech samples using speech synthesis technology.

[0018] A similarity grouping experiment is designed to let the test users listen to different synthesized speech samples, and the synthesized speech samples are grouped according to the subjective feelings of the test users, and a similarity matrix and a distance matrix are constructed.

[0019] The similarity matrix and the distance matrix are analyzed by using multidimensional scaling analysis and cluster analysis to extract representative speech samples.

[0020] The representative speech samples are analyzed to determine the representative speech samples that have a significant impact on the emotional response of the test user.

[0021] The preliminary perceptual image words of the subjective emotional response of the test users are collected, and the test users are asked to select the corresponding preliminary perceptual image words according to their subjective feelings about the representative voice samples.

[0022] The selected perceptual image words are subjected to semantic difference analysis, the differences of different perceptual image words in describing speech emotion response are quantified, significant speech perceptual image words are screened out, and speech emotion evaluation word samples are determined.

[0023] Its technical effects are: covering a variety of voice features such as speaking speed, pitch and emotional expression, and combining the design of the humanoid robot appearance, comprehensively considering various factors in the design process; using similarity grouping experiments, multidimensional scaling analysis and cluster analysis, in-depth analysis of users' subjective emotional responses is carried out to ensure that the extracted feature elements and representative samples are highly accurate and representative.

[0024] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the generation of appearance and voice combination video samples, the collection and analysis of words of the user's subjective emotional response triggered by the appearance and voice combination video samples, and the determination of the combination emotion evaluation word samples include:

[0025] Extract voice design elements that cover the voice characteristics of the humanoid robot and appearance design elements used to describe the appearance of the humanoid robot, generate a design plan for the combination of appearance and voice, and use speech synthesis technology and video production technology to generate appearance and voice combination video samples.

[0026] A similarity grouping experiment is designed to let the test users watch each appearance and voice combination video sample, and the test users are grouped according to the emotional word annotations of each appearance and voice combination video sample to construct a similarity matrix and a distance matrix.

[0027] The similarity matrix and the distance matrix are analyzed by using multidimensional scaling analysis and cluster analysis to extract representative appearance and voice combination video samples.

[0028] The representative appearance and voice combination video samples are analyzed to determine the representative appearance and voice combination video samples that have a significant impact on the emotional response of the test user.

[0029] Initial emotional words evoked by the combination of humanoid robot appearance and voice are collected, and test users are asked to select corresponding preliminary combination emotional image words based on their subjective feelings about the representative appearance and voice combination video samples.

[0030] The semantic difference analysis is performed on the preliminary combined perceptual image words to quantify the differences of different combined perceptual image words in describing combined emotional reactions, screen out significant combined perceptual image words, and determine the combined emotional evaluation word samples.

[0031] Its technical effect is: by combining appearance design elements and voice design elements, a combined video sample of appearance and voice is generated, which fully reflects the user's emotional response to the humanoid robot, overcomes the limitations of previous studies that only consider voice or appearance, and ensures that the user's multi-dimensional perceptual experience is fully considered.

[0032] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the speech emotion evaluation word samples are studied and experimented, and the feature elements extracted include:

[0033] The speech emotion evaluation word samples are vectorized to form a feature vector, n s The set of speech emotion evaluation word samples is

[0034] Each of the speech emotion evaluation words t si Using the scores on m emotional dimensions to express, we get the speech emotion score matrix Among them, S sij is the i-th speech emotion evaluation word t si The rating on the jth sentiment dimension.

[0035] Calculate the average score of the speech emotion evaluation words on all emotion dimensions

[0036]

[0037] Constructing speech emotion evaluation feature vector

[0038] The speech emotion scoring matrix is ​​standardized to obtain a speech emotion standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ s YesS s The mean vector of s YesS s The vector of standard deviations of .

[0039] Calculate the speech emotion covariance matrix of the standardized data in, Z s The transposed matrix of .

[0040] Perform eigenvalue decomposition C on the speech emotion covariance matrix s Vs =V s Λ s , get the speech emotion feature vector and speech emotion feature value, where V s is the eigenvector matrix, Λ s is a diagonal eigenvalue matrix.

[0041] Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the speech emotion principal component matrix W s =[v s1 ,v s2 ,…,v sk ].

[0042] Project the standardized data onto the principal component matrix to obtain the speech emotion feature vector F representing the characteristic elements s =Z s W s .

[0043] Its technical effects are: by vectorizing the speech emotion evaluation word samples to form feature vectors, the emotional features can be quantified and processed, and the subjective emotional response can be converted into actionable numerical data; the key emotional features are extracted, the dimensions of the emotional features are simplified, and the efficiency and accuracy of feature extraction are improved; the data is reduced in dimensionality through principal component analysis, which reduces the complexity of the data and retains most of the emotional information; by extracting and optimizing the speech emotion feature elements, high-quality input features are provided for the subsequent neural network model.

[0044] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the combined emotion evaluation word samples are studied and experimented, and the feature elements extracted include:

[0045] The combined sentiment evaluation word samples are vectorized to form a feature vector, n c The set of combined sentiment evaluation word samples is

[0046] Each of the combined sentiment evaluation words t ci Use the scores on m emotional dimensions to get the combined emotional score matrix Among them, C cij is the i-th combined sentiment evaluation word t ci The rating on the jth sentiment dimension.

[0047] Calculate the average score of the combined sentiment evaluation words on all sentiment dimensions

[0048]

[0049] Constructing a combined sentiment evaluation feature vector

[0050] The combined sentiment score matrix is ​​standardized to obtain a combined sentiment standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ c It is C c The mean vector of c It is C c The vector of standard deviations of .

[0051] Calculate the combined sentiment covariance matrix of the normalized data in, Z c The transposed matrix of .

[0052] Perform eigenvalue decomposition C on the combined emotion covariance matrix c V c =V c Λ c , get the combined emotion feature vector and speech emotion feature value, where V c is the eigenvector matrix, Λ c is a diagonal eigenvalue matrix.

[0053] Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the combined sentiment principal component matrix W c =[v c1 ,v c2 ,…,v ck ].

[0054] Project the standardized data onto the principal component matrix to obtain the combined emotional feature vector F representing the characteristic elements c =Z c W c .

[0055] Its technical effects are: vectorizing the combined sentiment evaluation word samples to form a feature vector, so that the combined sentiment features can be quantified and processed; by constructing a combined sentiment rating matrix, the ratings of each sentiment evaluation word in different sentiment dimensions are systematically represented, and the user's emotional response is fully captured; by performing eigenvalue decomposition on the combined sentiment covariance matrix, the most important sentiment features in the data are extracted to ensure the comprehensiveness and representativeness of feature extraction; by extracting and optimizing the combined sentiment feature elements, high-quality input features are provided for the subsequent neural network model, the performance and accuracy of the neural network model are improved, and the ability to predict user emotional responses is enhanced.

[0056] In a preferred embodiment of the present invention, in the above-mentioned method for designing humanoid robot emotions based on a neural network model, the scheme of conducting an orthogonal test on the characteristic elements and designing and generating the combination elements of appearance and voice includes:

[0057] The experimental goal of the orthogonal test is to enhance the user's emotional response by optimizing the appearance and voice combination of the humanoid robot.

[0058] Select the speech emotion feature factor and the combined emotion feature factor of the orthogonal test, wherein the speech emotion feature factor is composed of the speech emotion feature vector F s =Z s W s Indicates that the combined emotional feature factor is composed of the combined emotional feature vector F c =Z c W c express.

[0059] Different levels are set for the speech emotion feature factor and the combined emotion feature factor respectively.

[0060] Select an orthogonal array based on the number of factors and the level of each factor, and use the orthogonal array to generate a scheme for combining appearance and voice elements

[0061] According to the designed orthogonal table scheme, corresponding voice samples and combination samples are produced, user responses to the voice samples and the combination samples are collected, statistical analysis is performed, and the scheme of the final appearance and voice combination elements is optimized.

[0062] The technical effects are as follows: selecting speech emotion feature factors and combined emotion feature factors, and setting different levels respectively, using orthogonal tables to generate schemes of appearance and speech combination elements, reducing the number of experiments, improving experimental efficiency, and saving time and resources; making speech samples and combination samples according to the orthogonal table scheme, collecting user reactions and conducting statistical analysis to ensure that the optimization scheme can accurately reflect the user's emotional needs and preferences; obtaining the final scheme of appearance and speech combination elements through statistical analysis and optimization to ensure that the designed humanoid robot can better meet the user's emotional needs and improve user satisfaction and interactive experience; and improving the user's emotional response, enhancing the user's interactive experience with the robot, and improving the emotional acceptance and clarity of human-computer interaction by optimizing the appearance and speech combination of the humanoid robot.

[0063] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on a neural network model, the neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements is constructed by using a feedforward neural network and a back propagation algorithm, including:

[0064] The input vector X of the neural network model is s ,F c ], output vector Y = [E1, E2, ..., E m ], where F s is the speech emotion feature vector, F c is the combined sentiment feature vector, E i Rating of users' subjective emotional responses to humanoid robots.

[0065] The number of input layer nodes of the neural network model is the sum of the number of the speech emotion feature factors and the number of the combined emotion feature factors.

[0066] The hidden layer H of the neural network model is H=f(W h ·X+b h ), W h is the weight matrix of the hidden layer, b h is the bias vector and f is the activation function.

[0067] The number of output layer nodes of the neural network model is the number m of emotion dimensions.

[0068] Use mean squared error as loss function Among them, i is the sample index.

[0069] Perform forward propagation to calculate the output of the neural network model Among them, W o is the weight matrix of the output layer, b o is the bias vector.

[0070] Perform back propagation to calculate the gradient of the loss function with respect to the weights and biases

[0071]

[0072] Update weights using gradient descent Among them, η is the learning rate.

[0073] Its technical effects are as follows: by constructing a neural network model, an accurate mapping between the combined elements of appearance and voice and the user's subjective emotional response is achieved. The input vector of the neural network model includes a voice emotional feature vector and a combined emotional feature vector, and the output vector is the user's score of the subjective emotional response of the humanoid robot. The multi-dimensional prediction capability enables the model to consider multiple emotional dimensions at the same time and provide a more comprehensive emotional evaluation; the combination of the feedforward neural network and the back propagation algorithm enables the model to have adaptive learning capabilities. The back propagation algorithm calculates the gradient of the loss function relative to the weights and biases, and the gradient descent method is used to update the weights and biases, so that the model is continuously optimized during the training process to improve the prediction accuracy; the emotional model constructed by the feedforward neural network and the back propagation algorithm has many technical effects such as accurate mapping, multi-dimensional prediction, adaptive learning, data-driven, efficient optimization, flexible structure, complex feature expression, improved user experience and support for design decisions, providing strong technical support for the emotional design of humanoid robots.

[0074] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on a neural network model, the neural network model for constructing the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements using a feedforward neural network and a back propagation algorithm also includes:

[0075] Repeat the steps of forward propagation, back propagation and weight bias update until the loss function L converges to a preset value or reaches a predetermined number of training rounds to obtain a constructed neural network model.

[0076] Its technical effects are as follows: by repeating forward propagation and back propagation and continuously updating weights and biases, the model gradually reduces the prediction error, and the loss function L converges to a preset value. The iterative optimization process ensures that the prediction results of the model are becoming more and more accurate, while improving the stability of the model. During the training process, by setting a predetermined number of training rounds, the model is prevented from overfitting the training data, thereby improving the generalization ability of the model on new data. By repeating the training steps, the deep features and relationships in the data are fully explored, so that the model has stronger expressive ability and can adapt to diverse user emotional responses. By repeating the steps of forward propagation, back propagation and weight bias update until the loss function L converges or reaches the predetermined number of training rounds, the constructed neural network model not only performs well in terms of accuracy and stability, but also effectively prevents overfitting and underfitting, dynamically adapts to complex emotional relationships, achieves efficient training, supports real-time applications, and provides a reliable basis for design decisions.

[0077] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the optimization of the neural network model by using a multi-objective optimization algorithm includes:

[0078] Specific objectives of the multi-objective optimization are defined, wherein the specific objectives include at least one of maximizing the sentiment score, maximizing the design satisfaction, and maximizing the prediction accuracy.

[0079] For maximizing the sentiment score, construct the first objective function Among them, EmotionScore i is the sentiment score of the i-th sample, N1 is the total number of samples, and the negative sign indicates that the optimization goal is to maximize the sentiment score.

[0080] To maximize the design satisfaction, the second objective function is constructed Among them, DesignSatisfaction i is the design satisfaction score of the i-th sample, and the negative sign indicates that the optimization goal is to maximize the design satisfaction.

[0081] For the highest prediction accuracy, construct the third objective function PredictionError i is the prediction error of the ith sample.

[0082] Select a specified multi-objective optimization algorithm, apply the first objective function, the second objective function, the third objective function and the selected multi-objective optimization algorithm to the training process of the neural network model, optimize the objective function value, and obtain the optimized neural network model.

[0083] Its technical effects are: by defining three objective functions of maximizing emotional score, maximizing design satisfaction and maximizing prediction accuracy, multiple performance indicators of the neural network model are comprehensively optimized, so that the model finds the best balance between different objectives; by weighing and coordinating between different objectives, it is ensured that each objective in the optimization process receives reasonable attention, so that the final model can both meet emotional response and improve design satisfaction, while ensuring high prediction accuracy; by optimizing the neural network model with a multi-objective optimization algorithm, the comprehensive goals of maximizing emotional score, maximizing design satisfaction and maximizing prediction accuracy can be achieved, the accuracy, stability and user satisfaction of the model can be improved, the adaptability and robustness of the model can be enhanced, efficient design decision-making and training process can be supported, and ultimately the performance of humanoid robots in emotional design and user experience can be improved.

[0084] A humanoid robot emotion design system based on a neural network model, comprising:

[0085] The speech emotion evaluation word sample collection module is used to generate a synthetic speech sample of the humanoid robot, collect and analyze the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and determine the speech emotion evaluation word sample.

[0086] The combined emotion evaluation word sample collection module is used to generate appearance and voice combined video samples, collect and analyze the words of the user's subjective emotional response triggered by the appearance and voice combined video samples, and determine the combined emotion evaluation word samples.

[0087] The feature element extraction module is used to study and experiment on the speech emotion evaluation word samples and the combined emotion evaluation word samples to extract feature elements.

[0088] The scheme design module is used to conduct orthogonal experiments on the characteristic elements and design a scheme for generating appearance and voice combination elements.

[0089] The neural network model building module is used to use a feedforward neural network and a back-propagation algorithm to build a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the combined elements of appearance and voice.

[0090] The model optimization module is used to optimize the neural network model using a multi-objective optimization algorithm.

[0091] The beneficial effects of the present invention are:

[0092] The present invention systematically extracts characteristic elements through research and experiments on speech and appearance emotion evaluation word samples, ensures accurate extraction and characterization of emotion features through multidimensional scaling analysis, cluster analysis, vectorized representation and feature vector construction, and greatly improves the accuracy and effectiveness of feature extraction.

[0093] The present invention adopts orthogonal experimental design to generate a scheme of appearance and voice combination elements. Through the setting of experimental objectives, the selection of factors and levels, the use of orthogonal tables, and the final statistical analysis, the appearance and voice combination elements are optimized, and the user's emotional response and satisfaction are improved.

[0094] The present invention utilizes a feedforward neural network and a back-propagation algorithm to construct a model of the relationship between the user's subjective emotional response to a humanoid robot and the combined elements of appearance and voice. By setting the input, hidden, and output layers, defining the loss function, and applying the gradient descent method, the model is ensured to be able to effectively learn and predict the user's emotional response, thereby improving the accuracy and reliability of the model.

[0095] The present invention adopts a multi-objective optimization algorithm to optimize the neural network model. The specific goals defined include maximizing the emotional score, maximizing the design satisfaction and maximizing the prediction accuracy. By constructing the corresponding objective function and applying it to the training process of the neural network model, the overall performance of the model is optimized and the optimal balance between emotional response, design satisfaction and prediction accuracy is found. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0097] Figure 1 This is a flow chart of the humanoid robot emotion design method based on the neural network model of the present invention. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0099] Please refer to Figure 1 The first embodiment of the present invention provides a humanoid robot emotion design method based on a neural network model, which includes: generating a synthetic voice sample of a humanoid robot, collecting and analyzing evaluation words of the user's subjective emotional response induced by the synthetic voice sample, and determining a voice emotion evaluation word sample; generating an appearance and voice combination video sample, collecting and analyzing words of the user's subjective emotional response induced by the appearance and voice combination video sample, and determining a combination emotion evaluation word sample; studying and experimenting with the voice emotion evaluation word sample and the combination emotion evaluation word sample to extract feature elements; conducting an orthogonal test on the feature elements to design a scheme for generating appearance and voice combination elements; using a feedforward neural network and a back propagation algorithm to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements; and using a multi-objective optimization algorithm to optimize the neural network model.

[0100] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the generation of synthetic speech samples of the humanoid robot, the collection and analysis of evaluation words of the user's subjective emotional response induced by the synthetic speech samples, and the determination of speech emotion evaluation word samples include: selecting n synthetic speech samples covering different speech features, wherein the speech features include at least one of speech speed, pitch and emotional expression, and using speech synthesis technology to generate the synthetic speech samples; designing a similarity grouping experiment, allowing the test users to listen to different synthetic speech samples, grouping the synthetic speech samples according to the subjective feelings of the test users, constructing a similarity matrix and a distance matrix, and quantifying the subjective feelings of the test users to different synthetic speech samples. ; Use multidimensional scaling (MDS) and cluster analysis to analyze the similarity matrix and the distance matrix to extract representative speech samples; analyze the representative speech samples to determine the representative speech samples that have a significant impact on the emotional response of the test users; collect preliminary perceptual image words of the subjective emotional response of the test users, and let the test users select the corresponding preliminary perceptual image words according to their subjective feelings about the representative speech samples; perform semantic difference analysis on the selected perceptual image words, quantify the differences between different perceptual image words in describing speech emotional response, screen out significant speech perceptual image words, and determine speech emotional evaluation word samples.

[0101] Among them, the similarity matrix is an n×n symmetric matrix used to represent the similarity between each pair of speech samples; the distance matrix is ​​obtained by converting the similarity matrix D ij =1-S ij , used to represent the degree of dissimilarity between each pair of speech samples.

[0102] The steps of multidimensional scaling analysis include: calculating the centered distance matrix, calculating the mean of each row and column, and the mean of the entire matrix, The distance matrix D is converted into a dual-centered matrix B, Perform eigenvalue decomposition on the dual-centered matrix B to obtain eigenvalues ​​and eigenvectors B = VΛV T , where Λ is the diagonal matrix of eigenvalues ​​and V is the eigenvector matrix; select the largest first k eigenvalues ​​and corresponding eigenvectors, and calculate the low-dimensional coordinates Among them, V k is the matrix composed of the first k eigenvectors, Λ k is the corresponding eigenvalue diagonal matrix, and the low-dimensional coordinate X is the representation of the speech sample in the low-dimensional space.

[0103] The steps of clustering analysis using low-dimensional coordinates X include: using k-means clustering, initializing k cluster centers μ1, μ2, …, μ k ; For each sample x i , calculate its Euclidean distance to each cluster center, and assign it to the cluster corresponding to the nearest cluster center Among them, c i Represents sample x i Cluster labels; for each cluster j, calculate the mean of all samples assigned to the cluster and update the cluster center Among them, C j represents the sample set of cluster j, |C j ∣ represents the number of samples in cluster j; repeatedly assign samples to the nearest cluster center and update the cluster center until the cluster center no longer changes or the change is less than the set threshold.

[0104] After clustering is completed, each cluster contains several samples, and the center μ of each cluster is calculated. j , select a sample x from each cluster that is closest to the cluster center rep,j As a representative speech sample

[0105]

[0106] Semantic differential analysis using ANOVA: Calculating between-group variance in, is the overall average score of the jth perceptual image word, is the overall mean of all ratings, n j is the sample number of the jth perceptual image word; calculate the intra-group variance Among them, r ij is the score of the i-th speech sample on the j-th perceptual image word, is the average score of the jth perceptual image word; calculate the F value used to measure the ratio of between-group variance to within-group variance Among them, p is the number of groups, n is the total number of samples, and the F value is used to judge whether the difference of perceptual image words in describing speech emotional response is significant, so as to screen out representative words.

[0107] Its technical effects are: covering a variety of voice features such as speaking speed, pitch and emotional expression, and combining the design of the humanoid robot appearance, comprehensively considering various factors in the design process; using similarity grouping experiments, multidimensional scaling analysis (MDS) and cluster analysis to conduct in-depth analysis of users' subjective emotional responses to ensure that the extracted feature elements and representative samples are highly accurate and representative.

[0108] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the generation of appearance and voice combination video samples, the collection and analysis of words of user subjective emotional response triggered by the appearance and voice combination video samples, and the determination of the combination emotion evaluation word samples include: extracting voice design elements covering the voice characteristics of the humanoid robot and appearance design elements used to describe the appearance of the humanoid robot, generating an appearance and voice combination design scheme, and using speech synthesis technology and video production technology to generate appearance and voice combination video samples; designing a similarity grouping experiment, allowing the test users to watch each of the appearance and voice combination video samples, grouping them according to the emotional word labels of each of the appearance and voice combination video samples by the test users, and constructing a similarity matrix and a distance matrix ; Use multidimensional scaling analysis and cluster analysis to analyze the similarity matrix and the distance matrix, and extract representative appearance and voice combination video samples; analyze the representative appearance and voice combination video samples to determine the representative appearance and voice combination video samples that have a significant impact on the emotional response of the test users; collect initial emotional words used to describe the appearance and voice combination of humanoid robots, and let the test users select the corresponding preliminary combination of perceptual image words according to their subjective feelings about the representative appearance and voice combination video samples; perform semantic difference analysis on the preliminary combination of perceptual image words, quantify the differences between different combination of perceptual image words in describing the combination emotional response, screen out significant combination of perceptual image words, and determine the combination of emotional evaluation word samples.

[0109] Its technical effect is: by combining appearance design elements and voice design elements, a combined video sample of appearance and voice is generated, which fully reflects the user's emotional response to the humanoid robot, overcomes the limitations of previous studies that only consider voice or appearance, and ensures that the user's multi-dimensional perceptual experience is fully considered.

[0110] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the speech emotion evaluation word sample is studied and experimented, and the feature elements are extracted, including: vectorizing the speech emotion evaluation word sample to form a feature vector, n s The set of speech emotion evaluation word samples is Each of the speech emotion evaluation words t si Using the scores on m emotional dimensions to express, we get the speech emotion score matrix Among them, S sij is the i-th speech emotion evaluation word t si The score on the jth emotional dimension; Calculate the average score of the speech emotion evaluation words on all emotional dimensions Constructing speech emotion evaluation feature vector The speech emotion scoring matrix is ​​standardized to obtain a speech emotion standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ s YesS s The mean vector of s YesS s The standard deviation vector of ; Calculate the speech emotion covariance matrix of the standardized data in, Z s The transposed matrix of the speech emotion covariance matrix is ​​decomposed into eigenvalues ​​C s V s =V s Λ s , get the speech emotion feature vector and speech emotion feature value, where V s is the eigenvector matrix, Λ s It is a diagonal eigenvalue matrix; select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the speech emotion principal component matrix W s =[v s1 ,v s2 ,…,v sk ]; Project the standardized data onto the principal component matrix to obtain the speech emotion feature vector F representing the characteristic elements s =Z s W s .

[0111] Its technical effects are: by vectorizing the speech emotion evaluation word samples to form feature vectors, the emotional features can be quantified and processed, and the subjective emotional response can be converted into actionable numerical data; the key emotional features are extracted, the dimensions of the emotional features are simplified, and the efficiency and accuracy of feature extraction are improved; the data is reduced in dimensionality through principal component analysis, which reduces the complexity of the data and retains most of the emotional information; by extracting and optimizing the speech emotion feature elements, high-quality input features are provided for the subsequent neural network model.

[0112] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the combined emotion evaluation word sample is studied and experimented, and the feature element extraction includes: vectorizing the combined emotion evaluation word sample to form a feature vector, n c The set of combined sentiment evaluation word samples is Each of the combined sentiment evaluation words t ci Use the scores on m emotional dimensions to get the combined emotional score matrix Among them, C cij is the i-th combined sentiment evaluation word t ciThe score on the jth emotional dimension; Calculate the average score of the combined emotional evaluation words on all emotional dimensions Constructing a combined sentiment evaluation feature vector The combined sentiment score matrix is ​​standardized to obtain a combined sentiment standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ c It is C c The mean vector of c It is C c The standard deviation vector of ; Calculate the combined sentiment covariance matrix of the standardized data in, Z c The transposed matrix of the combined emotion covariance matrix is ​​decomposed into eigenvalues ​​C c V c =V c Λ c , get the combined emotion feature vector and speech emotion feature value, where V c is the eigenvector matrix, Λ c It is a diagonal eigenvalue matrix; select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the combined emotion principal component matrix W c =[v c1 ,v c2 ,…,v ck ]; Project the standardized data onto the principal component matrix to obtain the combined emotional feature vector F representing the characteristic elements c =Z c W c .

[0113] Its technical effects are: vectorizing the combined sentiment evaluation word samples to form a feature vector, so that the combined sentiment features can be quantified and processed; by constructing a combined sentiment rating matrix, the ratings of each sentiment evaluation word in different sentiment dimensions are systematically represented, and the user's emotional response is fully captured; by performing eigenvalue decomposition on the combined sentiment covariance matrix, the most important sentiment features in the data are extracted to ensure the comprehensiveness and representativeness of feature extraction; by extracting and optimizing the combined sentiment feature elements, high-quality input features are provided for the subsequent neural network model, the performance and accuracy of the neural network model are improved, and the ability to predict user emotional responses is enhanced.

[0114] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the scheme of conducting an orthogonal test on the characteristic elements and designing and generating the appearance and voice combination elements includes: determining that the experimental goal of the orthogonal test is to enhance the user's emotional response by optimizing the appearance and voice combination of the humanoid robot; selecting the voice emotion characteristic factor and the combined emotion characteristic factor of the orthogonal test, wherein the voice emotion characteristic factor is composed of the voice emotion feature vector F s =Z s W s Indicates that the combined emotional feature factor is composed of the combined emotional feature vector F c =Z c W c Representation; setting different levels for the speech emotion feature factor and the combined emotion feature factor; selecting an orthogonal table according to the number of factors and the level of each factor, and using the orthogonal table to generate a scheme for combining appearance and speech elements According to the designed orthogonal table scheme, corresponding voice samples and combination samples are produced, user responses to the voice samples and the combination samples are collected, statistical analysis is performed, and the scheme of the final appearance and voice combination elements is optimized.

[0115] Specifically, we select the L9(3^4) orthogonal array, which has 4 factors and 3 levels for each factor. The orthogonal array is expressed as Among them, A and B represent combined emotional feature factors (such as color and shape), and C and D represent voice emotional feature factors (such as speaking speed and pitch). Use speech synthesis technology to generate synthetic speech samples, and use speech synthesis technology and video production technology to generate appearance and voice combination video samples. Recruit subjects to watch and evaluate these samples, and record the user's emotional response and subjective evaluation of each combination sample. Perform statistical analysis on the collected data, calculate the contribution of each factor to the user's emotional response, and use methods such as analysis of variance (ANOVA) to evaluate the impact of each factor and its level on the user's emotional response. Based on the data analysis results, determine the best voice and combination plan.

[0116] The technical effects are as follows: selecting speech emotion feature factors and combined emotion feature factors, and setting different levels respectively, using orthogonal tables to generate schemes of appearance and speech combination elements, reducing the number of experiments, improving experimental efficiency, and saving time and resources; making speech samples and combination samples according to the orthogonal table scheme, collecting user reactions and conducting statistical analysis to ensure that the optimization scheme can accurately reflect the user's emotional needs and preferences; obtaining the final scheme of appearance and speech combination elements through statistical analysis and optimization to ensure that the designed humanoid robot can better meet the user's emotional needs and improve user satisfaction and interactive experience; and improving the user's emotional response, enhancing the user's interactive experience with the robot, and improving the emotional acceptance and clarity of human-computer interaction by optimizing the appearance and speech combination of the humanoid robot.

[0117] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on a neural network model, the neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements is constructed by using a feedforward neural network and a back propagation algorithm, including: the input vector X of the neural network model is [F s ,F c ], output vector Y = [E1, E2, ..., E m ], where F s is the speech emotion feature vector, F c is the combined sentiment feature vector, E i is the user's subjective emotional response score to the humanoid robot, such as intimacy, pleasure, credibility, etc.; the number of input layer nodes of the neural network model is the sum of the number of the speech emotion feature factor and the number of the combined emotion feature factor; the hidden layer H of the neural network model is f(W h ·X+b h ), W h is the weight matrix of the hidden layer, b h is the bias vector, f is the activation function; the neural network model can be set with several hidden layers, for example, the first hidden layer may include 64 neurons, the second hidden layer may include 32 neurons, and the first hidden layer may include 16 neurons; the number of output layer nodes of the neural network model is the number of emotional dimensions m; the mean square error is used as the loss function Where i is the sample index; forward propagation is performed to calculate the output of the neural network model Among them, W o is the weight matrix of the output layer, b o is the bias vector; perform back propagation and calculate the gradient of the loss function with respect to the weights and biases Update weights using gradient descent Among them, η is the learning rate.

[0118] Its technical effects are as follows: by constructing a neural network model, an accurate mapping between the combined elements of appearance and voice and the user's subjective emotional response is achieved. The input vector of the neural network model includes a voice emotional feature vector and a combined emotional feature vector, and the output vector is the user's score of the subjective emotional response of the humanoid robot. The multi-dimensional prediction capability enables the model to consider multiple emotional dimensions at the same time and provide a more comprehensive emotional evaluation; the combination of the feedforward neural network and the back propagation algorithm enables the model to have adaptive learning capabilities. The back propagation algorithm calculates the gradient of the loss function relative to the weights and biases, and the gradient descent method is used to update the weights and biases, so that the model is continuously optimized during the training process to improve the prediction accuracy; the emotional model constructed by the feedforward neural network and the back propagation algorithm has many technical effects such as accurate mapping, multi-dimensional prediction, adaptive learning, data-driven, efficient optimization, flexible structure, complex feature expression, improved user experience and support for design decisions, providing strong technical support for the emotional design of humanoid robots.

[0119] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the use of a feedforward neural network and a back propagation algorithm to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements also includes: repeating the steps of forward propagation, back propagation and weight bias update until the loss function L converges to a preset value or reaches a predetermined number of training rounds to obtain a constructed neural network model.

[0120] Its technical effects are as follows: by repeating forward propagation and back propagation and continuously updating weights and biases, the model gradually reduces the prediction error, and the loss function L converges to a preset value. The iterative optimization process ensures that the prediction results of the model are becoming more and more accurate, while improving the stability of the model. During the training process, by setting a predetermined number of training rounds, the model is prevented from overfitting the training data, thereby improving the generalization ability of the model on new data. By repeating the training steps, the deep features and relationships in the data are fully explored, so that the model has stronger expressive ability and can adapt to diverse user emotional responses. By repeating the steps of forward propagation, back propagation and weight bias update until the loss function L converges or reaches the predetermined number of training rounds, the constructed neural network model not only performs well in terms of accuracy and stability, but also effectively prevents overfitting and underfitting, dynamically adapts to complex emotional relationships, achieves efficient training, supports real-time applications, and provides a reliable basis for design decisions.

[0121] In a preferred embodiment of the present invention, in the above-mentioned humanoid robot emotion design method based on the neural network model, the optimization of the neural network model by using a multi-objective optimization algorithm includes: defining specific goals of the multi-objective optimization, wherein the specific goals include at least one of maximizing the emotion score, maximizing the design satisfaction, and maximizing the prediction accuracy; for maximizing the emotion score, constructing a first objective function Among them, EmotionScore i is the sentiment score of the ith sample, N1 is the total number of samples, and the negative sign indicates that the optimization goal is to maximize the sentiment score; for maximizing design satisfaction, the second objective function is constructed Among them, DesignSatisfaction i is the design satisfaction score of the i-th sample. The negative sign indicates that the optimization goal is to maximize the design satisfaction. For the highest prediction accuracy, the third objective function is constructed. PredictionError i is the prediction error of the ith sample; selecting a specified multi-objective optimization algorithm, applying the first objective function, the second objective function, the third objective function and the selected multi-objective optimization algorithm to the training process of the neural network model, optimizing the objective function value, and obtaining the optimized neural network model.

[0122] Specifically, the multi-objective optimization algorithm includes a genetic algorithm, a particle swarm optimization or a non-dominated sorting genetic algorithm. The objective function and the selected optimization algorithm are applied to the training process of the neural network model. The steps of the optimization process include: generating an initial solution from the solution space; evaluating each solution using the objective function; running the optimization algorithm, optimizing the objective function by iteratively updating the solution set; checking whether the optimization process converges to the Pareto frontier and generating the optimal solution set. Generate and analyze the Pareto frontier solution set and select the optimal solution; verify the effect of the optimization result through experiments; select the design solution that best meets the user's needs based on the optimization result and apply it to the design of the humanoid robot.

[0123] Its technical effects are: by defining three objective functions of maximizing emotional score, maximizing design satisfaction and maximizing prediction accuracy, multiple performance indicators of the neural network model are comprehensively optimized, so that the model finds the best balance between different objectives; by weighing and coordinating between different objectives, it is ensured that each objective in the optimization process receives reasonable attention, so that the final model can both meet emotional response and improve design satisfaction, while ensuring high prediction accuracy; by optimizing the neural network model with a multi-objective optimization algorithm, the comprehensive goals of maximizing emotional score, maximizing design satisfaction and maximizing prediction accuracy can be achieved, the accuracy, stability and user satisfaction of the model can be improved, the adaptability and robustness of the model can be enhanced, efficient design decision-making and training process can be supported, and ultimately the performance of humanoid robots in emotional design and user experience can be improved.

[0124] The second embodiment of the present invention provides a humanoid robot emotion design system based on a neural network model, which includes: a speech emotion evaluation word sample collection module, which is used to generate a synthetic speech sample of a humanoid robot, collect and analyze the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and determine the speech emotion evaluation word sample; a combined emotion evaluation word sample collection module, which is used to generate an appearance and speech combination video sample, collect and analyze the words of the user's subjective emotional response induced by the appearance and speech combination video sample, and determine the combined emotion evaluation word sample; a feature element extraction module, which is used to study and experiment with the speech emotion evaluation word sample and the combined emotion evaluation word sample to extract feature elements; a scheme design module, which is used to perform orthogonal experiments on the feature elements and design a scheme for generating appearance and speech combination elements; a neural network model construction module, which is used to use a feedforward neural network and a back propagation algorithm to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and speech combination elements; a model optimization module, which is used to optimize the neural network model using a multi-objective optimization algorithm.

[0125] The computer program product of the humanoid robot emotion design method and device based on the neural network model provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0126] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, it can execute the above-mentioned humanoid robot emotion design method based on the neural network model, so that the appearance and voice combination of the humanoid robot can be optimized by constructing a comprehensive relationship model. It not only focuses on the independent optimization of appearance and voice, but also explores the best combination of the two, thereby improving the overall experience of human-computer interaction.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for designing humanoid robot emotions based on a neural network model, characterized in that: include: Generate a synthetic speech sample of a humanoid robot, collect and analyze evaluation words of a user's subjective emotional response induced by the synthetic speech sample, and determine a speech emotion evaluation word sample; Generate appearance and voice combination video samples, collect and analyze the words of the user's subjective emotional response induced by the appearance and voice combination video samples, and determine the combination emotional evaluation word samples; Conducting research and experiments on the speech emotion evaluation word samples and the combined emotion evaluation word samples to extract characteristic elements; Conducting orthogonal experiments on the characteristic elements to design a scheme for generating appearance and voice combination elements; A feedforward neural network and back propagation algorithm are used to construct a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the combined elements of appearance and voice; A multi-objective optimization algorithm is used to optimize the neural network model.

2. The method for designing humanoid robot emotions based on a neural network model according to claim 1, characterized in that: The generating of the synthetic speech sample of the humanoid robot, collecting and analyzing the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and determining the speech emotion evaluation word samples include: Selecting n synthetic speech samples covering different speech features, wherein the speech features include at least one of speech speed, pitch and emotional expression, and generating the synthetic speech samples using speech synthesis technology; Designing a similarity grouping experiment, letting the test users listen to different synthesized speech samples, grouping the synthesized speech samples according to the test users' subjective feelings, and constructing a similarity matrix and a distance matrix; Using multidimensional scaling analysis and cluster analysis, the similarity matrix and the distance matrix are analyzed to extract representative speech samples; Analyzing the representative voice samples to determine the representative voice samples that have a significant impact on the emotional response of the test user; Collecting preliminary perceptual image words of subjective emotional responses of test users, and asking test users to select corresponding preliminary perceptual image words according to their subjective feelings about the representative voice samples; The selected perceptual image words are subjected to semantic difference analysis, the differences of different perceptual image words in describing speech emotion response are quantified, significant speech perceptual image words are screened out, and speech emotion evaluation word samples are determined.

3. The method for designing humanoid robot emotions based on a neural network model according to claim 1, characterized in that: The generating of the appearance and voice combination video sample, collecting and analyzing the words of the user's subjective emotional response induced by the appearance and voice combination video sample, and determining the combination emotional evaluation word sample comprises: Extracting speech design elements covering the speech characteristics of the humanoid robot and appearance design elements used to describe the appearance of the humanoid robot, generating a design scheme for the combination of appearance and speech, and generating a video sample for the combination of appearance and speech using speech synthesis technology and video production technology; Design a similarity grouping experiment, let the test users watch each appearance and voice combination video sample, group them according to the emotional words annotated by the test users for each appearance and voice combination video sample, and construct a similarity matrix and a distance matrix; Using multidimensional scaling analysis and cluster analysis, the similarity matrix and the distance matrix are analyzed to extract representative appearance and voice combination video samples; Analyze the representative appearance and voice combination video samples to determine the representative appearance and voice combination video samples that have a significant impact on the emotional response of the test user; Collecting initial emotional words used to describe the combination of the appearance and voice of the humanoid robot, and asking the test users to select the corresponding preliminary combination of emotional image words based on their subjective feelings about the representative appearance and voice combination video samples; The semantic difference analysis is performed on the preliminary combined perceptual image words to quantify the differences of different combined perceptual image words in describing combined emotional reactions, screen out significant combined perceptual image words, and determine the combined emotional evaluation word samples.

4. The method for designing humanoid robot emotions based on a neural network model according to claim 1, characterized in that: The speech emotion evaluation word samples are studied and experimented, and the feature elements extracted include: The speech emotion evaluation word samples are vectorized to form a feature vector, n s The set of speech emotion evaluation word samples is Each of the speech emotion evaluation words t si Using the scores on m emotional dimensions to express, we get the speech emotion score matrix Among them, S sij is the i-th speech emotion evaluation word t si The rating on the jth sentiment dimension; Calculate the average score of the speech emotion evaluation words on all emotion dimensions Constructing speech emotion evaluation feature vector The speech emotion scoring matrix is ​​standardized to obtain a speech emotion standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ s YesS s The mean vector of s YesS s The standard deviation vector of ; Calculate the speech emotion covariance matrix of the standardized data in, Z s The transposed matrix of Perform eigenvalue decomposition C on the speech emotion covariance matrix s V s =V s Λ s , get the speech emotion feature vector and speech emotion feature value, where V s is the eigenvector matrix, Λ s is a diagonal eigenvalue matrix; Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the speech emotion principal component matrix W s =[v s1 ,v s2 ,…,v sk ]; Project the standardized data onto the principal component matrix to obtain the speech emotion feature vector F representing the characteristic elements s =Z s W s .

5. The method for designing humanoid robot emotions based on a neural network model according to claim 4, characterized in that: The combined sentiment evaluation word samples are studied and experimented, and the feature elements extracted include: The combined sentiment evaluation word samples are vectorized to form a feature vector, n c The set of combined sentiment evaluation word samples is Each of the combined sentiment evaluation words t ci Use the scores on m emotional dimensions to get the combined emotional score matrix Among them, C cij is the i-th combined sentiment evaluation word t ci The rating on the jth sentiment dimension; Calculate the average score of the combined sentiment evaluation words on all sentiment dimensions Constructing a combined sentiment evaluation feature vector The combined sentiment score matrix is ​​standardized to obtain a combined sentiment standardized matrix Make the mean of each dimension of data 0 and the variance 1, where μ c It is C c The mean vector of c It is C c The standard deviation vector of ; Calculate the combined sentiment covariance matrix of the normalized data in, Z c The transposed matrix of Perform eigenvalue decomposition C on the combined emotion covariance matrix c V c =V c Λ c , get the combined emotion feature vector and speech emotion feature value, where V c is the eigenvector matrix, Λ c is a diagonal eigenvalue matrix; Select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the combined sentiment principal component matrix W c =[v c1 ,v c2 ,…,v ck ]; Project the standardized data onto the principal component matrix to obtain the combined emotional feature vector F representing the characteristic elements c =Z c W c .

6. The method for designing humanoid robot emotions based on a neural network model according to claim 5, characterized in that: The method of conducting an orthogonal test on the characteristic elements to design and generate a scheme for combining appearance and voice elements comprises: The experimental goal of the orthogonal test is to improve the user's emotional response by optimizing the appearance and voice combination of the humanoid robot; Select the speech emotion feature factor and the combined emotion feature factor of the orthogonal test, wherein the speech emotion feature factor is composed of the speech emotion feature vector F s =Z s W s Indicates that the combined emotional feature factor is composed of the combined emotional feature vector F c =Z c W c express; Setting different levels for the speech emotion feature factor and the combined emotion feature factor respectively; Select an orthogonal array based on the number of factors and the level of each factor, and use the orthogonal array to generate a scheme for combining appearance and voice elements According to the designed orthogonal table scheme, corresponding voice samples and combination samples are produced, user responses to the voice samples and the combination samples are collected, statistical analysis is performed, and the scheme of the final appearance and voice combination elements is optimized.

7. The method for designing humanoid robot emotions based on a neural network model according to claim 6, characterized in that: The neural network model for constructing the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements by using a feedforward neural network and a back propagation algorithm includes: The input vector X of the neural network model is s ,F c ], output vector Y = [E1, E2, ..., E m ], where F s is the speech emotion feature vector, F c is the combined sentiment feature vector, E i Rating of users’ subjective emotional responses to humanoid robots; The number of input layer nodes of the neural network model is the sum of the number of the speech emotion feature factors and the number of the combined emotion feature factors; The hidden layer H of the neural network model is H=f(W h ·X+b h ), W h is the weight matrix of the hidden layer, b h is the bias vector, f is the activation function; The number of output layer nodes of the neural network model is the number of emotion dimensions m; Use mean squared error as loss function Where i is the sample index; Perform forward propagation to calculate the output of the neural network model Among them, W o is the weight matrix of the output layer, b o is the bias vector; Perform back propagation to calculate the gradient of the loss function with respect to the weights and biases Update weights using gradient descent Among them, η is the learning rate.

8. The method for designing humanoid robot emotions based on a neural network model according to claim 7, characterized in that: The neural network model for constructing the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements by using the feedforward neural network and the back propagation algorithm also includes: Repeat the steps of forward propagation, back propagation and weight bias update until the loss function L converges to a preset value or reaches a predetermined number of training rounds to obtain a constructed neural network model.

9. The method for designing humanoid robot emotions based on a neural network model according to claim 8, characterized in that: The optimization of the neural network model by using a multi-objective optimization algorithm includes: Defining specific objectives of the multi-objective optimization, wherein the specific objectives include at least one of maximizing the sentiment score, maximizing the design satisfaction, and maximizing the prediction accuracy; For sentiment score maximization, construct the first objective function Among them, EmotionScore i is the sentiment score of the i-th sample, N1 is the total number of samples, and the negative sign indicates that the optimization goal is to maximize the sentiment score; To maximize the design satisfaction, the second objective function is constructed Among them, DesignSatisfaction i is the design satisfaction score of the i-th sample, and the negative sign indicates that the optimization goal is to maximize the design satisfaction; For the highest prediction accuracy, construct the third objective function PredictionError i is the prediction error of the i-th sample; Select a specified multi-objective optimization algorithm, apply the first objective function, the second objective function, the third objective function and the selected multi-objective optimization algorithm to the training process of the neural network model, optimize the objective function value, and obtain the optimized neural network model.

10. A humanoid robot emotion design system based on a neural network model, characterized in that: include: A speech emotion evaluation word sample collection module is used to generate a synthetic speech sample of the humanoid robot, collect and analyze the evaluation words of the user's subjective emotional response induced by the synthetic speech sample, and determine the speech emotion evaluation word sample; A combined emotion evaluation word sample collection module is used to generate appearance and voice combined video samples, collect and analyze words of user's subjective emotional response induced by the appearance and voice combined video samples, and determine the combined emotion evaluation word samples; A feature element extraction module, used to study and experiment the speech emotion evaluation word samples and the combined emotion evaluation word samples to extract feature elements; A scheme design module, used for conducting orthogonal experiments on the characteristic elements to design a scheme for generating appearance and voice combination elements; A neural network model building module, which is used to build a neural network model of the relationship between the user's subjective emotional response to the humanoid robot and the appearance and voice combination elements by using a feedforward neural network and a back propagation algorithm; The model optimization module is used to optimize the neural network model using a multi-objective optimization algorithm.

Images