Robot facial expression generation control method based on multi-particle swarm optimization algorithm

Abstract

The invention discloses a robot facial expression generation control method based on a multi-particle swarm optimization algorithm, and relates to the technical field of robot control. According to the method, motor parameters corresponding to facial expressions of a robot are abstracted into particles, populations and parameters are initialized, expressions are generated through physical mapping, and an RGB camera captures and processes images; the method comprises the following steps: firstly, obtaining a Pareto solution set, then respectively calculating expression accuracy, key motion unit activation degree and motion unit coordination degree by using three objective functions to obtain particle fitness, then updating the Pareto solution set and particle information, finally judging whether an end condition is met, and carrying out loop iteration until a natural and vivid target expression is generated. Manual calibration is not needed, the automation degree is high, the universality is good, computing resources can be saved, and the generated expressions conform to the human face movement rule and are vivid and real.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a robot facial expression generation control method based on a multi-particle swarm optimization algorithm. BACKGROUND

[0002] In the field of robot facial expression generation, a dynamic self-learning framework abstracts expression generation as an optimization problem of control parameters with physical constraints, and realizes global exploration and convergence of a high-dimensional solution space by constructing an objective function.

[0003] At present, there are various robot facial expression generation control methods on the market, but they all have certain shortcomings:

[0004] 1. The form of manual coding is used to write the pre-calibration control parameters corresponding to a specific expression into the controller, and the corresponding expression is generated by directly calling it when in use. This method requires a large amount of manual calibration for each expression in the early stage, and it is difficult to capture the details of the expression. The generated expression often falls into the uncanny valley effect, and the calibration method for each robot platform needs to be redesigned, which is time-consuming and laborious. The degree of automation is too low, the universality is poor, and through subjective manual coding, it is difficult to have a real and natural effect.

[0005] 2. The mapping relationship between the robot control points and the human facial muscle movement units is constructed, and then the human facial action coding system is used to complete the control of the robot. Data fitting, mixed shape model, deep neural network, etc. can be used to realize the construction of the mapping relationship, but this method has low interpretability, and the final expression effect is highly dependent on the process of fitting model and data quality, and the continuity and transition are not natural enough.

[0006] 3. Dynamic self-learning expression generation, this method automatically learns to generate facial expressions through an optimization feedback system. Part of it uses deep learning algorithms to extract features from a large amount of data sets, and part of it uses parameter optimization methods to continuously explore and evolve the current expression, but this method usually requires a lot of computing resources, and a single optimization goal often makes it difficult to achieve a natural and realistic expression effect, and it is easy to fall into a local optimal solution.

[0007] In summary, the existing robot facial expression generation control method still has obvious shortcomings in some aspects, and needs further improvement and innovation. SUMMARY

[0008] The purpose of the present application is to provide a robot facial expression generation control method based on a multi-particle swarm optimization algorithm.

[0009] To achieve the above purpose, the present application provides the following technical scheme:

[0010] A robot facial expression generation control method based on a multi-particle swarm optimization algorithm, comprising the following steps:

[0011] S1, initialize the population and set the parameters, regard the robot facial expression generated by each group of motor parameters as a particle individual, the position information of the particle is a vector Xi composed of motor parameters, the speed information of the particle is a vector Vi composed of the motor parameter change amount in the next unit time, and the fitness value of the particle is a vector Fi composed of three fitness values obtained by evaluating the objective function. Initialize the number of particle populations S, the iteration number G; initialize the Pareto solution set as an empty set; initialize the position vector and speed vector of each particle Pi, consider the physical constraints of the motor, and randomly generate a value normalized to [0, 1]; set the target expression such as happy, sad and angry.

[0012] S2, physical mapping and image capture processing, apply the position vector Xi (motor control parameter) of each particle Pi to the robot platform to generate the corresponding facial expression. Under fixed scene light, use an RGB camera to capture the image of the robot facial expression. The image is reduced, cropped, and the pixels in the key facial area are retained.

[0013] S3, multi-objective function design and calculation,

[0014] Objective function one: expression accuracy, input the captured robot facial expression image into the pre-trained expression recognition model, for the same expression, 10 times of continuous shooting are needed and input into the expression recognition model, and the output results are calculated by mean value to obtain the fitness value f of the objective function one i1 , which is used to evaluate the difference between the currently generated expression and the target expression.

[0015] Objective function two: key motion unit activation degree, based on the human facial motion coding system, the corresponding key muscle motion unit is determined for the target expression, then the mapping relationship between the key muscle motion unit and each motor on the robot hardware platform is confirmed to confirm the motor driving these key muscle motion units, and the sigmoid function

[0016] ;

[0017] Calculate the fitness value f of the key motion unit activation degree i2 , wherein α represents the driving degree of the motor corresponding to the key muscle motion unit, α0 represents the set threshold, ω m is the weight of each key motor, and k controls the steepness of the function near the threshold.

[0018] Objective function three: movement unit coordination degree, based on biological anatomy, aiming at different muscle groups, through the mapping relationship with the motor of the robot platform, converted into motor combination, the setting of reward and punishment coefficient, through the formula

[0019] ;

[0020] Calculate the fitness value f of the movement unit coordination degree i3 , wherein ω pq represents the pre-defined reward and punishment weight of each motor combination, AU p and AU q respectively represent the activation degree of the corresponding two motors in the combination, T p and T q respectively represent the pre-defined activation threshold of the two motors, and alpha represents the synergy gain coefficient.

[0021] The fitness values of the three objective functions are combined into the fitness vector Fi of the particle Pi: Fi = [fi1, fi2, fi3].

[0022] S4, update the Pareto solution set, compare the fitness Fi of the current particle Pi with the existing particles in the Pareto solution set (initially empty) one by one, judge the dominance relationship. If the current particle is not dominated by any particle in the solution set, add it to the solution set; if there is a particle in the solution set that is dominated by the current particle, kick out the particle. Get the updated Pareto solution set, which is used to save the optimal frontier solution set in the optimization process.

[0023] S5, update the particle information, select the individual best position value Xpbk and the group best position value Xgbk through the crowding degree, and use the formula

[0024] ;

[0025] ;

[0026] Update the velocity vector Vik+1 and the position vector Xik+1 of the next generation of each particle: wherein omega represents the inertia coefficient; r1 and r2 are random coefficients for increasing the diversity of the system solution set; c1 and c2 are individual learning coefficients and group learning coefficients respectively, which can be self-defined to adaptively change to adjust the optimization performance of the system.

[0027] S6, check the end condition and loop, after each particle of the current generation completes information update, the system judges whether the end condition is reached. If yes, stop the optimization iteration and output the Pareto solution set; if not, repeat step S2.

[0028] The beneficial effects of the present application are:

[0029] The three optimization objectives are designed in the method, and the physiological theory basis is combined therein, so that the generated robot facial expression conforms to the human facial movement law, and the flexibility and naturalness in the human expression generation process can be fully simulated. Through optimization of the three objective functions, the Pareto front solution set is recorded in the system learning process, and the high-quality front solutions in three directions are saved. In the later stage, the optimal solution can be selected according to the specific application or subjective factors. The self-learning process of the algorithm framework does not need manual calibration, and the optimization process can be automatically completed after starting to run, which saves a lot of manual labor. The algorithm framework can be applied to most bionic humanoid robot platforms, and can control the generation of various expressions, including basic expressions and some composite expressions, so it has excellent universality.

[0030] In the method, the number of iterations is determined as the end condition, which does not excessively occupy the computing resources in vain, and can explore the solution set meeting the requirements within a limited number of iterations through adaptive adjustment of learning parameters, record, and improve the performance of the finally generated expression while ensuring reasonable resource utilization.

[0031] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, and can be implemented according to the content of the specification, the following will be described in detail with the preferred embodiments of the present application and with the help of the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The flowchart of the control method shown in the present application. DETAILED DESCRIPTION

[0033] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. In the description of the present application, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through an intermediate medium, or internal communication of two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0034] As Figure 1 shown, the present application provides a robot facial expression generation control method based on a multi-particle swarm optimization algorithm, comprising the following steps:

[0035] S1, initialize the population and set the parameters, regard the robot facial expression generated by each group of motor parameters as a particle individual, the position information of the particle is a vector Xi composed of motor parameters, the speed information of the particle is a vector Vi composed of the motor parameter change amount in the next unit time, and the fitness value of the particle is a vector Fi composed of three fitness values obtained by evaluating the objective function. Initialize the number of particle populations S, the number of iterations G; initialize the Pareto solution set as an empty set; initialize the position vector and speed vector of each particle Pi, consider the physical constraints of the motor, and randomly generate values normalized to [0, 1]; set the target expression such as happy, sad, and angry.

[0036] S2, physical mapping and image capture processing, apply the position vector Xi (motor control parameters) of each particle Pi to the robot platform to generate the corresponding facial expression. Under fixed scene light, use an RGB camera to capture the image of the robot facial expression. Reduce, crop and retain the pixels in the key facial area.

[0037] S3, multi-objective function design and calculation,

[0038] Objective function one: expression accuracy rate, input the captured robot facial expression image into the pre-trained expression recognition model, for the same expression, need to continuously shoot 10 times and input into the expression recognition model, after the output result is calculated by mean, the fitness value f of objective function one is obtained i1 The objective function is used to evaluate the difference between the currently generated expression and the target expression.

[0039] Objective function two: key motion unit activation degree, based on human facial motion coding system, determine the corresponding key muscle motion unit for the target expression, then confirm the motor driving these key muscle motion units through the mapping relationship between the key muscle motion unit and each motor on the robot hardware platform, through the Sigmoid function

[0040] ;

[0041] Calculate the fitness value f of key motion unit activation degree i2 , where a represents the driving degree of the motor corresponding to the key muscle motion unit, a0 represents the set threshold, ω m is the weight of each key motor, k controls the steepness of the function near the threshold.

[0042] Objective function three: motion unit coordination degree, based on human anatomy, for different muscle combinations, through the mapping relationship with the robot platform motor, convert into motor combination, set the reward and punishment coefficient, through the formula

[0043] ;

[0044] Calculate the fitness value f of motion unit coordination degree i3 , where ω pq represents the reward and punishment weight of each motor combination defined in advance, AU p and AU q respectively represent the activation degree of the corresponding two motors in the combination, T p and T q respectively represent the activation threshold of the two motors defined in advance, a represents the synergy gain coefficient.

[0045] Combine the fitness values of the three objective functions into the fitness vector F of the particle Pi: Fi = f i1 , f i1 , f i1 ].

[0046] S4, update the Pareto solution set, compare the fitness Fi of the current particle Pi with the existing particles in the Pareto solution set (initially empty) one by one, and judge the dominance relationship. If the current particle is not dominated by any particle in the solution set, it is added to the solution set; if there is a particle in the solution set that is dominated by the current particle, the particle is kicked out. The updated Pareto solution set is obtained, which is used to save the optimal frontier solution set in the optimization process.

[0047] S5, update the particle information, select the individual best position value Xpbk and the group best position value Xgbk by crowding degree, and use the formula

[0048] ;

[0049] ;

[0050] Update the velocity vector Vik+1 and the position vector Xik+1 of each particle in the next generation: where ω represents the inertia coefficient; r1 and r2 are random coefficients for increasing the diversity of the system solution set; c1 and c2 are the individual learning coefficient and the group learning coefficient, respectively, which can be self-defined to adaptively change and adjust the optimization performance of the system.

[0051] S6, check the end condition and loop, after each particle in the current generation completes information update, the system judges whether the end condition is reached. If so, stop the optimization iteration and output the Pareto solution set; if not, repeat step S2.

[0052] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure.

[0053] The above-described embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1.A robot facial expression generation control method based on a multi-particle swarm optimization algorithm, characterized by, The method comprises the following steps: S1: initializing the population and setting parameters, taking the robot facial expression corresponding to each set of motor parameters as a particle individual, initializing the number of particle populations S, the number of iterations G, initializing the Pareto solution set as an empty set, initializing the position vector and velocity vector of each particle Pi and normalizing them to [0, 1], and setting the target expression; S2: physical mapping and image capture processing, applying the position vector Xi of each particle Pi to the robot platform to generate the corresponding facial expression, using an RGB camera to capture the image of the robot facial expression under fixed scene lighting, and reducing and cropping the image to retain the pixels of the key facial region; S3: Multi-objective function design and calculation, respectively through the target function one to calculate the expression accuracy to get fitness value f i1 , through the target function two to calculate the key motion unit activation degree to get fitness value f i2 , through the target function three to calculate the motion unit coordination degree to get fitness value f i3 , combine the three fitness values into the fitness vector Fi of the particle Pi; S4: updating the Pareto solution set, comparing the fitness Fi of the current particle Pi with the existing particles in the Pareto solution set one by one, judging the dominance relationship, if the current particle is not dominated by any particle in the solution set, adding the particle to the solution set, if there is a particle in the solution set that is dominated by the current particle, kicking out the particle, and obtaining the updated Pareto solution set; S5: update particle information, select individual best position value X by crowding degree pb k and population best position value X gb k and use the formula ； ； updating the velocity vector V of each particle for the next generation i k+1 and the position vector X i k+1 where ω represents the inertia coefficient, r1 and r2 are random coefficients, and c1 and c2 are the individual learning coefficient and the group learning coefficient, respectively; S6: checking the end condition and the loop effect, after each particle of the current generation completes information updating, judging whether the end condition is reached, if yes, stopping the optimization iteration, outputting the Pareto solution set, if not, repeating step S2. 2.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S3, the specific calculation process of the objective function one is as follows: input the captured and processed facial expression image of the robot into the pre-trained expression recognition model, continuously capture 10 times for the same expression and input the model, calculate the output result to obtain the fitness value f of the objective function one i1 . 3.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S3, the target function two calculates the activation degree of the key motion unit through a Sigmoid function ； wherein a denotes the driving degree of the motor corresponding to the key muscle motor unit, a0denotes a set threshold value, ω m is the weight of each key motor, and k controls the steepness of the function near the threshold value. 4.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S3, the target function three is calculated by the formula ； The degree of coordination of the computational motor units is calculated, where ω pq represents the predefined reward and penalty weights of each motor combination, AU p and AU q respectively represent the activation degree of the corresponding two motors in the combination, T p and T q respectively represent the predefined activation threshold of the two motors, and α represents the synergy gain coefficient. 5.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S1, the position information of the particle is a vector Xi composed of motor parameters, the velocity information of the particle is a vector Vi composed of the change amount of motor parameters in the next unit time, and the fitness value of the particle is a vector Fi composed of three fitness values evaluated by the target function. 6.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S5, the individual learning coefficient c1 and the group learning coefficient c2 can be adaptively changed to adjust the optimization performance of the system. 7.The robot facial expression generation control method based on a multi-particle swarm optimization algorithm according to claim 1, wherein In step S6, the end condition is to reach the set number of iterations G.

Images