Robot simulation eyeball control method, device, equipment and storage medium
By defining the eye movement template and preset emotion template corresponding to emotions, the control parameters of the robot simulates the eyeball are determined, and the problem of robots in the existing technology is difficult to truly simulate human eye expressions, realizing the vivid eyes of the robot under different emotions The expression of the relationship enhances the authenticity of emotional communication.
Patent Information
- Application Number
- CN202411067044.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-06
AI Technical Summary
When existing robots control facial expressions, especially eye movements, to express emotions, it is difficult to efficiently and realistically simulate human expressions, resulting in complex and costly expression control mechanisms, which compresses other functions of the robot.
By defining the eye movement template corresponding to emotions, and combining emotion values and intensity, the robot's eye expression in different emotions is determined. The preset emotion template is used to determine eye control parameters, such as blinking parameters, tearing parameters, eye rotation parameters and pupil parameters. , adjust the movements of the robot's simulated eyeballs in real time.
It realizes that the robot accurately and vividly expresses eye movements under different emotional states, enhances the realistic and realistic feeling of emotional expression, and allows the robot to convey emotions more effectively when interacting with people.
Smart Images

Figure CN118664618B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of robotics technology, and in particular to a method, device, equipment and storage medium for controlling simulated eyeballs of a robot. Background Art
[0002] With the development of the robotics industry, humanoid robots are becoming more and more popular among users. Humanoid robots have similar appearance features to humans and can perform some basic actions that imitate humans. In the research of humanoid robots, facial expression realization is an important basis for anthropomorphism.
[0003] Due to the complexity of human facial expressions, the current way to enable robots to imitate human expressions is to control the expression movement units one by one, which often requires complex structural design and control methods, making the entire expression control mechanism bulky, compressing the robot's other functions, and also increasing the robot's production cost. Summary of the invention
[0004] The embodiments of the present application provide a method, device, equipment and storage medium for controlling simulated eyeballs of a robot, which can solve the technical problem of how to improve the emotional expression effect of the robot's eyes.
[0005] In a first aspect, an embodiment of the present application provides a robot simulated eyeball control method, characterized by comprising:
[0006] Get the preset emotion template, which includes: emotion value, emotion intensity and eye movement.
[0007] The eyeball control parameters are determined according to the preset emotion template, and the eyeball control parameters include at least one of the following: a blinking parameter, a tearing parameter, an eyeball rotation parameter, and a pupil parameter.
[0008] Adjust the eye movements of the robot's simulated eyeballs according to the eyeball control parameters.
[0009] In one implementation, obtaining a preset emotion template includes: obtaining emotion training images to obtain an emotion training image set; annotating the emotion training images with emotion values, emotion intensity, and eye movements to obtain annotated emotion training images; and performing cluster analysis on the annotated emotion training images to obtain a preset emotion template.
[0010] In one implementation, the mapping relationship between the emotion value, emotion intensity, and eye movement is expressed as:
[0011] (E,S)→{P(i 1 ), P(i 2 ),… ,P(i n )};
[0012] Among them, E represents the emotion value, and the value range of E is an integer from 1 to 20; S represents the emotion intensity, and the value range of S is an integer from 1 to 5; P(i) represents eye movement, and n is a positive integer.
[0013] In one implementation, eye control parameters are determined according to a preset emotion template, including: performing emotion simulation control on a simulated eye according to the preset emotion template to obtain basic control parameters; performing cluster analysis on the basic control parameters to determine the basic control parameters that meet preset conditions as eye control parameters.
[0014] In one implementation, the eye movements of the robot's simulated eyeball are adjusted according to eyeball control parameters, including: according to the eyeball control parameters, using a first servo to control the robot's simulated eyeball to move in a horizontal direction; according to the eyeball control parameters, using a second servo to control the robot's simulated eyeball to move in a vertical direction; according to the eyeball control parameters, using a third servo to control the movement of the upper eyelid of the robot's simulated eyeball; according to the eyeball control parameters, using a fourth servo to control the movement of the lower eyelid of the robot's simulated eyeball.
[0015] In one implementation, adjusting the eye movements of the robot simulated eyeball according to the eyeball control parameters also includes: according to the eyeball control parameters, using a liquid pump to control the robot simulated eyeball to discharge or retract preset simulated tears.
[0016] In one implementation, the mapping relationship between the eye movement and the first steering gear, the second steering gear, the third steering gear, the fourth steering gear, and the liquid pump is expressed as:
[0017] P(i)→{D(j 1 )、D(j 2 ), D(j 3 )、D(j 4 )、D(j 5 )、D(j 6 )};
[0018] Where P(i) represents eye movement; D(j 1 ) represents the first servo; D(j 2 ) represents the second servo; D(j 3 ) represents the third servo; D(j 4 ) represents the fourth servo; D(j 5 ) represents the first liquid pump; D(j 6 ) represents the second liquid pump.
[0019] In a second aspect, an embodiment of the present application provides a robot simulated eyeball control device, which has the function of implementing the method in the first aspect or any possible implementation thereof. Specifically, the device includes a unit that implements the method in the first aspect or any possible implementation thereof.
[0020] In one embodiment, the device comprises:
[0021] The acquisition unit is used to acquire a preset emotion template, which includes: emotion value, emotion intensity and eye movement.
[0022] The processing unit is used to determine eye control parameters according to a preset emotion template, where the eye control parameters include at least one of the following: a blinking parameter, a tearing parameter, an eye rotation parameter, and a pupil parameter.
[0023] The processing unit is also used to adjust the eye movements of the robot's simulated eyeballs according to the eyeball control parameters.
[0024] In one embodiment, the acquisition unit is further used to acquire a preset emotion template, including: acquiring an emotion training image to obtain an emotion training image set;
[0025] The processing unit is also used to annotate the emotion training image with emotion value, emotion intensity and eye movement to obtain annotated emotion training image; and perform cluster analysis on the annotated emotion training image to obtain a preset emotion template.
[0026] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device implements a method according to any one of the implementation modes of the first aspect.
[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a computer device, the computer device implements the method of any one of the implementation methods of the first aspect above.
[0028] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes a method of any one of the implementation modes of the first aspect above.
[0029] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by defining eye movement templates corresponding to various emotions, and combining the emotion value and intensity, the eye expressions that the robot should show under different emotions are determined. Through these templates, the eye movements under different emotional states can be accurately defined and described; the eye control parameters (blinking parameters, tear parameters, eye rotation parameters and pupil parameters) are determined according to the preset emotion templates, and the emotion value and emotion intensity are converted into specific eye movement parameters, such as blinking frequency, speed and direction of eye rotation, pupil size, etc.; according to the eye control parameters, the robot's simulated eyeballs are adjusted and controlled in real time to show the preset eye movements. By adjusting the blinking, tearing, rotation and pupil states of the eyes, the robot can simulate the corresponding eye expressions under specific emotional states, so that the robot can convey emotions and communicate more vividly when interacting with people. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 This is a switching scene diagram of a robot expression provided in an embodiment of the present application.
[0031] Figure 2 It is a flow chart of a robot simulation eyeball control method provided in an embodiment of the present application.
[0032] Figure 3 This is an application scenario diagram of a liquid pump provided in an embodiment of the present application.
[0033] Figure 4 It is a schematic diagram of the structure of the device provided in the embodiment of the present application.
[0034] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0035] With the development of artificial intelligence, the technology of intelligent companion robots has made great progress. These robots have a friendly appearance and can perform voice communication, emotion recognition, activity reminders and other functions. They can interact with users in various ways to meet their social and entertainment needs.
[0036] Figure 1 This is a switching scene diagram of a robot expression provided in an embodiment of the present application.
[0037] like Figure 1 As shown in the figure, when the robot interacts with the user, it often needs to express the corresponding emotions through the change of facial expressions. For example, switching from a basic expression to a smiling face or hard work, etc.
[0038] However, current robots have many problems controlling facial expressions, especially controlling eye movements to express emotions, making it difficult to simulate human expressions efficiently and realistically.
[0039] In order to further illustrate the technical solution of the present application, specific embodiments are provided below.
[0040] Figure 2 It is a flow chart of a robot simulation eyeball control method provided in an embodiment of the present application.
[0041] like Figure 2 As shown, the above method includes the following steps S201~S203.
[0042] S201, obtaining a preset emotion template.
[0043] The preset emotion templates include: emotion value, emotion intensity and eye movement.
[0044] Emotional value: refers to the type or kind of a specific emotional state, such as joy, anger, sadness, etc. Emotional value describes the emotional state of a human being.
[0045] Emotional intensity: This refers to the intensity or degree of an emotion. Some emotions can be very strong, such as extreme sadness or extreme anger, while others can be milder.
[0046] Eye movements: Eye movements also play an important role in emotional expression. Different emotional states are often accompanied by specific eye movements, such as blinking frequency, changes in pupil size, speed and pattern of eye movement, etc.
[0047] These elements combined together can describe the physiological and behavioral characteristics of a human emotion.
[0048] S202: Determine eye control parameters according to a preset emotion template.
[0049] The eye control parameters include at least one of the following: a blink parameter, a tear parameter, an eye movement parameter, and a pupil parameter.
[0050] Suppose we select the emotion value as "joy", the emotion intensity as "medium", and set the eye action to "increase blinking frequency".
[0051] In this case, we can set the following eye control parameters: Blink Parameters: Increase the frequency of blinking, which means a dramatic increase in blinking movements to simulate the active expression of eyes when happy. Eye Movement Parameters: Increase the speed and amplitude of eye movement to show the activeness of sight when happy. Pupil Parameters: Slightly dilate to simulate excitement and comfort.
[0052] By setting these eye control parameters, the robot's simulated eyes can show eye movements in different emotional states, which helps to enhance the realism and authenticity of emotional expression.
[0053] S203, adjusting the eye movement of the robot's simulated eyeball according to the eyeball control parameters.
[0054] In one example, the control mechanism at the upper level (such as an artificial brain) can be controlled by sending emotion codes. The data interfaces that can be used are as follows:
[0055] (1 [Coding of control method]; 2 [Coding of interaction data]; 3 [Coding of emotion intensity]).
[0056] It can be understood that these three levels of interfaces can all be implemented through data interfaces: emotion control interface, motion control interface, and electromechanical control interface.
[0057] There are three types of control methods: emotional expression coding; direct eye combination movement coding; and local eye movement control coding.
[0058] The artificial brain can manipulate the artificial eyeball using these three control methods.
[0059] Different eye movements and expressions can be achieved according to different control mode coding (such as emotion expression coding, direct eye combination action coding, and local eye movement control coding).
[0060] Suppose there is an artificial intelligence robot with an artificial eyeball built into its head, and the eyeball system is controlled by an artificial brain. In different scenarios, the artificial brain can control the movement of the artificial eyeball through different coding methods:
[0061] Emotional expression coding: If the robot needs to show specific emotions, such as happiness, sadness, anger, etc., the artificial brain can send corresponding signals to the eye system through emotional expression coding, so that the eyeballs can show corresponding emotional expressions, such as blinking, widening eyes, etc.
[0062] Direct eye combination action coding: In some cases where specific eye combination actions are required, the artificial brain can use direct eye combination action coding to control the eye system to achieve specific eye movement combinations, such as widening the left eye and turning the right eye to the left.
[0063] Local eye movement control coding: If fine control of local eye movement is required, such as looking at a specific object, scanning the surrounding environment, etc., the artificial brain can use local eye movement control coding to send corresponding instructions to achieve fine eye movement control.
[0064] Each eye movement has specific actions that are interpreted as individual motor control components:
[0065] 1) Opening and closing of upper (lower) eyelids, speed, frequency, and number of repetitions.
[0066] 2) Tear gland secretion, intensity, and number of repetitions.
[0067] 3) Pupil zoom, intensity, and number of repetitions.
[0068] 4) Eye movement.
[0069] Through the control structure of this artificial eye system, the artificial brain can manipulate the artificial eye through different coding methods, thereby achieving rich and diverse eye movements and expressions, making the robot look more intelligent and realistic.
[0070] By adjusting the position, movement, size, shape and other parameters of the simulated eyeballs and combining them with other emotion expression signals, the simulated eyeballs can be effectively controlled to express emotions.
[0071] Assuming that the control method is coded as [emotional expression], the interaction data is coded as one of the following 20 types, and the addition of the emotion intensity coding can achieve eye control.
[0072] The following is a detailed introduction to emotion coding, corresponding eye movements, and emotion intensity coding in conjunction with Tables 1 to 6.
[0073] Table 1 introduces the codes and emotional meanings representing positive emotions.
[0074] Table 2 shows the coding and emotional meaning of negative emotions.
[0075] Table 3 shows the eye movements of happy emotions.
[0076] Table 4 shows the eye movements associated with sadness.
[0077] Table 5 shows the eye movements associated with anger.
[0078] Table 6 shows the eye movements associated with anxiety and disappointment.
[0079] Table 1
[0080]
[0081] Table 2
[0082]
[0083] Here are some eye movements that correspond to some emotions.
[0084] As shown in Tables 3 to 6 below.
[0085] Table 3
[0086]
[0087] Table 4
[0088]
[0089] Table 5
[0090]
[0091] Table 6
[0092]
[0093] There is a close connection between pupil changes and emotions. The following is a detailed explanation of the relationship between pupil changes and emotions:
[0094] 1. Emotional expression of pupil dilation.
[0095] When emotions change from neutral to excited and happy, pupils will unconsciously dilate. For example, when people see someone or something they like, their pupils will dilate. This reaction is driven by human physiological mechanisms. For example, when a man sees an attractive woman or a woman sees a handsome man, there will be a reaction of dilated pupils, which reflects people's physiological response to attraction. In certain activities, such as playing poker, if you get the expected good cards, emotional excitement will rise sharply, and the pupils will also dilate. This is a direct reflection of emotional state in pupil size.
[0096] 2. Emotional expression of pupil constriction.
[0097] On the contrary, when people see someone or something they don't like, their pupils will shrink. This is also a physiological reaction that is closely related to people's emotional state. For example, when people feel nervous or uneasy, their pupils will shrink. This change may indicate that people are experiencing some kind of stress or anxiety.
[0098] 3. The relationship between pupil changes and emotions.
[0099] The change of pupil is not only a physiological reaction, but also an important manifestation of emotional state. By observing the change of pupil size, we can understand a person's emotional state, such as excitement, happiness, tension, anxiety, etc. The association between pupil change and emotion is complex, and it is affected by many factors, such as light, drugs, diseases, etc. However, under the condition of controlling these factors, pupil change can be used as a reliable indicator to reflect people's emotional state.
[0100] 4. The physiological mechanism of pupil changes.
[0101] The changes in the pupil are controlled by the contraction and relaxation of the iris muscle. Contraction of the iris muscle will make the pupil smaller, while relaxation will make the pupil larger. This change is regulated by the autonomic nervous system, especially the sympathetic and parasympathetic nerves. When the sympathetic nerves are excited, the pupil will become larger, and when the parasympathetic nerves are excited, the pupil will become smaller. This change is closely related to people's emotional state, because the emotional state will affect the activity of the autonomic nervous system.
[0102] In one example, the automatic aperture technology of the camera is used to achieve the pupil zooming action. First, the camera needs to obtain the light information around the pupil of the robot's eye area in real time. This light information can be used to determine the light intensity, so as to determine whether the pupil size needs to be adjusted. The acquired light information will be passed to the processor for analysis. The processor can use a pre-set algorithm or rule to determine whether the pupil size needs to be adjusted based on the light intensity. If the light is strong, the pupil may need to be reduced; if the light is dim, the pupil may need to be enlarged. According to the analysis results of the processor, the pupil actuator is controlled to perform corresponding actions. The pupil actuator can be an electric actuator that adjusts the size of the robot's pupil by controlling the current or signal. According to the algorithm and control logic, the pupil actuator is controlled to gradually adjust the size of the pupil to adapt it to the current light intensity. The movement of contraction or expansion can be simulated on the pupil to achieve a pupil zooming effect similar to that of a real eye.
[0103] Through the above steps, the automatic aperture technology of the camera can be used to realize the zooming action of the robot's eye pupil, making the robot's eye look more vivid and realistic.
[0104] In one example, a micro-camera can be placed inside the pupil to enable image acquisition.
[0105] In one implementation, obtaining a preset emotion template includes: obtaining emotion training images to obtain an emotion training image set; annotating the emotion training images with emotion values, emotion intensity, and eye movements to obtain annotated emotion training images; and performing cluster analysis on the annotated emotion training images to obtain a preset emotion template.
[0106] In one implementation, the mapping relationship between the emotion value, emotion intensity, and eye movement is expressed as:
[0107] (E,S)→{P(i 1 ), P(i 2 ),… ,P(i n )}; (1)
[0108] Among them, E represents the emotion value, and the value range of E is an integer from 1 to 20; S represents the emotion intensity, and the value range of S is an integer from 1 to 5; P(i) represents eye movement, and n is a positive integer. For example, P(1) is blinking, P(1)=1 means the blink intensity is 1, P(2) is tearing, P(3) is eye movement, and P(4) is pupil dilation.
[0109] Formula 1 shows that an emotion value may be composed of multiple eye movements. The calling data interface of the simulated eyeball is (E, S, T, F), where T is the time to complete a single emotion performance and F is the number of times the emotion performance is repeated.
[0110] In one example, facial close-ups were screened from a large number of emotion training images, and the emotion value (E) and emotion intensity (S) of each picture as well as the specific eye movement intensity were marked. Cluster analysis was then performed on the relevant data of each emotion value, and finally the eye movement data that met the emotional judgment of most people was selected as the preset emotion template database.
[0111] When performing cluster analysis on the labeled emotion training images, the following methods can be used:
[0112] K-Means Clustering: This is one of the most commonly used clustering methods. It divides the data into K clusters so that each data point is assigned to the closest cluster to minimize the variance within the cluster. K-Means clustering can be used to divide emotional training images into different groups to obtain preset emotional templates.
[0113] Hierarchical Clustering: This method gradually merges data points into a hierarchical structure based on their similarity, forming a tree-like clustering structure. Hierarchical clustering can demonstrate the similarity between emotion training images and obtain clustering results of different scales.
[0114] Density-Based Clustering: This method clusters data points based on their density and can effectively handle data sets with noise and outliers. Density clustering can find clusters of any shape and is suitable for processing complex data distributions.
[0115] Spectral Clustering: Spectral clustering is a clustering method based on graph theory. It uses the similarity matrix between data points to perform eigenvalue decomposition and project the data points into the feature space for clustering. Spectral clustering is suitable for data distribution that is non-spherical or has a complex structure.
[0116] DBSCAN clustering (Density-Based Spatial Clustering of Applications with Noise): DBSCAN is a density-based clustering algorithm that can effectively identify clusters of any shape and can automatically process noisy data. DBSCAN is sensitive to the density and distribution of clusters and is suitable for processing data sets with different density areas.
[0117] Specifically, a suitable method is selected to perform cluster analysis of emotion training images according to the characteristics and requirements of the data set to obtain a preset emotion template, which is not limited here.
[0118] In one implementation, eye control parameters are determined according to a preset emotion template, including: performing emotion simulation control on a simulated eye according to the preset emotion template to obtain basic control parameters; performing cluster analysis on the basic control parameters to determine the basic control parameters that meet preset conditions as eye control parameters.
[0119] In this implementation, we first need to define a series of eye movement templates corresponding to different emotions, namely preset emotion templates. Each emotion template contains a set of characteristic parameters of eye movements, such as horizontal movement of the eyeball, vertical movement, blinking frequency, eyelid opening and closing speed, etc.
[0120] Once there is a preset emotion template, the system can select the corresponding emotion template according to the current emotional state or external input (such as emotion information obtained from the emotion recognition system). The system will determine the basic control parameters, that is, the basic movements of controlling eyeball simulation, based on the eye movement feature parameters defined in the selected emotion template.
[0121] Next, cluster analysis is performed on the basic control parameters. Cluster analysis is a data mining technique used to classify data points in a data set into different categories based on their feature similarities. In this implementation, the basic control parameters can be regarded as data points, and the basic control parameters corresponding to different emotions can be regarded as data sets with similar features.
[0122] Through cluster analysis, the system can classify basic control parameters with similar eye movement characteristics into the same category. Finally, the system will determine the eye control parameters that best suit the current emotion based on the preset emotion template and the results of cluster analysis. In this way, the process of determining eye control parameters based on the preset emotion template is realized, so that the artificial eye can express different emotional states with different eye movements.
[0123] In one implementation, the eye movements of the robot's simulated eyeball are adjusted according to eyeball control parameters, including: according to the eyeball control parameters, using a first servo to control the robot's simulated eyeball to move in a horizontal direction; according to the eyeball control parameters, using a second servo to control the robot's simulated eyeball to move in a vertical direction; according to the eyeball control parameters, using a third servo to control the movement of the upper eyelid of the robot's simulated eyeball; according to the eyeball control parameters, using a fourth servo to control the movement of the lower eyelid of the robot's simulated eyeball.
[0124] It can be understood that the movement control of the eyeball adopts spherical motor control technology, which requires two degrees of freedom and is controlled by two servos. The upper and lower eyelids are each controlled by a servo, and a total of four servos are required.
[0125] It is understandable that each eyeball can be controlled by an independent electronic control device. The electronic control devices of different eyeballs can communicate and exchange data in real time to achieve movement synchronization between the eyeballs. For example, wireless communication technology, such as Bluetooth or Wi-Fi, can be used to enable the electronic control devices of each eyeball to transmit control information to each other. The electronic control device of each eyeball can receive instructions through the main controller, and then perform corresponding eyeball movements according to the received instructions. The main controller can be responsible for integrating the emotional expression needs of all eyeballs, and sending unified commands to the electronic control devices of each eyeball through information communication.
[0126] Through the above method, each eyeball can be independently controlled, and the movement synchronization between the eyeballs can be ensured through information communication, so that the robot's eyes can express emotions and communicate with people in a more coordinated and vivid way.
[0127] In an example, the underlying interface can be opened here to directly define the data interface of artificial eye movement, as shown below:
[0128] (1 Action name; 2 Movement mode (movement route; acceleration in the acceleration phase; duration of acceleration; movement speed in the uniform phase; negative acceleration in the stop phase; duration of the deceleration phase); 3 Number of repetitions). The specific motor control can be set according to the actual situation.
[0129] In the eye movement control system, opening the underlying interface allows users to directly define the data interface of artificial eye movement, thereby customizing and controlling the various parameters and action implementations of eye movement.
[0130] In other words, providing such an interface allows the user to control various eye movements by defining parameters such as the action name, movement mode, and number of repetitions. For example, the eye movement path, acceleration, acceleration duration, movement speed, negative acceleration, deceleration duration, and other movement parameters, as well as the number of repetitions of the action, can be specified.
[0131] By opening up the underlying interface, users can flexibly customize and control eye movements through programming or other means to achieve various eye movements and expressions, thereby realizing more sophisticated and personalized eye control functions.
[0132] As an example, when the robot needs to look at a target on the right, the first servo will be adjusted according to the eye control parameters to move the robot's simulated eye horizontally to the right so that the eye focuses on the target.
[0133] As an example, assuming that the robot needs to look up or down at an object, according to the eye control parameters, the second servo can be controlled to move the robot's simulated eye in the vertical direction to achieve the action of looking up or looking down.
[0134] As an example, when it is necessary to simulate the action of blinking or opening eyes, according to the eye control parameters, the third servo can be adjusted to control the movement of the eyelid of the robot's simulated eyeball to achieve the effect of blinking or opening eyes.
[0135] As an example, similarly, according to the eyeball control parameters, the fourth servo can be controlled to move the lower eyelid of the robot's simulated eyeball to achieve movement of the lower eyelid of the eyeball, such as blinking, closing eyes and the like.
[0136] By adjusting the control parameters of these four servos, the robot can achieve various eye movements that simulate real eyeballs, thereby presenting more realistic and vivid eye expressions.
[0137] In one implementation, adjusting the eye movements of the robot simulated eyeball according to the eyeball control parameters also includes: according to the eyeball control parameters, using a liquid pump to control the robot simulated eyeball to discharge or retract preset simulated tears.
[0138] Combine the following Figure 3 Let me introduce it in detail.
[0139] Figure 3 This is an application scenario diagram of a liquid pump provided in an embodiment of the present application.
[0140] like Figure 3 As shown, a small water bag is used to store salt water. A thin hose connects the water bag, the liquid pump and the tear-releasing opening on the inner side of the upper eyelid. A drainage opening is left on the inner side of the lower eyelid to recycle tears into the water bag. The controller can control the speed and duration of the liquid pump to discharge and recycle salt water, thereby simulating effects such as slightly moist eyeballs, eyes filled with tears, and tears streaming down the face.
[0141] In one implementation, the mapping relationship between the eye movement and the first steering gear, the second steering gear, the third steering gear, the fourth steering gear, and the liquid pump is expressed as:
[0142] P(i)→{D(j1 )、D(j 2 ), D(j 3 )、D(j 4 )、D(j 5 )、D(j 6 )}; (2)
[0143] Where P(i) represents eye movement; D(j 1 ) represents the first servo; D(j 2 ) represents the second servo; D(j 3 ) represents the third servo; D(j 4 ) represents the fourth servo; D(j 5 ) represents the first liquid pump; D(j 6 ) represents the second liquid pump.
[0144] First, according to formula (2), all possible eye movements corresponding to emotions are actually performed on the simulated eyeball, and the corresponding P(i) values are judged. The relevant data of each eye movement are clustered and analyzed, and the data that conforms to the common sense and emotional judgment of most people are selected as the basic control parameters of eye movement. Then, according to formula (1), all possible situations are actually performed on the simulated eyeball, and the corresponding (E, S) values are judged. The relevant data of each emotional value are clustered and analyzed, and the eye movement data that conforms to the common sense and emotional judgment of most people are selected as the preset emotional template.
[0145] The above mainly introduces a robot simulation eyeball control method of an embodiment of the present application in conjunction with the accompanying drawings. It should be understood that although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence, these steps are not necessarily performed in sequence in the order shown in the figure. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a part of the steps or stages in other steps. The following is an introduction to a device of an embodiment of the present application in conjunction with the accompanying drawings. For the sake of brevity, when introducing the device below, it will be appropriately omitted, and the relevant content can refer to the relevant description in the method above, and will not be repeated.
[0146] Figure 4 It is a structural schematic diagram of a robot simulation eyeball control device provided in an embodiment of the present application.
[0147] like Figure 4 As shown, the device 1000 includes the following units.
[0148] The acquisition unit is used to acquire a preset emotion template, which includes: emotion value, emotion intensity and eye movement.
[0149] The processing unit is used to determine eye control parameters according to a preset emotion template, where the eye control parameters include at least one of the following: a blinking parameter, a tearing parameter, an eye rotation parameter, and a pupil parameter.
[0150] The processing unit is also used to adjust the eye movements of the robot's simulated eyeballs according to the eyeball control parameters.
[0151] In one implementation, the device 1000 further includes a storage unit 1003, which can be used to store instructions and / or data, so as to implement the method in the above embodiment.
[0152] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0153] Figure 5 Schematic diagram of the structure of the computer device provided in the embodiment of the present application. Figure 5 As shown, the computer device 3000 of this embodiment includes: at least one processor 3100 ( Figure 5 Only one) processor, memory 3200, and a computer program 3210 stored in the memory 3200 and executable on at least one processor 3100 are shown. When the processor 3100 executes the computer program 3210, the computer device implements the steps in the above embodiments.
[0154] The processor 3100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0155] The memory 3200 may be an internal storage unit of the computer device 3000 in some embodiments, such as a hard disk or memory of the computer device 3000. The memory 3200 may also be an external storage device of the computer device 3000 in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3000. Further, the memory 3200 may also include both an internal storage unit of the computer device 3000 and an external storage device. The memory 3200 is used to store an operating system, an application program, a boot loader (Boot Loader) data, and other programs, such as a program code of a computer program, etc. The memory 3200 may also be used to temporarily store data that has been output or is to be output.
[0156] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0157] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a computer device, the computer device can implement the steps in the above-mentioned method embodiments.
[0158] The embodiment of the present application provides a computer program product. When the computer program product is executed on a computer device, the computer device can implement the above-mentioned methods.
[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the computer device can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0160] It should be understood that the size of the sequence number of each step in the above-described embodiment does not mean the order of execution, and the order of execution of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiment of the present application. In the description, for the purpose of explanation rather than for limitation, specific details such as specific system structure, technology and the like are proposed, so as to thoroughly understand the embodiment of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed description of well-known systems, devices, circuits and methods is omitted to prevent unnecessary details from hindering the description of the present application.
[0161] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0162] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0163] Additionally, in the specification of the present application and the description of the appended claims, the terms “include,” “comprise,” “have,” and their variations all mean “including but not limited to,” unless otherwise specifically emphasized.
[0164] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0165] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0166] In the embodiments provided in the present application, it should be understood that the disclosed devices, computer equipment and methods can be implemented in other ways. For example, the device and computer equipment embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0167] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A robot simulation eyeball control method, It is characterized in that include: Acquire a preset emotion template, wherein the preset emotion template includes: emotion value, emotion intensity and eye movement; Determine eyeball control parameters according to a preset emotion template, wherein the eyeball control parameters include: blinking parameters, tearing parameters, eyeball rotation parameters and pupil parameters; The eye movements of the robot simulation eyeballs are adjusted according to the eyeball control parameters, each robot simulation eyeball is controlled by an independent electric control device, and the electric control devices use wireless communication technology to perform real-time communication and data exchange to achieve movement synchronization between the eyeballs; The step of adjusting the eye movement of the robot's simulated eyeball according to the eyeball control parameter further includes: The robot simulated eyeball comprises a water bag, a first conduit, a second conduit, a first liquid pump, a second liquid pump, an upper eyelid tear discharge hole, and a lower eyelid tear collection hole, wherein the first liquid pump is connected to the water bag via the first conduit, and the second liquid pump is connected to the water bag via the second conduit; According to the eyeball control parameters, the first liquid pump is used to control the robot simulated eyeball to discharge preset simulated tears from the upper eyelid tear hole, and the second liquid pump is used to collect the preset simulated tears from the lower eyelid tear hole.
2. The method according to claim 1, It is characterized in that The step of obtaining a preset emotion template includes: Acquire emotion training images to obtain an emotion training image set; Annotating the emotion training image with emotion value, emotion intensity and eye movement to obtain an annotated emotion training image; Cluster analysis is performed on the labeled emotion training images to obtain a preset emotion template.
3. The method according to claim 1, It is characterized in that The mapping relationship between the emotion value, emotion intensity and eye movement is expressed as: (E,S)→{P(i 1 ),P(i 2 ),… ,P(i n )}; Among them, E represents the emotion value, and the value range of E is an integer from 1 to 20; S represents the emotion intensity, and the value range of S is an integer from 1 to 5; P(i) represents eye movement, and n is a positive integer.
4. The method according to claim 1, It is characterized in that Determining the eye control parameters according to the preset emotion template includes: Performing emotion simulation control on the artificial eyeball according to the preset emotion template to obtain basic control parameters; A cluster analysis is performed on the basic control parameters, and the basic control parameters that meet preset conditions are determined as the eye control parameters.
5. The method according to claim 1, It is characterized in that The step of adjusting the eye movement of the robot's simulated eyeball according to the eyeball control parameter comprises: According to the eyeball control parameters, using the first servo to control the robot's simulated eyeball to move in the horizontal direction; According to the eyeball control parameters, using the second servo to control the robot's simulated eyeball to move in the vertical direction; According to the eyeball control parameters, using a third servo to control the movement of the upper eyelid of the robot's simulated eyeball; According to the eyeball control parameters, the fourth servo is used to control the movement of the lower eyelid of the robot's simulated eyeball.
6. The method according to claim 5, It is characterized in that The mapping relationship between the eye movement and the first steering gear, the second steering gear, the third steering gear, the fourth steering gear, and the liquid pump is expressed as: P(i)→{D(j 1 ),D(j 2 ), D(j 3 ),D(j 4 ),D(j 5 ),D(j 6 )}; Where P(i) represents eye movement; D(j 1 ) represents the first servo; D(j 2 ) represents the second servo; D(j 3 ) represents the third servo; D(j 4 ) represents the fourth servo; D(j 5 ) represents the first liquid pump; D(j 6 ) represents the second liquid pump.
7. A robot simulation eyeball control device, It is characterized in that include: An acquisition unit, used to acquire a preset emotion template, wherein the preset emotion template includes: emotion value, emotion intensity and eye movement; A processing unit, used to determine eyeball control parameters according to a preset emotion template, wherein the eyeball control parameters include: blinking parameters, tearing parameters, eyeball rotation parameters and pupil parameters; The processing unit is further used to adjust the eye movements of the robot simulation eyeballs according to the eyeball control parameters, each robot simulation eyeball is controlled by an independent electric control device, and the electric control devices use wireless communication technology to perform real-time communication and data exchange to achieve movement synchronization between the eyeballs; The step of adjusting the eye movement of the robot's simulated eyeball according to the eyeball control parameter further includes: The robot simulated eyeball comprises a water bag, a first conduit, a second conduit, a first liquid pump, a second liquid pump, an upper eyelid tear discharge hole, and a lower eyelid tear collection hole, wherein the first liquid pump is connected to the water bag via the first conduit, and the second liquid pump is connected to the water bag via the second conduit; According to the eyeball control parameters, the first liquid pump is used to control the robot simulated eyeball to discharge preset simulated tears from the upper eyelid tear hole, and the second liquid pump is used to collect the preset simulated tears from the lower eyelid tear hole.
8. A computer device, It is characterized in that The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device implements the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer device, the method according to any one of claims 1 to 6 is implemented.
