Intelligent robot with emotion recognition, character setting and interaction functions

Abstract

The present invention proposes an intelligent robot with emotion recognition, character personality setting, and interaction functions. The robot has a dual-brain structure on the left and right. The left brain obtains the first optimal path planning through a neural network model completed by training. The right brain extracts effective features from the first face image data on the first optimal path planning to obtain the first face recognition feature. An intelligent interactive emotion model is constructed according to the first face recognition feature and the corresponding first emotion personality interaction plan set. The second optimal path planning of the intelligent robot and the second emotion personality interaction plan for the target obtained by processing the second face image data on the second optimal path planning are used to conduct emotion personality interaction with people according to the second emotion personality interaction plan. Through the effective processing of the real-time path and the utilization level of the image, the robot can accurately set the personality of the faces that may be encountered, thereby improving the user experience.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to an intelligent robot with emotion recognition, character setting and interaction functions. Background Art

[0002] An emotional robot is to endow a computer or robot with human-like emotions by artificial methods and technologies, so that it has the ability to express, recognize and understand joy, anger, sorrow and happiness, and to imitate, extend and expand human emotions.

[0003] In the prior art, an emotional robot uses face recognition technology to give corresponding response instructions to corresponding people, so as to achieve the effect of intelligent companionship. At the same time, the technology for face recognition is already sufficiently developed, but such robots are significantly backward in using multi-dimensional information technology. For example, interacting only by recognizing face image data is particularly monotonous, and it is impossible to predict face images in advance on the optimal operation path of the robot, so as to select the character setting mode of the emotional robot in advance to obtain a better user experience of emotional interaction.

[0004] Therefore, the existing emotional robots have the following problems:

[0005] First, only face data is used for robot character setting and interactive operation instruction output, without predicting the face image features of people in advance, and without considering the influence of the flow of people on the optimized path;

[0006] Second, the face recognition on the adaptation path planning of the image is not optimized. The face images obtained by the robot in the operation map will change slightly, making the model's recognition of such images appear to have a low accuracy rate;

[0007] Third, most of the computing modules used in emotional robots are relatively single. Concentrating all software and hardware in one computing terminal will significantly lead to insufficient computing power, and the setting of the robot's waist is relatively ordinary, unable to disperse the heavy load in the vertical direction, resulting in low reliability and stability of the internal motors of the robot, and reducing the service life of the emotional robot.

[0008] Therefore, there is an urgent need to design an intelligent robot with emotion recognition, character setting and interaction functions that can improve the degree of freedom and service life of the robot's waist and use the face image data on the planned path for prediction processing. Summary of the Invention

[0009] To solve the above technical problems, the present invention proposes an intelligent robot with emotion recognition, character setting and interaction functions.

[0010] In the first aspect of the present invention, a robot control method with emotion recognition, character personality setting, and interaction functions is provided, characterized in that the method includes:

[0011] Obtain the first activity area coordinate data and the first pedestrian flow data of the robot, and obtain the corresponding first optimal path planning;

[0012] Construct a path planning neural network model using the first activity area coordinate data, the first pedestrian flow data, and the first optimal path planning;

[0013] Obtain the first face image data on the path according to the first optimal path planning, and perform effective feature extraction on the first face image data to obtain the first face recognition feature;

[0014] Construct an intelligent interactive emotion model according to the first face recognition feature and the set first emotion personality interaction plan;

[0015] Receive the second activity area coordinate data and the second pedestrian flow data of the robot, obtain the second optimal path planning according to the path planning neural network model, and input the second face recognition feature obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain the second emotion personality interaction plan;

[0016] The robot sets its personality according to the second emotion personality interaction plan.

[0017] Further, the first activity area coordinate data or the second activity area coordinate data is two-dimensional coordinate system distance data, a two-dimensional coordinate system is constructed using the accuracy value of the distance sensor, the first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first activity area coordinate data or the second activity area per unit time, and the accuracy value is 0.2 - 1 centimeter.

[0018] Further, the method for performing effective feature extraction on the first face image data or the second face image data is image conversion, and the first face image data is converted into the first face recognition feature after conversion.

[0019] Further, the path planning neural network model uses a convolutional neural network CNN or a fully connected neural network DNN.

[0020] Further, the path planning neural network model uses an improved convolutional neural network CNN.

[0021] Further, the intelligent interactive emotion model uses a classifier based on the Fisher criterion or a support vector machine method.

[0022] Further, the classifier based on the Fisher criterion adopts an improved classifier based on the Fisher criterion.

[0023] There is also provided an intelligent robot with emotion recognition, character personality setting and interaction functions, including a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that:

[0024] The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, where:

[0025] The dedicated data acquisition module: acquires the coordinate data of the first activity area of the robot and the first pedestrian flow data, also acquires the selected first optimal path planning, and acquires the first face image data on the path according to the first optimal path planning, and performs effective feature extraction on it to obtain the first face recognition feature;

[0026] The cloud database module: is used to store the trained path planning neural network model and the trained intelligent interaction emotion model;

[0027] The left brain terminal includes a sensor module, a left brain data acquisition module and a left brain neural network planning module:

[0028] The sensor module: includes a distance sensor;

[0029] The left brain data acquisition module: is used to acquire the coordinate data of the second activity area of the robot and the second pedestrian flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion personality interaction module;

[0030] The left brain neural network planning module: communicates with the dedicated database module to obtain the coordinate data of the first activity area, the first pedestrian flow data and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module;

[0031] The right brain terminal includes a voice interaction module, an expression driving module, a machine vision module and a right brain emotion personality interaction module;

[0032] The right brain emotion personality interaction module: receives the first face recognition feature and the corresponding set first emotion personality interaction plan to construct an intelligent interaction emotion model, and uploads the intelligent interaction emotion model to the cloud database module; also receives the second face recognition feature, and calls the intelligent interaction emotion model in the cloud database module to process the second face recognition feature to obtain the second emotion personality interaction plan, and performs interaction operation control on the robot according to the second emotion personality interaction plan, and performs robot personality setting.

[0033] Furthermore, the left brain terminal and the right brain terminal achieve collaborative operation through internal communication.

[0034] Furthermore, the brushless motor for the degree of freedom of the robot's waist passes through the outer ring bearing.

[0035] The present invention not only uses face data for robot personality setting and interactive operation instruction output, but also anticipates the face image features of people in advance, and considers the influence of the flow of people on the optimized path; and performs optimized processing of face recognition on the adapted path planning for the image, considering that the face images obtained by the robot during operation will change slightly, and designs an optimized image conversion calculation method to improve the accuracy of the model in recognizing such images; the emotional robot in the present invention is provided with left and right brain terminals capable of internal communication, and places large-scale calculations on the cloud computing terminal to prevent difficulties caused by insufficient computing power, and the setting that the brushless motor for the degree of freedom of the robot's waist passes through the outer ring bearing can disperse the heavy load in the vertical direction, improve the reliability and stability of the internal motor of the robot, and increase the service life of the emotional robot.

[0036] More embodiments and improvement effects of the present invention will be further introduced in combination with the drawings and specific embodiments. Brief Description of the Drawings

[0037] Figure 1 is a flowchart of the robot control method for emotion recognition, character personality setting and interaction function of the present invention;

[0038] Figure 2 is a schematic diagram of the right brain terminal of the present invention;

[0039] Figure 3 is a schematic diagram of the left brain terminal of the present invention;

[0040] Figure 4 is a schematic diagram of the front of the robot used in the present invention;

[0041] Figure 5 is a schematic diagram of the structure of the electronic device according to the embodiment of the present invention. Detailed Embodiments

[0042] Next, in combination with the drawings and specific embodiments, the invention will be further described.

[0043] To solve the above technical problems, the present invention proposes an intelligent robot with emotion recognition, character personality setting and interaction functions.

[0044] In the first aspect of the present invention, a robot control method with emotion recognition, character personality setting and interaction functions is provided, characterized in that the method includes:

[0045] Obtain the first active area coordinate data and the first pedestrian flow data of the robot, and obtain the corresponding first optimal path planning;

[0046] Use the first active area coordinate data, the first pedestrian flow data, and the first optimal path planning to construct a path planning neural network model;

[0047] Obtain the first face image data on the path according to the first optimal path planning, and perform effective feature extraction on the first face image data to obtain the first face recognition feature;

[0048] Construct an intelligent interactive emotion model according to the first face recognition feature and the set first emotion personality interaction plan;

[0049] Receive the second active area coordinate data and the second pedestrian flow data of the robot, obtain the second optimal path planning according to the path planning neural network model, and input the second face recognition feature obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain the second emotion personality interaction plan;

[0050] The robot sets its personality according to the second emotion personality interaction plan.

[0051] In this embodiment, the first emotion personality interaction plan or the second emotion personality interaction plan can be internally set as several modes of the emotional robot. Specifically, there can be three interaction modes, which can be divided into a happy emotion personality interaction plan, a coquettish emotion personality interaction plan, and a comforting emotion personality interaction plan.

[0052] Further, the first active area coordinate data or the second active area coordinate data is two-dimensional coordinate system distance data, a two-dimensional coordinate system is constructed using the accuracy value of the distance sensor, the first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first active area coordinate data or the second active area per unit time, and the accuracy value is 0.2 - 1 centimeter.

[0053] The coordinate data of the first active area or the second active area in this embodiment is set by the accuracy of the distance sensor. If the accuracy value is 1 cm, the robot's active area is converted into a coordinate system with a coordinate of 1 cm. Specifically, a certain origin (0, 0) is set, and the robot's active area can be (1, 5). When the robot performs path planning, the starting point and the ending point of the activity are selected. A series of coordinates of the robot from the starting point to the ending point are used as the coordinate data of the active area. For example, if the robot moves from the origin (0, 0) to the end point (1, 2), the coordinate data of the active area can be a vector composed of four coordinates (0, 0), (0, 1), (0, 2), and (1, 2). The composition method is to directly connect them as (0, 0, 0, 1, 0, 2, 1, 2). All the coordinate data of the active area have the same dimension, and zeros are filled in the insufficient places. If the number of vector eigenvalue is all 10, the above coordinate data of the active area is filled with zeros as (0, 0, 0, 1, 0, 2, 1, 2, 0, 0).

[0054] Further, the method for effectively extracting features from the first face image data or the second face image data is image conversion, and its formula is:

[0055]

[0056] Equation c m+1 is the converted image, I is the first face image data or the second face image data, E is the identity matrix, W i is the conversion submatrix of dimension i, r i is a constant parameter greater than 0, is an assumed variable, E T is the transpose of the identity matrix, is the transpose of the conversion submatrix of dimension i, r is the differential change image data, r is generally random white noise image data, r is inversely proportional to the accuracy of the distance sensor, and the first face image data is converted into the first face recognition feature after conversion.

[0057] Further, the path planning neural network model adopts a convolutional neural network CNN or a fully connected neural network DNN.

[0058] Further, the path planning neural network model adopts an improved convolutional neural network CNN, and its activation function m(x) is:

[0059]

[0060] In the formula, n is the number of planned paths, W n (T) is the target optimization function, and x is the input quantity of the activation function.

[0061] In this embodiment, since the CNN neural network is a conventional algorithm and its activation function and loss function determine the training effect of the model, in the present invention, the activation function is improved to take into account the planned path of the present invention, making the optimization process of path planning more accurate.

[0062] Further, the intelligent interactive emotion model adopts a classifier based on the Fisher criterion or a support vector machine method.

[0063] Further, the classifier based on the Fisher criterion adopts an improved classifier based on the Fisher criterion, and its calculation formula is as follows:

[0064]

[0065] A is the first face recognition feature or the second face recognition feature, R(S) is the output emotional personality interaction plan, W T is the normal vector perpendicular to the hyperplane, P is the accuracy of the distance sensor, the range is 0.2 - 1, c is the mean value of the face image pixels of the first optimal planned path or the mean value of the face image pixels of the second optimal planned path, n is the number of planned paths, G i is the average gray value of the face image pixels of the path that can be planned.

[0066] In this embodiment, the happy emotional personality interaction plan, the coquettish emotional personality interaction plan, and the comforting emotional personality interaction plan respectively correspond to the values of R(S) being greater than 0, equal to 0, and less than 0.

[0067] There is also provided an intelligent robot with emotion recognition, role personality setting and interaction functions, including a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that:

[0068] In this embodiment, both the left brain terminal and the right brain terminal are equipped with an internal communication module to ensure the coordination and computing power compensation of the two terminals, so as to achieve load balancing and reduce the failure rate.

[0069] The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, where:

[0070] The dedicated data acquisition module: acquires the coordinate data of the first activity area of the robot and the first pedestrian flow data, also acquires the selected first optimal path plan, and acquires the first face image data on the path according to the first optimal path plan, and performs effective feature extraction on it to obtain the first face recognition feature;

[0071] The cloud database module: is used to store the trained path planning neural network model and the trained intelligent interactive emotion model;

[0072] The left brain terminal includes a sensor module, a left brain data acquisition module, and a left brain neural network planning module:

[0073] The sensor module: includes a distance sensor;

[0074] The left brain data acquisition module: is used to acquire the coordinate data of the second activity area of the robot and the second pedestrian flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion and personality interaction module;

[0075] The left brain neural network planning module: communicates with the dedicated database module to obtain the coordinate data of the first activity area, the first pedestrian flow data, and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module;

[0076] The right brain terminal includes a voice interaction module, an expression driving module, a machine vision module, and a right brain emotion and personality interaction module;

[0077] The right brain emotion and personality interaction module: receives the first face recognition feature and the corresponding set first emotion and personality interaction plan to construct an intelligent interaction emotion model, and uploads the intelligent interaction emotion model to the cloud database module; also receives the second face recognition feature, and calls the intelligent interaction emotion model in the cloud database module to process the second face recognition feature to obtain a second emotion and personality interaction plan, and performs interactive operation control on the robot according to the second emotion and personality interaction plan, and performs robot personality setting.

[0078] Further, the left brain terminal and the right brain terminal use internal communication to achieve collaborative operation.

[0079] Further, the brushless motor of the robot waist degree of freedom passes through an outer ring bearing.

[0080] The present invention not only uses face data for robot personality setting and interactive operation instruction output, but also anticipates the face image features of people in advance, and considers the influence of pedestrian flow on the optimized path; and performs optimized processing on face recognition for image adaptation path planning, considering that the face images obtained by the robot during operation will change slightly, and designs an optimized image conversion calculation method to improve the accuracy of model recognition of such images; the emotional robot in the present invention is provided with left and right brain terminals capable of internal communication, and places large-scale calculations on the cloud computing terminal to prevent difficulties caused by insufficient computing power, and the setting that the brushless motor of the robot waist degree of freedom passes through an outer ring bearing can spread the vertical direction heavy load, improve the reliability and stability of the internal motor of the robot, and increase the service life of the emotional robot.

[0081] The electronic device may include a processor 401 and a memory 402 storing computer program instructions. Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0082] The memory 402 may include a mass storage for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 402 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 402 may be internal or external to the data processing device. In a specific embodiment, the memory 402 is a non-volatile solid state memory. In a specific embodiment, the memory 402 includes a read-only memory (ROM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0083] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the above-mentioned control methods for the obstetrician-gynecologist dedicated arm lamp.

[0084] In one example, the electronic device may further include a communication interface 403 and a bus 410. Among them, as Figure 3 shown, the processor 401, the memory 402, and the communication interface 403 are connected through the bus 410 to complete communication with each other.

[0085] The communication interface 403 is mainly used to implement communication between the modules, devices, units, and / or devices in the embodiments of the present invention.

[0086] The bus 410 includes hardware, software, or both, and couples components of the electronic device to each other. By way of example and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, the bus 410 can include one or more buses. Although embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

[0087] Of course, it can be understood that each embodiment of the present invention can achieve one of the effects alone, and a combination of multiple embodiments of the present invention can achieve all of the above effects. However, it is not required that each embodiment of the present invention achieve all of the above advantages and effects, because each embodiment of the present invention can constitute an independent technical solution and make one or more contributions to the prior art.

[0088] For the part of the module structure not specifically defined in the present invention, reference shall be made to the content recorded in the prior art. The prior art mentioned in the foregoing background art part and the specific embodiment part of the present invention can be regarded as a part of the present invention for understanding the meaning of some technical features or parameters. The protection scope of the present invention shall be subject to the content actually recorded in the claims.

Claims

1. A robot control method with emotion recognition, character setting and interactive functions, characterized in that: The method comprises: Obtaining coordinate data of a first activity area and first human flow data of the robot, and obtaining a corresponding first optimal path planning; Constructing a path planning neural network model using the first activity area coordinate data, the first human flow data, and the first optimal path planning; The path planning neural network model adopts an improved convolutional neural network CNN, and its activation function m(x) is: Where n is the number of planned paths, W n (T) is the target optimization function, and x is the input of the activation function; Acquire first facial image data on the path according to the first optimal path planning, and perform effective feature extraction on the first facial image data to obtain first facial recognition features; Constructing an intelligent interactive emotion model according to the first face recognition feature and the set first emotion and personality interaction scheme; The intelligent interactive emotion model adopts a classifier or a support vector machine method based on the Fisher criterion; The classifier based on the Fisher criterion adopts an improved classifier based on the Fisher criterion, and its calculation formula is as follows: A is the first face recognition feature or the second face recognition feature, R(S) is the output emotion and personality interaction scheme, W T is the normal vector perpendicular to the hyperplane, P is the distance sensor accuracy, ranging from 0.2 to 1, c is the mean value of the face image pixels of the first optimal planning path or the mean value of the face image pixels of the second optimal planning path, n is the number of planning paths, G i is the average gray value of the face image pixels for which the path can be planned; Receiving the coordinate data of the second activity area and the second human flow data of the robot, obtaining a second optimal path planning according to the path planning neural network model, and inputting the second face recognition features obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain a second emotion and personality interaction plan; The robot sets its personality according to the second emotion-personality interaction plan.

2. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 1, characterized in that: The first activity area coordinate data or the second activity area coordinate data is two-dimensional coordinate system distance data, and the two-dimensional coordinate system is constructed using the accuracy value of the distance sensor. The first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first activity area coordinate data or the second activity area per unit time, and the accuracy value is 0.2-1 cm.

3. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 2, characterized in that: The method for effectively extracting features from the first face image data or the second face image data is image conversion, and the formula is: Formula c m+1 is the converted image, I is the first face image data or the second face image data, E is the unit matrix, W i is the i-dimensional transformer, r i is a constant parameter greater than 0, is the assumed variable, E T is the transpose of the identity matrix, is the i-dimensional transformer transpose, r is the difference change image data, r is generally random white noise image data, r is inversely proportional to the accuracy of the distance sensor, and the first face image data is converted into the first face recognition feature.

4. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 1, characterized in that: The path planning neural network model adopts a convolutional neural network CNN or a fully connected neural network DNN.

5. An intelligent robot with emotion recognition, character setting and interaction functions, used to implement the method according to claim 1, comprising a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that: The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, wherein: The dedicated data acquisition module: acquires coordinate data of the first activity area of ​​the robot and first human flow data, and also acquires a selected first optimal path plan, and acquires first face image data on the path according to the first optimal path plan, and performs effective feature extraction on the first face image data to obtain a first face recognition feature; The cloud database module is used to store the trained path planning neural network model and the trained intelligent interactive emotion model; The left brain terminal includes a sensor module, a left brain data acquisition module and a left brain neural network planning module: The sensor module comprises a distance sensor; The left brain data acquisition module is used to obtain the coordinate data of the second activity area of ​​the robot and the second human flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion and personality interaction module; The left-brain neural network planning module communicates with the dedicated database module to obtain the first activity area coordinate data, the first traffic data and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module; The right brain terminal includes a voice interaction module, an expression drive module, a machine vision module and a right brain emotion and personality interaction module; The right brain emotion and personality interaction module: receives the first facial recognition feature and the corresponding first emotion and personality interaction plan to construct an intelligent interactive emotion model, and uploads the intelligent interactive emotion model to the cloud database module; also receives the second facial recognition feature, and calls the intelligent interactive emotion model of the cloud database module, processes the second facial recognition feature to obtain a second emotion and personality interaction plan, performs interactive operation control on the robot according to the second emotion and personality interaction plan, and sets the robot's personality.

6. An intelligent robot with emotion recognition, character setting and interactive functions as claimed in claim 5, characterized in that: The left-brain terminal and the right-brain terminal realize collaborative operation by utilizing internal communication.

7. An intelligent robot with emotion recognition, character setting and interactive functions as claimed in claim 6, characterized in that: The robot waist freedom brushless motor passes through the outer ring bearing.

Images