Intelligent robot with emotion recognition, role character setting and interaction functions

Abstract

The invention provides an intelligent robot with emotion recognition, role character setting and interaction functions, the robot has a left-and-right double-brain structure, and the left brain obtains a first optimal path plan through a trained neural network model; the right brain performs effective feature extraction on the first face image data on the first optimal path planning to obtain a first face recognition feature, and constructs an intelligent interaction emotion model according to the first face recognition feature and a correspondingly set first emotion character interaction scheme, and processing a second optimal path plan of the intelligent robot and second face image data on the second optimal path plan to obtain a second emotional character interaction scheme of the target, and performing emotional character interaction with people according to the second emotional character interaction scheme. Through effective processing of a real-time path and utilization level of an image, accurate character setting of a robot on a face which may be encountered is improved, so that the experience feeling of a user is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an intelligent robot with emotion recognition, character setting and interaction functions. Background Art

[0002] Emotional robots are computers or robots that are endowed with human-like emotions using artificial methods and technologies, enabling them to express, recognize and understand joy, sorrow and anger, and to imitate, extend and expand human emotions.

[0003] In the existing technology, emotional robots use facial recognition technology to respond to corresponding instructions to the corresponding people, thereby achieving the effect of intelligent care. At the same time, the technology for facial recognition is already sufficiently developed, but this type of robot still lags behind in the use of multi-dimensional information technology. For example, relying solely on the recognition of facial image data to interact is particularly monotonous, and it is impossible to predict facial images in advance on the robot's optimal operating path, thereby selecting the emotional robot's personality setting mode in advance to obtain emotional interaction with a better user experience.

[0004] Therefore, existing emotional robots have the following problems:

[0005] First, only facial data is used to set the robot's personality and output interactive operation instructions, without predicting the facial image features of the person in advance, and without considering the impact of human flow on the optimization path;

[0006] Second, the image is not optimized for face recognition in path planning. The face images obtained by the robot in the running diagram will change slightly, making the model's recognition accuracy of such images appear low.

[0007] Third, the computing modules used in most emotional robots are relatively simple. Concentrating all software and hardware in one computing terminal will obviously lead to insufficient computing power. In addition, the waist setting of the robot is relatively general and cannot spread the heavy load in the vertical direction, resulting in low reliability and stability of the robot's internal motor, reducing the service life of the emotional robot.

[0008] To this end, it is urgent to design an intelligent robot with emotion recognition, character personality setting and interactive functions that can improve the robot's waist freedom and service life and use the facial image data on the planned path for predictive processing. Summary of the invention

[0009] In order to solve the above technical problems, the present invention proposes an intelligent robot with emotion recognition, character personality setting and interactive functions.

[0010] In a first aspect of the present invention, a robot control method with emotion recognition, character setting and interaction functions is provided, characterized in that the method comprises:

[0011] Obtaining coordinate data of a first activity area and first human flow data of the robot, and obtaining a corresponding first optimal path planning;

[0012] Constructing a path planning neural network model using the first activity area coordinate data, the first human flow data, and the first optimal path planning;

[0013] Acquire first facial image data on the path according to the first optimal path planning, and perform effective feature extraction on the first facial image data to obtain first facial recognition features;

[0014] Constructing an intelligent interactive emotion model according to the first face recognition feature and the set first emotion and personality interaction scheme;

[0015] Receiving the coordinate data of the second activity area and the second human flow data of the robot, obtaining a second optimal path planning according to the path planning neural network model, and inputting the second face recognition features obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain a second emotion and personality interaction plan;

[0016] The robot sets its personality according to the second emotion-personality interaction plan.

[0017] Furthermore, the first activity area coordinate data or the second activity area coordinate data is two-dimensional coordinate system distance data, and the two-dimensional coordinate system is constructed using the accuracy value of the distance sensor. The first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first activity area coordinate data or the second activity area per unit time, and the accuracy value is 0.2-1 cm.

[0018] Furthermore, a method for effectively extracting features from the first facial image data or the second facial image data is image conversion, and the first facial image data is converted into a first facial recognition feature.

[0019] Furthermore, the path planning neural network model adopts a convolutional neural network CNN or a fully connected neural network DNN.

[0020] Furthermore, the path planning neural network model adopts an improved convolutional neural network CNN.

[0021] Furthermore, the intelligent interactive emotion model adopts a classifier based on Fisher criterion or a support vector machine method.

[0022] Furthermore, the classifier based on the Fisher criterion adopts an improved classifier based on the Fisher criterion.

[0023] Also provided is an intelligent robot with emotion recognition, character setting and interactive functions, including a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that:

[0024] The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, wherein:

[0025] The dedicated data acquisition module: acquires coordinate data of the first activity area of ​​the robot and first human flow data, and also acquires a selected first optimal path plan, and acquires first face image data on the path according to the first optimal path plan, and performs effective feature extraction on the first face image data to obtain a first face recognition feature;

[0026] The cloud database module is used to store the trained path planning neural network model and the trained intelligent interactive emotion model;

[0027] The left brain terminal includes a sensor module, a left brain data acquisition module and a left brain neural network planning module:

[0028] The sensor module comprises a distance sensor;

[0029] The left brain data acquisition module is used to obtain the coordinate data of the second activity area of ​​the robot and the second human flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion and personality interaction module;

[0030] The left-brain neural network planning module communicates with the dedicated database module to obtain the first activity area coordinate data, the first traffic data and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module;

[0031] The right brain terminal includes a voice interaction module, an expression drive module, a machine vision module and a right brain emotion and personality interaction module;

[0032] The right brain emotion and personality interaction module: receives the first facial recognition feature and the corresponding first emotion and personality interaction plan to construct an intelligent interactive emotion model, and uploads the intelligent interactive emotion model to the cloud database module; also receives the second facial recognition feature, and calls the intelligent interactive emotion model of the cloud database module, processes the second facial recognition feature to obtain a second emotion and personality interaction plan, performs interactive operation control on the robot according to the second emotion and personality interaction plan, and sets the robot's personality.

[0033] Furthermore, the left-brain terminal and the right-brain terminal achieve collaborative operation by utilizing internal communication.

[0034] Furthermore, the robot waist freedom brushless motor passes through an outer ring bearing.

[0035] The present invention not only uses facial data to set the robot's personality and output interactive operation instructions, but also predicts the facial image features of the characters in advance, and takes into account the impact of human flow on the optimized path; and optimizes the image to adapt to face recognition in path planning, considering that the facial images obtained by the robot during operation will produce some changes, designs an optimized image conversion calculation method to improve the accuracy of the model in recognizing such images; the emotional robot in the present invention is provided with left and right brain terminals that can communicate internally, and places large-scale calculations on cloud computing terminals to prevent difficulties caused by insufficient computing power, and the robot's waist freedom brushless motor can diffuse vertical heavy loads through the setting of an outer ring bearing, thereby improving the reliability and stability of the robot's internal motor and increasing the service life of the emotional robot.

[0036] More embodiments and improved effects of the present invention will be further introduced in conjunction with the drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the robot control method for emotion recognition, character personality setting and interactive functions of the present invention;

[0038] Figure 2 is a schematic diagram of the right brain terminal of the present invention;

[0039] Figure 3 is a schematic diagram of the left brain terminal of the present invention;

[0040] Figure 4 is a front view schematic diagram of a robot used in the present invention;

[0041] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0043] In order to solve the above technical problems, the present invention proposes an intelligent robot with emotion recognition, character personality setting and interactive functions.

[0044] In a first aspect of the present invention, a robot control method with emotion recognition, character setting and interaction functions is provided, characterized in that the method comprises:

[0045] Obtaining coordinate data of a first activity area and first human flow data of the robot, and obtaining a corresponding first optimal path planning;

[0046] Constructing a path planning neural network model using the first activity area coordinate data, the first human flow data, and the first optimal path planning;

[0047] Acquire first facial image data on the path according to the first optimal path planning, and perform effective feature extraction on the first facial image data to obtain first facial recognition features;

[0048] Constructing an intelligent interactive emotion model according to the first face recognition feature and the set first emotion and personality interaction scheme;

[0049] Receiving the coordinate data of the second activity area and the second human flow data of the robot, obtaining a second optimal path planning according to the path planning neural network model, and inputting the second face recognition features obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain a second emotion and personality interaction plan;

[0050] The robot sets its personality according to the second emotion-personality interaction plan.

[0051] In this embodiment, the first emotion-personality interaction scheme or the second emotion-personality interaction scheme can be internally set as several modes of the emotional robot, specifically three interaction modes, which can be divided into a happy emotion-personality interaction scheme, a coquettish emotion-personality interaction scheme, and a comforting emotion-personality interaction scheme.

[0052] Furthermore, the first activity area coordinate data or the second activity area coordinate data is two-dimensional coordinate system distance data, and the two-dimensional coordinate system is constructed using the accuracy value of the distance sensor. The first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first activity area coordinate data or the second activity area per unit time, and the accuracy value is 0.2-1 cm.

[0053] The first activity area coordinate data or the second activity area coordinate data in this embodiment is set by the accuracy of the distance sensor. If the accuracy value is 1cm, the robot activity area is converted into a coordinate system with a coordinate of 1cm. Specifically, a certain origin (0, 0) is set, and the robot activity area can be (1, 5). When the robot performs path planning, the activity starting point and the end point are selected, and a series of coordinates from the activity starting point to the end point of the robot are used as the activity area coordinate data. For example, if the robot goes from the origin (0, 0) to the end point (1, 2), the activity area coordinate data can be a vector composed of four coordinates (0.0)(0, 1)(0, 2)(1, 2), and the construction method is to directly connect (0, 0, 0, 1, 0, 2, 1, 2). All activity area coordinate data have the same dimension, and the insufficient places are filled with zeros. If the number of vector eigenvalues ​​is all 10, the above activity area coordinate data is filled with zeros (0, 0, 0, 1, 0, 2, 1, 2, 0, 0).

[0054] Furthermore, the method for effectively extracting features from the first face image data or the second face image data is image conversion, and the formula is:

[0055]

[0056] Formula c m+1 is the converted image, I is the first face image data or the second face image data, E is the unit matrix, W i is the i-dimensional transformer, r i is a constant parameter greater than 0, is the assumed variable, E T is the transpose of the identity matrix, is the i-dimensional transformer transpose, r is the difference change image data, r is generally random white noise image data, r is inversely proportional to the accuracy of the distance sensor, and the first face image data is converted into the first face recognition feature.

[0057] Furthermore, the path planning neural network model adopts a convolutional neural network CNN or a fully connected neural network DNN.

[0058] Furthermore, the path planning neural network model adopts an improved convolutional neural network CNN, and its activation function m(x) is:

[0059]

[0060] Where n is the number of planned paths, W n (T) is the target optimization function, and x is the input of the activation function.

[0061] In this embodiment, since the CNN neural network is a conventional algorithm, its activation function and loss function determine the training effect of the model. Therefore, the activation function is improved in the present invention to adapt to the path planning considerations of the present invention, so that the path planning optimization processing is more accurate.

[0062] Furthermore, the intelligent interactive emotion model adopts a classifier based on Fisher criterion or a support vector machine method.

[0063] Furthermore, the classifier based on the Fisher criterion adopts an improved classifier based on the Fisher criterion, and its calculation formula is as follows:

[0064]

[0065] A is the first face recognition feature or the second face recognition feature, R(S) is the output emotion and personality interaction scheme, W T is the normal vector perpendicular to the hyperplane, P is the distance sensor accuracy, ranging from 0.2 to 1, c is the mean value of the face image pixels of the first optimal planning path or the mean value of the face image pixels of the second optimal planning path, n is the number of planning paths, G i is the average gray value of the face image pixels for which the path can be planned.

[0066] In this embodiment, the happy emotional personality interaction scheme, the coquettish emotional personality interaction scheme, and the comforting emotional personality interaction scheme correspond to R(S) values ​​greater than 0, equal to 0, and less than 0, respectively.

[0067] Also provided is an intelligent robot with emotion recognition, character setting and interactive functions, including a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that:

[0068] In this embodiment, both the left-brain terminal and the right-brain terminal are equipped with an internal communication module to ensure the coordination and computing power compensation of the two terminals, so as to balance the load and reduce the failure rate.

[0069] The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, wherein:

[0070] The dedicated data acquisition module: acquires coordinate data of the first activity area of ​​the robot and first human flow data, and also acquires a selected first optimal path plan, and acquires first face image data on the path according to the first optimal path plan, and performs effective feature extraction on the first face image data to obtain a first face recognition feature;

[0071] The cloud database module is used to store the trained path planning neural network model and the trained intelligent interactive emotion model;

[0072] The left brain terminal includes a sensor module, a left brain data acquisition module and a left brain neural network planning module:

[0073] The sensor module comprises a distance sensor;

[0074] The left brain data acquisition module is used to obtain the coordinate data of the second activity area of ​​the robot and the second human flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion and personality interaction module;

[0075] The left-brain neural network planning module communicates with the dedicated database module to obtain the first activity area coordinate data, the first traffic data and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module;

[0076] The right brain terminal includes a voice interaction module, an expression drive module, a machine vision module and a right brain emotion and personality interaction module;

[0077] The right brain emotion and personality interaction module: receives the first facial recognition feature and the corresponding first emotion and personality interaction plan to construct an intelligent interactive emotion model, and uploads the intelligent interactive emotion model to the cloud database module; also receives the second facial recognition feature, and calls the intelligent interactive emotion model of the cloud database module, processes the second facial recognition feature to obtain a second emotion and personality interaction plan, performs interactive operation control on the robot according to the second emotion and personality interaction plan, and sets the robot's personality.

[0078] Furthermore, the left-brain terminal and the right-brain terminal achieve collaborative operation by utilizing internal communication.

[0079] Furthermore, the robot waist freedom brushless motor passes through an outer ring bearing.

[0080] The present invention not only uses facial data to set the robot's personality and output interactive operation instructions, but also predicts the facial image features of the characters in advance, and takes into account the impact of human flow on the optimized path; and optimizes the image to adapt to face recognition in path planning, considering that the facial images obtained by the robot during operation will produce some changes, designs an optimized image conversion calculation method to improve the accuracy of the model in recognizing such images; the emotional robot in the present invention is provided with left and right brain terminals that can communicate internally, and places large-scale calculations on cloud computing terminals to prevent difficulties caused by insufficient computing power, and the robot's waist freedom brushless motor can diffuse vertical heavy loads through the setting of an outer ring bearing, thereby improving the reliability and stability of the robot's internal motor and increasing the service life of the emotional robot.

[0081] The electronic device may include a processor 401 and a memory 402 storing computer program instructions. Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0082] Memory 402 may include a large capacity memory for data or instructions. For example, but not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside a data processing device. In a specific embodiment, memory 402 is a non-volatile solid-state memory. In a specific embodiment, memory 402 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0083] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the arm light control methods for obstetricians and gynecologists in the above embodiments.

[0084] In one example, the electronic device may further include a communication interface 403 and a bus 410. Figure 3 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0085] The communication interface 403 is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiment of the present invention.

[0086] Bus 410 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0087] Of course, it can be understood that each embodiment of the present invention can achieve one of the effects alone, and a combination of multiple embodiments of the present invention can achieve all of the above effects, but it is not required that every embodiment of the present invention achieve all of the above advantages and effects, because each embodiment of the present invention can constitute a separate technical solution and make one or more contributions to the prior art.

[0088] The present invention does not particularly specify the structure of some modules, which shall be subject to the contents recorded in the prior art. The prior art mentioned in the above background technology section and the specific embodiment section of the present invention can be used as part of the present invention to understand the meaning of some technical features or parameters. The scope of protection of the present invention shall be subject to the contents actually recorded in the claims.

Claims

1. A robot control method with emotion recognition, character setting and interactive functions, characterized in that: The method comprises: Obtaining coordinate data of a first activity area and first human flow data of the robot, and obtaining a corresponding first optimal path planning; Constructing a path planning neural network model using the first activity area coordinate data, the first human flow data, and the first optimal path planning; Acquire first facial image data on the path according to the first optimal path planning, and perform effective feature extraction on the first facial image data to obtain first facial recognition features; Constructing an intelligent interactive emotion model according to the first face recognition feature and the set first emotion and personality interaction scheme; Receiving the coordinate data of the second activity area and the second human flow data of the robot, obtaining a second optimal path planning according to the path planning neural network model, and inputting the second face recognition features obtained by performing effective feature extraction on the second face image data according to the second optimal path planning into the intelligent interactive emotion model to obtain a second emotion and personality interaction plan; The robot sets its personality according to the second emotion-personality interaction plan.

2. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 1, characterized in that: The first activity area coordinate data or the second activity area coordinate data is two-dimensional coordinate system distance data, and the two-dimensional coordinate system is constructed using the accuracy value of the distance sensor. The first pedestrian flow data or the second pedestrian flow data is the number of people passing through the first activity area coordinate data or the second activity area per unit time, and the accuracy value is 0.2-1 cm.

3. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 2, characterized in that: The method for effectively extracting features from the first facial image data or the second facial image data is image conversion, and the first facial image data is converted into a first facial recognition feature.

4. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 1, characterized in that: The path planning neural network model adopts a convolutional neural network CNN or a fully connected neural network DNN.

5. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 4, characterized in that: The path planning neural network model adopts an improved convolutional neural network (CNN).

6. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 5, characterized in that: The intelligent interactive emotion model adopts a classifier based on Fisher criterion or a support vector machine method.

7. A robot control method with emotion recognition, character setting and interactive functions as claimed in claim 6, characterized in that: The classifier based on Fisher criterion adopts an improved classifier based on Fisher criterion.

8. An intelligent robot with emotion recognition, character setting and interactive functions, comprising a computing cloud terminal, a left brain terminal and a right brain terminal, characterized in that: The computing cloud terminal includes a dedicated data acquisition module and a cloud database module, wherein: The dedicated data acquisition module: acquires coordinate data of the first activity area of ​​the robot and first human flow data, and also acquires a selected first optimal path plan, and acquires first face image data on the path according to the first optimal path plan, and performs effective feature extraction on the first face image data to obtain a first face recognition feature; The cloud database module is used to store the trained path planning neural network model and the trained intelligent interactive emotion model; The left brain terminal includes a sensor module, a left brain data acquisition module and a left brain neural network planning module: The sensor module comprises a distance sensor; The left brain data acquisition module is used to obtain the coordinate data of the second activity area of ​​the robot and the second human flow data, and call the path planning neural network model in the cloud database module to generate the second optimal path planning, and communicate with the right brain emotion and personality interaction module; The left-brain neural network planning module communicates with the dedicated database module to obtain the first activity area coordinate data, the first traffic data and the first optimal path planning to construct a path neural network model, and transmits the path neural network model to the cloud database module; The right brain terminal includes a voice interaction module, an expression drive module, a machine vision module and a right brain emotion and personality interaction module; The right brain emotion and personality interaction module: receives the first facial recognition feature and the corresponding first emotion and personality interaction plan to construct an intelligent interactive emotion model, and uploads the intelligent interactive emotion model to the cloud database module; also receives the second facial recognition feature, and calls the intelligent interactive emotion model of the cloud database module, processes the second facial recognition feature to obtain a second emotion and personality interaction plan, performs interactive operation control on the robot according to the second emotion and personality interaction plan, and sets the robot's personality.

9. An intelligent robot with emotion recognition, character setting and interactive functions as claimed in claim 8, characterized in that: The left-brain terminal and the right-brain terminal realize collaborative operation by utilizing internal communication.

10. An intelligent robot with emotion recognition, character setting and interactive functions as claimed in claim 8, characterized in that: The robot waist freedom brushless motor passes through the outer ring bearing.

Images