A target selection model and robot interaction system for robot interaction
By using the target selection model and interaction system in the robot interaction system, combining interaction demand information and neural network to screen the target objects, the robot's active interaction in complex human flow situations is achieved, and the problem of poor robot initiative in the existing technology is solved, and the improvement of improved robotics' Service efficiency and quality.
Patent Information
- Application Number
- CN202111496103.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-09
AI Technical Summary
Existing robots have poor initiative in complex human-flow occasions and are unable to effectively initiate human-computer interaction, resulting in unsatisfactory service results.
Adopt a target selection model and robot interaction system for robot interaction. By defining interaction requirements information, a neural network model is built, target objects are screened in combination with on-site information, and effective interaction between robots and target objects is achieved through navigation, tracking and greeting modules. .
It improves the initiative and efficiency of robot services, ensures the success rate and quality of interaction, adapts to complex occasions, reduces the idle time of robots, and improves the human-computer interaction experience.
Smart Images

Figure CN114399529B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robots, and in particular to a target selection model for robot interaction and a robot interaction system. Background Art
[0002] Robot services have been applied in many fields. However, the robots currently used in the service field have poor initiative and single functions. Especially in complex situations with human flow, robots cannot actively initiate human-machine interaction, and the robot service effect is not ideal. Summary of the invention
[0003] In order to solve the defect in the prior art that robots are not suitable for complex human flow situations, the present invention proposes a target selection model for robot interaction and a robot interaction system.
[0004] One of the purposes of the present invention adopts the following technical solution:
[0005] A target selection model for robot interaction, wherein obtaining the model comprises the following steps:
[0006] S1. Information used to determine the interaction needs between humans and robots is defined as interaction need information, which includes at least one of voice, body language and facial expression;
[0007] Extract the corresponding interaction demand information and interaction results from the work records of the robot and the human, select part of the interaction demand information as samples, and mark the interaction expected values for the samples according to the interaction results corresponding to the samples; construct a first neural network with the interaction demand information as input and the interaction expected value as output;
[0008] S2. Train the first neural network based on the labeled samples to obtain the trained first neural network as a target selection model.
[0009] The second object of the present invention adopts the following technical solution:
[0010] A robot interaction system includes a target selection module, in which the target selection model as claimed in claim 1 is provided; the target selection module is used to obtain the interaction demand information of on-site personnel and calculate the interaction expectation value of the on-site personnel in combination with the target selection model; the target selection module screens target objects from the on-site personnel according to the interaction expectation value.
[0011] Preferably, it also includes a target navigation module; the target navigation module is connected to the target selection module; the target navigation module is used to obtain the motion information of the target object, and generate a navigation path for the robot to encounter the target object in combination with the robot position and the motion information of the target object.
[0012] Preferably, the target navigation module is also used to control the robot to wait in place when the robot encounters a moving obstacle, and to count the robot's waiting time in place, and to notify the target selection module to reselect the target object when the robot's waiting time in place reaches a set waiting time threshold.
[0013] Preferably, it also includes a target tracking module, which is connected to the target selection module; the target tracking module is provided with a target loss judgment condition, which includes: the target object is out of the robot's field of view, the target object moves away from the robot and / or the target object enters an uninterruptible state; the uninterruptible state includes: a phone call state and a human-to-human interaction state;
[0014] The target tracking module is used to track the target object in real time and determine whether the target object meets the target loss judgment condition. The target tracking module is used to notify the target selection module to reselect the target object when the target object meets the target loss judgment condition.
[0015] Preferably, it also includes a microphone array and a camera device, and the target selection module and the target navigation module are respectively connected to the microphone array and the camera device. The target selection module obtains interaction demand information through the microphone array and the camera device, and the target navigation module obtains the position of the target object based on the sound source information and / or image information.
[0016] Preferably, it also includes a greeting module and a response module; the greeting module is provided with a mapping relationship between multiple groups of greetings and greeting conditions, the greeting conditions include at least one of weather, date and scene, the greeting module is used to filter greetings in combination with the greeting conditions to actively initiate human-computer interaction; the response module is used to respond according to the information output by the target object; the information output by the target object is voice, text or body language.
[0017] Preferably, it also includes a display screen, which is also connected to a greeting module, in which at least some greetings in the greeting module are associated with interactive images, and the display screen synchronously displays corresponding interactive images according to the greetings output by the greeting module; the display screen adopts a touch display screen, and the display screen is also connected to a response module, so that people can interact with the robot through the display screen; the display screen is located on the face of the robot.
[0018] Preferably, the target selection module is used to select the on-site personnel whose first interaction expectation value is greater than or equal to the set expectation threshold as the target object, or the target selection module is used to select the personnel with the highest interaction expectation value in the set screening area as the target object.
[0019] Preferably, the target selection module is also provided with a priority response condition, which includes: actively greeting the robot and actively intercepting the robot; the target selection module is also used to detect in real time whether there is a person who meets the priority response conditions on the scene. If a person who meets the priority response conditions appears, the target selection module will replace the target object with the person who meets the priority response conditions.
[0020] The advantages of the present invention are:
[0021] (1) The target selection model obtained through training can be combined with the interaction demand information of any person on site to determine the interaction expectation value, so as to automatically screen out people who have the interaction demand with the robot from the on-site personnel according to the interaction expectation value, so that the robot can actively search for the people who have the interaction demand with the robot, thereby initiating the interaction and improving the initiative of the robot service.
[0022] (2) The annotation information in the annotated samples, namely the interaction expected value, can be manually annotated by domain experts. The annotated interaction expected value is associated with the actual interaction result, ensuring the adaptability of the interaction requirement information and the interaction expected value in the annotated samples, thereby ensuring the reliability of the final target selection model.
[0023] (3) By setting the interactive demand information, the screening of specific groups of people who meet the robot service scenarios is realized, so that the final target objects are people who have the expectation of communicating with the robot and meet the theme of the activity, thereby ensuring the efficiency of the robot service and the service quality and activity effect.
[0024] (4) The target selection module is used to select the on-site personnel whose first interaction expectation value is greater than or equal to the set expectation threshold as the target object, so as to improve the efficiency of screening the target object. By setting the expected threshold, the accuracy of screening the target object is also guaranteed, thereby ensuring the success rate of the robot actively initiating interaction. Alternatively, the target selection module is used to select the personnel with the highest interaction expectation value in the set screening area as the target object. Specifically, the screening area can be an area within the robot's field of view, such as a circular area or a fan-shaped area with the robot as the center, so as to ensure that the robot can quickly encounter the target object after screening out the target object, thereby initiating interaction and ensuring the success rate of active interaction.
[0025] (5) The setting of the target navigation module facilitates the robot to quickly and accurately encounter the target object determined by the target selection module, thereby ensuring the implementation basis of the robot's active interaction.
[0026] (6) The setting of the waiting time threshold in the target navigation module avoids the robot from wasting a lot of time on the same target object, improves the flexibility of the robot in screening target objects, and thus further improves the robot's service efficiency.
[0027] (7) By setting the target loss judgment condition, the uncontrollability of humans is taken into account, so that when the target object is lost, a new target object can be found in time, which is conducive to reducing the idle time of the robot and improving the service efficiency of the robot. The target loss judgment condition can be flexibly set according to the scene, etc., thereby improving the flexibility of the robot in selecting the target object and making the robot more adaptable to the service occasion.
[0028] (8) By setting the priority response conditions, the robot can respond to people who initiate interaction needs at any time when there is no interaction with people, that is, when the robot is idle or searching for a target object. In other words, the robot prioritizes passive interaction needs, thereby improving people's subjective experience.
[0029] (9) The setting of greetings and greeting conditions makes it convenient for the robot to initiate active interaction according to the actual situation and greet the target object in an anthropomorphic manner, thereby improving the target object's interactive experience and further enhancing the target object's interest in the robot, thereby improving the robot's service quality and success rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A flowchart of a training method for a target selection model for robot interaction;
[0031] Figure 2 This is a module connection diagram of a robot interaction system. DETAILED DESCRIPTION
[0032] A target selection model for robot interaction
[0033] This embodiment proposes a target selection model for robot interaction, and obtaining the model includes the following steps.
[0034] S1. Information used to determine the interaction needs between humans and robots is defined as interaction need information, which includes at least one of voice, body language and facial expression;
[0035] The corresponding interaction demand information and interaction results are extracted from the work records of the robot and human interaction, part of the interaction demand information is screened as samples, and the interaction expected values are marked on the samples according to the interaction results corresponding to the samples; a first neural network is constructed with the interaction demand information as input and the interaction expected value as output.
[0036] S2. Train the first neural network based on the labeled samples to obtain the trained first neural network as a target selection model.
[0037] In this embodiment, the target selection model obtained through training can be combined with the interaction demand information of any on-site personnel to determine the interaction expectation value, so as to automatically screen people who have interaction needs with the robot from the on-site personnel according to the interaction expectation value, so that the robot can actively search for the people who have interaction needs with the robot, thereby initiating interaction by the robot, thereby improving the initiative of the robot service.
[0038] In this embodiment, the annotation information in the labeled samples, i.e., the interaction expectation value, can be manually labeled, i.e., labeled by domain experts. The labeled interaction expectation value is associated with the actual interaction result, ensuring the adaptability of the interaction demand information and the interaction expectation value in the labeled samples, thereby ensuring the reliability of the final target selection model.
[0039] In specific implementation, the interaction requirement information may be set as a set consisting of multiple reference items, each of which includes multiple sub-items, for example:
[0040]
[0041]
[0042]
[0043]
[0044] Specifically, in the above sub-items, "physical movements to interact with the robot" include waving, funny movements, etc. Combined with the above reference items, it can be seen that the target object selected by the target selection model is a person who has the need to interact with the robot, that is, a target object, at least a person who summons the robot, makes movements to the robot, or stares at the robot for a long time / frequently, that is, the target object selected by the target selection model must be a person who is interested in the robot.
[0045] In specific implementation, according to the robot service scenario, the interaction demand information can also be set to include specific groups of people. For example, for the Children's Day event, you can set:
[0046]
[0047] For the March 8th Women's Day event, you can set:
[0048]
[0049] In this way, by setting the interactive demand information, the screening of specific groups of people who meet the robot service scenarios is realized, so that the final target objects screened out are people who have the expectation to communicate with the robot and meet the theme of the activity, thereby ensuring the efficiency of the robot service as well as the service quality and activity effect.
[0050] A robot interaction system
[0051] A robot interaction system proposed in this embodiment includes a target selection module, a target navigation module, a target tracking module, a greeting module, a response module and a display screen.
[0052] The target selection module is provided with the target selection model mentioned above. The target selection module is used to obtain the interaction demand information of the on-site personnel and calculate the interaction expectation value of the on-site personnel in combination with the target selection model. The target selection module selects the target object from the on-site personnel according to the interaction expectation value.
[0053] Specifically, in this embodiment, the target selection module is used to select the on-site personnel whose first interaction expectation value is greater than or equal to the set expectation threshold as the target object, so as to improve the efficiency of screening the target object. By setting the expectation threshold, the accuracy of screening the target object is also guaranteed, thereby ensuring the success rate of the robot actively initiating interaction.
[0054] In specific implementation, the target selection module may be configured to select a person with the highest interaction expectation value in a set screening area as a target object. Specifically, the screening area may be an area within the robot's field of view, such as a circular area or a sector area with the robot as the center, to ensure that the robot can quickly encounter the target object after screening out the target object, thereby initiating interaction and ensuring the success rate of active interaction.
[0055] The target navigation module is connected to the target selection module. The target navigation module is used to obtain the motion information of the target object, and generate a navigation path for the robot to encounter the target object in combination with the robot position and the motion information of the target object. Specifically, when the target object is stationary, the target navigation module obtains the optimal path for the robot to move toward the target object as the navigation path based on the relative position between the robot and the target object and the on-site obstacle information through the existing path optimization algorithm. When the target object moves, the target navigation module generates the motion trajectory of the target object within a preset time value in combination with the motion direction and motion speed of the target object, and the target navigation module obtains the position where the robot encounters the target object based on the relative position between the robot and the motion trajectory and the motion speed of the robot, and records it as the encounter position. The target navigation module obtains the optimal path for the robot to move toward the target object as the navigation path based on the relative position between the robot and the encounter position and the on-site obstacle information through the existing path optimization algorithm.
[0056] Specifically, in this embodiment, the robot interaction system performs visual monitoring of the scene through a camera device. The camera device adopts a depth camera. Each frame of image taken by the depth camera includes information of four dimensions, namely, three color channels of red, green, and blue and one depth channel. After obtaining the image, a target detection model based on a deep neural network, such as the existing YOLOV3 model, is used to identify pedestrians in the image and determine their position information. By tracking the changing history of the positions of pedestrians in the venue in multiple frames of video images, the motion trajectory and speed information of the pedestrians are obtained, thereby obtaining the motion trajectory.
[0057] In this embodiment, the robot interaction system also includes a microphone array, so as to locate the sound source through the microphone array, thereby locking the target object for summoning the robot through voice.
[0058] It is worth noting that since the robot carrying the robot interaction system may serve in an environment with complex human traffic, that is, it may frequently encounter moving obstacles, the robot needs to adjust the navigation path in real time according to the moving obstacles encountered during movement. The generation and adjustment of the navigation path are both existing technologies and will not be elaborated here.
[0059] In this embodiment, when the robot encounters a large-area moving obstacle such as a large number of pedestrians, in order to ensure the safety of the pedestrians and the robot, the target navigation module controls the robot to wait in place so that the robot can continue to move after the large-area moving obstacle leaves the navigation path.
[0060] The target tracking module is connected to the target selection module. The target tracking module is provided with target loss judgment conditions, which include: the target object is out of the robot's field of view, the target object moves away from the robot and / or the target object enters an uninterrupted state. The uninterrupted state includes: a calling state, a human-to-human interaction state, and a human-to-robot interaction state. The above three states can be collectively referred to as a chat state. In specific settings, if there is only one robot in a certain area, the human-to-robot interaction state does not need to be set in the uninterrupted state. In specific implementation, the target loss judgment conditions and the uninterrupted state can be manually edited and adjusted.
[0061] The target tracking module is used to track the target object in real time and determine whether the target object meets the target loss judgment condition. The target tracking module is used to notify the target selection module to reselect the target object when the target object meets the target loss judgment condition.
[0062] In this embodiment, the uncontrollability of humans is taken into account by setting the target loss judgment condition, so that a new target object can be found in time when the target object is lost, which is conducive to reducing the idle time of the robot and improving the service efficiency of the robot.
[0063] In this embodiment, the target loss judgment condition can be flexibly set according to the scene, etc., so as to improve the flexibility of the robot in selecting the target object and make the robot more adaptable to the service occasion.
[0064] In specific implementation, the target navigation module can also be used to control the robot to wait in place when the robot encounters a moving obstacle such as a pedestrian; the target navigation module is also used to count the robot's waiting time in place, and to notify the target selection module to reselect the target object when the robot's waiting time in place reaches the set waiting time threshold. The setting of the waiting time threshold avoids the robot from wasting a lot of time on the same target object, improves the robot's flexibility in selecting target objects, and thus further improves the robot's service efficiency.
[0065] In this embodiment, the target selection module is also provided with a priority response condition, which includes: actively calling the robot and actively intercepting the robot; the target selection module is also used to detect in real time whether there is a person who meets the priority response condition at the scene. If a person who meets the priority response condition appears, the target selection module will replace the target object with the person who meets the priority response condition. In this way, by setting the priority response condition, the robot can respond to the person who initiates the interaction demand at any time when there is no interaction with the person, that is, when the robot is in an idle state or in a state of searching for the target object, that is, the robot prioritizes the passive interaction demand, thereby improving the subjective feeling of the person.
[0066] The greeting module has multiple sets of mapping relationships between greetings and greeting conditions. The greeting conditions include at least one of the weather, date and scene. The greeting module is used to filter greetings based on the greeting conditions to actively initiate human-computer interaction. For example, in summer, when the robot finds the target object, it can say "It's very hot today, you must need a cold drink"; on Children's Day, the robot can say to the children "Today is your holiday, happy holidays" and so on.
[0067] In this embodiment, a display screen can also be set on the robot, such as the face, so that the robot can display a corresponding picture when greeting. For example, when the robot says "Today is your holiday, happy holidays" to a child, the display screen can be set to display a dynamic love picture, etc., to enhance the visual experience and subjective feelings of the target object. Specifically, in this embodiment, the display screen is also connected to a greeting module, and at least part of the greetings in the greeting module are associated with interactive images. The display screen synchronously displays the corresponding interactive images according to the greetings output by the greeting module.
[0068] The setting of greetings and greeting conditions makes it convenient for the robot to initiate active interactions according to actual conditions and greet the target object in an anthropomorphic manner, thereby improving the target object's interactive experience and further enhancing the target object's interest in the robot, thereby improving the robot's service quality and success rate.
[0069] The response module is used to respond according to the information output by the target object. The information output by the target object is voice, text or body language. Specifically, the robot can obtain the semantic input of the target object through the array microphone, obtain text input through the keyboard or touch screen, and obtain body language input through the camera device.
[0070] In this embodiment, the display screen is a touch display screen, and the display screen is also connected to a response module, so that people can interact with the robot through the display screen.
[0071] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A robot interaction system, It is characterized in that It includes a target selection module, which has a target selection model built in it; the target selection module is used to obtain the interaction demand information of the on-site personnel, and calculate the interaction expectation value of the on-site personnel in combination with the target selection model; the target selection module selects the target object from the on-site personnel according to the interaction expectation value; The acquisition of the target selection model comprises the following steps: S1. Recording information used to determine the interaction needs between humans and robots as interaction need information, which includes at least one of voice, body language, and facial expression; Extract the corresponding interaction demand information and interaction results from the work records of the robot and the human, select part of the interaction demand information as samples, and mark the interaction expected values for the samples according to the interaction results corresponding to the samples; construct a first neural network with the interaction demand information as input and the interaction expected value as output; S2. Train the first neural network based on the labeled samples to obtain the trained first neural network as a target selection model; The robot interaction system further includes a target navigation module; the target navigation module is connected to the target selection module; the target navigation module is used to obtain the motion information of the target object, and generate a navigation path for the robot to encounter the target object in combination with the robot position and the motion information of the target object; The robot also includes a target tracking module, which is connected to the target selection module; the target tracking module is provided with a target loss judgment condition, which includes: the target object is out of the robot's field of view, the target object moves away from the robot and / or the target object enters an uninterruptible state; the uninterruptible state includes: a phone call state and a human-to-human interaction state; The target tracking module is used to track the target object in real time and determine whether the target object meets the target loss judgment condition. The target tracking module is used to notify the target selection module to reselect the target object when the target object meets the target loss judgment condition.
2. The robot interaction system according to claim 1, It is characterized in that The target navigation module is also used to control the robot to wait in place when the robot encounters a moving obstacle, and to count the robot's waiting time in place, and to notify the target selection module to reselect the target object when the robot's waiting time in place reaches the set waiting time threshold.
3. The robot interaction system according to claim 2, It is characterized in that It also includes a microphone array and a camera device. The target selection module and the target navigation module are respectively connected to the microphone array and the camera device. The target selection module obtains interaction demand information through the microphone array and the camera device, and the target navigation module obtains the position of the target object based on the sound source information and / or image information.
4. The robot interaction system according to claim 1, It is characterized in that It also includes a greeting module and a response module; the greeting module is provided with a mapping relationship between multiple groups of greetings and greeting conditions, the greeting conditions include at least one of weather, date and scene, and the greeting module is used to select greetings in combination with the greeting conditions to actively initiate human-computer interaction; The response module is used to respond according to the information output by the target object; the information output by the target object is voice, text or body language.
5. The robot interaction system according to claim 4, It is characterized in that It also includes a display screen, which is also connected to a greeting module. At least part of the greetings in the greeting module are associated with interactive images. The display screen synchronously displays the corresponding interactive images according to the greetings output by the greeting module. The display screen adopts a touch display screen. The display screen is also connected to a response module, and a person can interact with the robot through the display screen. The display screen is located on the face of the robot.
6. The robot interaction system according to claim 1, It is characterized in that The target selection module is used to select the on-site personnel whose first interaction expectation value is greater than or equal to the set expectation threshold as the target object, or the target selection module is used to select the personnel with the highest interaction expectation value in the set screening area as the target object.
7. The robot interaction system according to claim 1, It is characterized in that The target selection module also has priority response conditions, which include: actively greeting the robot and actively intercepting the robot; the target selection module is also used to detect in real time whether there is a person who meets the priority response conditions on the scene. If a person who meets the priority response conditions appears, the target selection module will replace the target object with the person who meets the priority response conditions.
Citation Information
Cited By
Emergency monitoring robot combined with Raspberry Pi distortion removal monitoring
CN119648594A
An emergency monitoring robot combined with Raspberry Pi for distortion monitoring
CN119648594B
