A method for determining whether a person in the surroundings of a robot is interested in the robot

Abstract

The present application belongs to the field of artificial intelligence robot behavior interaction technology, and discloses a method for judging whether a person in the periphery of a robot is interested in the robot, comprising the following steps: step 1, information preprocessing; step 2, calculating the distance from the person in the periphery of the robot to the periphery of the robot; step 3, judging whether the person is stationary; step 4, judging whether the person is close to the robot; step 5, judging whether the person meets the condition of being interested in the robot; and step 6, finally judging whether the person is interested in the robot. The present application combines visual information and other information, and judges whether a person in the periphery of a robot is interested in the robot based on the face orientation, so as to actively communicate with the interested user and improve the intelligence of the service robot.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence robot behavior interaction technology, and in particular relates to a method for determining whether people around a robot are interested in the robot. Background Technology

[0002] Service robots are primarily used for communication with users. Typically, users first input a voice command (such as "Hello, tell me about this place"), and then the robot performs a specific action based on that command. However, everyday life also offers examples of robots proactively seeking out users for interaction, such as salespeople actively engaging with customers interested in a product. Therefore, robots that can proactively communicate with users are more intelligent than those that can only passively receive user commands. But which users should robots proactively seek out for communication? Naturally, they should seek out users who are interested in the robot itself. However, current robot technology lacks the ability to determine whether people in the vicinity are interested in the robot, thus preventing proactive communication. Summary of the Invention

[0003] The purpose of this invention is to provide a method for determining whether people around a robot are interested in the robot, in order to solve the above-mentioned technical problems.

[0004] To address the aforementioned technical problems, the present invention provides a specific technical solution for a method of determining whether people around a robot are interested in the robot:

[0005] A method for determining whether people around a robot are interested in the robot includes the following steps:

[0006] Step 1: Information preprocessing;

[0007] Step 2: Calculate the distance from people around the robot to the robot's surroundings;

[0008] Step 3: Determine if the person is still;

[0009] Step 4: Determine if the person is close to the robot;

[0010] Step 5: Determine if the person meets the criteria for being interested in robots;

[0011] Step 5.1: Determine whether the person meets the condition of being interested in robots in Scheme 1;

[0012] Step 5.2: Determine whether the person meets the condition of being interested in robots in Scheme 2;

[0013] Step 5.3: Determine whether the person meets the condition of being interested in robots in Scheme 3;

[0014] Step 6: Finally, determine whether the person is interested in the robot.

[0015] Furthermore, step 1 includes the following specific steps:

[0016] Step 1.1: Construct a coordinate system with the ground at the robot's location as the origin. The horizontal direction of the robot is the x-axis, the rightward direction is the positive x-axis, the horizontal direction is the y-axis, and the direction directly in front of the robot is the positive y-axis. The vertical direction of the robot is the z-axis, and the top is the positive z-axis.

[0017] Step 1.2: The vision module sends a message to the robot to indicate whether the faces of people around the robot are facing the robot. If face is true, it means that the person is facing the robot; if face is false, it means that the person is not facing the robot.

[0018] Step 1.3: The vision module sends out the position information x and y of the people around the robot, which represent the horizontal position coordinates of the people respectively;

[0019] Step 1.4: The vision module issues the IDs of people around the robot. The ID is a unique identifier for people around the robot. For the same person, the ID remains unchanged and is a positive number, starting from 1 and increasing sequentially.

[0020] Step 1.5: Construct a set S = {id, x, y, face} representing character information based on the above information;

[0021] Step 1.6: The time frequency at which the robot collects the above information is f, which is the total amount of information generated in 1 second;

[0022] Step 1.7: Radius R1, time T1 seconds, and threshold θ1. In Scheme 1, if the proportion of people walking within a circle of radius R1 with their faces turned towards the robot exceeds θ1 within time T1, and their walking trajectory is closer to the robot, then it is determined that people around the robot are interested in the robot; otherwise, it is determined that they are not interested. The total amount of information generated within time T1 is a set. There are a total of T1f sets S in set E1;

[0023] Step 1.8: Radius R2, time T2 seconds, and threshold θ2. In Scheme 2, if the proportion of people whose faces are turned towards the robot while stationary within a circle of radius R2 for time T2 exceeds θ2, then the people around the robot are considered interested in it; otherwise, they are considered uninterested. The total amount of information generated within time T2 is a set. There are a total of T2f sets S in set E2;

[0024] Step 1.9: Radius R3, time T3 seconds, and threshold θ3. In Scheme 3, if a person is within a circle of radius R3 for time T3 and the proportion of their face facing the robot exceeds θ3 within that time, then the person around the robot is considered interested in the robot; otherwise, they are considered uninterested. The total amount of information generated within time T3 is a set. There are a total of T3f sets S in set E2;

[0025] Step 1.10: Threshold Ψ for whether the character is stationary.

[0026] Furthermore, step 2 includes the following specific steps:

[0027] Step 2.1: Extract the set S = {id, x, y, face} of people around the robot in real time;

[0028] Step 2.2: The distance between the person and the robot is...

[0029] Furthermore, step 3 includes the following specific steps:

[0030] Let S1 = {id, x1, y1, face1} represent the set corresponding to the previous sampling time for a certain person, and S2 = {id, x2, y2, face2} represent the set corresponding to the next sampling time. The distance between the previous and next sampling times is...

[0031]

[0032] It is then determined that the person was stationary between the previous sampling time and the next sampling time.

[0033] Furthermore, step 4 includes the following specific steps:

[0034] Let S1 = {id, x1, y1, face1} represent the set corresponding to a person at the previous sampling time, and S2 = {id, x2, y2, face2} represent the set corresponding to a person at the next sampling time. If the distance between the person and the robot at the previous sampling time is greater than the distance between the person and the robot at the next sampling time...

[0035]

[0036] The trajectory of the person between the two sampling times is determined to be moving closer to the robot.

[0037] Furthermore, step 5.1 includes the following specific steps:

[0038] Step 5.1.1: Extract the sampling information E of person i within a circle of radius R1 within a time period not less than T1.i ={S 1i S 2i S 3i S Ni}, N=(T1f], where N represents the smallest integer not less than T1f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni ;

[0039] Step 5.1.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1;

[0040] Condition 1: If

[0041]

[0042] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0043]

[0044] Condition 3: If

[0045]

[0046] Step 5.1.3: If conditions 1, 2, and 3 are met, then it is determined that person i is interested in the robot.

[0047] Furthermore, step 5.2 includes the following specific steps:

[0048] Step 5.2.1: Extract the sampling information E of person i within a circle of radius R2 within a time interval not less than T2. i ={S 1i S 2i S 3i S Ni}, N=(T2f], where N represents the smallest integer not less than T2f, and the first element is S. 1i The second element is S 2i The Nth element is S.Ni ;

[0049] Step 5.2.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1;

[0050] Condition 1: If

[0051]

[0052] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0053]

[0054] Condition 3: If

[0055]

[0056] Step 5.2.3: If conditions 1, 2, and 3 are met, then it is determined that person i is interested in the robot. Further, step 5.3 includes the following specific steps:

[0057] Step 5.3.1: Extract the sampling information E of person i within a circle with radius R3 for a time period not less than T3. i ={S 1i S 2i S 3i S Ni}, N=(T3f], where N represents the smallest integer not less than T3f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni ;

[0058] Step 5.3.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N;

[0059] Condition 1: If

[0060]

[0061] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0062]

[0063] Step 5.3.3: If conditions 1 and 2 are met, then it is determined that person i is interested in the robot.

[0064] Furthermore, step 6 includes the following specific steps:

[0065] Step 6.1: If any of the conditions in schemes 1, 2, and 3 are met, then determine that the people around the robot are interested in the robot;

[0066] Step 6.2: If any of the conditions in schemes 1, 2, and 3 are not met, then it is determined that the people around the robot are not interested in the robot.

[0067] The method for determining whether people around a robot are interested in the robot according to the present invention has the following advantages: The present invention combines visual and other information, and determines whether people around the robot are interested in the robot based on facial orientation, thereby enabling proactive communication with interested users and improving the intelligence of service robots. Attached Figure Description

[0068] Figure 1 This is a schematic diagram illustrating the method of the present invention for determining whether people around the robot are interested in the robot;

[0069] Figure 2 This is a schematic diagram of the information preprocessing process of the present invention;

[0070] Figure 3 This is a schematic diagram of the coordinate system construction of the present invention. Detailed Implementation

[0071] To better understand the purpose, structure, and function of this invention, the following description, in conjunction with the accompanying drawings, provides a more detailed account of a method for determining whether people around a robot are interested in the robot.

[0072] like Figure 1 As shown, this invention designs a method based on facial orientation to determine whether people around a robot are interested in the robot, applicable to service robots. This invention provides three methods for determining whether people around a robot (hereinafter referred to as "people") are interested in the robot, as follows:

[0073] (1) As the character walks closer to the robot, the character's face is turned toward the robot for a long time;

[0074] (2) The character remains stationary near the robot, and the character's face is turned toward the robot for an extended period of time;

[0075] (3) The person stays near the robot for a long time and faces the robot for a long time;

[0076] If any one of the above three conditions is met, it is determined that the person is interested in the robot; otherwise, it is determined that the person is not interested in the robot.

[0077] Specifically, such as Figure 1 As shown, a method for determining whether people around a robot are interested in the robot according to the present invention includes the following steps:

[0078] Step 1: As Figure 2 As shown, information preprocessing;

[0079] Step 1.1: As Figure 3 As shown, construct a coordinate system with the ground at the robot's location as the origin, the robot's horizontal direction as the x-axis, the robot's rightward direction as the positive x-axis, the robot's horizontal direction as the y-axis, and the robot's front as the positive y-axis. The robot's vertical direction is the z-axis, with the top as the positive z-axis.

[0080] Step 1.2: The vision module sends a message to the robot to indicate whether the faces of people around the robot are facing the robot. If face is true, it means that the person is facing the robot; if face is false, it means that the person is not facing the robot.

[0081] Step 1.3: The vision module sends out the position information x and y of the people around the robot, which represent the horizontal position coordinates of the people respectively;

[0082] Step 1.4: The vision module issues the IDs of people around the robot. The ID is a unique identifier for people around the robot. For the same person, the ID remains unchanged and is a positive number, starting from 1 and increasing sequentially.

[0083] Step 1.5: Construct a set S = {id, x, y, face} representing character information based on the above information;

[0084] Step 1.6: The time frequency at which the robot collects the above information is f, which is the total amount of information generated in 1 second;

[0085] Step 1.7: Radius R1, time T1 seconds, and threshold θ1. In Scheme 1, if the proportion of people walking within a circle of radius R1 with their faces turned towards the robot exceeds θ1 within time T1, and their walking trajectory is closer to the robot, then it is determined that people around the robot are interested in the robot; otherwise, it is determined that they are not interested. The total amount of information generated within time T1 is a set. There are a total of T1f sets S in set E1;

[0086] Step 1.8: Radius R2, time T2 seconds, and threshold θ2. In Scheme 2, if the proportion of people whose faces are turned towards the robot while stationary within a circle of radius R2 for time T2 exceeds θ2, then the people around the robot are considered interested in it; otherwise, they are considered uninterested. The total amount of information generated within time T2 is a set. There are a total of T2f sets S in set E2;

[0087] Step 1.9: Radius R3, time T3 seconds, and threshold θ3. In Scheme 3, if a person is within a circle of radius R3 for time T3 and the proportion of their face facing the robot exceeds θ3 within that time, then the person around the robot is considered interested in the robot; otherwise, they are considered uninterested. The total amount of information generated within time T3 is a set. There are a total of T3f sets S in set E2;

[0088] Step 1.10: Threshold Ψ for whether the character is stationary.

[0089] Step 2: Calculate the distance from people around the robot to the robot's surroundings;

[0090] Step 2.1: Extract the set S = {id, x, y, face} of people around the robot in real time;

[0091] Step 2.2: The distance between the person and the robot is...

[0092] Step 3: Determine if the person is still;

[0093] Let S1 = {id, x1, y1, face1} represent the set corresponding to the previous sampling time for a certain person, and S2 = {id, x2, y2, face2} represent the set corresponding to the next sampling time. The distance between the previous and next sampling times is...

[0094]

[0095] Then it is determined that the person was stationary between the previous sampling time and the next sampling time;

[0096] Step 4: Determine if the person is close to the robot;

[0097] Let S1 = {id, x1, y1, face1} represent the set corresponding to a person at the previous sampling time, and S2 = {id, x2, y2, face2} represent the set corresponding to a person at the next sampling time. If the distance between the person and the robot at the previous sampling time is greater than the distance between the person and the robot at the next sampling time...

[0098]

[0099] The trajectory of the person between the two sampling times is determined to be moving closer to the robot.

[0100] Step 5: Determine if the person meets the criteria for being interested in robots;

[0101] Step 5.1: Determine if the person meets the condition of being interested in the robot in Scheme 1:

[0102] Step 5.1.1: Extract the sampling information E of person i within a circle of radius R1 within a time period not less than T1. i ={S 1i S 2i S 3i S Ni}, N=(T1f], where N represents the smallest integer not less than T1f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni .

[0103] Step 5.1.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1;

[0104] Condition 1: If

[0105]

[0106] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0107]

[0108] Condition 3: If

[0109]

[0110] Step 5.1.3: If conditions 1, 2, and 3 are met, then it is determined that person i is interested in the robot.

[0111] Step 5.2: Determine whether the person meets the condition of being interested in robots in Scheme 2;

[0112] Step 5.2.1: Extract the sampling information E of person i within a circle of radius R2 within a time interval not less than T2. i ={S 1i S 2i S 3i S Ni}, N=(T2f], where N represents the smallest integer not less than T2f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni .

[0113] Step 5.2.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1;

[0114] Condition 1: If

[0115]

[0116] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0117]

[0118] Condition 3: If

[0119]

[0120] Step 5.2.3: If conditions 1, 2, and 3 are met, then it is determined that person i is interested in the robot.

[0121] Step 5.3: Determine whether the person meets the condition of being interested in robots in Scheme 3;

[0122] Step 5.3.1: Extract the sampling information E of person i within a circle with radius R3 for a time period not less than T3. i ={S 1i S 2i S 3i S Ni}, N=(T3f], where N represents the smallest integer not less than T3f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni .

[0123] Step 5.3.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N.

[0124] Condition 1: If

[0125]

[0126] Condition 2: Statistical face ji (j = 1, 2, 3, ..., N) is the number of true values, denoted as M. If

[0127]

[0128] Step 5.3.3: If conditions 1 and 2 are met, then it is determined that person i is interested in the robot.

[0129] Step 6: Finally, determine whether the person is interested in the robot;

[0130] Step 6.1: If any of the conditions in schemes 1, 2, and 3 are met, then it is determined that the people around the robot are interested in the robot.

[0131] Step 6.2: If any of the conditions in schemes 1, 2, and 3 are not met, then it is determined that the people around the robot are not interested in the robot.

[0132] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method of determining whether a person in the periphery of a robot is interested in the robot, characterized by, Includes the following steps: Step 1: Information preprocessing; Step 1.1: Construct a coordinate system with the ground where the robot is located as the origin, the horizontal direction of the robot as the x-axis, the rightward direction of the robot as the positive x-axis, the horizontal direction of the robot as the y-axis, the front of the robot as the positive y-axis, the vertical direction of the robot as the z-axis, and the top as the positive z-axis. Step 1.2: The vision module sends a command indicating whether the faces of people around the robot are facing the robot. If `face` is true, it means the person is facing the robot; if it is false, it means the person is not facing the robot. Step 1.3: The vision module sends out the position information x and y of the people around the robot, which represent the horizontal and vertical position coordinates of the people, respectively; Step 1.4: The vision module issues the IDs of people around the robot. The ID is a unique identifier for people around the robot. For the same person, the ID remains unchanged and is a positive number, starting from 1 and increasing sequentially. Step 1.5: Construct a set S = {id, x, y, face} representing the person's information based on the above information; Step 1.6: Set the time frequency for the robot to collect the above information to f, which is the total amount of information generated in 1 second; Step 1.7: Set the radius R1, time T1 seconds, and threshold θ1. In Scheme 1, if the proportion of people walking within a circle of radius R1 with their faces turned towards the robot exceeds θ1 within time T1, and their walking trajectory is closer to the robot, then the people around the robot are considered interested in it; otherwise, they are considered uninterested. The total amount of information generated within time T1 is a set. There are a total of T1f sets S in set E1; Step 1.8: Set radius R2, time T2 seconds and threshold θ2, i.e. if the proportion of the people in the scheme 2 who are stationary within the circle with radius R2 and face the robot within T2 seconds exceeds θ2, it is determined that the people around the robot are interested in the robot, otherwise it is determined that they are not interested; the total amount of information generated within T2 seconds is a set There are T2f sets S in the set E2. Step 1.9: Set radius R3, time T3, and threshold θ3, i.e. if the proportion of the people in the scheme 3 within the circle with radius R3 facing the robot within T3 time exceeds θ3, it is determined that the people around the robot are interested in the robot, otherwise it is determined that they are not interested; the total amount of information generated within T3 time is a set There are T3f sets S in the set E3. In the above steps, radius R1 <R2<R3; Step 1.10: Set the threshold Ψ for whether the character is stationary; Step 2: Calculate the distance from people around the robot to the robot; Step 3: Determine if the person is still; Step 4: Determine if the person is close to the robot; Step 5: Determine if the person meets the criteria for being interested in robots; Step 5.1: Determine whether the person meets the condition of being interested in robots in Scheme 1; Step 5.2: Determine whether the person meets the condition of being interested in robots in Scheme 2; Step 5.3: Determine whether the person meets the condition of being interested in robots in Scheme 3; Step 6: Finally, determine whether the person is interested in the robot.

2. The method of determining whether a character around a robot is interested in the robot according to claim 1, wherein Step 2 includes the following specific steps: Step 2.1: Collecting the information of the people around the robot in real time S = {id, x, y, face}; Step 2.2: The distance between the people around the robot and the robot is 3. The method for determining whether people around a robot are interested in the robot according to claim 2, characterized in that, Step 3 includes the following specific steps: Let S1 = {id, x1, y1, face1} represent the set corresponding to the previous sampling time of a certain person, and S2 = {id, x2, y2, face2} represent the set corresponding to the next sampling time. Let the distance between the previous and next sampling times be... It is then determined that the person was stationary between the previous sampling time and the next sampling time.

4. The method of determining whether a character around a robot is interested in the robot according to claim 3, wherein, Step 4 includes the following specific steps: Let S1 = {id, x1, y1, face1} represent the set corresponding to a person at the previous sampling time, and S2 = {id, x2, y2, face2} represent the set corresponding to a person at the next sampling time. If the distance between the person and the robot at the previous sampling time is greater than the distance between the person and the robot at the next sampling time... The trajectory of the person between the two sampling times is determined to be moving closer to the robot.

5. The method of determining whether a character around a robot is interested in the robot according to claim 4, wherein, Step 5.1 includes the following specific steps: Step 5.1.1: Extract the sampling information E of person i within a circle of radius R1 within a time interval not less than T1. i ={S 1i S 2i S 3i S Ni }, N = (T1f], where N represents the smallest integer not less than T1f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni ; Step 5.1.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1; Condition 1: If Condition 2: count the number of faces ji the number of times (j = 1, 2, 3, …, N) is true, set as M, if Condition 3: If Step 5.1.3: If conditions 1, 2, and 3 are met simultaneously, then it is determined that person i is interested in the robot.

6. The method of determining whether a character around a robot is interested in the robot according to claim 4, wherein, Step 5.2 includes the following specific steps: Step 5.2.1: Extract the sampling information E of person i within a circle of radius R2 within a time interval not less than T2. i ={S 1i S 2i S 3i S Ni }, N=(T2f], where N represents the smallest integer not less than T2f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni ; Step 5.2.2: Let the j-th element be S. ji ={id, x ji y ji ,face ji At this point, the distance between the person and the robot is... The (j+1)th element is S j+1i ={id, x j+1i y j+1i ,face j+1i At this point, the distance between the person and the robot is... j = 1, 2, 3, ..., N-1; Condition 1: If Condition 2: Statistical face ji The number of true values, j = 1, 2, 3, ..., N, let M be the number of true values. Condition 3: If Step 5.2.3: If conditions 1, 2, and 3 are met simultaneously, then it is determined that person i is interested in the robot.

7. The method of determining whether a character around a robot is interested in the robot according to claim 4, wherein, Step 5.3 includes the following specific steps: Step 5.3.1: Extract the sampling information E of person i within a circle with radius R3 for a time period not less than T3. i ={S 1i S 2i S 3i S Ni }, N=(T3f], where N represents the smallest integer not less than T3f, and the first element is S. 1i The second element is S 2i The Nth element is S. Ni ; Step 5.3.2: Let the jth element be S ji = {id, x ji , y ji , face ji}, where the distance between the character and the robot is j = 1, 2, 3, …, N; Condition 1: If Condition 2: the number of conditions where face is true, j = 1, 2, 3,..., N, is denoted as M, if ji M > 0.5N Step 5.3.3: If conditions 1 and 2 are met simultaneously, then it is determined that person i is interested in the robot.

8. The method of determining whether a character around a robot is interested in the robot according to claim 1, wherein, Step 6 includes the following specific steps: Step 6.1: If any of the conditions in schemes 1, 2, and 3 are met, then determine that the people around the robot are interested in the robot; Step 6.2: If any of the conditions in schemes 1, 2, and 3 are not met, then it is determined that the people around the robot are not interested in the robot.

Images