Interaction method of home care robot based on posture adjustment and projection calibration

Abstract

The present invention relates to the field of smart home technology, and more specifically to an interaction method for a home care robot based on posture adjustment and projection calibration. The present invention includes S1, establishment of a global coordinate system; S2, gesture awakening of the care robot; S3, trajectory planning of the care robot; S4, recognition of the user's posture; S5, selection of the projection position; S6, posture adjustment and projection calibration; S7, gesture recognition and object confirmation; S8, trajectory planning of the target object; S9, picking up and returning the target object; and S10, resetting the care robot. Through the establishment of a global coordinate system, posture adjustment and projection calibration, gesture recognition and object confirmation, the present invention can achieve a clearer projection area for human-computer interaction, more accurate gesture recognition, and more convenient confirmation of the target object during the home care process.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to an interaction method for a home care robot based on posture adjustment and projection calibration. Background Art

[0002] Patent number US15589319 discloses a method for interacting with distance holographic objects based on finger direction. The method uses user control of a 3D holographic object among multiple 3D holographic objects projected in the air above a central point by a projector. Although 3D holographic projection is used, this method is difficult to implement in everyday scenarios, especially in home environments, and is inconvenient to operate and has significant limitations. The use of 3D holographic projection is also costly. Compared to this method, projecting within a plane in everyday scenarios can reduce costs. A computer is used to extrapolate the user's fingertip's direction to draw the holographic intersection line between the user's fingertip and the 3D projected holographic object. Compared to this method, the finger pointing method is used to map the finger's position, find the intersection point with the image, then magnify the area, and add a red marker on the projected image, which makes it easier for users to identify the correct finger pointing. Furthermore, in existing smart elderly care scenarios, interaction between elderly users and robots is a relatively troublesome issue because these users cannot accurately express their needs and intentions through operation or voice input. Summary of the Invention

[0003] The technical problem to be solved by the present invention is: to overcome the shortcomings of the existing technology and provide an interaction method for a home care robot based on posture adjustment and projection calibration. Through the establishment of a global coordinate system, posture adjustment and projection calibration, gesture recognition and object confirmation, the human-computer interaction projection area can be clearer, gesture recognition can be more accurate, and target object confirmation can be more convenient during the home care process.

[0004] The technical solution of the present invention is:

[0005] A method for interacting with a home care robot based on posture adjustment and projection calibration comprises the following steps:

[0006] S1. Establishment of the global coordinate system: The accompanying robot 3D scans the spatial layout of each room and establishes the global coordinate system; and establishes the local coordinate system for the target object. Both the global coordinate system and the local coordinate system are stored locally in the accompanying robot;

[0007] S2. Gesture wake-up of the accompanying robot: Each room is equipped with a wake-up device that scans gestures in real time. The wake-up device and the accompanying robot are linked for control. When the wake-up device detects a specific gesture from the user, the accompanying robot is activated.

[0008] S3. Accompanying robot trajectory planning: The accompanying robot plans the optimal route based on the global coordinates and quickly reaches the user's room along the optimal route to prepare for interaction.

[0009] S4. User posture recognition: The accompanying robot uses a binocular vision system to capture the user's posture and determine the user's optimal interaction angle;

[0010] S5. Selection of projection position: The accompanying robot uses a projector to project at the optimal interaction angle, and the projection content includes the directory where the target item is located;

[0011] S6. Posture Adjustment and Projection Calibration: The companion robot adjusts its posture by adjusting the distance between itself and the user, thereby controlling the clarity of the projection area and ensuring the accuracy of gesture recognition within the projection area.

[0012] S7, gesture recognition and item confirmation: The user uses gestures to find the directory where the target item is located in the projection area by clicking step by step, and confirms the corresponding target item by double-clicking;

[0013] S8, Target object trajectory planning: The accompanying robot plans the optimal route in the global coordinates based on the local coordinates of the target object, quickly reaches the room where the target object is located, and makes preparations before picking it up;

[0014] S9, Pick up and return the target item: The accompanying robot picks up the corresponding target item, returns to the original route according to the trajectory planning, and delivers the target item to the user;

[0015] S10. Reset of the accompanying robot: The wake-up device detects a specific gesture from the user, controls the accompanying robot to return to the original room, reset, and wait for the next wake-up.

[0016] Preferably, the wake-up device in step S2 and step S10 includes a camera and a visual detection system. The camera collects the user's gestures and compares them with the gestures stored in the visual detection system. If the comparison results are consistent, the accompanying robot is started or reset.

[0017] Preferably, the visual detection system dynamically identifies the specific hand gestures in step S2 and step S10 by comparing the key points of the hand in the previous frame image and the key points of the hand in the next frame image.

[0018] Preferably, in step S4, the user postures are divided into standing, sitting, walking, and lying postures, and projection areas of different orientations are selected according to different user postures.

[0019] Preferably, the projection area in step S4 includes the ground, wall, ceiling, palm, and curtain, and the projection area also allows modification of the interaction angle according to the user's finger pointing.

[0020] Preferably, the content projected in step S5 includes a target item directory, an entertainment directory, a management directory, and a life directory.

[0021] Preferably, the projection content and the gesture posture in step S5 use the same coordinate system. The accompanying robot moves to the intersection area with the image according to the gesture posture, enlarges the intersection area through a specific gesture posture, and adds a red marking point on the projection area to mark the target object pointed to by the user.

[0022] Preferably, the posture adjustment in step S6 includes adjusting the accompanying robot forward, backward, left, and right. During the posture adjustment process, the projection area remains constant and the projection clarity gradually changes.

[0023] Preferably, the directory where the target item is located in step S7 establishes a one-to-one correspondence with the local coordinates of the target item. The user marks the location of the required target item in the directory and confirms it by double-clicking. The accompanying robot is linked to the control cabinet where the target item is located for control, and the control cabinet makes preparations before picking it up.

[0024] Preferably, in step S9, the corresponding target item is picked up, the accompanying robot is equipped with a tray, the control cabinet cooperates with the robotic arm, the robotic arm clamps the target item onto the tray, and the accompanying robot transports the target item to the user's room through the tray and moves it in front of the user.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] By establishing a global coordinate system, each room, user location, and target object can be accurately located;

[0027] Through posture adjustment and projection calibration, the distance between the accompanying robot and the user is continuously adjusted, making the human-machine interaction projection area clearer and gesture recognition more accurate;

[0028] Through gesture recognition and item confirmation, the system uses a double-click confirmation method in the catalog to quickly confirm the target item, making it convenient for elderly people at home to use it independently. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 It is a structural schematic diagram of the present invention.

[0031] Figure 2 It is a structural diagram of the establishment of the global coordinate system.

[0032] Figure 3 This is one of the structural diagrams for selecting the projection position.

[0033] Figure 4 This is the second structural diagram for selecting the projection position.

[0034] Figure 5 This is the third structural diagram for selecting the projection position.

[0035] Figure 6 It is a structural diagram of posture adjustment and projection calibration.

[0036] Figure 7 It is a structural diagram of gesture recognition and object confirmation.

[0037] In the figure: 1. Accompanying robot; 2. Wake-up device; 3. User; 4. Target object; 5. Ground; 6. Wall; 7. Ceiling. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0039] Example 1

[0040] like Figure 1 As shown, this embodiment provides an interaction method for a home care robot based on posture adjustment and projection calibration, comprising the following steps:

[0041] S1. Establishment of global coordinate system: The accompanying robot 13D scans the spatial layout of each room and establishes global coordinates; and establishes local coordinates for the target object 4. Both global coordinates and local coordinates are stored in the local accompanying robot 1, such as Figure 2 As shown;

[0042] S2. Gesture wake-up of the accompanying robot: Each room is equipped with a wake-up device 2, which scans gestures in real time. The wake-up device 2 is linked with the accompanying robot 1 for control. When the wake-up device 2 detects a specific gesture from the user 3, the accompanying robot 1 is activated.

[0043] S3, companion robot trajectory planning: companion robot 1 plans the best route based on the global coordinates, and quickly reaches the room where user 3 is located along the best route to prepare for the interaction;

[0044] S4. Recognition of the posture of user 3: the accompanying robot 1 uses the binocular vision system to collect the posture of user 3 and determine the optimal interaction angle of user 3;

[0045] S5. Selection of projection position: The accompanying robot 1 uses a projector to project at an optimal interaction angle, and the projected content includes the directory where the target item 4 is located;

[0046] S6, posture adjustment and projection calibration: The accompanying robot 1 adjusts its posture by adjusting the distance between itself and the user 3, thereby controlling the clarity of the projection area to ensure the accuracy of gesture recognition within the projection area;

[0047] S7, gesture recognition and item confirmation: User 3 uses gestures to search for the directory where the target item 4 is located in the projection area by clicking step by step, and confirms the corresponding target item 4 by double-clicking;

[0048] S8, trajectory planning for the target item 4: The accompanying robot 1 plans the best route in the global coordinates based on the local coordinates of the target item 4, quickly reaches the room where the target item 4 is located, and makes preparations before picking it up;

[0049] S9, picking up and returning the target item 4: The accompanying robot 1 picks up the corresponding target item 4, returns to the original route according to the trajectory planning, and delivers the target item 4 to the user 3;

[0050] S10, resetting the accompanying robot 1: the wake-up device 2 detects a specific gesture made by the user 3, controls the accompanying robot 1 to return to the original room, reset, and wait for the next wake-up.

[0051] Working principle:

[0052] The present invention establishes a global coordinate system to accurately locate each room, the location of the user 3, and the location of the target object 4;

[0053] The present invention uses posture adjustment and projection calibration to continuously adjust the distance between the accompanying robot 1 and the user 3, thereby making the human-machine interaction projection area clearer and the gesture recognition more accurate;

[0054] The present invention uses gesture recognition and item confirmation, and adopts a directory double-click confirmation method to quickly realize the confirmation of the target item 4, which is convenient for the elderly at home to use independently.

[0055] Example 2

[0056] Based on Example 1, the wake-up device 2 in step S2 and step S10 includes a camera and a visual detection system. The camera collects the gestures of the user 3 and compares them with the gestures stored in the visual detection system. If the comparison results are consistent, the accompanying robot 1 is started or reset.

[0057] Preferably, the visual detection system dynamically identifies the specific hand gestures in step S2 and step S10 by comparing the key points of the hand in the previous frame image and the key points of the hand in the next frame image.

[0058] Preferably, in step S4 , the postures of the user 3 are divided into: standing, sitting, walking, and lying, and projection areas of different orientations are selected according to different postures of the user 3 .

[0059] Preferably, the projection area in step S4 includes the ground 5, the wall 6, the ceiling 7, the palm, and the curtain. The projection area also allows the modification of the interaction angle according to the finger pointing of the user 3, such as Figure 3-Figure 5 As shown;

[0060] Preferably, the projected content in step S5 includes a target item directory, an entertainment directory, a management directory, and a life directory, such as Figure 7 shown.

[0061] Preferably, the projection content and the gesture posture in step S5 use the same coordinate system. The accompanying robot 1 moves to the intersection area with the image according to the gesture posture, amplifies the intersection area through a specific gesture posture, and adds a red mark point on the projection area to mark the target object 4 pointed by the user 3, such as Figure 6 shown.

[0062] Preferably, the posture adjustment in step S6 includes adjusting the accompanying robot 1 forward, backward, left, and right. During the posture adjustment process, the projection area remains constant and the projection clarity gradually changes.

[0063] Preferably, in step S7, the directory where the target item 4 is located establishes a one-to-one correspondence with the local coordinates of the target item 4. The user 3 marks the location of the target item 4 in the directory and double-clicks to confirm. The accompanying robot 1 is linked to the control cabinet where the target item 4 is located, and the control cabinet makes preparations before taking it, such as Figure 7 shown.

[0064] Preferably, in step S9, the corresponding target item 4 is picked up, the accompanying robot 1 is equipped with a tray, the control cabinet cooperates with the robotic arm, the robotic arm clamps the target item 4 onto the tray, and the accompanying robot 1 transports the target item 4 to the room where the user 3 is located through the tray and moves it in front of the user 3.

[0065] It should be noted that no matter when and where, even if the user 3 is sick and lying on his back in bed, he can still interact with the robot through this method.

[0066] The companion robot 1 and user 3 can communicate accurately anytime and anywhere. Using its onboard projector, the companion robot 1 projects a video onto an area appropriate for user 3's position, visually displaying the content of the conversation. User 3 uses their finger to select content on the projected area, and the companion robot 1 uses binocular structured light to capture the finger's direction and select the content on the projected area.

[0067] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and such modifications or substitutions shall be within the scope of the present invention. Any person skilled in the art who is familiar with the present invention may easily conceive of changes or substitutions within the technical scope disclosed in the present invention, and such changes or substitutions shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An interactive method for a home care robot based on posture adjustment and projection calibration, characterized in that: The steps include: S1. Establishment of a global coordinate system: The accompanying robot (1) 3D scans the spatial layout of each room and establishes global coordinates; and establishes local coordinates for the target object (4). Both the global coordinates and the local coordinates are stored in the local accompanying robot (1); S2. Gesture awakening of the accompanying robot: each room is equipped with an awakening device (2), which scans the gestures in real time. The awakening device (2) and the accompanying robot (1) are controlled in a linkage manner. When the awakening device (2) detects that the user (3) has made a specific gesture, the accompanying robot (1) is activated. S3. Trajectory planning of the accompanying robot: The accompanying robot (1) plans the best route according to the global coordinates, and quickly reaches the room where the user (3) is located according to the best route, and makes preparations before the interaction; S4. Recognition of user posture: the accompanying robot (1) uses a binocular vision system to collect the posture of the user (3) and determine the optimal interaction angle of the user (3); S5. Selection of projection position: the accompanying robot (1) uses a projector to project at an optimal interaction angle, and the projection content includes the directory where the target item (4) is located; S6, posture adjustment and projection calibration: the accompanying robot (1) adjusts its posture by adjusting the distance between itself and the user (3), and thereby controls the clarity of the projection area to ensure the accuracy of gesture recognition in the projection area; S7, gesture recognition and item confirmation: the user (3) uses gestures to search for the directory where the target item (4) is located in the projection area by clicking step by step, and confirms the corresponding target item (4) by double-clicking; S8, trajectory planning of the target item (4): the accompanying robot (1) plans the best route in the global coordinates according to the local coordinates of the target item (4), quickly reaches the room where the target item (4) is located, and makes preparations before taking it; S9, taking and returning the target item (4): the accompanying robot (1) takes the corresponding target item (4), returns along the original route according to the trajectory planning, and delivers the target item (4) to the user (3); S10, resetting of the accompanying robot (1): the awakening device (2) detects that the user (3) has made a specific gesture, and controls the accompanying robot (1) to return to the original room and reset, waiting for the next awakening.

2. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 1, characterized in that: The awakening device (2) in step S2 and step S10 includes a camera and a visual detection system. The camera collects the gestures of the user (3) and compares them with the gestures stored in the visual detection system. If the comparison results are consistent, the accompanying robot (1) is started or reset.

3. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 2, characterized in that: Regarding the specific hand gestures in step S2 and step S10, the visual detection system dynamically identifies the specific hand gestures by comparing the key points of the hand in the previous frame image and the key points of the hand in the next frame image.

4. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 1, characterized in that: In step S4, the postures of the user (3) are divided into standing, sitting, walking, and lying postures, and projection areas of different orientations are selected according to the different postures of the user (3).

5. The interaction method of the home care robot based on posture adjustment and projection calibration according to claim 1 or 4, characterized in that: The projection area in step S4 includes the ground (5), the wall (6), the ceiling (7), the palm, and the screen. The projection area also allows the modification of the interaction angle according to the finger pointing of the user (3).

6. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 5, characterized in that: The projected content in step S5 includes a target item (4) directory, an entertainment directory, a management directory, and a life directory.

7. The interaction method of a home care robot based on posture adjustment and projection calibration according to claim 1 or 6, characterized in that: In step S5, the projection content and the gesture posture use the same coordinate system, and the accompanying robot (1) moves to the intersection area with the image according to the gesture posture, amplifies the intersection area through a specific gesture posture, and adds a red marking point on the projection area to mark the target object (4) pointed to by the user (3).

8. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 7, characterized in that: The posture adjustment in step S6 includes the accompanying robot (1) moving forward, backward, left, and right for adjustment. During the posture adjustment process, the projection area remains constant and the projection clarity gradually changes.

9. The interaction method of a home care robot based on posture adjustment and projection calibration according to claim 1 or 8, characterized in that: The directory where the target item (4) is located in step S7 establishes a one-to-one correspondence with the local coordinates of the target item (4). The user (3) marks the location of the required target item (4) in the directory and confirms it by double-clicking. The accompanying robot (1) is linked to the control cabinet where the target item (4) is located for control, and the control cabinet is prepared before taking it.

10. The interactive method of a home care robot based on posture adjustment and projection calibration according to claim 9, characterized in that: In the step S9, the corresponding target item (4) is picked up, the accompanying robot (1) is equipped with a tray, the control cabinet cooperates with the robotic arm, the robotic arm clamps the target item (4) onto the tray, and the accompanying robot (1) transports the target item (4) to the room where the user (3) is located through the tray and moves it in front of the user (3).

Images