Robot behavior control method with man-machine interaction function

Abstract

The invention provides a robot behavior control method with a man-machine interaction function, relates to the technical field of robots, and aims to interact with a user through a robot, identify actions of the user through a robot terminal, find out non-standard actions and assist the user to correct the actions through the robot. The method can be suitable for various motion calibration scenes, such as dance motion calibration, fitness motion calibration, rehabilitation motion calibration and the like. The accuracy and efficiency of training can be improved.

Description

Technical Field

[0001] The present invention relates to robot technology, and in particular to a robot behavior control method with human-machine interaction function. Background Art

[0002] In modern life, people are paying more and more attention to their health and skills. Whether it is rehabilitation to restore body functions, fitness to shape a good body, or the pursuit of exquisite performance in dance art, accurate movement calibration is the key link.

[0003] However, traditional motion calibration methods have many limitations. Most of them rely on one-on-one manual guidance, which not only consumes a lot of manpower and time costs, but also the levels of instructors vary widely, making it difficult to ensure the accuracy and consistency of calibration.

[0004] Therefore, a solution is urgently needed to break through this bottleneck. Summary of the invention

[0005] The present invention provides a robot behavior control method with a human-machine interaction function, which can improve the efficiency and accuracy of action calibration.

[0006] A first aspect of the present invention provides a robot behavior control method with a human-machine interaction function, comprising a robot body and an interaction device carrier, wherein the robot body is provided with a plurality of electrode sheets, and the interaction device carrier is provided with metal parts corresponding to the electrode sheets, including: The robot recognizes the user's current action based on the camera module of the robot body, and records the user's non-compliant actions based on the comparison results between the current action and the standard action; Determine the calibration part corresponding to the non-compliant action, and activate the electrode sheet corresponding to the calibration part to make it fit with the corresponding metal part of the interactive device carrier; Control the robot to perform standard actions and generate reminder information.

[0007] Optionally, in a possible implementation of the first aspect, identifying the current action of the user according to the shooting module of the robot body, and recording the non-compliant action of the user in combination with the comparison result of the current action and the standard action, includes: Acquire image data collected by the shooting module and extract key points of the human body in the image data; Determine the user's current action based on the relative positions and current angles of each key point; The standard position and standard angle of the preset point corresponding to the standard action are retrieved, and the standard position and standard angle are compared with the relative position and the current angle respectively, and the current action whose comparison similarity is less than a threshold is determined to be a non-compliant action.

[0008] Optionally, in a possible implementation of the first aspect, before identifying the current action of the user according to the shooting module of the robot body and recording the non-compliant action of the user in combination with the comparison result of the current action and the standard action, the method further includes: Identify the actual height of the user based on the camera module and obtain the ratio between the user's hand and height; According to the initial limb length and height of the robot body, combined with the proportional relationship and the height value, the adjustment parameters of the limb parts of the robot body are calculated, and the limb parts include hands and legs; The limbs of the robot body are adjusted according to the adjustment parameters so that the height of the robot body is consistent with the height of the user.

[0009] Optionally, in a possible implementation of the first aspect, in the process of identifying the current action of the user according to the shooting module of the robot body, and recording the non-compliant action of the user in combination with the comparison result of the current action and the standard action, the process further includes: Identify the key points of the human body in multiple image frames collected by the shooting module, and calculate the displacement values ​​of the same key points in adjacent frames; Calculate the average value of multiple displacement values, determine the preset standard displacement value corresponding to the robot body, and obtain the adjustment coefficient according to the ratio of the average value to the standard displacement value; The standard speed of the robot body is offset-adjusted according to the product of the adjustment coefficient and the standard speed of the robot body.

[0010] Optionally, in a possible implementation of the first aspect, in the process of identifying the current action of the user according to the shooting module of the robot body, and recording the non-compliant action of the user in combination with the comparison result of the current action and the standard action, the process further includes: Obtain the proficiency level input by the user, and control the robot's body motion according to the preset speed corresponding to the proficiency level; The user's current speed is collected in real time to obtain the speed difference between the preset speed and the current speed. When the speed difference is greater than the threshold, the robot body movement is controlled according to the preset speed that is one level lower than the current proficiency level.

[0011] Optionally, in a possible implementation manner of the first aspect, determining a calibration part corresponding to the non-compliant action, and activating an electrode sheet corresponding to the calibration part to make it fit with a corresponding metal part of an interactive device carrier, includes: Identify the actual position of the metal component corresponding to the electrode sheet according to the image data collected by the camera module; The optimal path is generated by taking the current position of the electrode sheet at the calibration site as the starting point and the actual position of the metal component as the end point; The calibration part of the robot body is controlled to move according to the preferred path, and the corresponding electrode sheet is activated to perform the bonding operation on the electrode sheet and the metal part at the calibration part.

[0012] Optionally, in a possible implementation of the first aspect, controlling the movement of the calibration part of the robot body according to the preferred path includes: Obtain the distance value between the corresponding electrode sheet and the metal part in the calibration position during the movement; When the distance value is greater than the distance threshold, the calibration part is controlled to move according to the preset speed; When the distance value is less than or equal to the distance threshold, the preset speed is offset according to the product of the ratio of the distance value to the distance threshold and the preset speed, and the movement of the calibration part is controlled according to the offset speed.

[0013] Optionally, in a possible implementation of the first aspect, controlling the robot body to perform a standard action and generating reminder information includes: Control the robot body to perform standard movements, and collect the user's force values ​​at the corresponding positions of each electrode sheet based on the force feedback module; When the force value is not within the standard force value range, a reminder message is generated.

[0014] Optionally, in a possible implementation manner of the first aspect, when the applied force value is not within the standard applied force value range, the process of generating the reminder information further includes: When the applied force value is less than the minimum value of the interval, a first adjustment value corresponding to the difference between the minimum value of the interval and the applied force value is obtained, and the first adjustment value is added to the current current value of the corresponding electrode sheet; When the applied force value is greater than the maximum value of the interval, a second adjustment value corresponding to the difference between the applied force value and the maximum value of the interval is obtained, and the second adjustment value is subtracted from the current current value of the corresponding electrode sheet.

[0015] Optionally, in a possible implementation of the first aspect, an average adjustment value of the first adjustment value and / or the second adjustment value of each electrode sheet after calibration is completed is obtained, and a position corresponding to the electrode sheet whose average adjustment value is greater than a preset threshold is determined as a position prone to error; The average of the historical current values ​​of the error-prone high-incidence positions is calculated, and the current value of the corresponding electrode sheet is set to the average value during the next interactive operation.

[0016] A second aspect of the present invention provides a robot behavior control system with a human-machine interaction function, comprising a robot body and an interaction device carrier, wherein the robot body is provided with a plurality of electrode sheets, and the interaction device carrier is provided with metal parts corresponding to the electrode sheets, including: The recognition module is used to recognize the user's current action according to the shooting module of the robot body, and record the user's non-compliant action in combination with the comparison result between the current action and the standard action; A fitting module is used to determine the calibration part corresponding to the non-compliant action, and activate the electrode sheet corresponding to the calibration part to fit it with the corresponding metal part of the interactive device carrier; The calibration module is used to control the robot body to perform standard actions and generate reminder information.

[0017] The beneficial effects of the present invention are as follows: By installing cameras at multiple locations on the robot body and using image fusion technology to obtain complete user motion information, the real-time detected motion posture is compared with the pre-built standard motion library, and the similarity between the two is accurately calculated, so as to accurately identify the user's non-compliant motion. This precise motion recognition and recording method provides a reliable basis for subsequent motion calibration, greatly improving the pertinence and effectiveness of calibration. For example, in dance motion calibration, it can accurately capture the subtle deviations of each joint of the dancer, helping them to quickly improve their movements and enhance their dance expressiveness; in fitness scenarios, it can promptly detect the wrong movements of fitness enthusiasts and reduce the risk of sports injuries; for rehabilitation patients, it can record their motion recovery in detail, providing strong data support for the adjustment of rehabilitation treatment plans.

[0018] According to the user's height value and the hand-to-height ratio obtained by the shooting module, combined with the initial limb length and height of the robot body, the adjustment parameters of the robot body's limbs (hands and legs) are accurately calculated to achieve an accurate match between the robot body height and the user's height, greatly improving the convenience and comfort of human-computer interaction. At the same time, the robot body's movement speed can be adjusted in a variety of ways. It can be based on the displacement values ​​of the key points of the human body in the multi-frame images collected by the shooting module, and the ratio of the average value to the standard displacement value can be calculated to obtain the adjustment coefficient to offset the robot's standard speed; it can also be based on the preset speed corresponding to the proficiency level input by the user, combined with the current speed of the user collected in real time, when the speed difference is greater than the threshold, the robot's movement speed can be flexibly adjusted. This personalized speed adaptation mechanism enables the robot to closely follow the user's movement rhythm and provide customized services for users with different proficiency levels and movement speeds.

[0019] After determining the calibration part corresponding to the user's non-compliant action, the robot body accurately controls the corresponding electrode sheet to fit the metal parts on the interactive device carrier. During the fitting process, the distance value between the electrode sheet and the metal part is monitored in real time, and the movement speed of the calibration part is dynamically adjusted to ensure the efficiency and accuracy of the fitting. When the user follows the robot to perform standard actions, the force feedback module collects the user's force value at the corresponding position of each electrode sheet in real time. When the force value deviates from the standard force value range, the current of the electrode sheet is automatically adjusted. When the force is insufficient, the current is increased to enhance the adsorption force to assist the user in applying force. When the force is too large, the current is reduced to reduce the user's force, thereby helping the user to better complete the standard action, improve the training effect, and ensure training safety. In addition, by calculating the average adjustment value of the adjustment value after the calibration of each electrode sheet, the high-incidence position prone to error is determined, and the electrode sheet current value is pre-set at the next interactive operation according to the average historical current value of the position, so as to further optimize the calibration effect of the prone-to-error position. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention; Figure 2 is a flow chart of a robot behavior control method with human-computer interaction function provided by an embodiment of the present invention; Figure 3 It is a structural schematic diagram of a robot behavior control system with human-computer interaction function provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 , is a schematic diagram of an application scenario provided by an embodiment of the present invention. Specifically, in this embodiment, the robot can interact with the user, identify the user's movements through the robot terminal, find out the irregular movements, and assist the user in calibrating the movements through the robot. It can be applied to various movement calibration scenarios, such as dance movement calibration, fitness movement calibration, rehabilitation movement calibration, etc.

[0023] This embodiment includes a robot body and an interactive device carrier, the robot body is provided with a plurality of electrode sheets, and the interactive device carrier is provided with metal parts corresponding to the electrode sheets. Among them, the interactive device carrier can be a piece of clothing with metal parts, and the clothing can be sewn or embedded with thin and light metal patches with good magnetic conductivity at key joints, such as shoulders, elbows, wrists, hips, knees and ankles. The shape and size of these metal patches are optimized to ensure that there is enough adsorption area with the robot electrode sheets without affecting the normal activities of the user. For example, the metal patch on the wrist is designed to be relatively small and flexible, and does not hinder the rotation of the wrist. When the user wears the interactive device carrier, the electrode sheets in the robot body and the metal parts in the interactive device carrier fit together, and the user can be assisted in calibrating the action by controlling the robot body.

[0024] See also Figure 2 , is a flow chart of a robot behavior control method with human-computer interaction function provided by an embodiment of the present invention, Figure 2 The execution subject of the method shown may be software and / or hardware devices. The execution subject of the present application may include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, user equipment may include but is not limited to computers, smart phones, personal digital assistants (PDA) and the electronic devices mentioned above. Network equipment may include but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. It includes steps S101 to S103, as follows: S101, identifying the user's current action according to the shooting module of the robot body, and recording the user's non-compliant action in combination with the comparison result between the current action and the standard action.

[0025] In practical applications, the shooting module can be a camera, and multiple cameras can be installed at multiple locations on the robot body, such as the chest, arms, back, head, etc., to form a distributed viewing system. Cameras at different locations can capture user actions from different angles, and obtain complete user action information through image fusion technology.

[0026] When training the user's movements, the robot can show a set of movements, and the user follows the training. In this process, the user's movements can be identified, and all non-standard movements can be recorded. Non-standard movements are user movements that are inconsistent with standard movements. Standard movements can be the corresponding standard movements of the user's current training. The robot obtains the movement instructions corresponding to the standard movements in a variety of ways. First, it can be connected to external devices (such as smart speakers, mobile phone apps, etc.), and the user selects a preset movement sequence (such as fitness movement packages, dance movement combinations, etc.) on the APP and sends it to the robot. Second, the robot itself has the function of movement programming. The user can program the movement on the robot's operation interface by gestures, touch screens, etc., and the robot records and stores these movement instructions. Third, the robot can also obtain the latest movement library from the cloud server. These movement libraries are produced and uploaded by professional fitness coaches, dance choreographers, etc., and contain movements of various types and difficulty levels. When the robot completes an action demonstration, it can collect the user's movements and compare them with the standard movements currently demonstrated, so as to identify and record the user's non-standard movements.

[0027] Based on the above embodiment, the specific implementation of step S101 may be: Obtain image data collected by the shooting module and extract key points of the human body in the image data.

[0028] Specifically, the camera on the robot body can be used to shoot the user's movements from multiple angles in real time to obtain a continuous sequence of images, ensuring that the movement information of the human joints and limbs can be fully captured. Then, through deep learning detection models such as OpenPose and HRNet, the characteristic patterns of different parts of the human body are learned to accurately detect the key points of the human body. Among them, the key points are the key joints in the human body, such as the shoulder, elbow, wrist, hip, knee, ankle and other key joints.

[0029] The user's current action is determined based on the relative positions and current angles of each key point.

[0030] The skeleton structure can be constructed through the connection relationship between each key point. By analyzing the relative position and current angle of each joint point in the skeleton structure, the current posture of the human body can be determined. For example, by calculating the angle and distance between the joint points of the arm, it can be determined whether the arm is in a straight state or a bent state, and the degree of bending.

[0031] The standard position and standard angle of the preset point corresponding to the standard action are retrieved, and the standard position and standard angle are compared with the relative position and the current angle respectively, and the current action whose comparison similarity is less than a threshold is determined to be a non-compliant action.

[0032] In practical applications, image sequences of various standard actions and corresponding preset point coordinate information can be collected and annotated in advance to build a standard action library. The action posture detected in real time is compared with the actions in the standard action library, the similarity between the two is calculated, and an appropriate threshold is set. Actions with a similarity less than the threshold are judged as non-standard actions.

[0033] Among them, the preset point refers to the preset joint point. When comparing, the position distance and angle of the corresponding joint point can be compared respectively, and the similarity values ​​of the distance and angle are calculated and then added and fused to obtain the similarity after the fusion calculation, thereby improving the accuracy of the judgment.

[0034] When recording non-compliant actions, the detected human key point coordinates and related feature data, such as joint angles, can be saved in the form of text files or databases. In order to facilitate the distinction of key points, each key point can be assigned a unique number corresponding to it. When recording, the key point number, coordinate value in the image, feature value, etc. can be saved for subsequent query and analysis.

[0035] In addition, in order to improve the synchronization of the actions during the interaction, before step S101, the height of the robot body can also be adjusted in combination with the height of the user to keep it consistent with the height of the user, thereby improving the efficiency and convenience of human-computer interaction. Specifically, the height of the robot body can be adjusted through the following embodiments: The actual height of the user is identified by the camera module, and the ratio between the user's hand and height is obtained.

[0036] Specifically, when obtaining the user's height value, the shooting module can be used to shoot a full-body frontal image of the user, ensuring that the user's feet are standing steadily on the same plane, the body is straight, and the camera is perpendicular to the user and within a suitable distance range to obtain a clear and complete human body image. Then, a deep learning human key point detection algorithm, such as OpenPose, is used to process the collected images and identify key nodes on the user's body, such as the top of the head and ankles. According to the coordinate information of the detected key points of the top of the head and ankles, the pixel distance between the two points is calculated. Since the imaging principle and shooting parameters of the camera are known, the pixel distance can be converted into the actual height value through the mapping relationship between pixels and actual distances established in advance.

[0037] When obtaining the proportional relationship between the user's hand and height, the specific key points of the hand, such as the acromion point and the fingertips, can be determined. Then the pixel distance from the acromion point to the fingertips in the image is measured and compared with the calculated pixel distance of the user's height to obtain the proportional relationship between the user's hand and height, so that the user can better cooperate with the robot.

[0038] According to the initial limb length and height of the robot body, combined with the proportional relationship and the height value, the adjustment parameters of the limb parts of the robot body are calculated, and the limb parts include hands and legs.

[0039] The adjustment parameter is a value corresponding to the adjustment amplitude of the robot's limb length. After obtaining the ratio of the user's height and limbs, the value of the limb adjustment when the robot is adjusted to the corresponding height of the user can be calculated based on this value.

[0040] Due to the importance of leg length in determining height, the height difference between the user and the robot can be calculated, and the height difference can be adjusted by the leg length, that is, it can be used as the adjustment value of the robot's legs. The adjustment value of the hand can be obtained by subtracting the initial hand length of the robot from the product of the ratio of the user's hand to height and the height value.

[0041] The limbs of the robot body are adjusted according to the adjustment parameters so that the height of the robot body is consistent with the height of the user.

[0042] The hand limbs and leg limbs of the robot body can adopt a retractable structure, and the length of the hand and leg can be adjusted by controlling the motor drive.

[0043] During step S101, in order to achieve personalized customized training for users, the movement speed of the robot body during training can also be adjusted. By comparing the error between the actual movement speed and the user's movement speed that is expected to be followed, the movement speed of the robot body is continuously adjusted, so that the robot body can more accurately and smoothly follow the changes in the user's movement speed, thereby improving the accuracy of training.

[0044] Specifically in some embodiments, the movement speed of the robot body can be adjusted by the following steps: Identify the key points of the human body in multiple image frames collected by the shooting module, and calculate the displacement values ​​of the same key points in adjacent frames.

[0045] By using the shooting module to capture user movements in real time, the user's body parts in multiple image frames are identified and tracked, and the position information of each key point of the user's body and its changes over time are obtained. The characteristic parameters related to the movement speed can be extracted, that is, the displacement values ​​of each key point in adjacent frames.

[0046] The average value of multiple displacement values ​​is calculated to determine the preset standard displacement value corresponding to the robot body, and the adjustment coefficient is obtained according to the ratio of the average value to the standard displacement value.

[0047] It is understandable that, since there are multiple key points, when determining the user's speed, it can be determined by the average value of multiple key points. In practical applications, the standard displacement value corresponding to the standard speed during the action can be configured for the robot body in advance, and the adjustment coefficient for offset can be obtained by the ratio of the average value and the standard displacement value, and the standard speed of the robot body can be offset and adjusted by the adjustment coefficient.

[0048] The standard speed of the robot body is offset-adjusted according to the product of the adjustment coefficient and the standard speed of the robot body.

[0049] By offsetting the standard speed, the speed of the robot can be adjusted according to the difference between the user's movement speed and the robot's current movement speed to keep it consistent with the user's speed. For example, if the user's movement speed increases, the robot's movement speed can be increased accordingly; if the user's movement speed slows down, the robot's movement speed can be reduced.

[0050] In some other embodiments, the movement speed of the robot body can also be adjusted by the following steps: Obtain the proficiency level input by the user and control the robot's body movements according to the preset speed corresponding to the proficiency level.

[0051] In actual applications, users can input their proficiency level in a specific activity (such as dance, fitness, rehabilitation training, etc.). The proficiency level can be the user's subjective assessment of their ability level in the activity, such as elementary, intermediate, advanced, etc., or it can be determined by previous tests or training records.

[0052] The corresponding speed is preset for each proficiency level. For example, the preset speed corresponding to the elementary proficiency level is slower, which is suitable for beginners to gradually familiarize themselves with the movements and rhythm; the preset speed corresponding to the advanced proficiency level is faster, which is suitable for users with a certain foundation and ability to conduct more challenging training.

[0053] The user's current speed is collected in real time to obtain the speed difference between the preset speed and the current speed. When the speed difference is greater than the threshold, the robot body movement is controlled according to the preset speed that is one level lower than the current proficiency level.

[0054] When collecting the user's current speed, it can be obtained through the image frames collected by the shooting module, and it can also be determined by the displacement values ​​of key points. After determining the current speed, it can be compared with the preset speed corresponding to the user's current proficiency level to calculate the speed difference between the two. This difference reflects the difference between the user's actual movement speed and the system's expected speed.

[0055] You can also set a speed difference threshold. This threshold is determined based on the nature and requirements of the activity and is used to determine whether the difference between the user's current speed and the preset speed is too large. When the calculated speed difference is greater than this threshold, it means that the user may not be able to keep up with the preset speed corresponding to the current proficiency level, and may have movement difficulties or other problems. Therefore, in this case, the speed of the robot body can be adjusted to a preset speed that is one level lower than the user's current proficiency level. The purpose of this is to make the robot's speed more in line with the user's actual ability, help the user to better complete the activity, and avoid situations where the user is injured or unable to continue training due to excessive speed. For example, if the user is currently at an intermediate proficiency level, when the speed difference is greater than the threshold, the robot will move at the preset speed corresponding to the primary proficiency level, providing the user with a more suitable training rhythm.

[0056] S102, determining the calibration part corresponding to the non-compliant action, and activating the electrode sheet corresponding to the calibration part to make it fit with the corresponding metal part of the interactive device carrier.

[0057] The calibration part is the body part corresponding to the irregular action, such as the arm. It is understandable that after the irregular body part of the user is determined, after the user wears the interactive device carrier, the electrode sheet of the robot body can be fitted with the metal parts of the corresponding part in the interactive device carrier, so that the corresponding part in the robot body can be controlled to move according to the action instruction, and the user can be driven to perform the action, so as to realize the calibration of the user's action.

[0058] It should be noted that the electrodes on the robot body do not generate magnetism when not powered on. The reason why the calibration position must be determined before the electrodes at the corresponding position are powered on is to enable the robot body to apply force to specific parts more concentratedly to achieve precise control and calibration of the action. For example, when performing fine hand movements, only the electrodes near the wrist and arm joints are powered on. The robot can accurately adjust the force and movement trajectory of these parts to avoid unnecessary interference from powering on other parts and improve the accuracy of action execution.

[0059] Specifically, after determining the calibration position, the position of the metal patch corresponding to the calibration position in the interactive device carrier can be determined. After the robot body adjusts its own position, the corresponding electrode sheet and metal parts will be adsorbed and bonded. During the bonding process, the selected electrode sheet can be energized to generate magnetism. The startup process is the power-on process.

[0060] Based on the above embodiment, the specific implementation of step S102 may be: The actual position of the metal component corresponding to the electrode sheet is identified based on the image data collected by the shooting module.

[0061] Specifically, the position of the metal parts on the user's clothing can be identified by the camera. For example, by extracting the features of the metal parts in the image (such as shape, color, texture, etc.), the specific position of the metal parts in the image can be accurately determined. Then, the actual position of the metal parts corresponding to the corresponding electrode sheets can be determined. The coordinates of the center point of the metal parts can be converted through the mapping relationship between pixels and actual distances to obtain the corresponding actual position.

[0062] The optimal path is generated with the current position of the electrode sheet at the calibration site as the starting point and the actual position of the metal component as the end point.

[0063] The current position of the electrode sheet on the calibration part (such as the user's arm, leg, or other body part that needs to be fitted with the electrode sheet) is clearly defined as the starting point of the path. This position can be obtained by the robot body using its own encoder, gyroscope, and other sensors. The actual position of the metal part is used as the end point of the path. Based on the position information of the starting point and the end point, a path planning algorithm (such as the Dijkstra algorithm, etc.) is used to generate a path from the current position of the electrode sheet to the actual position of the metal part. The preferred path is the optimal path from the current position to the actual position, which can be the shortest path.

[0064] In the process of generating a path, various factors can also be considered, such as collecting surrounding obstacles and avoiding them when planning the path.

[0065] The calibration part of the robot body is controlled to move according to the preferred path, and the corresponding electrode sheet is activated to perform the bonding operation on the electrode sheet and the metal part at the calibration part.

[0066] According to the generated preferred path, instructions are sent to the control system of the robot body to control the robot calibration part (such as the robotic arm, etc.) to move along the path. The robot's drive system will accurately adjust the angle and movement of each joint according to the instructions, so that the calibration part gradually approaches the metal part along the preferred path. During the movement of the robot's calibration part, the electrode sheet associated with the calibration part can be activated. The electrode sheet can be powered to generate magnetism so as to interact with the metal part. When the calibration part moves near the metal part, due to the physical properties of the electrode sheet (such as magnetism), the electrode sheet will attract the metal part to achieve bonding. The purpose of this bonding operation is to ensure that the electrode sheet is tightly connected to the metal part so that it can work normally in subsequent activities and realize effective interaction and collaboration between the robot and the user.

[0067] In some embodiments, the calibration part of the robot body can be controlled to move according to the preferred path by the following steps: The distance value between the corresponding electrode sheet and the metal part in the calibration part during the movement is obtained. When the distance value is greater than the distance threshold, the calibration part is controlled to move according to the preset speed. When the distance value is less than or equal to the distance threshold, the preset speed is offset according to the ratio of the distance value to the distance threshold and the product of the preset speed, and the calibration part is controlled to move according to the offset speed.

[0068] The robot body controls the movement of the calibration part according to the preferred path to achieve the process of bonding the electrode sheet and the metal part, and the distance between the two can be continuously monitored. In this process, the calibration part of the robot will constantly change its position, and the relative position of the electrode sheet and the metal part will also change dynamically.

[0069] When the distance value obtained is greater than the distance threshold, it means that the electrode sheet and the metal part are still relatively far away. At this time, the calibration part can be controlled to move according to the preset speed. This preset speed can be a relatively fast and safe speed, the purpose of which is to allow the robot body to quickly approach the metal part, improve the efficiency of the fitting operation, and avoid problems such as collisions caused by excessive speed. The distance threshold can be determined based on factors such as the characteristics of the electrode sheet and the metal part, the fitting requirements, and the robot's motion performance.

[0070] When the distance between the electrode sheet and the metal part is less than or equal to the distance threshold, it means that the two are close. In this case, in order to more accurately control the bonding process between the electrode sheet and the metal part and avoid impact or inaccurate bonding due to excessive speed, the moving speed of the calibration part needs to be adjusted.

[0071] Calculate the ratio of the distance value to the distance threshold, and then multiply this ratio by the preset speed to get a speed offset. The principle of this calculation method is that as the electrode sheet and the metal part get closer (the distance value gets smaller and smaller), the speed offset will also decrease accordingly, so that the movement speed of the calibration part will gradually decrease. For example, if the distance value is half of the distance threshold, then the speed offset is half of the preset speed, that is, the movement speed of the calibration part will become half of the original preset speed.

[0072] According to the calculated speed offset, the preset speed is adjusted to obtain the offset speed. Then, the calibration part is controlled to continue to move according to the offset speed, so that the electrode sheet can approach the metal part at a slower and more precise speed, and finally achieve accurate and smooth fitting operation.

[0073] Through the above method, the moving speed of the calibration part can be dynamically adjusted according to the distance value between the electrode sheet and the metal part, which can improve the accuracy and safety of the bonding while ensuring the bonding efficiency.

[0074] S103, controlling the robot body to perform standard actions and generating reminder information.

[0075] After completing the fitting operation, the robot body can control the corresponding joint movement according to the action instructions corresponding to the standard action, leading the user to perform the action. For example, when performing the "raise your hand to a horizontal position" action, the robot's shoulder motor and elbow motor work together to smoothly lift the user's arm according to the preset motion trajectory and speed.

[0076] During the process of calibrating the user's movements, the robot body can continuously monitor the user's movement status through various sensors, and generate reminder information when the movement status deviates from the standard status, thereby reminding the user to calibrate.

[0077] In actual applications, the reminder information can be voice reminder information. For example, the robot body can have a built-in speech synthesis module. When it recognizes that the user's action is not in place, it will play a pre-recorded or real-time synthesized prompt voice through the speaker. For example, "Your arm is not stretched enough, please stretch it a little more", etc., clearly and clearly telling the user what needs to be improved.

[0078] In the above way, accurate calibration of actions can be achieved through the interaction between the robot and the user.

[0079] Based on the above embodiment, the specific implementation of step S103 may be: The robot body is controlled to perform standard movements, and the force applied by the user at the corresponding position of each electrode is collected based on the force feedback module. When the force applied is not within the standard force applied range, a reminder message is generated.

[0080] Among them, the force feedback module can be a force sensor, and the force value refers to the force value applied by the user. A force sensor can be installed at the part of the robot body that contacts the user to sense the magnitude and direction of the force applied by the user during the following action. For example, when the robot drives the user to raise his arm, if the force applied by the user is obviously too large or too small, and there is a huge difference with the force required for the standard action, it may indicate that the user's force is not applied correctly or the muscle control is not coordinated, resulting in non-standard actions. At this time, a reminder message can be generated to remind the user.

[0081] The standard force value range can be a pre-set value range corresponding to normal force application. For example, when performing a joint stretching movement in rehabilitation training, a specific force value range is set based on the normal range of motion and muscle strength requirements of the joint to ensure that the user trains within the correct force range.

[0082] In addition, while reminding, the robot body will also adjust the motion parameters of its own joints according to the degree of difference and appropriately apply external force to assist the user in correcting the action. When the applied force value is less than the minimum value of the interval, obtain the first adjustment value corresponding to the difference between the minimum value of the interval and the applied force value, and add the first adjustment value to the current current value of the corresponding electrode sheet. When the applied force value is greater than the maximum value of the interval, obtain the second adjustment value corresponding to the difference between the applied force value and the maximum value of the interval, and subtract the second adjustment value from the current current value of the corresponding electrode sheet.

[0083] Specifically, the force values ​​collected by the force feedback module at each position of the user can be continuously monitored and compared with the standard force value interval. When it is found that the force value at a certain position is less than the minimum value of the standard force value interval, it indicates that the force applied by the user at this position is insufficient and does not reach the force level required by the standard action. In order to adjust the user's force application, the difference between the minimum value of the interval and the current force value can be calculated. Then, according to a pre-set corresponding relationship (for example, a mapping table of differences and adjustment values ​​established by experimental data), a first adjustment value corresponding to the difference is obtained. This first adjustment value represents the amplitude by which the current of the corresponding electrode sheet needs to be adjusted in order to make the force value reach the standard interval. After obtaining the first adjustment value, the current current value of the corresponding electrode sheet can be operated, and the first adjustment value is added to the current current value, thereby changing the electromagnetic adsorption force, increasing the adsorption force of the electrode sheet at this position, and guiding the user to move in the correct direction of action. For example, if the user's arm is not lifted at an angle sufficient, the robot body increases the adsorption force of the shoulder electrode sheet to assist the user in lifting his arm to a suitable position.

[0084] Similarly, when it is detected that the force value at the corresponding position of a certain electrode sheet is greater than the maximum value of the standard force value interval, it means that the force applied by the user at this position is too large and exceeds the reasonable range allowed by the standard action. This excessive force may cause problems such as deformation of the movement and increased risk of injury. At this time, the difference between the force value and the maximum value of the interval can be calculated, and a second adjustment value corresponding to the difference can be obtained based on a pre-set corresponding relationship. This second adjustment value is to adjust the electrode sheet current to reduce the user's force. Subtract the second adjustment value from the current value of the corresponding electrode sheet. The reduction in the electrode sheet current will weaken the adsorption force it generates, reduce the force applied to the user at this position, and return the force value to the standard force value interval.

[0085] By adjusting the electrode current according to the relationship between the applied force value and the standard range, it is possible to automatically take measures to intervene and adjust when the user applies abnormal force, thereby helping the user to better complete standard movements and improve training effects, while ensuring the user's safety and avoiding adverse consequences caused by improper force application.

[0086] Based on the above embodiments, this solution also includes the following embodiments: Obtain the average adjustment value of the first adjustment value and / or the second adjustment value of each electrode sheet after calibration, determine that the corresponding position of the electrode sheet whose average adjustment value is greater than a preset threshold is a high-incidence position of error, calculate the mean of the historical current values ​​of the high-incidence position of error, and set the current value of the corresponding electrode sheet to the mean during the next interactive operation.

[0087] After the calibration operation is completed, the first adjustment value (when the force value is less than the minimum value of the interval) and / or the second adjustment value (when the force value is greater than the maximum value of the interval) generated by each electrode during the entire calibration process due to the mismatch between the user's force value and the standard force value interval can be recorded. These adjustment values ​​reflect the changes made to the electrode current in order to make the force meet the standard.

[0088] For each electrode sheet, the average value of all first adjustment values ​​and / or second adjustment values ​​is calculated to obtain the average adjustment value of the electrode sheet. The average adjustment value comprehensively reflects the degree and trend of abnormal force application at the corresponding position of the electrode sheet during this calibration process.

[0089] In practical applications, a preset threshold can be set in advance, and the average adjustment value of each electrode sheet can be compared with the threshold. If the average adjustment value of an electrode sheet is greater than the preset threshold, it means that the force applied to the corresponding position of the electrode sheet is abnormally frequent and to a large extent during this calibration, and the position is determined to be a high-incidence position for errors. For example, in rehabilitation training, if the average adjustment value of the electrode sheet at the wrist is large, it means that the user often deviates from the standard range in the force applied in wrist-related movements, and the wrist position is a high-incidence position for errors. This helps to accurately locate the parts of the user's body that are prone to force problems during the execution of the action. High-incidence positions for errors are the parts of the user's body that are prone to force problems.

[0090] For the identified high-incidence error-prone positions, the current value records of the corresponding electrode sheets in the past multiple interactive operations at that position can be queried and collected, that is, the historical current values. By performing statistical analysis on these historical current values, their average values ​​are calculated. These historical current values ​​reflect the current adjustment used to correct the force application problem at that position during different calibration processes. For example, after many rehabilitation training sessions, the system records the current value of the electrode sheet at the wrist in each training session. By calculating the average of these values, a current value with reference significance can be obtained.

[0091] Before the next interactive operation (such as a new round of rehabilitation training), the current value of the electrode sheet corresponding to the position prone to error can be set to the calculated historical current average. The purpose of this is to optimize the current of the electrode sheet at the position prone to error in advance based on past experience. Because past calibration operations have shown that this current average is more effective in guiding the force applied at this position to approach the standard range. For example, at the beginning of a new round of rehabilitation training, the current of the electrode sheet at the wrist is directly set to the previously calculated historical average, so as to help users meet the standard force requirements in wrist movements more quickly and improve training efficiency and effectiveness.

[0092] See also Figure 3 , is a schematic diagram of the structure of a robot behavior control system with a human-computer interaction function provided by an embodiment of the present invention, the robot behavior control system with a human-computer interaction function comprises: It includes a robot body and an interactive device carrier. The robot body is provided with a plurality of electrode sheets, and the interactive device carrier is provided with metal parts corresponding to the electrode sheets, including: The recognition module is used to recognize the user's current action according to the shooting module of the robot body, and record the user's non-compliant action in combination with the comparison result between the current action and the standard action; A fitting module is used to determine the calibration part corresponding to the non-compliant action, and activate the electrode sheet corresponding to the calibration part to fit it with the corresponding metal part of the interactive device carrier; The calibration module is used to control the robot body to perform standard actions and generate reminder information.

[0093] Figure 3 The apparatus of the embodiment shown can be used to perform Figure 2 The implementation principles and technical effects of the steps in the method embodiment shown are similar and will not be repeated here.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot behavior control method with human-computer interaction function, characterized in that: It includes a robot body and an interactive device carrier. The robot body is provided with a plurality of electrode sheets, and the interactive device carrier is provided with metal parts corresponding to the electrode sheets, including: The robot recognizes the user's current action based on the camera module of the robot body, and records the user's non-compliant actions based on the comparison results between the current action and the standard action; Determine the calibration part corresponding to the non-compliant action, and activate the electrode sheet corresponding to the calibration part to make it fit with the corresponding metal part of the interactive device carrier; Control the robot to perform standard actions and generate reminder information.

2. The method according to claim 1, characterized in that The robot's camera module identifies the user's current action and compares the current action with the standard action to record the user's non-compliant actions, including: Acquire image data collected by the shooting module and extract key points of the human body in the image data; Determine the user's current action based on the relative positions and current angles of each key point; The standard position and standard angle of the preset point corresponding to the standard action are retrieved, and the standard position and standard angle are compared with the relative position and the current angle respectively, and the current action whose comparison similarity is less than a threshold is determined to be a non-compliant action.

3. The method according to claim 1, characterized in that Before the current action of the user is recognized according to the shooting module of the robot body and the non-compliant action of the user is recorded in combination with the comparison result between the current action and the standard action, the following is also included: Identify the actual height of the user based on the camera module and obtain the ratio between the user's hand and height; According to the initial limb length and height of the robot body, combined with the proportional relationship and the height value, the adjustment parameters of the limb parts of the robot body are calculated, and the limb parts include hands and legs; The limbs of the robot body are adjusted according to the adjustment parameters so that the height of the robot body is consistent with the height of the user.

4. The method according to claim 1, characterized in that In the process of identifying the user's current action according to the shooting module of the robot body and recording the user's non-compliant action in combination with the comparison result of the current action and the standard action, it also includes: Identify the key points of the human body in multiple image frames collected by the shooting module, and calculate the displacement values ​​of the same key points in adjacent frames; Calculate the average value of multiple displacement values, determine the preset standard displacement value corresponding to the robot body, and obtain the adjustment coefficient according to the ratio of the average value to the standard displacement value; The standard speed of the robot body is offset-adjusted according to the product of the adjustment coefficient and the standard speed of the robot body.

5. The method according to claim 1, characterized in that In the process of identifying the user's current action according to the shooting module of the robot body and recording the user's non-compliant action in combination with the comparison result of the current action and the standard action, it also includes: Obtain the proficiency level input by the user, and control the robot's body motion according to the preset speed corresponding to the proficiency level; The user's current speed is collected in real time to obtain the speed difference between the preset speed and the current speed. When the speed difference is greater than the threshold, the robot body movement is controlled according to the preset speed that is one level lower than the current proficiency level.

6. The method according to claim 1, characterized in that Determine the calibration part corresponding to the non-compliant action, activate the electrode sheet corresponding to the calibration part, and make it fit with the corresponding metal part of the interactive device carrier, including: Identify the actual position of the metal component corresponding to the electrode sheet according to the image data collected by the camera module; The optimal path is generated by taking the current position of the electrode sheet at the calibration site as the starting point and the actual position of the metal component as the end point; The calibration part of the robot body is controlled to move according to the preferred path, and the corresponding electrode sheet is activated to perform the bonding operation on the electrode sheet and the metal part at the calibration part.

7. The method according to claim 6, characterized in that Controlling the movement of the calibration part of the robot body according to the preferred path includes: Obtain the distance value between the corresponding electrode sheet and the metal part in the calibration position during the movement; When the distance value is greater than the distance threshold, the calibration part is controlled to move according to the preset speed; When the distance value is less than or equal to the distance threshold, the preset speed is offset according to the product of the ratio of the distance value to the distance threshold and the preset speed, and the movement of the calibration part is controlled according to the offset speed.

8. The method according to claim 1, characterized in that Control the robot to perform standard actions and generate reminder information, including: Control the robot body to perform standard movements, and collect the user's force values ​​at the corresponding positions of each electrode sheet based on the force feedback module; When the force value is not within the standard force value range, a reminder message is generated.

9. The method according to claim 8, characterized in that When the applied force value is not within the standard applied force value range, the process of generating a reminder message also includes: When the applied force value is less than the minimum value of the interval, a first adjustment value corresponding to the difference between the minimum value of the interval and the applied force value is obtained, and the first adjustment value is added to the current current value of the corresponding electrode sheet; When the applied force value is greater than the maximum value of the interval, a second adjustment value corresponding to the difference between the applied force value and the maximum value of the interval is obtained, and the second adjustment value is subtracted from the current current value of the corresponding electrode sheet.

10. The method according to claim 9, characterized in that Also includes: Obtaining an average adjustment value of the first adjustment value and / or the second adjustment value of each electrode sheet after calibration, and determining that the corresponding position of the electrode sheet whose average adjustment value is greater than a preset threshold is a position prone to error; The average of the historical current values ​​of the error-prone high-incidence positions is calculated, and the current value of the corresponding electrode sheet is set to the average value during the next interactive operation.

Images