Meal assistance robot cooperative control method and device, equipment and medium

Abstract

The application relates to a meal assisting robot cooperative control method and device, equipment and medium, wherein the method comprises the following steps: acquiring a front face image and a target distance between an end effector of a SCARA mechanical arm in a meal assisting robot and a face; determining a center point position and a preset face key point position in the front face image; determining a corresponding direction vector when a meal feeding task is performed based on the target distance, the center point position and the preset face key point position; calculating target angular velocities of joints in the SCARA mechanical arm according to size parameters of the SCARA mechanical arm in the meal assisting robot and the direction vector; and controlling the movement of the SCARA mechanical arm based on the target angular velocities and a relative position relationship between the preset face key point position and a mouth. The whole scheme can realize efficient and stable feeding control.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a collaborative control method, device, computer equipment, storage medium, and computer program product for a meal-assisting robot. Background Technology

[0002] As the global population ages and the number of disabled elderly continues to grow, this group suffers from upper limb motor impairments and loses the ability to eat independently due to neurological diseases and other reasons. As a result, meal-assisting robots that assist with eating in the home environment have become a key need to solve care problems and improve the quality of life for patients. The market has placed higher demands on the functionality and stability of such robots.

[0003] From a technical perspective, current common home care meal-assisting robots have many limitations in terms of mechanical structure and visual perception. Mechanically, most employ multi-degree-of-freedom serial robotic arm structures. While these offer a large workspace and good operational flexibility, the drive system is distributed across various joints, resulting in a large overall inertia. During high-speed movement in dynamic tracking, this easily leads to response lag and significant vibration. Ordinary parallel robots, while possessing high rigidity and fast response characteristics, have limited workspace and struggle to adapt to the multi-pose, non-fixed-coordinate feeding scenarios required in a home environment. Regarding visual perception, most existing meal-assisting robot systems rely on external cameras for positioning. These external cameras are susceptible to environmental interference, and meal-assisting robots based on conventional image processing and tracking technologies cannot achieve stable feeding.

[0004] It is evident that, due to deficiencies in mechanical structure and visual perception, traditional technologies make it difficult to achieve efficient and stable collaborative control of meal-assisting robots in the complex and ever-changing home environment. Summary of the Invention

[0005] Therefore, it is necessary to provide a collaborative control method, device, computer equipment, storage medium, and computer program product for a meal-assisting robot that supports efficient and stable feeding, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a collaborative control method for a meal-assisting robot. The method includes: Acquire a frontal image of a human face and the target distance between the end effector of the SCARA (Selective Compliance Assembly Robot Arm) robotic arm in the meal-assisting robot and the human face; Determine the center point and preset facial key point positions in the frontal image of the face; Based on the target distance, center point position, and preset facial key point positions, determine the direction vector corresponding to the feeding task; Based on the size parameters and orientation vector of the SCARA robotic arm in the meal-assisting robot, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated. The movement of the SCARA robotic arm is controlled based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth.

[0007] In one embodiment, the direction vector for performing the feeding task is determined based on the target distance, the center point position, and the preset facial key point positions, including: Based on the center point position and the preset facial key point positions, obtain the visual transmission error value; Based on the visual transmission error value and the target distance, the direction vector corresponding to the feeding task is determined.

[0008] In one embodiment, obtaining the visual transmission error value based on the center point position and the preset facial key point positions includes: The key facial point is determined as the tip of the nose in the middle of the face; Obtain the first coordinate of the center point in the image coordinate system and the second coordinate of the tip of the nose in the image coordinate system; Calculate the offset between the first and second coordinates to obtain the visual transmission error value.

[0009] In one embodiment, the calculation of the target angular velocity corresponding to the joints in the SCARA robotic arm, based on the size parameters and orientation vector of the SCARA robotic arm in the dining robot, includes: The direction vector is abstracted into a velocity direction vector using fuzzy PID; Based on the size parameters and velocity direction vector of the SCARA robotic arm in the meal-assisting robot, the target angular velocity corresponding to the joints in the SCARA robotic arm is calculated.

[0010] In one embodiment, the calculation of the target angular velocity corresponding to the joints in the SCARA robotic arm, based on the size parameters and velocity direction vector of the SCARA robotic arm in the meal-assisting robot, includes: Obtain the first rotation angle of the first joint and the second rotation angle of the second joint in the current SCARA robotic arm; Based on the size parameters, extract the lengths of the first and second robotic arms in the SCARA robotic arm; Based on the first rotation angle, the second rotation angle, the first robotic arm length, the second robotic arm length, and the direction vector, a set of equations for solving the joint angular velocity is generated. Solve the system of equations for calculating the joint angular velocity to obtain the first angular velocity of the first joint and the second angular velocity of the second joint.

[0011] In one embodiment, the set of equations for calculating the joint angular velocity is as follows:

[0012] In the formula, The length of the first robotic arm. The length of the second robotic arm. For the first rotation angle, For the second rotation angle, The first angular velocity, The second angular velocity, and These are the first direction offset value and the second direction distance value corresponding to the direction vector, respectively.

[0013] Secondly, this application also provides a collaborative control device for a meal-assisting robot. The device includes: The data acquisition module is used to acquire frontal images of the human face and the target distance between the SCARA robotic arm end effector and the human face in the meal-assisting robot; The point determination module is used to determine the position of the center point and the preset facial key point positions in a frontal image of a face; The vector calculation module is used to determine the direction vector corresponding to the feeding task based on the target distance, center point position, and preset facial key point positions. The angular velocity calculation module is used to calculate the target angular velocity of the joints in the SCARA robotic arm based on the size parameters and direction vector of the SCARA robotic arm in the dining robot. The control module is used to control the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between preset facial key points and the mouth.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps: Acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face; Determine the center point and preset facial key point positions in the frontal image of the face; Based on the target distance, center point position, and preset facial key point positions, determine the direction vector corresponding to the feeding task; Based on the size parameters and orientation vector of the SCARA robotic arm in the meal-assisting robot, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated. The movement of the SCARA robotic arm is controlled based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth.

[0015] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps: Acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face; Determine the center point and preset facial key point positions in the frontal image of the face; Based on the target distance, center point position, and preset facial key point positions, determine the direction vector corresponding to the feeding task; Based on the size parameters and orientation vector of the SCARA robotic arm in the meal-assisting robot, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated. The movement of the SCARA robotic arm is controlled based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth.

[0016] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps: Acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face; Determine the center point and preset facial key point positions in the frontal image of the face; Based on the target distance, center point position, and preset facial key point positions, determine the direction vector corresponding to the feeding task; Based on the size parameters and orientation vector of the SCARA robotic arm in the meal-assisting robot, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated. The movement of the SCARA robotic arm is controlled based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth.

[0017] The aforementioned collaborative control method, device, computer equipment, storage medium, and computer program products for the meal-assisting robot acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face; determine the center point position and preset facial key point positions in the frontal image of the human face; determine the direction vector corresponding to the feeding task based on the target distance, center point position, and preset facial key point positions; calculate the target angular velocity corresponding to the joints in the SCARA robotic arm according to the size parameters and direction vector of the SCARA robotic arm in the meal-assisting robot; and control the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth. Throughout the process, the direction vector pointing from the target point to the center of the image is first determined based on the face image and distance, solving the problem of accurate tracking and providing reliable guidance for feeding positioning; then, this direction vector is decomposed into each joint of the SCARA robotic arm, and the joint rotation speed is calculated in combination with the size parameters, allowing the end effector to move smoothly and stably according to the predetermined direction vector, achieving efficient and stable feeding. Attached Figure Description

[0018] Figure 1 This is an application environment diagram of the collaborative control method for a meal-assisting robot in one embodiment; Figure 2 This is a flowchart illustrating the collaborative control method for a meal-assisting robot in one embodiment; Figure 3 This is a schematic diagram showing the center point position and preset facial key point positions of a frontal face image in one embodiment; Figure 4 This is a mapping diagram of a SCARA-structured robotic arm in a planar coordinate system in one embodiment; Figure 5 This is a flowchart illustrating the collaborative control method for a meal-assisting robot in another embodiment; Figure 6 This is a schematic diagram illustrating the application experiment of the collaborative control method for the meal-assisting robot in one embodiment; Figure 7 This is a structural block diagram of the collaborative control device for the meal-assisting robot in one embodiment; Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] The collaborative control method for the meal-assisting robot provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, controller 102 is connected to the meal-assisting robot 104. Controller 102 responds to user operations, acquires a frontal image of the face, and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the face; determines the center point position and preset facial key point positions in the frontal image of the face; based on the target distance, center point position, and preset facial key point positions, determines the direction vector corresponding to the feeding task; according to the size parameters and direction vector of the SCARA robotic arm in the meal-assisting robot, calculates the target angular velocity corresponding to the joints in the SCARA robotic arm; and controls the movement of the SCARA robotic arm in the meal-assisting robot 104 based on the target angular velocity.

[0021] In one embodiment, such as Figure 2 As shown, a collaborative control method for a meal-assisting robot is provided, which can be applied to... Figure 1 Taking controller 102 as an example, the following steps are included: S100: Acquires a frontal image of a human face and the target distance between the SCARA robotic arm end effector and the human face in the meal-assisting robot.

[0022] In this step, a frontal image of the user's face is acquired using a camera mounted on the feeding robot. Specifically, the camera can be mounted on the SCARA robotic arm's end effector. This camera features high-definition imaging, clearly capturing detailed facial features and providing accurate data for subsequently determining the location of facial key points. Simultaneously, a laser rangefinder is used to measure the target distance between the SCARA robotic arm's end effector and the user's face. Laser rangefinders are characterized by high measurement accuracy and fast response speed, enabling real-time acquisition of accurate distance information. The purpose of this step is to provide necessary data for subsequently determining the direction vector for the feeding task. Since the user's face may move during actual assisted feeding, accurately acquiring the facial image and distance information is a prerequisite for precise control of the robotic arm's movement. More specifically, this target distance refers to the distance between the SCARA robotic arm's end effector and preset facial key points on the user's face. Taking the tip of the nose as an example, the target distance here refers to the distance between the SCARA robotic arm's end effector and the user's face.

[0023] S200: Determine the position of the center point and the preset facial key point positions in the frontal image of the face.

[0024] The center point of the entire frontal facial image is determined. Since the spoon and camera of a feeding robot are generally integrated into the end effector of a SCARA robotic arm, the position of the spoon on the end effector is simplified to the center point of the frontal facial image to improve localization and data processing efficiency. Preset facial key point positions are key point positions pre-defined on the face, specifically the tip of the nose, corners of the eyes, corners of the mouth, etc. Preferably, the tip of the nose is used, as most open-source large facial models have nose tip recognition; therefore, using the nose tip as the preset facial key point position is beneficial for subsequent facial coordinate calculation.

[0025] S300: Based on the target distance, center point position, and preset facial key point positions, determine the direction vector corresponding to the feeding task.

[0026] Since the camera and the robotic arm's end effector are located in the same coordinate system, such as Figure 3 As shown, the center point (N) of the camera can be considered as the end effector (the coordinate point of the spoon). By calculating the offset of the facial key points (the coordinate point of the nose tip (M)) from the center point (N) in the image coordinate system, and combining this with the target distance measured by the laser rangefinder, the direction vector corresponding to the feeding task is determined using vector operation rules. This direction vector represents the direction in which the robotic arm's end effector needs to move, solving the tracking problem caused by user facial movement and robotic arm response delay in traditional robotic arm control methods, and providing crucial directional guidance for subsequent precise control of the robotic arm's movement.

[0027] S400: Based on the size parameters and direction vector of the SCARA robotic arm in the meal-assisting robot, calculate the target angular velocity corresponding to the joints in the SCARA robotic arm.

[0028] like Figure 4 As shown, the arm portion of the SCARA robotic arm has two degrees of freedom, plotted in a coordinate system. One joint (the first joint) rotates around point O, and the second joint (the second joint) rotates around point A. The rotation at point O affects the end effector via line BZ perpendicular to OB, and the effect of point A affects the end effector via line BQ perpendicular to AB; the vector sum of these two is BB'. The length of the line connecting OB is L, which is the direction vector corresponding to the error value calculated through points A and B. Based on the dimensional parameters of the SCARA robotic arm, such as the lengths of each joint, this specifically includes the length of the first robotic arm. Second robotic arm length Based on the determined direction vector, and using kinematic principles and vector decomposition methods, the direction vector is decomposed into individual joints. By establishing a mathematical model and applying relevant kinematic formulas, the target angular velocities corresponding to the joints in the SCARA robotic arm are calculated. This step transforms the overall motion direction into specific motion parameters for each joint, providing a basis for controlling the coordinated movement of the robotic arm's joints and ensuring that the end effector moves according to the predetermined velocity vector.

[0029] S500: Controls the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between preset facial key points and the mouth.

[0030] After calculating the tracking and resolution information between the spoon position and the preset facial key points (such as the tip of the nose) by solving the target angular velocity, and considering that actual feeding requires accurately delivering food to the mouth, secondary control is further performed based on the positional relationship between the preset facial key point positions and the mouth. Specifically, the relative positional relationship between the preset facial key point (tip of the nose) and the mouth in the facial coordinate system is first determined. This relative positional relationship can be preset or determined based on a facial feature recognition algorithm. During the control process, the calculated target angular velocity is sent to the drive system of the SCARA robotic arm. Based on the received target angular velocity signal and the relative positional relationship between the preset facial key point and the mouth, the drive system fine-tunes the rotational speed of the motors of each joint, enabling the coordinated movement of each joint of the SCARA robotic arm, driving the end effector to move in a determined direction vector, and ultimately accurately delivering food to the mouth.

[0031] The aforementioned collaborative control method for the meal-assisting robot acquires a frontal image of the face and the target distance between the end effector of the SCARA robotic arm in the robot and the face; determines the center point position and preset facial key point positions in the frontal image of the face; based on the target distance, center point position, and preset facial key point positions, determines the direction vector corresponding to the feeding task; calculates the target angular velocity of the joints in the SCARA robotic arm according to the size parameters and direction vector; and controls the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between the preset facial key points and the mouth. Throughout the process, the direction vector pointing from the target point to the center of the image is first determined based on the face image and distance, solving the problem of accurate tracking and providing reliable guidance for feeding positioning; then, this direction vector is decomposed into each joint of the SCARA robotic arm, and the joint rotation speed is calculated in combination with the size parameters, allowing the end effector to move stably according to the predetermined direction vector, achieving efficient and stable feeding.

[0032] In one embodiment, such as Figure 5As shown, based on the target distance, center point position, and preset facial key point positions, the direction vector for performing the feeding task is determined as follows: S320: Obtain the visual transmission error value based on the center point position and the preset facial key point positions.

[0033] The preset facial key point positions can be the tip of the nose, the corner of the eye, the corner of the mouth, etc. Preferably, the tip of the nose can be selected as the preset facial key point position. The reason for choosing the tip of the nose as the preset facial key point is that most open-source face models have the function of nose tip recognition, which makes this solution widely adaptable and can be adapted to most face recognition models on the market. At the same time, the coordinates of the feeding spoon are calculated by solving the coordinates of the tip of the nose, and choosing the tip of the nose as the key point facilitates subsequent calculation and control. The center point position is usually a reference point determined based on the overall features of the face image in the image coordinate system. This point represents the center position of the face in the image. The visual transmission error value is obtained by calculating the offset between the nose tip coordinate point (M) and the center point position in the image coordinate system. Specifically, the offset of these two points in the Y-axis and Z-axis directions in the image coordinate system can be calculated to obtain the error value of the face in the Y-axis and Z-axis directions. This is because the positional offset of a face in an image is directly reflected in the relative position of the nose tip coordinate point and the center point. By calculating this relative relationship, the error of the face in the corresponding axis can be accurately obtained.

[0034] S340: Determine the direction vector corresponding to the feeding task based on the visual transmission error value and the target distance.

[0035] The target distance can be calculated using a laser rangefinder, representing the straight-line distance between the feeding spoon and the target location (such as the feeding point on a face). Laser rangefinders offer advantages such as high precision and fast response, accurately acquiring target distance information. The X-axis error value is determined by combining the target distance calculated by the laser rangefinder with relevant algorithms. Since the laser rangefinder measures straight-line distance, through certain geometric relationships and algorithmic processing, this distance information can be converted into an error value in the X-axis direction, thus comprehensively acquiring the error situation of the face in three-dimensional space (X, Y, Z axes). Combining the visual transmission error values ​​(Y-axis and Z-axis error values) with the target distance (X-axis error), a specific algorithm calculates and determines the direction vector corresponding to performing the feeding task. This direction vector is a three-dimensional vector containing the direction information of the feeding spoon's movement in space, guiding the robotic arm to accurately move the feeding spoon to the target position.

[0036] In one embodiment, obtaining the visual transmission error value based on the center point position and the preset facial key point positions includes: Step 1: Determine the preset facial key point as the tip of the nose in the middle of the face.

[0037] In this embodiment, the tip of the nose is selected as the preset facial key point after comprehensive consideration. From a technical adaptability perspective, most open-source face models currently have nose tip recognition capabilities, which allows this solution to be widely adapted to various face recognition models on the market without requiring complex adjustments for specific models, thus reducing the difficulty and cost of technical implementation. From the perspective of subsequent calculation and control, the coordinates of the feeding spoon are calculated using the coordinates of the nose tip. Choosing the nose tip as the key point facilitates the establishment of a unified coordinate system, simplifies the calculation process, and improves the stability and reliability of the system.

[0038] Step 2: Obtain the first coordinate of the center point in the image coordinate system and the second coordinate of the tip of the nose in the image coordinate system.

[0039] For details, please refer to [link / reference]. Figure 3 The center point of the camera is considered as the end effector (the coordinate point of the spoon (N), the center point of the image). This center point has specific coordinate values ​​in the image coordinate system, denoted as the first coordinate. Simultaneously, the coordinate values ​​of the nose tip A in the image coordinate system are obtained using facial recognition technology, denoted as the second coordinate. In obtaining these two coordinates, it is necessary to ensure the uniformity and accuracy of the image coordinate system to guarantee the reliability of subsequent calculations. The image coordinate system is typically a two-dimensional coordinate system established with the upper left corner of the image as the origin, the positive X-axis pointing horizontally to the right, and the positive Y-axis pointing vertically downwards.

[0040] Step 3: Calculate the offset between the first and second coordinates to obtain the visual transmission error value.

[0041] Based on the first coordinate of the center point and the second coordinate of the nose tip obtained in step 2, the offset between them is calculated using coordinate operations. Specifically, the difference between the first and second coordinates along the X and Y axes is calculated, and this difference represents the offset in the image coordinate system. This offset reflects the positional deviation of the nose tip relative to the center point in the image. The principle that the error values ​​of the Y and Z axes of the face can be calculated by determining the offsets of points M and N in the image coordinate system makes this offset a crucial component of the visual transmission error.

[0042] In one embodiment, the calculation of the target angular velocity corresponding to the joints in the SCARA robotic arm, based on the size parameters and orientation vector of the SCARA robotic arm in the dining robot, includes: Step 1: Abstract the direction vector into a velocity direction vector using fuzzy PID.

[0043] Fuzzy PID combines fuzzy control with traditional PID control. Traditional PID control adjusts system error through three components: proportional (P), integral (I), and derivative (D). However, when dealing with complex and nonlinear meal-assistance robot systems, parameter tuning is difficult, making it hard to achieve ideal control results. Fuzzy control, on the other hand, can dynamically adjust PID parameters based on the real-time system state using fuzzy rules. In this embodiment, fuzzy PID is used to process the direction vector to meet the motion control requirements of the meal-assistance robot.

[0044] Specifically, see Figure 4 Based on the current characteristics of the direction vector BB', the PID parameters (proportional, integral, and derivative coefficients) are dynamically adjusted using internally preset fuzzy rules. After the fuzzy PID adjusts the parameters, it is equivalent to transforming the original direction vector BB' into a new velocity direction vector. This new vector not only retains the original approximate direction information but also incorporates appropriate velocity attributes based on the system's real-time status (vector magnitude represents rotational speed, and direction represents forward and reverse rotation), enabling more effective guidance of the robotic arm's movement and allowing it to move towards the target (the face position) with a more suitable speed and direction.

[0045] Step 2: Based on the size parameters and velocity direction vector of the SCARA robotic arm in the meal-assisting robot, calculate the target angular velocity corresponding to the joints in the SCARA robotic arm.

[0046] like Figure 4 As shown, the SCARA robotic arm has two degrees of freedom, with one joint rotating around point O and the other around point A. The rotation at point O affects the end effector perpendicular to the line BZ connecting OB, and the influence of point A on the end effector is perpendicular to the line BQ connecting AB. The sum of these two vectors is the direction vector (in this step, the velocity direction vector). Meanwhile, the dimensional parameters of the SCARA robotic arm in the dining robot are fixed, including the length of each link and the position of the joints. These parameters form the basis for kinematic calculations.

[0047] Based on the structural characteristics and dimensional parameters of the SCARA robotic arm, an inverse kinematics algorithm is used to calculate the target angular velocities corresponding to the joints. Inverse kinematics deduces the angles and angular velocities of each joint from the position and velocity information of the robotic arm's end effector. Using the velocity direction vector as the desired velocity input to the end effector, and combining this with the robotic arm's dimensional parameters, a kinematic equation is established. This equation describes the relationship between the end effector velocity and the joint angular velocities. By solving this kinematic equation, the target angular velocities corresponding to one and two joints can be obtained. For example, matrix operations can be used to transform the kinematic equation into matrix form, and the solutions for the joint angular velocities can be obtained through operations such as inverting the matrix.

[0048] In one embodiment, the calculation of the target angular velocity corresponding to the joints in the SCARA robotic arm, based on the size parameters and velocity direction vector of the SCARA robotic arm in the meal-assisting robot, includes: Step 1: Obtain the first rotation angle of the first joint and the second rotation angle of the second joint in the current SCARA robotic arm.

[0049] The device's motors read the rotation angle information of each joint in real time, thereby obtaining the first rotation angle of the first joint and the second rotation angle of the second joint in the SCARA robotic arm at the current moment. The device's motors have angle reading capabilities, which can accurately feed back the rotation angle data of the joints, providing the basic data for subsequent calculations. These two rotation angles are the basis for the key parameters (α, β) in the subsequent calculation of the joint angular velocity equations. Only by accurately obtaining the current rotation angle of the joint can the position and attitude information of the robotic arm's end effector be further determined, and then the target angular velocity of the joint can be solved.

[0050] Step 2: Based on the size parameters, extract the lengths of the first and second robotic arms in the SCARA robotic arm.

[0051] During the design phase of the meal-assisting robot, the dimensional parameters of the SCARA robotic arm were precisely determined and stored in the robot's control system. These dimensional parameters include the length of the first robotic arm. Second robotic arm length The control system can extract these two key length parameters from stored data based on preset identifiers or instructions. Specifically, the SCARA robotic arm consists of a first robotic arm, a second robotic arm, and an end effector, with clearly defined and fixed relative positions between each part. The first robotic arm, serving as the basic support for the entire robotic arm, is fixedly connected at one end to the robot's base, providing stable support and an initial motion reference for the entire robotic arm. The second robotic arm is connected to the first robotic arm via a rotary joint, which enables the second robotic arm to rotate relative to the first robotic arm in the horizontal plane. The other end of the second robotic arm, the end furthest from the first robotic arm, is connected to the end effector. The end effector is the part of the SCARA robotic arm that directly contacts the object being manipulated (such as tableware, food, etc.), and it is installed at the end of the second robotic arm furthest from the first robotic arm. Through the coordinated movement of the first and second robotic arms, the end effector can achieve flexible positioning and operation in three-dimensional space, thereby completing various tasks during the meal assistance process, such as grasping tableware, picking up food, and accurately delivering food to the user's mouth.

[0052] Step 3: Based on the first rotation angle, the second rotation angle, the first robotic arm length, the second robotic arm length, and the direction vector, generate a set of equations for solving the joint angular velocity.

[0053] See also Figure 4 Let the angular velocity of joint o (corresponding to the first joint) be... The angular velocity of joint A (corresponding to the second joint) is Calculate the velocity vector. And by taking the velocity vector at point B out separately and interpreting it, we get the following formula:

[0054] The end effector reference point B is considered the origin, and the target point B' has coordinates (X, Y). Here, X represents the visual transmission error value, which is acquired by the vision system and transmitted to the control system; Y represents the distance between the end effector and the target, which can be acquired by a high-precision laser rangefinder and fed back to the control system. Since the device's motors can read the rotation angles of each joint in real time, the following can be calculated using appropriate trigonometric functions: Figure 3 middle , The value of . and These are related to the rotation angles of the second and first joints, as well as the geometry of the robotic arm.

[0055] Based on the concept of vector addition, equations are established in the x-axis and y-axis directions respectively. In the x-axis direction, the velocity component of the second robotic arm's end effector in the X-axis direction is... The velocity component of the first robotic arm's end effector in the X-axis direction The sum equals the coordinate value X of the target point B' in the x-axis direction, that is... The velocity component of the second robotic arm's end effector in the y-axis direction. The velocity component of the first robotic arm's end effector in the y-axis direction The sum equals the y-coordinate value of the target point B' in the y-axis direction, that is... Therefore, the following set of equations for solving the joint angular velocity is obtained:

[0056] In the formula, The length of the first robotic arm. The length of the second robotic arm. For the first rotation angle, For the second rotation angle, The first angular velocity, The second angular velocity, and These are the first direction offset value and the second direction distance value corresponding to the direction vector, respectively.

[0057] Step 4: Solve the system of equations for joint angular velocity to obtain the first angular velocity of the first joint and the second angular velocity of the second joint.

[0058] In the system of equations , , , Since X and Y are all known quantities, the solution can be obtained. , This refers to the rotational speed of each joint. The control system uses mathematical algorithms (such as methods for solving linear equations) to solve this system of equations. By substituting known parameter values, the result is obtained through calculation. and The value is the first angular velocity of the first joint and the second angular velocity of the second joint.

[0059] In practical applications, the collaborative control of the meal-assisting robot proposed in this application is applied to the meal-assisting robot, and its overall latency is reduced to less than 0.2s. Compared with traditional methods, it significantly improves the stability and real-time performance of dynamic tracking, with a real-time performance improvement of about 90%.

[0060] Furthermore, in practical applications, to evaluate the success rate and feeding efficiency of the device, it can be done through methods such as... Figure 6 The experiment involved 30 scooping, tracking, and feeding tests on three types of food (rice (solid granular food), rice porridge (semi-liquid food), and soup (fluid food)). The experiment was conducted in a standard indoor environment simulating a home setting. Ordinary lighting and a temperature of 25±2℃ were used to test the scooping of each food. The criterion for a successful experiment was that the food delivered to the user's mouth by the device must be intact and not spilled outside the bowl.

[0061] In practical applications, different feeding speeds are adopted when controlling the movement of the SCARA robotic arm to meet the different feeding needs of different food types, i.e., different feeding parameters are corresponding to them. Specifically, three feeding parameter modes can be set, including low-speed safety mode, standard mode, and efficiency mode. The low-speed safety mode is suitable for porridge, liquid food, and soft food. The speed of the robotic arm's distal spoon can be set to 20 mm / s, which decreases to 6 mm / s in the last 30 mm of the food approaching the mouth, and the stable pause time after reaching the mouth is set to 700 ms. The standard mode is suitable for rice and ordinary soft food. The speed of the robotic arm's distal spoon can be set to 30 mm / s, which decreases to 10 mm / s in the last 25 mm of the food approaching the mouth, and the pause time after reaching the mouth is set to 400 ms. The efficiency mode is suitable for more formed soft solid food. The speed of the robotic arm's distal spoon can be set to 40 mm / s, which decreases to 15 mm / s in the last 20 mm of the food approaching the mouth, and the pause time after reaching the mouth is set to 250 ms.

[0062] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps. Based on the same inventive concept, this application also provides a collaborative control device for a dining robot to implement the aforementioned collaborative control method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the collaborative control device for dining robots provided below can be found in the limitations of the collaborative control method for dining robots described above, and will not be repeated here.

[0063] In one embodiment, such as Figure 7 As shown, this application also provides a collaborative control device for a meal-assisting robot. The device includes: The data acquisition module 100 is used to acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face. The point determination module 200 is used to determine the position of the center point and the position of preset facial key points in a frontal image of a face. The vector calculation module 300 is used to determine the direction vector corresponding to the feeding task based on the target distance, center point position and preset facial key point position; The angular velocity calculation module 400 is used to calculate the target angular velocity of the joints in the SCARA robotic arm based on the size parameters and direction vector of the SCARA robotic arm in the dining robot. The control module 500 is used to control the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between preset facial key points and the mouth.

[0064] In one embodiment, the vector calculation module 300 is further configured to obtain a visual transmission error value based on the center point position and the preset facial key point position; and determine the direction vector corresponding to the feeding task based on the visual transmission error value and the target distance.

[0065] In one embodiment, the vector calculation module 300 is further configured to determine the preset facial key point as the tip of the nose in the face; obtain the first coordinate of the center point position in the image coordinate system and the second coordinate of the tip of the nose in the image coordinate system; calculate the offset value between the first coordinate and the second coordinate to obtain the visual transmission error value.

[0066] In one embodiment, the angular velocity solving module 400 is further used to abstract the direction vector into a velocity direction vector using fuzzy PID; and to calculate the target angular velocity corresponding to the joint in the SCARA robotic arm based on the size parameters of the SCARA robotic arm and the velocity direction vector in the dining robot.

[0067] In one embodiment, the angular velocity solving module 400 is further configured to obtain the first rotation angle of the first joint and the second rotation angle of the second joint in the current SCARA robotic arm; extract the length of the first robotic arm and the length of the second robotic arm in the SCARA robotic arm according to the size parameters; generate a set of joint angular velocity solving equations based on the first rotation angle, the second rotation angle, the length of the first robotic arm, the length of the second robotic arm and the direction vector; solve the set of joint angular velocity solving equations to obtain the first angular velocity of the first joint and the second angular velocity of the second joint.

[0068] In one embodiment, the set of equations for calculating the joint angular velocity is as follows:

[0069] In the formula, The length of the first robotic arm. The length of the second robotic arm. For the first rotation angle, For the second rotation angle, The first angular velocity, The second angular velocity, and These are the first direction offset value and the second direction distance value corresponding to the direction vector, respectively.

[0070] The modules in the aforementioned collaborative control device for the meal-assisting robot can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0071] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a collaborative control method for a meal-assisting robot. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0072] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described collaborative control method for the meal-assisting robot.

[0073] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described collaborative control method for the meal-assisting robot.

[0074] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described collaborative control method for a meal-assisting robot.

[0075] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0076] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A collaborative control method for a meal-assisting robot, characterized in that, The method includes: Acquire a frontal image of a human face and the target distance between the end effector of the SCARA robotic arm in the meal-assisting robot and the human face; Determine the position of the center point and the positions of preset facial key points in the frontal image of the face; Based on the target distance, the center point position, and the preset facial key point positions, determine the direction vector corresponding to the feeding task; Based on the size parameters of the SCARA robotic arm in the meal-assisting robot and the direction vector, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated. The movement of the SCARA robotic arm is controlled based on the target angular velocity and the relative positional relationship between preset facial key points and the mouth.

2. The method according to claim 1, characterized in that, The determination of the direction vector corresponding to the feeding task based on the target distance, the center point position, and the preset facial key point positions includes: Based on the center point position and the preset facial key point position, obtain the visual transmission error value; Based on the visual transmission error value and the target distance, the direction vector corresponding to the feeding task is determined.

3. The method according to claim 2, characterized in that, The step of obtaining the visual transmission error value based on the center point position and the preset facial key point positions includes: The key facial point is determined as the tip of the nose in the middle of the face; Obtain the first coordinates of the center point in the image coordinate system and the second coordinates of the tip of the nose in the image coordinate system; The offset between the first coordinate and the second coordinate is calculated to obtain the visual transmission error value.

4. The method according to claim 1, characterized in that, The step of calculating the target angular velocity corresponding to the joints in the SCARA robotic arm based on the size parameters of the SCARA robotic arm and the direction vector includes: The direction vector is abstracted into a velocity direction vector using fuzzy PID; Based on the size parameters of the SCARA robotic arm in the meal-assisting robot and the velocity direction vector, the target angular velocity corresponding to the joint in the SCARA robotic arm is calculated.

5. The method according to claim 4, characterized in that, The step of calculating the target angular velocity corresponding to the joints in the SCARA robotic arm based on the size parameters of the SCARA robotic arm and the velocity direction vector includes: Obtain the first rotation angle of the first joint and the second rotation angle of the second joint in the current SCARA robotic arm; Based on the aforementioned size parameters, the lengths of the first and second robotic arms in the SCARA robotic arm are extracted. Based on the first rotation angle, the second rotation angle, the first robotic arm length, the second robotic arm length, and the direction vector, a set of joint angular velocity calculation equations is generated; Solve the set of equations for calculating the joint angular velocity to obtain the first angular velocity of the first joint and the second angular velocity of the second joint.

6. The method according to claim 5, characterized in that, The set of equations for solving the joint angular velocity is as follows: ， In the formula, The length of the first robotic arm. The length of the second robotic arm. For the first rotation angle, For the second rotation angle, The first angular velocity, The second angular velocity, and These are the first direction offset value and the second direction distance value corresponding to the direction vector, respectively.

7. A collaborative control device for a meal-assisting robot, characterized in that, The device includes: The data acquisition module is used to acquire frontal images of the human face and the target distance between the SCARA robotic arm end effector and the human face in the meal-assisting robot; The point determination module is used to determine the position of the center point and the position of preset facial key points in the frontal image of the face; The vector calculation module is used to determine the direction vector corresponding to the feeding task based on the target distance, the center point position, and the preset facial key point position. The angular velocity calculation module is used to calculate the target angular velocity corresponding to the joints in the SCARA robotic arm based on the size parameters of the SCARA robotic arm in the meal-assisting robot and the direction vector. The control module is used to control the movement of the SCARA robotic arm based on the target angular velocity and the relative positional relationship between preset facial key points and the mouth.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images