A control method and system for a ball game robot

Abstract

The present disclosure provides a control method and system of a ball game robot, and relates to the technical field of motion robot control, which acquires badminton court environment image data and off-court motion image data; based on the environment image data and off-court motion image data, a deep learning model is used to identify the positions of the badminton, the robot and the motion personnel; according to the position information of the badminton, the robot and the motion personnel, a tracking algorithm is used to realize the reconstruction of the badminton flight trajectory, and the badminton hitting point and the ball landing point are predicted; the motion state of each mechanism of the robot and the body IMU data are acquired, and the robot is predicted in posture and motion state based on the motion state of each mechanism and the body IMU data; based on the predicted robot posture, motion state, badminton hitting point and ball landing point, the robot dynamic balance control algorithm and the MPC control algorithm are used to realize the balance control and the hitting action control of the robot itself respectively.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of sports robot control, in particular to a control method and system of a ball sports robot. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In the field of badminton sports, with the rapid development of artificial intelligence and robot technology, the research and development of badminton sports robots has gradually become a hot spot. These robots aim to simulate real game environments, provide personalized training or exercise, and improve the technical level of sports personnel and increase the exercise and entertainment methods of badminton enthusiasts. However, badminton sports require sports personnel to have fast reactions, stable movements, and precise hitting skills. In training, the improvement of these skills requires a large amount of repeated practice and targeted feedback.

[0004] Traditional training methods often lack personalized guidance, and the frequent ball landing and restarting of the training process during training consumes a lot of time, thereby reducing training efficiency. The existing control of badminton robots has limitations in simulating real game environments and providing personalized training, which limits their application in efficient training. The specific limitations are as follows:

[0005] (1) Structural design: The structure of the current badminton sports robot is complex, requiring high precision, which increases the cost and lacks obstacle avoidance ability.

[0006] (2) Movement speed and stability: Traditional badminton sports robots have deficiencies in movement speed and stability, and cannot effectively simulate the fast movement and stable hitting of humans.

[0007] (3) Control algorithm: Existing control algorithms such as dynamic equation fitting, BP neural network, and PID control algorithm have limitations in precision and stability, are greatly affected by the environment and the shape and specifications of the shuttlecock, and are sensitive to system models, making parameter selection difficult.

[0008] (4) Hitting accuracy: Existing robots are prone to problems of unstable center of gravity during hitting, affecting the stability of the robot and the accuracy of the hitting. SUMMARY

[0009] The control method and system of the ball game robot are proposed to solve the above problems, a double-wheel foot and single-arm with racket structure are designed, the trajectory of the badminton is fitted and predicted through deep learning, and the posture of the robot is predicted, the tracking of the badminton game and the real-time accurate positioning of the position in the field are realized, the robot is controlled to realize the rapid decision and accurate completion of various hitting actions, and the response speed and control precision of the robot are improved.

[0010] According to some embodiments, the technical scheme is adopted in the present disclosure as follows:

[0011] A control method of a ball game robot comprises:

[0012] Obtaining the image data of the badminton court environment and the image data of the off-court movement;

[0013] Based on the image data of the environment and the image data of the off-court movement, the positions of the badminton, the robot and the sports personnel are identified by using a deep learning model;

[0014] According to the position information of the badminton, the robot and the sports personnel, the tracking algorithm is used to realize the reconstruction of the flight trajectory of the badminton, and the hitting point and the falling point of the badminton are predicted;

[0015] Obtaining the motion state of each mechanism of the robot and the body IMU data, and predicting the posture and motion state of the robot based on the motion state of each mechanism and the body IMU data;

[0016] Based on the predicted posture and motion state of the robot, the hitting point and the falling point of the badminton, the dynamic balance control algorithm and the MPC control algorithm are used to realize the balance control and the hitting action control of the robot itself respectively;

[0017] Among them, the dynamic motion model is constructed based on the posture and motion state of the robot, the optimal joint angle and driving wheel speed are calculated by using the MPC control algorithm based on the dynamic model and the predicted hitting point and falling point data of the badminton, and the robot is driven to complete various hitting actions according to the obtained joint angle and driving wheel speed.

[0018] According to some embodiments, the technical scheme is adopted in the present disclosure as follows:

[0019] A control system of a ball game robot comprises:

[0020] The data acquisition module is used for acquiring the image data of the badminton court environment and the image data of the off-court movement;

[0021] A badminton trajectory prediction module is configured to identify the positions of the badminton, the robot and the sports personnel based on environmental image data and off-court sports image data by using a deep learning model; and to reconstruct the flight trajectory of the badminton by using a tracking algorithm based on the position information of the badminton, the robot and the sports personnel, and to predict the hitting point and the landing point of the badminton.

[0022] A robot motion posture prediction module is configured to acquire the motion states of each mechanism of the robot and the body IMU data, and to predict the posture and the motion state of the robot based on the motion states of each mechanism and the body IMU data.

[0023] A control module is configured to implement the balance control and the hitting action control of the robot itself by using a robot dynamic balance control algorithm and an MPC control algorithm, respectively, based on the predicted robot posture, motion state, hitting point and landing point of the badminton.

[0024] In this way, a dynamic motion model is constructed based on the robot posture and motion state, the optimal joint angle and drive wheel speed are calculated by using the MPC control algorithm based on the dynamic model and the predicted hitting point and landing point of the badminton, and the robot is driven to complete various hitting actions according to the obtained joint angle and drive wheel speed.

[0025] According to some embodiments, the present disclosure adopts the following technical solutions:

[0026] A ball game robot, which is designed as a double-wheel foot, a single-arm with a racket, and a single arm is a mechanical arm, a racket is arranged at the head of the mechanical arm, the joints of the mechanical arm are four degrees of freedom, each joint is driven by a motor, including an end motor, a third motor, a second motor and a first motor, and each joint is equipped with a sensor.

[0027] The double-wheel foot is a master-slave wheel driving structure, including a standing motor, a carbon fiber square tube and a large wheel driving motor assembly, the double-wheel foot and the mechanical arm are installed on a main body of the robot, a battery compartment and a battery swing motor are arranged on a vertical shaft below the main body of the robot, and the battery compartment and the battery swing motor can rotate around the vertical shaft

[0028] According to some embodiments, the present disclosure adopts the following technical solutions:

[0029] A non-transitory computer readable storage medium is configured to store computer instructions, which are executed by a processor to implement the control method of the ball game robot.

[0030] According to some embodiments, the present disclosure adopts the following technical solutions:

[0031] An electronic device comprises a processor, a memory and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the control method of the ball game robot.

[0032] Compared with the prior art, the beneficial effects of the present disclosure are:

[0033] The control method of the ball game robot of the present disclosure realizes efficient movement and accurate ball hitting on the badminton court through the innovative double-wheel foot design and single-arm with racket structure. The battery compartment serves as a counterweight, enhancing automatic balance control. In addition, the robot adopts multiple visual perception modes, such as eyes on the body and eyes outside the body, to improve tracking and response capabilities for badminton movements. The invention also relates to a method of controlling the robot, including using edge computing processors to achieve real-time communication for fast decision-making and precise actions. This real-time communication architecture enables fast data transmission and processing, improving the response speed and control accuracy of the robot. Through these improvements, the badminton robot can better serve sports training, entertainment and exercise, meet the needs of different users, overcome the limitations of existing technologies, and provide a new way of badminton training and entertainment.

[0034] The control method of the ball game robot of the present disclosure adopts edge computing nodes, integrating image recognition algorithms, badminton trajectory fitting and prediction, hitting point and landing point prediction, and robot posture prediction. Dynamic balance control is realized on the body controller, and through the MPC (Model Predictive Control) algorithm, the robot's drive wheels and joint actuators can complete various hitting actions. The badminton robot of the present disclosure exhibits flexibility, lightness, stability, convenience and high efficiency, enabling fast movement, jumping and performing various high-difficulty hitting actions, significantly improving overall performance, adaptability and safety.

[0035] The control method of the ball game robot of the present disclosure, the robot dynamic balance control algorithm can effectively adjust the battery compartment position, optimize the center of gravity distribution, thereby improving the stability and maneuverability of the robot, so that it can perform tasks accurately and safely in complex environments. This control method not only improves the performance of the robot, but also enhances its adaptability and safety in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which form a part of this disclosure, are included to provide a further understanding of the present disclosure, the illustrative embodiments of the present disclosure, and their description are used to explain the present disclosure, and do not constitute an improper limitation on the present disclosure.

[0037] Figure 1A control flow diagram of the badminton robot according to an embodiment of the present disclosure;

[0038] Figure 2 A squatting diagram of the badminton robot according to an embodiment of the present disclosure;

[0039] Figure 3 A standing diagram of the badminton robot according to an embodiment of the present disclosure;

[0040] Figure 4 A badminton robot with racket mechanical arm according to an embodiment of the present disclosure;

[0041] Figure 5 A single-sided wheel-foot squatting diagram of the badminton robot according to an embodiment of the present disclosure;

[0042] Figure 6 A single-sided wheel-foot standing diagram of the badminton robot according to an embodiment of the present disclosure;

[0043] Figure 7 A battery compartment dynamic balance control different position of the badminton robot according to an embodiment of the present disclosure;

[0044] Figure 8 A coordinate conversion diagram of the court binocular stereo vision detection according to an embodiment of the present disclosure;

[0045] Figure 9 A detection and calibration diagram of the badminton court according to an embodiment of the present disclosure.

[0046] Wherein, 1, carbon fiber tube, 2, racket, 3, carbon fiber racket rod, 4, carbon fiber support, 5, end motor, 6, No. 3 motor 6, 7, first carbon fiber connecting plate, 8, No. 2 motor, 9, second carbon fiber connecting plate, 10, conductive slip ring, 11, carbon fiber tube, 12, No. 1 motor, 13, aluminum alloy pull rod, 14, standing motor, 15, first carbon fiber square tube, 16, second carbon fiber square tube, 17, sensor, 18, large wheel drive motor assembly, 19, large wheel, 20, body, 21, battery swing motor, 22, battery compartment. DETAILED DESCRIPTION

[0047] The present disclosure will be further described below in conjunction with the drawings and embodiments.

[0048] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0049] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components and / or combinations thereof, but do not preclude the presence or addition of one or more other features, steps, operations, devices, components, and / or combinations thereof.

[0050] Embodiment 1

[0051] In one embodiment of the present disclosure, a ball game robot is provided, as shown in Figure 2 The ball game robot is designed as a double-wheel foot, single-arm racket, as shown in Figure 3 The single arm is a mechanical arm, and a racket 2 is arranged at the head of the mechanical arm. The joints of the mechanical arm are four degrees of freedom, each joint is driven by a motor, including an end motor 5, a third motor 6, a second motor 8 and a first motor 12. Each joint is equipped with a sensor.

[0052] Further, the racket 2 is composed of a racket and a carbon fiber racket rod 3. The carbon fiber racket rod 3 is connected with the mechanical arm through a carbon fiber support 4. The end motor 5 is installed at the bottom of the carbon fiber support 4. A third motor is installed on one side of the carbon fiber support 4. The third motor 6 is connected with one end of a first carbon fiber connecting plate 7. The other end of the first carbon fiber connecting plate 7 is connected with a second carbon fiber connecting plate 9 through one end of the second carbon fiber connecting plate 9. The other end of the second carbon fiber connecting plate 9 is connected with the top of a carbon fiber tube 11. A conductive slip ring 10 is arranged on the carbon fiber tube 1. The first motor is arranged at the bottom of the carbon fiber tube 1.

[0053] As shown in Figure 5 The carbon fiber tube 1 is externally provided with a body 20. Double-wheel foot structures are installed on both sides of the body 20. A battery swing motor 21 is arranged below the body 20, and a battery compartment 22 is arranged on the vertical shaft below. The battery compartment is electrically connected with the battery swing motor 21.

[0054] Further, the main structure of the mechanical arm is made of carbon fiber material, which has high strength and low density characteristics, helping to reduce the weight of the mechanical arm while maintaining the required strength and rigidity. In view of the requirements of various types of badminton hitting, and as far as possible with less joints to achieve, therefore the mechanical arm adopts 4 degrees of freedom joints to meet the needs, each joint is driven by a motor, allowing the mechanical arm to move and position in multiple directions. Joint and drive system: the end motor is responsible for the fine motion of the end effector (such as racket) of the mechanical arm, realizing the hitting action. The third motor as a main joint of the mechanical arm, is responsible for the motion in the vertical plane, such as up and down swing. The second motor is responsible for the horizontal motion of the mechanical arm, such as left and right swing, providing direction control for hitting. The first motor as the base joint of the mechanical arm, is responsible for the overall rotation motion, enabling the mechanical arm to face different directions. Each joint is equipped with sensors for accurate measurement of the angle and speed of the joint, providing feedback to the control system to ensure accurate control.

[0055] The battery compartment serves as a dynamic counterweight, optimizing the center of gravity distribution of the robot by adjusting its front and rear swing position, achieving automatic balance control. This dynamic balance control is a significant innovation over traditional static counterweight design, enabling the robot to maintain stability in various motion states. The ball sport robot of the present invention adopts advanced control strategy, integrating double-wheel foot balance control and motion control, through IMU sensor real-time monitoring of the robot's posture, combined with dynamic adjustment of the battery compartment, achieving stability and accuracy in fast movement. Through the IMU sensor real-time monitoring of the robot's posture, combined with the dynamic adjustment of the battery compartment, the robot can maintain balance in various motion states. This control strategy is achieved by dynamically adjusting the position and angle of the battery compartment, directly affecting the center of gravity distribution of the robot, thus having an important influence on its stability and maneuverability. At the same time, in view of the rapid response requirement of badminton sports, the robot calculates the shortest path to reach the target point, and uses the flexibility of double-wheel foot to fine-tune, through iterative algorithm to dynamically adjust the joint parameters of the mechanical arm, ensuring the accuracy and adaptability of hitting.

[0056] As shown in Figure 4 , the double-wheel foot is a master-slave wheel drive structure, including a standing motor 14, a carbon fiber square tube, and a large wheel drive motor assembly. The double-wheel foot and the mechanical arm are installed on both sides of the body main body, and the battery compartment and the battery swing motor are arranged on the vertical shaft below the body main body, which can rotate around the vertical shaft.

[0057] The double-wheel foot structure can achieve both squatting and standing postures. In the wheel-foot structure, the bottom is provided with a large wheel 19, the large wheel 19 is connected to the second carbon fiber square tube 16, a sensor 17 and a large wheel driving motor assembly 18 are arranged on the second carbon fiber square tube 16, the top of the second carbon fiber square tube 16 is connected to the first carbon fiber square tube 15 and the aluminum alloy pull rod 13 respectively, and the standing motor 14 is installed at the head of the first carbon fiber square tube 15.

[0058] The aluminum alloy pull rod 13 and the first carbon fiber square tube 15 can rotate around the second carbon fiber square tube 16.

[0059] Further, precise speed and position control is achieved through the master-slave wheel driving structure, improving the flexibility and stability of the robot. The standing motor is used to adjust the overall height of the robot, realizing the standing and squatting actions to adapt to different ball hitting heights. The carbon fiber square tube serves as the support structure of the mechanical arm, providing the necessary strength and stability while maintaining lightweight. The large wheel driving motor assembly is responsible for driving the large wheel of the robot, realizing the movement and jumping action. Through precise control of the standing motor and the large wheel driving motor, the robot can realize the jumping action to complete the high-difficulty ball hitting.

[0060] The working principle of the battery compartment counterweight dynamic balance control is as follows:

[0061] The dynamic balance control of the double-wheel soccer robot is achieved by dynamically adjusting the position and angle of the battery compartment. This adjustment directly affects the center of gravity distribution of the robot, thus having an important influence on its stability and maneuverability. The design of the battery compartment allows its front and rear shells to rotate around a vertical axis within a certain angle range, so that the battery compartment can act as a dynamic counterweight, optimizing the center of gravity distribution of the robot through its position change, including:

[0062] 1) Battery compartment angle adjustment:

[0063] The design of the battery compartment allows it to rotate forward and backward around a vertical axis. This rotation changes the vertical position of the battery compartment's center of mass, thereby adjusting the robot's center of gravity height and position.

[0064] 2) Center of gravity position calculation:

[0065] The center of gravity of the robot is the weighted average of the center of mass positions of all components. The rotation of the battery compartment changes its center of mass position, thereby affecting the center of gravity position of the entire robot.

[0066] 3) Balance control strategy:

[0067] The robot is equipped with a gyroscope sensor to monitor its attitude changes in real time. The control system analyzes these data and calculates the adjustments needed to maintain or achieve balance. Then, the control system drives the motor to adjust the position of the battery compartment to the calculated optimal angle.

[0068] Further, the calculation process of the battery compartment counterweight dynamic balance control is as follows:

[0069] Assumed conditions: 1) Battery compartment mass mb

[0070] 2) The height of the center of gravity of the robot without the battery compartment hr

[0071] 3) The total mass of the robot mr

[0072] 4) The vertical distance from the center of mass of the battery compartment to the rotation axis d

[0073] 5) The rotation angle of the battery compartment θ.

[0074] Step 1. The effect of battery compartment rotation on the height of the center of gravity:

[0075] When the battery compartment rotates around the vertical axis by an angle θ, θ its center of mass in the vertical direction will change, and this change can be calculated by a trigonometric function:

[0076] (θ)Δ h = d ⋅sin( θ )

[0077] where Δ h represents the effect of battery compartment rotation on the height of the center of gravity of the robot.

[0078] Step 2. Calculation of the new center of gravity position:

[0079] Using the effect of battery compartment rotation on the height of the center of gravity, the new position of the robot is calculated hcg ′:

[0080] hcg ′= hcg +Δ h

[0081] where h hcg is the original center of gravity position of the robot without the battery compartment.

[0082] Step 3. Moment calculation:

[0083] Calculate the moment generated by the battery compartment swing M ( θ ):

[0084] M ( θ )=mb ⋅ d ⋅sin( θ )

[0085] Where torque is the product of the battery compartment mass, the distance from the center of mass to the rotation axis, and the sine of the rotation angle.

[0086] Step 4. Stability Considerations: To ensure the stability of the robot, the swing speed and acceleration of the battery compartment need to be limited. This can be achieved by controlling the size of the torque, ensuring that it does not exceed the threshold for stable operation of the robot.

[0087] Step 5. Energy Consumption: The swing of the battery compartment will consume energy, which needs to be considered in the design. Energy consumption can be estimated by the following formula:

[0088] E =∫ M ( θ )⋅ dt

[0089] Where ω is the angular velocity of the battery compartment rotation.

[0090] Step 6. Application of Closed-Loop Feedback Control:

[0091] System Monitoring: Gyroscopes and accelerometers monitor the tilt angle and center of gravity position of the robot.

[0092] Error Calculation: Calculate the error between the desired center of gravity position and the actual center of gravity position.

[0093] Control Algorithm: Apply a PID control algorithm to adjust the rotation angle of the battery compartment based on the error θ .

[0094] Dynamic Adjustment: Monitor the robot's posture in real-time and dynamically adjust the position of the battery compartment based on real-time data to maintain the balance of the robot.

[0095] Assuming the robot detects a backward tilt during fast movement, the closed-loop feedback control process is as follows:

[0096] 1) The gyroscope detects the backward tilt and feeds back to the control system.

[0097] 2) Error Calculation: The control system calculates the error between the backward tilt angle and the horizontal state.

[0098] 3) Control Algorithm: The PID controller adjusts the amount based on the error calculation and decides the angle that the battery compartment needs to move forward θ .

[0099] Execute Adjustment: The motor adjusts the battery compartment to the new angle, reducing the backward tilt and restoring balance. ​

[0100] Stability check: During adjustment, torque and energy consumption are continuously monitored to ensure that the adjustment is within a safe range.

[0101] Dynamic adjustment: If the recline continues or changes, the control system continues to adjust the battery compartment position based on real-time data.

[0102] Example 2

[0103] In one embodiment of the present disclosure, a control method for a ball game robot is provided, comprising:

[0104] Step 1: Obtain the image data of the badminton court environment and the motion image data outside the court;

[0105] Step 2: Based on the environmental image data and the motion image data outside the court, use a deep learning model to identify the positions of the badminton, robot, and sports personnel;

[0106] Step 3: According to the position information of the badminton, robot, and sports personnel, use a tracking algorithm to reconstruct the flight trajectory of the badminton and predict the hitting point and landing point of the badminton;

[0107] Step 4: Obtain the motion state of each mechanism of the robot and the body IMU data, and based on the motion state of each mechanism and the body IMU data, predict the attitude and motion state of the robot;

[0108] Step 5: Based on the predicted robot attitude, motion state, badminton hitting point, and landing point, use the robot dynamic balance control algorithm and MPC control algorithm respectively to realize the balance control and hitting action control of the robot itself;

[0109] Wherein, a dynamic motion model is constructed based on the robot attitude and motion state, and based on the dynamic model and the predicted badminton hitting point and landing point data, the MPC control algorithm is used to calculate the optimal joint angle and drive wheel speed, and the robot is driven to complete various hitting actions according to the obtained joint angle and drive wheel speed.

[0110] As an embodiment, the control method for a ball game robot of the present disclosure combines the visual perception of "eye on the body mode" and "eye outside the body mode", uses high frame rate and high resolution cameras and binocular stereo vision system, realizes accurate capture and real-time tracking and motion prediction of badminton three-dimensional motion, and optimizes the motion path and hitting strategy of the robot in different areas of the court. The combination of close-range and long-range visual data provides the robot with comprehensive badminton tracking capability.

[0111] The eye-in-hand mode is that the robot is equipped with a binocular camera, which can capture the close-range flight trajectory of the shuttlecock and realize the obstacle avoidance function in the visual detection range.

[0112] Further, the eye-in-hand mode is that the robot is equipped with a binocular camera, which can capture the close-range flight trajectory of the shuttlecock and realize the obstacle avoidance function in the visual detection range.

[0113] Further, the eye-in-hand mode is that the robot is equipped with a binocular camera, which can capture the close-range flight trajectory of the shuttlecock and realize the obstacle avoidance function in the visual detection range.

[0114] Specifically, first, the two cameras at each end of the badminton court are calibrated to determine the intrinsic parameters (focal length, distortion coefficient, etc.) and extrinsic parameters (relative position and angle between the two cameras). Common calibration methods include chessboard calibration.

[0115] The images captured by the left and right cameras are corrected so that their parallax lines are horizontally aligned. This step is completed through the stereoscopic correction of the images to eliminate the deviation during the installation of the binocular camera.

[0116] In combination with the current mainstream target detection deep learning model YOLO, the badminton court environment image data and the motion image data collected by the external camera outside the court are used for detection of the badminton, the robot, and the sports personnel. The detection of the robot is performed through aruco detection.

[0117] The existing YOLO model is used for recognition and detection of the badminton, the robot, and the sports personnel to obtain feature points, which are matched. For the matched feature points, the parallax in the left and right images is calculated, the depth of the target is calculated through the parallax and the camera parameters, and the three-dimensional coordinates (X, Y, Z) of the target points are converted into the position in the actual space according to the depth and the coordinate system of the left and right cameras.

[0118] Specifically, for the matched feature points, the position difference (parallax) in the left and right images is calculated, which is usually represented by a parallax map to represent the depth relationship between the target and the camera. The smaller the parallax, the farther the target; the larger the parallax, the closer the target. Through the parallax and the camera parameters (such as focal length and baseline distance, i.e. the distance between the two cameras), the depth (distance) of the target can be calculated using the formula:

[0119]

[0120] Finally, the three-dimensional coordinates (X, Y, Z) of the target point are converted into the actual space position according to the coordinate system of the depth and left and right cameras.

[0121] In addition, based on the perspective transformation and geometric model detection, the positions of the four corner points of the field are obtained, the perspective transformation matrix is calculated, the court is mapped to the front view, and the position of the standard court model is calibrated.

[0122] The above-mentioned position information recognition of badminton, robot and sports personnel is realized through the existing YOLO model, and then the tracking algorithm (Kalman filter) is used for real-time trajectory tracking, and finally the falling point of the badminton and the hitting point and the position information data of the robot are output.

[0123] Further, the present disclosure constructs a low-latency, high-reliability, high-security real-time communication architecture, which utilizes 5G network and low-latency wireless protocol technologies to realize fast information exchange between the robot controller and the perception and execution modules. Advanced edge computing and control algorithms are used to achieve efficient movement and accurate hitting on the badminton court. Edge computing node 1 is responsible for processing visual data, and through the information of the robot body camera and the off-court camera, it realizes accurate tracking and trajectory prediction of the badminton. Edge computing node 2 processes the motion state data of the robot, including speed, position and IMU data, to predict the attitude of the robot and perform dynamic balance control. In the badminton robot of the present disclosure, the deployment of edge computing nodes is the key to efficient data processing and real-time control. Edge computing nodes 1 and 2 undertake different processing tasks to optimize the performance and response speed of the robot.

[0124] As an embodiment, the specific process of the control method of the ball game robot of the present disclosure is as follows:

[0125] (1) System initialization

[0126] Start the system: start the robot system, including the robot body camera, the off-court camera, the speed and position sensors of each mechanism, the body IMU (inertial measurement unit) and the battery compartment position sensor.

[0127] Self-checking program: execute the self-checking program to ensure that all hardware devices are working properly, including communication links, sensor data streams and actuator states.

[0128] (2) Data acquisition

[0129] Robot body camera data acquisition: acquire environmental information around the robot through the robot body camera for badminton trajectory prediction.

[0130] Off-site camera data collection: Collect data of badminton, sports robots and sports personnel through off-site cameras, providing long-distance visual perception.

[0131] Robot mechanism speed and position data collection: Collect real-time data of robot mechanism speed and position, as well as IMU data, for robot motion control and attitude estimation.

[0132] Battery compartment position data collection: Monitor the position of the battery compartment to ensure stable power supply.

[0133] (3) Edge computing node 1

[0134] Badminton visual detection data processing: Extract key features of badminton such as speed, direction and rotation based on badminton visual detection data.

[0135] Through the analysis of badminton movement data, the trajectory of badminton is fitted and its future position is predicted.

[0136] Badminton hitting point and landing point prediction: Based on the trajectory and movement characteristics of badminton, the hitting point and landing point are predicted to provide decision support for robot hitting action.

[0137] Further, the process of badminton trajectory fitting and future position prediction:

[0138] 1. Data collection and preprocessing

[0139] Image acquisition: Use the robot's binocular camera and external camera to capture the movement image of badminton.

[0140] Image preprocessing: Denoising, enhancement and other processing are performed on the image to improve the accuracy of feature extraction.

[0141] 2. Feature extraction and recognition using YOLO

[0142] Key point detection: Detect key points such as the edge or specific landmark points of badminton in the image.

[0143] Feature matching: Match feature points in consecutive frames to track the movement of badminton.

[0144] 3. Three-dimensional trajectory reconstruction

[0145] Stereo vision processing: Combine the data of the binocular vision system to calculate the three-dimensional coordinates of badminton.

[0146] Disparity calculation: Use the disparity information between left and right images, combined with camera parameters, to calculate the depth information of badminton.

[0147] 4. Trajectory fitting

[0148] Polynomial Model: Use a polynomial model to fit the trajectory of the shuttlecock.

[0149]

[0150] where, and are the vertical and horizontal positions of the shuttlecock, are polynomial coefficients.

[0151] Gaussian Model: Use a Gaussian distribution to describe the probabilistic nature of the shuttlecock's trajectory.

[0152]

[0153] where, is the mean, representing the predicted trajectory position, is the standard deviation, representing the uncertainty of the prediction.

[0154] Physical Model: Consider the effects of gravity and air resistance to establish the kinematic equations of the shuttlecock.

[0155]

[0156]

[0157] where, and are the horizontal and vertical velocities, is the acceleration of gravity, is the drag coefficient, is the mass of the shuttlecock.

[0158] 5. Trajectory Prediction

[0159] State Space Model: Construct a state space model of the shuttlecock, including state variables such as position, velocity, and acceleration.

[0160]

[0161]

[0162] where, is the state vector, is the control input, is the observation vector.

[0163] Prediction Algorithm: If the target is successfully tracked in consecutive frames, the target's position information can be updated and the target can be tracked in subsequent frames, while applying methods such as Kalman filtering or particle filtering to predict and correct the trajectory of the shuttlecock.

[0164] Specifically, the existing extended Kalman filter is used to filter the observation data, reduce measurement noise, and improve the accuracy of trajectory prediction. At the same time, the badminton kinematics equation is established, and the parameters are solved by the least squares method to realize trajectory tracking and prediction.

[0165] Specifically, the badminton kinematics equation is established, and the parameters are solved by the least squares method to realize trajectory tracking and prediction, including:

[0166] 1) Model establishment: According to the flight characteristics of the badminton, the aerodynamics model of the badminton is established. This usually involves considering the influence of air resistance, gravity, lift and other factors on the trajectory of the badminton.

[0167] 2) Parameter solving: The least squares method is used to solve the parameters of the badminton kinematics equation. This involves collecting a series of observation data, and then estimating the model parameters by minimizing the error between the observation data and the model prediction.

[0168] Further, 1) Model error and measurement error: In the prediction process, due to the existence of model error and measurement error, the maximum likelihood estimation of the state of the badminton can be calculated by Bayesian filtering, such as extended Kalman filter.

[0169] 2) Nonlinear filter: For nonlinear models, extended Kalman filter may not be able to provide high-precision estimates. In this case, consider using unscented Kalman filter (UKF) or particle filter (PF) to improve the accuracy of the estimate.

[0170] As an embodiment, the present disclosure can update and predict in real time:

[0171] 1) Update trajectory points: In order to improve the real-time and accuracy of the prediction, the method of updating the trajectory points can be adopted, that is, on the basis of the least squares method, new observation points are constantly added and the oldest points are removed to refit the trajectory curve.

[0172] 2) Error analysis: By comparing the errors of different prediction methods, the method with the smallest error is selected. For example, the least squares method of updating the trajectory points can significantly reduce the prediction error and improve the accuracy of the prediction.

[0173] Through the above steps, the trajectory of the badminton can be effectively tracked and predicted, so as to provide accurate motion information for the badminton robot or other related applications.

[0174] The above process is applied to each frame of the video sequence to realize continuous tracking of small targets such as badminton, and the algorithm is optimized in combination with the flight characteristics of the badminton.

[0175] Through the above detailed process, the robot can accurately fit the trajectory of the shuttlecock and predict its future position, providing scientific decision support for the robot's hitting action. This trajectory prediction method based on multi-modal perception can fully utilize the advantages of different sensors, improving the robustness and accuracy of the system.

[0176] (4) Edge computing node 2

[0177] Obtain the robot vision detection result (recognized position), the running speed of each mechanism, and the IMU data of the body, and comprehensively process the vision detection data and IMU data of the robot for the robot's attitude estimation and motion control.

[0178] Robot pose prediction: predict the robot's pose at future time, provide basis for dynamic balance control.

[0179] Specifically, existing physical models (such as Newton-Euler equations) and machine learning algorithms (such as neural networks) are used to predict the future pose of the robot.

[0180] (5) Obtain the motion state of each mechanism of the robot and the body IMU data, and predict the pose and motion state of the robot based on the motion state of each mechanism and the body IMU data;

[0181] Based on the predicted robot pose, motion state, shuttlecock hitting point and falling point, respectively use robot dynamic balance control algorithm and MPC control algorithm to realize the balance control and hitting action control of the robot itself;

[0182] Among them, based on the robot pose and motion state, a dynamic motion model is constructed, based on the dynamic model and the predicted shuttlecock hitting point and falling point data, the MPC control algorithm is used to calculate the optimal joint angle and drive wheel speed, and the robot is driven to complete various hitting actions according to the obtained joint angle and drive wheel speed.

[0183] Adjust the position of the battery compartment to optimize the distribution of the robot's center of gravity and improve the stability and accuracy of the hitting.

[0184] As an embodiment, the obstacle avoidance operation of the robot motion process is realized, and the cost function is affected by the target direction, the current direction of the robot, and the previously selected direction, the selection preference of the robot is adjusted by adjusting the weight, and the motion obstacle avoidance of the robot is realized.

[0185] 1) Define a cost function, which is affected by three factors: target direction (predicted position of the shuttlecock), current direction of the robot, and previously selected direction.

[0186] 2) The cost function can be expressed as:

[0187] wherein, are weight coefficients used to adjust the influence of each factor on the obstacle avoidance decision. 目标 is the deviation between the target direction and the current direction of the robot, 当前 is the deviation between the current direction of the robot and the motion direction, 之前 is the deviation between the previously selected direction and the current direction of the robot.

[0188] By adjusting the weight coefficients , the robot's obstacle avoidance strategy can be adjusted. For example, if rapid approach to the target is more important, the value of w 1 can be increased. The adjustment of weights can be dynamically adjusted according to the current state of the robot, historical behavior and environmental characteristics.

[0189] Further, the cost of different obstacle avoidance paths is calculated according to the cost function, and the path with the minimum cost is selected as the obstacle avoidance path. The calculation of the obstacle avoidance path can combine the dynamic motion model of the robot, considering the speed, acceleration and turning limit of the robot.

[0190] Specifically, the path planning algorithm (such as A* algorithm, RRT algorithm, etc.) is used to plan the movement trajectory of the robot according to the obstacle avoidance path. Path planning needs to consider the dynamics and kinematics constraints of the robot to ensure the feasibility of the path.

[0191] The planned obstacle avoidance path is converted into control instructions of the robot, such as the speed and steering angle of the motor.

[0192] These instructions are executed by the control system of the robot to achieve obstacle avoidance.

[0193] The execution of the robot's obstacle avoidance is monitored in real time, and the obstacle avoidance strategy is adjusted through sensor feedback.

[0194] Further, if the obstacle avoidance execution is not ideal, the cost function and obstacle avoidance path are recalculated for dynamic adjustment.

[0195] The effect of the obstacle avoidance algorithm is evaluated through simulation or actual operation, including the success rate of obstacle avoidance, the degree of path optimization and the response time, etc. The obstacle avoidance algorithm is further optimized according to the evaluation results.

[0196] And, it is judged in real time whether the distance between the robot and the obstacle will collide, if the collision risk is large, the motion direction of the robot needs to be modified immediately or the motion of the robot needs to be stopped.

[0197] Finally, the system resets: after completing the hitting action, the system enters the reset state, preparing for the next hitting action or system shutdown.

[0198] The above specific embodiment describes the whole process from system initialization to data acquisition, processing, control algorithm execution, and robot action execution, ensuring that the robot can accurately and efficiently complete the badminton hitting task. Through the application of the robot dynamic balance control algorithm and the MPC control algorithm, the robot can maintain dynamic balance while achieving precise motion control and hitting action.

[0199] Embodiment 3

[0200] In an embodiment of the present disclosure, a control system of a ball game robot is provided, comprising:

[0201] A data acquisition module is configured to acquire image data of an environment in a badminton court and image data of a motion outside the court.

[0202] A badminton trajectory prediction module is configured to identify positions of a badminton, the robot, and a motion person based on the image data of the environment and the image data of the motion outside the court by using a deep learning model; and to reconstruct a flight trajectory of the badminton by using a tracking algorithm based on position information of the badminton, the robot, and the motion person, and to predict a hitting point and a landing point of the badminton.

[0203] A robot motion posture prediction module is configured to acquire motion states of each mechanism of the robot and body IMU data, and to predict a posture and a motion state of the robot based on the motion states of each mechanism and the body IMU data.

[0204] A control module is configured to achieve balance control and hitting action control of the robot by using a robot dynamic balance control algorithm and an MPC control algorithm based on predicted postures and motion states of the robot, a hitting point, and a landing point of the badminton.

[0205] In the control module, a dynamic motion model is constructed based on the postures and motion states of the robot, and optimal joint angles and driving wheel speeds are calculated by using the MPC control algorithm based on the dynamic model and predicted data of the hitting point and the landing point of the badminton, and the robot is driven to complete various hitting actions according to the obtained joint angles and driving wheel speeds.

[0206] Embodiment 4

[0207] In an embodiment of the present disclosure, a non-transitory computer readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the control method of the ball game robot.

[0208] Embodiment 5

[0209] An embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the control method of the ball robot.

[0210] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0211] These computer program instructions can also be loaded into a computer or other programmable data processing device to cause a series of operation steps to be executed on the computer or other programmable data processing device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0212] Although the specific embodiments of the present disclosure are described above with reference to the accompanying drawings, the present disclosure is not limited to the above embodiments, and various modifications or changes can be made to the embodiments without departing from the scope of the present disclosure.

Claims

1. A control method of a ball game robot, characterized by, The method comprises the following steps: Obtain the image data of the environment inside the badminton court and the motion image data outside the court; Based on the environment image data and the motion image data outside the court, use a deep learning model to identify the positions of the badminton, the robot and the sports personnel; According to the positions of the badminton, the robot and the sports personnel, use a tracking algorithm to reconstruct the flight trajectory of the badminton and predict the hitting point and the landing point of the badminton; Obtain the motion state of each mechanism of the robot and the IMU data of the body, and predict the attitude and motion state of the robot based on the motion state of each mechanism and the IMU data of the body; Based on the predicted attitude and motion state of the robot, the hitting point and the landing point of the badminton, use the dynamic balance control algorithm and the MPC control algorithm to realize the balance control and the hitting action control of the robot respectively; Among them, based on the attitude and motion state of the robot, a dynamic motion model is constructed, based on the dynamic model and the predicted hitting point and landing point of the badminton, the MPC control algorithm is used to calculate the optimal joint angle and drive wheel speed, and the robot is driven to complete various hitting actions according to the obtained joint angle and drive wheel speed; Use multiple sensors to obtain the motion state of each mechanism of the robot and the IMU data of the body, the motion state includes the speed and attitude position of the robot, predict the next time attitude of the robot through the speed, attitude position and trajectory data of the badminton, and control the balance of the robot itself by using the robot dynamic balance control algorithm according to the attitude prediction of the robot and the current position and angle of the battery compartment, and the control system drives the motor to adjust the battery compartment to the calculated optimal angle to optimize the center of gravity distribution.

2. The control method of a ball game robot according to claim 1, wherein The built-in binocular camera of the robot is used to collect the environment image data inside the badminton court, the external cameras at both ends of the court are used to collect the motion image data, the YOLO model is used for identification and detection of the badminton, the robot and the sports personnel, the feature points are obtained, and the feature points are matched, the parallax in the left and right images is calculated for the matched feature points, the depth of the target is calculated through the parallax and the camera parameters, and the three-dimensional coordinates (X, Y, Z) of the target point are converted into the position in the actual space according to the depth and the coordinate system of the left and right cameras.

3. The control method of a ball game robot according to claim 1, wherein The badminton kinematics equation is established, the trajectory tracking and prediction of the badminton are realized based on the position information of the badminton, the robot and the sports personnel, the parameters are solved by the least square method, and finally the hitting point and the landing point of the badminton and the position information data of the robot are output.

4. The control method of a ball game robot according to claim 1, wherein The motion model of the robot is established, including the dynamics and kinematics model, the control strategy is optimized according to the predicted attitude and motion state of the robot, the hitting point and the landing point of the badminton, the MPC control algorithm is used to calculate the optimal joint angle and drive wheel speed, the drive wheels and joint actuators of the robot are driven, the robot completes various hitting actions, and the data is fed back in real time to adjust the control strategy to cope with environmental changes and prediction errors.

5. The control method of a ball game robot according to claim 1, wherein The battery compartment can rotate forward and backward around the vertical axis of the robot, and the rotation changes the position of the center of mass of the battery compartment in the vertical direction, thereby adjusting the height and position of the center of gravity of the robot. The center of gravity of the robot is the weighted average of the center of mass of all components. The rotation of the battery compartment changes its center of mass, thereby adjusting the center of gravity of the entire robot. The robot is equipped with an IMU sensor that monitors its attitude changes in real time. The IMU data is used to calculate the optimal angle of the center of gravity adjustment required to maintain or achieve balance. The system controls the motor to adjust the position of the battery compartment according to the calculated optimal angle.

6. A control system for a sports ball robot, implementing a control method for a sports ball robot as claimed in any one of claims 1 to 5, characterized in that It comprises: a data acquisition module for acquiring badminton court environment image data and off-court motion image data; a badminton trajectory prediction module for identifying the positions of the badminton, robot and sports personnel based on the environment image data and off-court motion image data using a deep learning model; reconstructing the flight trajectory of the badminton using a tracking algorithm based on the position information of the badminton, robot and sports personnel, and predicting the hitting point and landing point of the badminton; a robot motion posture prediction module for acquiring the motion state of each mechanism of the robot and the IMU data of the body, and predicting the posture and motion state of the robot based on the motion state of each mechanism and the IMU data of the body; a control module for implementing balance control and hitting action control of the robot itself using dynamic balance control algorithm and MPC control algorithm based on the predicted robot posture, motion state, badminton hitting point and landing point; wherein a dynamic motion model is constructed based on the robot posture and motion state, and the optimal joint angle and drive wheel speed are calculated using the MPC control algorithm based on the dynamic model and the predicted badminton hitting point and landing point data, and the robot completes various hitting actions according to the obtained joint angle and drive wheel speed.

7. A system for controlling a sports robot as claimed in claim 6, wherein The ball game robot is designed with double-wheel feet and a single arm with a racket. The single arm is a mechanical arm, and a racket is arranged at the head of the mechanical arm. The joints of the mechanical arm are four degrees of freedom, each joint is driven by a motor, including an end motor, a No. 3 motor, a No. 2 motor and a No. 1 motor, and each joint is equipped with a sensor. The double-wheel feet are driven by a master-slave wheel structure, including a standing motor, a carbon fiber square tube and a large wheel drive motor assembly. The double-wheel feet and the mechanical arm are installed on the main body of the machine body, and a battery compartment and a battery swing motor are arranged on the vertical shaft below the main body of the machine body, which can rotate around the vertical shaft.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium is used to store computer instructions, which are executed by the processor to implement the control method of the ball game robot according to any one of claims 1-5.

9. An electronic device, comprising: It comprises: a processor, a memory and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to make the electronic device execute the control method of the ball game robot according to any one of claims 1-5.

Images