A smart tennis ball automatic ball retrieval device and method based on multi-sensor fusion

Abstract

This invention relates to an intelligent automatic tennis ball retrieval device based on multi-sensor fusion, comprising: a mobile platform; a multi-sensor fusion module including a lidar, a visual sensor, an infrared sensor, and an ultrasonic sensor; a ball-retrieval execution module including a ball-retrieval arm, a roller brush mechanism, a conveying channel, and a ball storage bin; a main control unit of the control module for coordinating data interaction and command issuance among the various units; a data fusion unit for preprocessing, fusing, and verifying data from each sensor, and outputting tennis ball coordinates and obstacle information; a navigation control unit for planning a path based on a map and positioning data and controlling the movement of the device; an execution control unit for controlling the actions of the ball-retrieval arm, the roller brush mechanism, and the mobile platform; and a power module including a battery pack and a charging management module. This invention achieves high-precision environmental perception and tennis ball recognition through multi-sensor fusion, combines an autonomous navigation algorithm to achieve full-court coverage movement, and optimizes the ball-retrieval mechanism to improve retrieval efficiency.

Description

Technical Field

[0001] This invention relates to the field of sports equipment technology, and in particular to an intelligent automatic tennis ball retrieval device and method based on multi-sensor fusion. Background Technology

[0002] During tennis training or matches, a large number of tennis balls are scattered across the court. Manually retrieving these balls is not only inefficient and increases the workload for trainers and staff, but it also disrupts the training rhythm and affects training effectiveness. To address this issue, some automatic ball-retrieval devices have emerged on the market, but existing technology still has many shortcomings. Existing automatic ball-retrieval devices are mainly divided into two categories: one is a purely mechanical device that requires manual pushing or pulling. It relies on a rotating roller to drive a brush to roll the tennis balls into a storage bin. This type of device lacks autonomous movement capability, cannot achieve full-court automatic coverage, has a limited retrieval range, and is extremely inefficient in complex court environments (such as those with obstacles or scattered tennis balls). The other type is a semi-automatic ball-retrieval device with simple navigation functions. It uses a single sensor (such as an infrared sensor or a visual sensor) for tennis ball detection and environmental perception. However, infrared sensors are easily affected by environmental factors such as light and court color, resulting in low accuracy in tennis ball recognition and serious issues of missed or incorrect retrieval. Visual sensors experience a significant drop in recognition performance in low-light conditions or when the tennis ball is similar in color to the background, and they cannot effectively perceive the distance to obstacles, making them prone to collisions and damage. Furthermore, the existing ball-collecting mechanism design has shortcomings: the success rate of picking up tennis balls that roll close to the ground or are partially embedded in the court gaps is generally below 80%; the ball storage compartments mostly rely on mechanical limit switches to monitor capacity, which has a large error (±5 balls), often resulting in ineffective operation after the compartment is full or ball jamming due to failure to empty it in time. Therefore, developing an intelligent automatic tennis ball-collecting device and method based on multi-sensor fusion, integrating high-precision environmental perception, fully autonomous navigation, efficient ball collection, and intelligent status monitoring, has become a key requirement to overcome the current technological bottlenecks. Summary of the Invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the problems of insufficient autonomous movement capability, low environmental perception accuracy, poor tennis ball recognition accuracy, low ball picking efficiency and lack of intelligent monitoring function in the existing technology. The present invention provides an intelligent automatic tennis ball picking device and method based on multi-sensor fusion. It achieves high-precision environmental perception and tennis ball recognition through multi-sensor fusion, and achieves full-court coverage movement by combining autonomous navigation algorithm, and optimizes the ball picking mechanism to improve picking efficiency.

[0004] To address the aforementioned technical problems, this invention provides an intelligent automatic tennis ball retrieval device based on multi-sensor fusion, comprising: Mobile platform; The multi-sensor fusion module includes a lidar, a visual sensor, an infrared sensor, and an ultrasonic sensor. The lidar is installed at the top center of the mobile platform to acquire 3D point cloud data of the field to construct an environmental map and identify obstacles. The visual sensor is installed at the front of the mobile platform to acquire images and identify tennis balls using a target detection algorithm. The infrared sensors are installed on both sides of the bottom of the mobile platform to detect tennis balls that are close to the ground or at low height. The ultrasonic sensors are installed on the four end faces of the mobile platform to detect nearby obstacles for emergency obstacle avoidance. The ball-collecting execution module includes a ball-collecting arm, a roller brush mechanism, a conveying channel, and a ball storage bin. The ball-collecting arm is installed at the front end of the mobile platform and is equipped with an electric push rod for driving extension and retraction adjustment. A servo motor is installed at the end of the ball-collecting arm for angle adjustment. The roller brush mechanism is installed between the outlet of the conveying channel and the ball storage bin. It uses the friction generated by rotation to roll in the tennis ball and convey it to the ball storage bin. The control module includes a main control unit, a data fusion unit, a navigation control unit, and an execution control unit. The main control unit coordinates data interaction and command issuance among the various units. The data fusion unit preprocesses, fuses, and verifies data from various sensors, and outputs tennis ball coordinates and obstacle information. The navigation control unit plans a path based on the map and positioning data and controls the movement of the device. The execution control unit controls the actions of the ball-collecting arm, the roller brush mechanism, and the moving platform. The power module, including the battery pack and charging management module, is used to supply power to each module.

[0005] Furthermore, the mobile platform includes a chassis, a drive unit, and a navigation unit. The drive unit includes a DC geared motor, wheels, and a reducer. The DC geared motor is connected to the wheels through the reducer. The navigation unit includes a GPS positioning module and an inertial measurement unit. The GPS positioning module acquires the absolute position, and the inertial measurement unit collects attitude angle and acceleration data. The two are fused to achieve positioning and navigation. Furthermore, the visual sensing device includes a supplementary lighting module, which consists of LED beads arranged in a ring around the camera lens. The module detects ambient light using a light intensity sensor and automatically activates when the light intensity is below a set threshold. The brightness is adjusted via a PWM signal.

[0006] Furthermore, an infrared beam sensor is installed on the conveying channel to detect the tennis ball conveying status.

[0007] Furthermore, a weighing module is installed inside the ball storage chamber to detect the weight of the stored balls; a breathable mesh is installed on the top of the ball storage chamber, a transparent observation window is provided on the side, and a detachable base plate is installed at the bottom.

[0008] Furthermore, it also includes a human-machine interaction module, which includes a touch screen and a buzzer. The touch screen is used to set parameters and display the device status, and the buzzer is used to issue a warning signal when the ball storage bin is full or malfunctions.

[0009] Secondly, in order to solve the above problems, the present invention provides an intelligent tennis ball automatic ball retrieval method based on multi-sensor fusion, applied to the device described in the first aspect, comprising the following steps: S1: Set the ball-collecting range, path mode parameters, and full-load threshold via the touch screen. The control module then activates each sensor and drive unit to perform a system self-test. S2: The lidar collects three-dimensional information of the field and combines it with GPS positioning data to build a field map. The initial ball-picking path is planned according to the map. Multiple sensors simultaneously collect environmental data. The data fusion unit fuses the data from lidar, vision, infrared and ultrasonic sensors, removes interference data, and obtains information on obstacle distribution and potential tennis ball positions. S3: The navigation control module controls the device to move autonomously according to the planned path. During the movement, the ultrasonic sensor detects nearby obstacles in real time. If an obstacle is detected, the path is adjusted to avoid the obstacle. At the same time, the visual sensor and infrared sensor continuously detect the tennis ball. After the visual sensor collects the image, it identifies the tennis ball through the target detection algorithm. Combined with the infrared sensor data, it confirms the position of the tennis ball close to the ground and transmits the coordinate information of the tennis ball to the main control unit. S4: The main control unit calculates the required extension length and angle of the ball-collecting arm based on the ball's coordinate information, controls the ball-collecting arm to adjust to the target position, and starts the roller brush mechanism to roll in the ball. The ball slides down the conveyor channel to the ball storage bin. If multiple balls are detected, they are picked up in order of distance from closest to furthest. S5: The weighing module in the ball storage bin collects the total weight of the stored balls in real time. The main control unit calculates the real-time number of stored balls based on the preset standard weight of a single tennis ball. When the number of stored balls reaches the preset full bin threshold, an early warning signal is issued, the ball picking operation is suspended, and the return path planning is initiated. S6: When the preset ball-collecting range is completed or the user manually stops, the device automatically returns to the starting position, shuts down all execution modules, and enters standby mode.

[0010] Furthermore, the specific process of data fusion in step S2 includes: Data preprocessing: filtering and denoising the lidar point cloud, correcting distortion in the visual image, performing analog-to-digital conversion and filtering on the analog signal output from the infrared sensor, and removing outliers from the ultrasonic sensor data. Data fusion: Using the center of the tennis ball bounding box identified by the visual sensor as the initial coordinates, the corresponding three-dimensional coordinates detected by the LiDAR are used as observations, and fused through extended Kalman filtering to output the three-dimensional coordinates of the tennis ball; Data verification: If the infrared sensor detects a reflected signal at the corresponding coordinates and the ultrasonic sensor does not detect an obstacle, then the coordinates are confirmed as a valid tennis ball position; otherwise, it is marked as a suspected target. Furthermore, the full capacity threshold mentioned in step S5 is the rated capacity value of the ball storage bin. When the total weight of the stored balls detected by the weighing module reaches the weight corresponding to the threshold, the main control unit controls the buzzer to emit a warning sound, and at the same time the touch screen displays the full capacity prompt information. The device pauses the ball picking operation and plans the path back to the starting position.

[0011] Furthermore, during the autonomous movement process described in step S3, when the ultrasonic sensor detects that the distance to an obstacle is less than a first threshold, the mobile platform decelerates; when the detected distance to an obstacle is less than a second threshold, the mobile platform stops urgently and replans its path, wherein the first threshold is greater than the second threshold.

[0012] Compared with the prior art, the above-described technical solution of the present invention has the following advantages: (1) The intelligent tennis ball automatic ball picking device based on multi-sensor fusion of the present invention realizes automatic identification, positioning, picking and storage of tennis balls through the coordinated cooperation of mobile platform, multi-sensor fusion module, ball picking execution module, control module and power supply module, solving the problems of low accuracy and poor efficiency of existing devices. It is suitable for tennis training and competition scenarios. In addition, the device has autonomous movement capability and does not require manual pushing. It can effectively overcome the problems of single sensor being susceptible to environmental interference and low recognition accuracy, and significantly improve the accuracy of environmental perception and tennis ball recognition.

[0013] (2) The intelligent tennis ball automatic ball retrieval method based on multi-sensor fusion of the present invention achieves a complete closed-loop process from parameter configuration to automatic operation and safe return through the organic coordination of six steps: initialization setting, environmental perception and map construction, autonomous movement and tennis ball detection, automatic ball retrieval operation, capacity monitoring and early warning, and operation completion. This method enables the device to autonomously complete the ball retrieval operation of the entire court without human intervention, and the ball retrieval efficiency is high, which is an improvement over manually pushed mechanical devices. At the same time, the system self-check, fault alarm, full capacity early warning and other mechanisms ensure the safety and reliability of the operation. Attached Figure Description

[0014] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0015] Figure 1 This is a schematic diagram of the structure of the intelligent tennis ball automatic ball-collecting device based on multi-sensor fusion in a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of the multi-sensor fusion module in this invention; Figure 3 This is a flowchart of the intelligent tennis ball automatic ball retrieval system based on multi-sensor fusion in this invention; Explanation of reference numerals in the accompanying drawings: 1. Moving platform; 2. LiDAR; 3. Vision sensor; 4. Infrared sensor; 5. Ultrasonic sensor; 6. Ball picking arm; 7. Roller brush mechanism; 8. Conveying channel; 9. Ball storage bin; 10. Infrared beam sensor. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention. Example

[0017] Reference Figure 1-2 As shown, the present invention provides an intelligent tennis ball automatic retrieval device based on multi-sensor fusion, comprising: Mobile platform 1 includes a chassis, a drive unit, and a navigation unit. The drive unit includes a DC geared motor, wheels, and a reducer; the DC geared motor is connected to the wheels via the reducer. The navigation unit includes a GPS positioning module and an inertial measurement unit. The GPS positioning module acquires absolute position, and the inertial measurement unit collects attitude angle and acceleration data. The two are fused to achieve positioning and navigation. The drive unit adopts a differential drive structure, with the DC geared motor rigidly connected to the wheels via the reducer, achieving stable power output and flexible steering for the device. Specifically, the drive unit uses two 57BLDC type 12V DC geared motors (power 200W, reduction ratio 1:50), and is equipped with Φ15cm rubber inflatable wheels (tire pressure 2.5bar), and the steering is achieved by differential drive; the power of the DC geared motor is ≥150W, the reduction ratio is ≥1:50, and the wheel diameter is ≥12cm; The navigation unit uses a UBLOXNEO-7M GPS module (positioning accuracy ±1m, update frequency 1Hz, receiver sensitivity -165dBm) and an MPU6050 IMU (sampling frequency 100Hz, accelerometer accuracy ±2g, gyroscope accuracy ±250° / s). The IMU is fixed to the center of the chassis with an aluminum bracket to reduce vibration interference. The GPS positioning accuracy is ≤±1m, and the IMU sampling frequency is ≥50Hz. A combined navigation scheme of GPS positioning module and inertial measurement unit (IMU) is adopted. High-precision positioning and navigation are achieved through data fusion of the two, improving positioning accuracy compared to GPS navigation alone. This effectively solves the problems of positioning drift and large path deviation in existing devices, ensuring accurate movement and full coverage of the device in complex scenarios such as a standard tennis court.

[0018] The multi-sensor fusion module includes a lidar 2, a vision sensor, an infrared sensor 4, and an ultrasonic sensor 5. The lidar 2 is installed at the top center of the mobile platform 1 to acquire 3D point cloud data of the site to construct an environmental map and identify obstacles. The lidar selected is an RPLIDAR A3 (range range 0.1-25m, sampling frequency 10Hz, angular resolution 0.15°), which is fixed to the top center of the chassis with Φ8 bolts (installation height 45cm). After 360° scanning calibration, the site boundary recognition error is ≤5cm. The lidar range is ≥0.1-20m and the sampling frequency is ≥5Hz. The visual sensor 3 is installed at the front end of the mobile platform 1 to acquire images and identify tennis balls through a target detection algorithm; specifically, the visual sensor is a high-definition camera with a resolution of ≥1080P, which is used in conjunction with a target detection module based on the YOLO series algorithm and is installed at the front end of the mobile platform 1 at a height of 30-50cm from the ground. Infrared sensors 4 are installed on both sides of the bottom of the mobile platform 1 to detect tennis balls that are close to the ground or at low height. The infrared sensors 4 are of type GP2Y0A21YK, 10cm from the front edge. Two sensors are symmetrically installed on both sides of the bottom of the mobile platform 1, 5-10cm from the ground. The detection direction is at a 15° angle to the ground. After potentiometer calibration, the output voltage is 1.6V at 50cm. The detection distance of the infrared sensors is 20-150cm. The ultrasonic sensor 5 is installed on the four end faces of the mobile platform 1 at the front, back, left, and right to detect nearby obstacles for emergency obstacle avoidance. The ultrasonic sensor 5 has a ranging range of 2-400cm and an accuracy of ±0.5cm. Four of them are installed on the four end faces of the mobile platform. Four HC-SR04 ultrasonic sensors are installed on the four end faces of the chassis (30cm from the ground, with the detection direction perpendicular to the end face). The ball-collecting execution module includes a ball-collecting arm 6, a roller brush mechanism 7, a conveying channel 8, and a ball storage bin 9. The ball-collecting arm 6 is installed at the front end of the mobile platform 1. An electric push rod 61 is provided on the ball-collecting arm 6 for driving and adjusting its extension and retraction. A servo motor 62 is provided at the end of the ball-collecting arm 6 for angle adjustment. The roller brush mechanism 7 is installed between the outlet of the conveying channel 8 and the ball storage bin 9. It uses the friction generated by rotation to roll up the tennis ball and convey it to the ball storage bin 9. Specifically, the ball-collecting arm 6 is telescopic, driven by two-stage electric push rods 61, with a stroke of 0-30cm and a telescopic speed of ≥3cm / s. The end is adjusted by a servo motor 62 to achieve an angle of ±30° with an adjustment accuracy of ≤1°. The roller brush mechanism includes a roller with a diameter of 8-12cm, a nylon brush, and a DC motor. The brush density is ≥40 brushes / cm², and the motor speed is 80-120r / min. Specifically, the conveying channel 8 is an inclined channel with an inner diameter of 10-15cm, made of ABS material, 50cm in length, and an inclination angle of 25-35°. The inner wall of the conveying channel 8 is coated with a 0.1mm thick polytetrafluoroethylene coating. The inlet is connected to the roller outlet through a Φ15cm rubber hose, and the outlet flange is connected to the top of the ball storage chamber 9. In addition, an E3Z-LS63 type infrared beam sensor 10 is installed in the middle of the conveying channel 8 to detect the tennis ball conveying status. Its detection distance is 12cm, and it outputs a low level when the tennis ball passes through. The ball storage chamber 9 has an internal weighing module for detecting the weight of the stored balls; the top of the ball storage chamber 9 is equipped with a breathable mesh, the side has a transparent observation window, and the bottom has a removable base plate. The ball storage chamber 9 is a cylindrical structure with a capacity of ≥40 standard tennis balls, and its internal weighing module has an accuracy of ≥0.1g. The control module includes a main control unit, a data fusion unit, a navigation control unit, and an execution control unit. The main control unit coordinates data interaction and command issuance among the various units. The data fusion unit preprocesses, fuses, and verifies data from various sensors, outputting tennis ball coordinates and obstacle information. The navigation control unit plans a path based on map and positioning data and controls the movement of the device. The execution control unit controls the actions of the ball-collecting arm 6, the roller brush mechanism 7, and the moving platform 1. Preferably, the main control chip is an ARM architecture chip with a main frequency ≥300MHz. The data fusion module uses an extended Kalman filter (EKF) algorithm to fuse multi-sensor data, outputting three-dimensional coordinates of the tennis ball with an accuracy of ±2cm. The navigation control module uses GPS+IMU combined navigation to achieve path planning and dynamic obstacle avoidance. The execution control module uses PWM control technology with a response delay ≤100ms. The power module includes a 12V lithium polymer battery pack and a charging management module, with a battery capacity ≥12Ah, supporting wired charging and wireless charging with a power ≥5W, and has a built-in BMS battery management system.

[0019] The power module, including a battery pack and a charging management module, is used to power each module. The battery pack is a 12V lithium polymer battery pack with a capacity of ≥12Ah, supporting wired charging and wireless charging with a power of ≥5W. The charging management module has a built-in BMS battery management system.

[0020] The visual sensor is equipped with a supplementary lighting module, which consists of LED beads arranged in a ring around the camera lens. An ambient light sensor detects the ambient light, and the module automatically activates when the illuminance falls below a set threshold. Brightness is adjusted via a PWM signal. Specifically, the supplementary lighting module consists of 3-5 LED beads with a power of ≥2W, working in conjunction with the illuminance sensor to achieve automatic brightness adjustment. It automatically activates when the illuminance is ≤300 lux, and the brightness adjustment range is 10%-100%.

[0021] It also includes a human-machine interaction module, which comprises a touchscreen and a buzzer. The touchscreen is used to set parameters and display the device status, while the buzzer provides an audible warning signal when the ball storage compartment 9 is full or malfunctioning. The touchscreen allows for setting parameters such as the ball-collecting range and path mode, as well as real-time display of device status (battery level, ball storage capacity). The buzzer provides an audible warning when the compartment is full or malfunctioning. This design lowers the barrier to entry for the device, enabling users to easily monitor its operation and promptly handle abnormal situations, thus improving the human-machine interaction experience and practicality.

[0022] Based on the above structure, the intelligent tennis ball automatic retrieval device with multi-sensor fusion first receives parameters set by the user, such as the retrieval range, path mode, and full load threshold, through the human-machine interaction module. The main control unit of the control module initiates a system self-test, confirming that key components such as the lidar 2, vision sensor, infrared sensor 4, ultrasonic sensor 5, drive motor, and battery pack are functioning normally before entering standby mode.

[0023] After self-testing, the device enters the environmental perception and map building phase. LiDAR 2 scans the site 360° at a 10Hz frequency, collecting 3D point cloud data; the GPS positioning module obtains the device's absolute position; and the inertial measurement unit (IMU) collects attitude angle and acceleration data. The navigation control unit integrates GPS and IMU data to achieve combined positioning, and simultaneously constructs a rasterized site map based on the LiDAR point cloud data, planning a spiral traversal path based on an improved A* algorithm.

[0024] During autonomous movement, the multi-sensor fusion module works synchronously: LiDAR 2 is responsible for environmental map construction and long-distance obstacle recognition; the visual sensor detects the tennis ball's position in real time using the YOLOv5s algorithm; infrared sensor 4 detects tennis balls that are close to the ground or at low altitudes; and ultrasonic sensor 5 monitors nearby obstacles to achieve dynamic obstacle avoidance. The data fusion unit preprocesses, fuses, and verifies the data from the four sensors, outputting the tennis ball's three-dimensional coordinates with an accuracy of ±2cm.

[0025] When a valid tennis ball is detected, the navigation control module moves the control unit to the vicinity of the target. The execution control unit drives the electric push rod 61 and servo motor 62 of the ball-collecting arm 6 to adjust the extension length and pitch angle, aligning the roller brush mechanism 7 with the tennis ball. The roller brush mechanism 7 rotates, and the friction between the nylon brush and the tennis ball draws the ball in. The tennis ball slides down the conveyor channel 8 into the ball storage chamber 9. The infrared beam sensor 10 in the middle of the conveyor channel 8 confirms that the tennis ball has been conveyed, and the device continues to the next target.

[0026] The HX711 weighing module at the bottom of the ball storage compartment 9 monitors the weight of the stored balls in real time, and the main control unit calculates the real-time number of balls stored based on the standard weight of a single tennis ball. When the full capacity threshold is reached, a buzzer sounds an alarm, the touchscreen displays the full capacity information, the device pauses the ball retrieval operation, and automatically plans a path back to the starting position.

[0027] After the operation is completed or the user manually stops it, the device returns to the starting position, shuts down all execution modules, and enters standby mode. The user can open the removable bottom plate of the ball storage compartment 9 to empty the tennis balls and then restart the next round of operation. Example

[0028] like Figure 3 As shown, this invention discloses an intelligent tennis ball automatic ball retrieval method based on multi-sensor fusion, applied to the device described in Embodiment 1, comprising the following steps: S1: Initialization settings: Set the ball picking range, path mode parameters and full load threshold through the touch screen. The control module starts each sensor and drive unit to perform system self-test. The self-test includes sensor communication, motor rotation and battery voltage. The self-test takes ≤10s. In case of a fault, an alarm will be sounded and the fault type will be displayed. S2: Environmental perception and map building: LiDAR 2 collects three-dimensional information of the venue and combines it with GPS positioning data to build a venue map. The initial ball picking path is planned according to the map. Multiple sensors simultaneously collect environmental data. The data fusion unit performs fusion processing on the data from LiDAR 2, visual sensor, infrared sensor 4 and ultrasonic sensor 5, removes interference data, and obtains information on obstacle distribution and potential tennis ball positions. S3: Autonomous Movement and Tennis Detection: The navigation control module controls the device to move autonomously according to the planned path. During the movement, the ultrasonic sensor 5 detects nearby obstacles in real time. If an obstacle is detected, the path is adjusted to avoid the obstacle. At the same time, the visual sensor and infrared sensor 4 continuously detect the tennis ball. After the visual sensor collects the image, it identifies the tennis ball through the target detection algorithm. Combined with the data from the infrared sensor 4, it confirms the position of the tennis ball close to the ground and transmits the coordinate information of the tennis ball to the main control unit. S4: Automatic ball picking operation: The main control unit calculates the required extension length and angle of the ball picking arm 6 based on the ball coordinate information, controls the ball picking arm 6 to adjust to the target position, and starts the roller brush mechanism 7 to roll in the ball. The ball slides down the conveyor channel 8 to the ball storage bin 9. If multiple balls are detected, they are picked up in order from closest to furthest. S5: Capacity monitoring and early warning: The weighing module in the ball storage bin 9 collects the total weight of the stored balls in real time. The main control unit calculates the real-time number of stored balls based on the preset standard weight of a single tennis ball. When the number of stored balls reaches the preset full bin threshold, an early warning signal is issued, the ball picking operation is suspended, and return path planning is initiated. S6: Operation complete: When the preset ball-collecting range is completed or the user manually stops, the device automatically returns to the starting position, shuts down all execution modules, and enters standby mode.

[0029] The specific process of data fusion in step S2 includes: Data preprocessing: filtering and denoising the point cloud of LiDAR 2, distortion correction of the image of the visual sensor, analog-to-digital conversion and filtering of the analog signal output by infrared sensor 4, and removal of outliers from the data of ultrasonic sensor 5. Data fusion: Using the center of the tennis ball's bounding box identified by the visual sensor as the initial coordinates, the corresponding three-dimensional coordinates detected by the LiDAR are used as observations. The data is then fused using an extended Kalman filter (EKF) to output the three-dimensional coordinates of the tennis ball. The specific integration process is as follows: The state equations are as follows: X(k) = A*X(k-1) + B*u(k) + w(k); X(k) is the three-dimensional coordinate state vector (X,Y,Z)T of the tennis ball at time k; A is the state transition matrix, B is the control matrix, and w(k) is the process noise; The observation equation is Z(k) = H*X(k) + v(k); Z(k) is the three-dimensional coordinate observation vector observed by lidar 2 at time k; H is the observation matrix, and v(k) is the observation noise; Through the updated iteration of the extended Kalman filter (EKF), the three-dimensional coordinates (X,Y,Z) of the tennis ball are output after fusion, with a Z-axis error ≤0.05m.

[0030] Data verification: If the infrared sensor 4 detects a reflected signal at the corresponding coordinates and the ultrasonic sensor 5 does not detect an obstacle, then the coordinates are confirmed as a valid tennis ball position; otherwise, it is marked as a suspected target, and the control device moves to a closer distance for re-detection.

[0031] During the autonomous movement described in step S3, when the ultrasonic sensor 5 detects that the distance to the obstacle is less than the first threshold, the mobile platform 1 decelerates; when the distance to the obstacle is detected to be less than the second threshold, the mobile platform 1 stops urgently and replans its path, where the first threshold is greater than the second threshold.

[0032] The specific steps of path planning in step S3 are as follows: Map modeling: The standard tennis court (23.77m×10.97m) is rasterized with a grid size of 0.2m×0.2m. Obstacle grids (objects with Z-axis > 0.3m exist within the grid) are marked using LiDAR point cloud data. Cost function optimization: Based on the traditional algorithm cost function f(n)=g(n)+h(n), a turning cost coefficient θ(n) is added, i.e. f(n)=g(n)+1.2*h(n)+θ(n), where θ(n) is the angle cost between the current path and the target direction (θ=0 when the angle is 0°, and θ=5 when the angle is 90°), reducing unnecessary turning; Dynamic obstacle avoidance: The obstacle grid map is updated every 500ms during movement. When a new obstacle grid is detected on the planned path, local path replanning is triggered. The Dijkstra algorithm is used to quickly generate obstacle avoidance paths. The obstacle avoidance response time is ≤100ms, ensuring no collisions when the movement speed is 0.5m / s.

[0033] In step S5, the full capacity threshold is the rated capacity of the ball storage bin 9. When the total weight of the stored balls detected by the weighing module reaches the weight corresponding to the threshold, the main control unit controls the buzzer to sound an alarm, and at the same time the touch screen displays the full capacity prompt information. The device pauses the ball picking operation and plans the path back to the starting position.

[0034] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A smart tennis ball automatic retrieval device based on multi-sensor fusion, characterized in that, include: Mobile platform; The multi-sensor fusion module includes a lidar, a visual sensor, an infrared sensor, and an ultrasonic sensor. The lidar is installed at the top center of the mobile platform to acquire 3D point cloud data of the field to construct an environmental map and identify obstacles. The visual sensor is installed at the front of the mobile platform to acquire images and identify tennis balls using a target detection algorithm. The infrared sensors are installed on both sides of the bottom of the mobile platform to detect tennis balls that are close to the ground or at low height. The ultrasonic sensors are installed on the four end faces of the mobile platform to detect nearby obstacles for emergency obstacle avoidance. The ball-collecting execution module includes a ball-collecting arm, a roller brush mechanism, a conveying channel, and a ball storage bin. The ball-collecting arm is installed at the front end of the mobile platform and is equipped with an electric push rod for driving extension and retraction adjustment. A servo motor is installed at the end of the ball-collecting arm for angle adjustment. The roller brush mechanism is installed between the outlet of the conveying channel and the ball storage bin. It uses the friction generated by rotation to roll in the tennis ball and convey it to the ball storage bin. The control module includes a main control unit, a data fusion unit, a navigation control unit, and an execution control unit. The main control unit coordinates data interaction and command issuance among the various units. The data fusion unit preprocesses, fuses, and verifies data from various sensors, and outputs tennis ball coordinates and obstacle information. The navigation control unit plans a path based on the map and positioning data and controls the movement of the device. The execution control unit controls the actions of the ball-collecting arm, the roller brush mechanism, and the moving platform. The power module, including the battery pack and charging management module, is used to supply power to each module.

2. The intelligent tennis ball automatic retrieval device based on multi-sensor fusion according to claim 1, characterized in that, The mobile platform includes a chassis, a drive unit, and a navigation unit. The drive unit includes a DC geared motor, wheels, and a reducer. The DC geared motor is connected to the wheels through the reducer. The navigation unit includes a GPS positioning module and an inertial measurement unit. The GPS positioning module obtains the absolute position, and the inertial measurement unit collects attitude angle and acceleration data. The two are fused to achieve positioning and navigation.

3. The intelligent tennis ball automatic retrieval device based on multi-sensor fusion according to claim 1, characterized in that, The visual sensing device includes a supplementary lighting module, which consists of LED beads arranged in a ring around the camera lens. The module detects ambient light using a light intensity sensor and automatically activates when the light intensity is below a set threshold. The brightness is adjusted via a PWM signal.

4. The intelligent tennis ball automatic retrieval device based on multi-sensor fusion according to claim 1, characterized in that, Infrared beam sensors are installed on the conveying channel to detect the tennis ball conveying status.

5. The intelligent tennis ball automatic retrieval device based on multi-sensor fusion according to claim 1, characterized in that, The ball storage chamber is equipped with a weighing module for detecting the weight of the stored balls; the top of the ball storage chamber is equipped with a breathable net, the side is provided with a transparent observation window, and the bottom is equipped with a detachable base plate.

6. The intelligent tennis ball automatic retrieval device based on multi-sensor fusion according to claim 1, characterized in that, It also includes a human-machine interaction module, which includes a touch screen and a buzzer. The touch screen is used to set parameters and display the device status, and the buzzer is used to issue a warning signal when the ball storage bin is full or malfunctions.

7. A method for automatically retrieving intelligent tennis balls based on multi-sensor fusion, applied to the device described in any one of claims 1-6, characterized in that, Includes the following steps: S1: Set the ball-collecting range, path mode parameters, and full-load threshold via the touch screen. The control module then activates each sensor and drive unit to perform a system self-test. S2: The lidar collects three-dimensional information of the field and combines it with GPS positioning data to build a field map. The initial ball-picking path is planned according to the map. Multiple sensors simultaneously collect environmental data. The data fusion unit fuses the data from lidar, vision, infrared and ultrasonic sensors, removes interference data, and obtains information on obstacle distribution and potential tennis ball positions. S3: The navigation control module controls the device to move autonomously according to the planned path. During the movement, the ultrasonic sensor detects nearby obstacles in real time. If an obstacle is detected, the path is adjusted to avoid the obstacle. At the same time, the visual sensor and infrared sensor continuously detect the tennis ball. After the visual sensor collects the image, it identifies the tennis ball through the target detection algorithm. Combined with the infrared sensor data, it confirms the position of the tennis ball close to the ground and transmits the coordinate information of the tennis ball to the main control unit. S4: The main control unit calculates the required extension length and angle of the ball-collecting arm based on the ball's coordinate information, controls the ball-collecting arm to adjust to the target position, and starts the roller brush mechanism to roll in the ball. The ball slides down the conveyor channel to the ball storage bin. If multiple balls are detected, they are picked up in order of distance from closest to furthest. S5: The weighing module in the ball storage bin collects the total weight of the stored balls in real time. The main control unit calculates the real-time number of stored balls based on the preset standard weight of a single tennis ball. When the number of stored balls reaches the preset full bin threshold, an early warning signal is issued, the ball picking operation is suspended, and the return path planning is initiated. S6: When the preset ball-collecting range is completed or the user manually stops, the device automatically returns to the starting position, shuts down all execution modules, and enters standby mode.

8. The method according to claim 7, characterized in that, The specific process of data fusion in step S2 includes: Data preprocessing: filtering and denoising the lidar point cloud, correcting distortion in the visual image, performing analog-to-digital conversion and filtering on the analog signal output from the infrared sensor, and removing outliers from the ultrasonic sensor data. Data fusion: Using the center of the tennis ball bounding box identified by the visual sensor as the initial coordinates, the corresponding three-dimensional coordinates detected by the LiDAR are used as observations, and fused through extended Kalman filtering to output the three-dimensional coordinates of the tennis ball; Data verification: If the infrared sensor detects a reflected signal at the corresponding coordinates and the ultrasonic sensor does not detect an obstacle, the coordinates are confirmed as a valid tennis ball position; otherwise, it is marked as a suspected target, and the control device moves to a closer distance for re-detection.

9. The method according to claim 7, characterized in that, The full capacity threshold mentioned in step S5 is the rated capacity value of the ball storage bin. When the total weight of the stored balls detected by the weighing module reaches the weight corresponding to this threshold, the main control unit controls the buzzer to sound an alarm, and at the same time the touch screen displays the full capacity prompt information. The device pauses the ball picking operation and plans a path back to the starting position.

10. The method according to claim 7, characterized in that, During the autonomous movement described in step S3, when the ultrasonic sensor detects that the distance to an obstacle is less than a first threshold, the mobile platform decelerates; when the detected distance to an obstacle is less than a second threshold, the mobile platform stops urgently and replans its path, where the first threshold is greater than the second threshold.

Images