Intelligent omni-directional ball recovery robot based on slam path planning and visual identification

Abstract

The invention discloses an intelligent omni-directional ball recycling robot based on slam path planning and visual identification, and belongs to the technical field of intelligent robotics.In the ball recycling robot, a moving chassis is combined with a camera shooting assembly to move to a ball picking position, and then balls on the ground are clamped through a ball grabbing mechanism; the ball grabbing mechanism is used for grabbing balls and overturning and placing the balls into the ball storage mechanism, the ball storage mechanism ascends to the ball storage position or descends to the ball receiving position through the lifting mechanism, automatic ball picking of large balls such as volleyballs can be achieved, the ball grabbing mechanism grabs, overturns and picks up the balls, the ball storage mechanism ascends and descends in the moving process, and the ball picking efficiency is improved.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent robots, in particular to an intelligent omnidirectional ball recovery robot based on slam path planning and visual recognition. Background Art

[0002] At present, most of the ball-picking robots on the market are for small balls such as table tennis, tennis, and golf balls, but there are no special ball-picking robots for medium and large balls such as volleyballs. Small balls are small in size and low in cost, and a large number of balls are used during training. Small balls are easy to pick up and store, so there are many mature products in this area on the market. However, few people are researching machines to collect large balls because they are too large.

[0003] With the in-depth implementation of the strategy of building a strong sports nation and the continuous outstanding achievements of the Chinese women's volleyball team, the participation and competitive level of volleyball have been continuously improved, various training activities have become increasingly frequent, and the use of volleyballs in training and competitions has increased significantly. In this context, the extensive use of volleyballs has led to the frequent need to pick up the balls. The traditional way of picking up the balls, which mainly relies on manual picking up, can no longer meet the needs of efficient and continuous training. This not only consumes the physical strength of the athletes, but also takes up precious training time. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent omnidirectional ball recovery robot based on slam path planning and visual recognition to solve the above problems, which can realize automatic ball picking of large balls such as volleyballs.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] An intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition, comprising a steerable and movable mobile chassis and a camera assembly arranged on the mobile chassis; and also comprising the following contents:

[0007] The ball catching mechanism includes a flip frame, a flip motor, a ball clamping frame and a ball clamping assembly capable of clamping a ball. The flip frame is arranged at the front end of the mobile chassis. The ball clamping frame is rotatably arranged on the flip frame, and the ball clamping frame can be flipped on the front and rear vertical planes. The flip motor is transmission-connected to the ball clamping frame. The ball clamping assembly is arranged on the ball clamping frame. The ball clamping assembly can be flipped between a ball clamping position and a ball releasing position under the drive of the flip motor. The ball clamping position is located in front of the mobile chassis, and the ball releasing position is located above the mobile chassis.

[0008] The lifting mechanism includes a lifting frame and a lifting assembly, the lifting frame is arranged at the rear end of the mobile chassis, and the lifting assembly is arranged on the lifting frame and the mobile chassis;

[0009] The ball storage mechanism includes a ball barrel member, a front plate member and a front ball pushing assembly, the ball barrel member is slidably connected to the lifting frame up and down, the ball barrel member is connected to a lifting assembly, and the ball barrel member can be lifted and lowered between a ball receiving position and a ball storing position under the drive of the lifting assembly, and the ball storing position is higher than the ball receiving position, and the ball receiving position is below the ball releasing position, the ball barrel member is extended and arranged in the front and rear directions, and a slide extending in the front and rear directions is provided at the bottom of the inner wall of the ball barrel member, and the slide protrudes from the front end of the ball barrel member by a certain length to form a ball receiving portion, the front plate member is slidably connected to the front end of the ball barrel member front and back, and the front plate member is extended in the up and down directions and is arranged in front of the ball receiving portion, the front plate member is connected to a front pushing assembly, and the front plate member can be moved backward a certain distance and reset under the drive of the front push ball assembly.

[0010] It also includes a power supply and a controller for power supply, and the controller is electrically connected to the mobile chassis, the camera assembly, the flip motor, the ball clamping assembly, the lifting assembly, and the front ball pushing assembly. The controller is connected to the travel motor of the mobile chassis to drive the omnidirectional wheels to travel. The controller is connected to the lifting motor of the lifting assembly to drive the ball storage mechanism to rise or fall. The front ball pushing assembly is a ball pushing cylinder, which is equipped with an air source, an air circuit, and an electromagnetic valve, etc. It can also be implemented by using an electric push rod with reference to existing technologies. All of them are existing technologies and will not be described one by one here.

[0011] The ball clamping assembly of this application is explained by taking the opening and closing clamping structure as an example. Other replacement schemes include adsorption, etc., which are all existing technologies and will not be described in detail here. The ball clamping assembly includes an opening and closing driver, a claw arm and a face claw. The two claw arms are respectively arranged on the left and right sides of the ball clamping frame through their rear ends. The two claw arms are connected to the opening and closing driver to perform opening and closing actions under the drive of the opening and closing driver. The front ends of the two claw arms are respectively provided with face claws to clamp the ball. Among them, the opening and closing driver is an opening and closing cylinder component, which is equipped with an air source, an air circuit and an electromagnetic valve, etc. It can also be implemented by a motor with reference to the existing technology. They are all existing technologies and will not be described in detail here.

[0012] As mentioned above, the chassis is moved to the ball picking position in combination with the camera assembly, and then the ball on the ground is clamped by the ball grabbing mechanism, and the ball is turned over and placed into the ball storage mechanism. The ball storage mechanism rises to the ball storage position or descends to the ball receiving position through the lifting mechanism, so that automatic ball picking of large balls such as volleyballs can be achieved.

[0013] Based on the above scheme, in an improved scheme, in the ball recovery robot, the front and rear ends of the bottom of the inner side of the face claw are respectively configured with lower rollers, and the lower rollers can be arranged to roll inward, and the front and rear ends of the top of the inner side of the face claw are respectively configured with upper rollers, and the upper rollers can be arranged to roll inward. In this way, adding four rollers can make the deviated ball return to the center point, which plays a role in correcting the ball position deviation.

[0014] Based on the above scheme, in an improved scheme, in the ball recovery robot, the lower roller is a bull's eye wheel, and when the face claw is in a clamping state at the ball clamping position, the four bull's eye wheels can lift the ball so that there is a certain gap between the ball and the ground. In this way, at the moment of clamping the ball, the ball is a certain gap from the ground, and the mobile chassis is immediately controlled to start moving, and the ball will not be dragged to the ground, and then the mobile chassis will perform operations such as flipping and releasing the ball during the movement of the mobile chassis, reducing the time waiting for the flipped ball to leave the ground and improving the efficiency of picking up the ball.

[0015] Based on the above scheme, in an improved scheme, in the ball recovery robot, the ball storage mechanism also includes a rear plate and a switch driver, the I end of the rear plate can be rotatably arranged at the side edge of the rear end of the ball cylinder, the I end of the rear plate is connected to the switch driver, and the II end of the rear plate can be driven by the switch driver to rotate between the center and the outer edge of the ball cylinder. Among them, the switch driver is a switch servo, and can also be implemented by pushing and pulling with a motor or an electric push rod with reference to existing technologies, which are all existing technologies and will not be described one by one here. In this way, a ball cylinder opening switch structure is formed to ensure that the opening is in an open or closed state.

[0016] Based on the above scheme, in an improved scheme, in the ball recovery robot, the ball storage mechanism also includes a rear push frame and a rear push ball assembly, the rear push frame is slidably connected to the side edge of the rear end of the ball cylinder member, the I end of the rear plate member is rotatably arranged at the rear of the rear push frame with the side edge of the rear end face of the rear push frame as the center, the switch driver is arranged on the rear push frame, the rear push frame is connected with the rear push assembly, and the rear push frame can move forward a certain distance and reset under the drive of the rear push ball assembly. Among them, the rear push ball assembly is a ball push cylinder member, which is equipped with an air source, an air circuit and an electromagnetic valve, etc., and can also be realized by using an electric push rod with reference to existing technologies, which are all existing technologies and will not be described one by one here. In this way, a ball push structure is formed so that the ball is pushed forward to the ball receiving portion, so that people can directly take the ball from it.

[0017] Based on the above scheme, in an improved scheme, in the ball recovery robot, the rear push frame is equipped with an L-shaped limiter, the I end of the limiter is arranged on the rear end surface of the rear push frame, the II end of the limiter protrudes in the circumferential direction, the limiter and the rear push frame form a U-shaped groove, and the rear plate is inserted into the U-shaped groove when the II end of the rear plate rotates to the center position of the ball cylinder. In this way, the limiter plays a role and improves the fixing stability.

[0018] Based on the above scheme, in an improved scheme, in the ball recovery robot, the ball storage mechanism also includes a pulley track, two pulley tracks are arranged relatively on the left and right sides of the slideway, the pulley track includes a plurality of pulleys, the plurality of pulleys are evenly arranged along the front-back direction, the pulleys can be arranged to roll forward and backward, and a certain smooth spacing is spaced between two adjacent pulleys, so that the ball can stop between a pair of two adjacent pulleys. In this way, the ball can be pushed to move smoothly in the ball barrel under the action of external force, and the ball can stay at a certain position in a static state, avoiding the ball from sliding due to a slight movement change of the mobile chassis, thereby improving the overall stability.

[0019] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0020] The intelligent all-round ball recovery robot based on SLAM path planning and visual recognition of the present invention is combined with a D435i depth camera and a YOLOv8 target detection algorithm. The efficient intelligent ball picking robot moves to the ball picking position by combining a mobile chassis with a camera assembly, and then clamps the ball on the ground through a ball grabbing mechanism, and flips the ball and places it into a ball storage mechanism. The ball storage mechanism rises to a ball storage position or descends to a ball receiving position through a lifting mechanism, and can realize automatic ball picking of large balls such as volleyballs. The ball grabbing mechanism grabs, flips and picks up, and the ball storage mechanism rises and falls during the moving process, thereby improving the ball picking efficiency. The present invention can solve the problem of inefficient manual ball picking in volleyball courts, improve training efficiency, ensure that athletes have more time and energy to invest in skill improvement and physical recovery, and provide intelligent solutions for sports training at the same time, and promote the modernization of sports facility management. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural schematic diagram of the ball recovery robot of the present invention.

[0022] Figure 2 yes Figure 1 Mobile chassis layout diagram.

[0023] Figure 3 yes Figure 1 Schematic diagram of the omnidirectional wheel structure.

[0024] Figure 4 yes Figure 1 Top view of the ball catching mechanism.

[0025] Figure 5 yes Figure 1 Schematic diagram of the lifting mechanism structure.

[0026] Figure 6 yes Figure 1 Schematic diagram of the ball storage mechanism structure.

[0027] In the attached drawings, 1. mobile chassis, 2. ball grabbing mechanism, 3. lifting mechanism, 4. ball storage mechanism, 5. camera assembly. DETAILED DESCRIPTION

[0028] The specific implementation of the invention is further described below with reference to the accompanying drawings.

[0029] As mentioned above, the intelligent all-round ball recovery robot based on slam path planning and visual recognition of the present application has a basic solution and an improved solution. For example, a roller is added to the basic solution to correct the ball position deviation feature, and a feature of clamping and lifting the ball to a certain distance from the ground is added to the basic solution, etc. The following will take the best examples of all feature combinations as examples for further supplementary explanation.

[0030] like Figure 1-Figure 6 As shown, the intelligent all-round ball recovery robot of the present application includes: an independently suspended omnidirectional wheel chassis (mobile chassis) 1, a ball grabbing mechanism 2 arranged on the independently suspended omnidirectional wheel chassis 1, a double-rope synchronous lifting mechanism 3, a ball storage mechanism 4 and a camera assembly (visual control part) 5 arranged on the lifting mechanism, and a control system (intelligent control system) respectively connected to the independently suspended omnidirectional wheel chassis 1, the ball grabbing mechanism 2, the lifting mechanism 3 and the ball storage mechanism 4.

[0031] The independent suspension omni-directional wheel chassis 1 includes a chassis frame 11, an independent suspension system 13 and control system hardware. The control system hardware includes: a small computer (such as a PLC controller) 121, a mid360 laser radar 122, a lower computer main control board 123, a gas circuit solenoid valve 124, a circuit power supply (power supply) 125, an emergency stop switch 126, and a binocular camera 127.

[0032] Specifically, the chassis frame is mainly composed of fiberglass board, aluminum square tube, and printed parts. The chassis aluminum square tube is a square aluminum square tube with a thickness of 1.5mm and a cross-section of 20mm×20mm. In order to protect the wheels and provide impact resistance, 320mm long aluminum square tubes are added around as anti-collision bars, and printed parts are embedded to protect the wheels from impact. A 220mm long aluminum square tube is installed at the front end for installing the ball grabbing mechanism, and 4mm thick fiberglass boards are left on both sides to place electronic components.

[0033] Preferably, the chassis body is in a tic-tac-toe shape, which is firm and stable as a whole and can withstand the vertical force transmitted from the upper mechanism. Considering that the machine is a fully automatic machine, uncontrollable problems may occur, and it is necessary to improve the machine's anti-collision ability. Anti-collision bars are installed around the chassis. At the same time, the anti-collision bars can also limit the aluminum square tubes extending out of the tic-tac-toe shape of the main body to prevent them from bending and deforming. Fiberglass boards are used to connect different aluminum square tubes to support weight and reduce deformation of the mechanism. At the same time, fiberglass boards have the advantages of high temperature resistance, flame retardancy, corrosion resistance, high tensile strength, good electrical insulation, and good processing performance. It can not only reduce the noise generated by the machine during movement, but also increase its service life.

[0034] Preferably, in order to move flexibly and stably in a small space and adapt to various complex environments, four omnidirectional wheels 131 are preferably selected, and a first drive motor (travel motor) 132 connected thereto is provided at each omnidirectional wheel 131, so as to accurately control the travel direction and travel speed of the independently suspended omnidirectional wheel chassis 1. Of course, in other embodiments, considering the size of the ball recovery robot or other practical application requirements, the number of different omnidirectional wheels can also be other, and the first drive motor adapted thereto can also be adjusted in quantity and position according to the number and orientation of the omnidirectional wheels, which will not be described one by one here.

[0035] Preferably, in order to keep the robot stable during driving and reduce the overall impact of abnormal road conditions, the suspension system is increased to increase the shock absorption function of the wheel system. The suspension is connected by the central warehouse 133 as the overall connection center, and the front end of the central warehouse 133 is connected to the first drive motor 132. In order to fix the suspension stroke in the longitudinal direction, sliders 134 are fixed on the left and right sides of the central warehouse 133, which are connected to the slide rails 135 fixed on the outer bracket 137. In order to make the wheel set and the frame elastically connected and alleviate and suppress the impact of uneven roads on the vehicle body, a shock absorbing spring 136 is set at the upper end of the central warehouse 133, and a top plate 138 is installed above the shock absorbing spring 136, and the top plate 138 is fixed on the outer bracket 137. According to the fourth strength theory sqrt(σ1^2+σ2^2+σ3^2-σ1σ2-σ2σ3-σ3σ1)<[σ] analysis, glass fiber board is selected as the top plate material.

[0036] The ball grabbing mechanism 2 comprises a clamping claw (surface claw) 21 , a roller 22 , a claw arm 23 , a cylinder (opening and closing cylinder member) 24 , a second driving motor 25 , a meshing gear 26 , and a clamping claw bracket (ball clamping bracket) 27 .

[0037] Preferably, the clamping jaw 21 is connected to the clamping jaw 23, and the clamping jaw 23 is connected to the cylinder 24. The cylinder 24 can move the clamping jaw 23 in the horizontal direction to provide opening and closing power for the clamping jaw. The clamping jaw moves inward to close and moves outward to open, so as to control the clamping jaw to take or release the ball.

[0038] Preferably, the diameter of the volleyball is 200mm to 210mm. In order to enable the clamp 21 to firmly clamp the ball, the clamp opening and closing stroke is 300mm. The clamp 21 adopts a cylindrical arc or spherical arc with a radius of 100mm to better fit the spherical surface to prevent the ball from leaving the grasping range due to a small deviation from the center point.

[0039] Preferably, a clamping bracket 27 is provided above the cylinder 24, and the clamping bracket 27 is rotatably arranged through a fixed axis. One end of the clamping bracket 27 is connected to the meshing gear 26, and the second driving motor (flipping motor) 25 transmits the torque to the clamping bracket 27 through the meshing gear, so that the front end clamping part can rotate along the fixed axis, and the clamped ball is transported to the interior of the robot and sent to the ball storage mechanism 4.

[0040] Preferably, in order to improve the success rate of taking the ball, four rollers are added at the four corners of the clamping jaw 21 to make the deviated ball return to the center point, thereby correcting the ball position deviation.

[0041] The lifting components of the lifting mechanism 3 are equipped with a lifting motor and a steel cable, etc. The lifting frame is composed of four 20*20 aluminum square tubes with a wall thickness of 1mm and a height of 500mm as columns 31, which are fixed in pairs at the front and back through aluminum square tubes 34 of the same model with a length of 186mm and glass fiber board connectors. A slide rail 32 is fixed on the column aluminum square tube. There is a fixed pulley 33 on the left and right sides of the upper end of the lifting mechanism 3, a left pulley group 35 with a longitudinal and a transverse fixed pulley on the lower left, a longitudinal fixed pulley 37 on the lower right, and a wire collection groove 38 connected to the third drive motor (lifting motor) 39.

[0042] Preferably, four aluminum square tube columns 31 with a length of 560 mm are erected vertically to carry the slide rails 32 to limit the movement trajectory, and guide rail limiters are added at both ends of the guide rails 32 to prevent their displacement and avoid excessive movement of the slider to cause separation from the slide rail. Two aluminum square tubes 34 with a length of 186 mm are used to fix the 560 mm aluminum square tube columns 31 in groups of two through fiberglass boards to prevent inward bending and deformation, thereby improving overall stability.

[0043] Preferably, the lifting mechanism 3 is operated by two steel cables driving the ball storage mechanism 4 to move, one end of the two steel cables are fixed together on the wire taking-up groove 38, and the other end bypasses the movable pulley assembly to be connected to the ball storage mechanism, wherein, the side close to the wire taking-up groove bypasses the movable pulley assembly and is directly connected to the wire taking-up groove, and the side away from the wire taking-up groove passes through the fixed pulley to bypass the running track of the ball storage mechanism and is connected to the wire taking-up groove 38, and the two ends are synchronously pulled up by the third drive motor 39 to avoid tilting and causing the slider to jam due to different lifting speeds on both sides.

[0044] Preferably, in order to finally gather the steel cables at both ends into the wire-collecting groove 38, an array of pulley blocks is used to change the direction of the steel cables gathering, wherein the fixed pulley block 33 directs the steel cables downward, and the pulley 37 at the right end directs the steel cables toward the wire-collecting groove 38, and the fixed pulley block 35 at the left end first changes the direction of the steel cables to the horizontal direction by the longitudinal fixed pulley, and then guides the direction of the steel cables to the wire-collecting groove 38 by the horizontal fixed pulley.

[0045] The ball storage mechanism 4 includes a ball cylinder, a front plate and a front ball push assembly. In order to allow the ball to pass smoothly in the cylinder and reduce unnecessary displacement, the ball cylinder as a whole is a 250mm cylindrical frame structure, which is horizontally mounted on the lifting mechanism 3. An SMC cylinder 412 with a stroke of 200mm is fixed above and below the cylinder (ball cylinder). The ball push plate (front plate) 411 and the ball baffle plate (rear plate) 424 are both connected to the carbon fiber guide rod 413 on both sides. The carbon fiber guide rod is connected to the ball storage cylinder (ball cylinder) through a linear bearing. The ball baffle plate 424 at the rear of the cylinder is connected to the fourth drive servo (switch servo) 421. A slide is provided in the ball storage cylinder. The slide can be a semi-cylindrical structure or two guide rails. The ball storage mechanism as a whole is mainly made of fiberglass boards, and the joints of the boards are reinforced with six-sided nuts.

[0046] Preferably, in order to enable the ball storage mechanism 4 to store two balls at a time to reduce the number of round trips of the machine, a cylinder (front ball pushing cylinder) 412 is provided at the bottom of the ball storage mechanism 4, and a retractable ball pushing plate 411 moves horizontally to push the first stored ball into the ball storage tube, and at the same time pushes two balls out at one time when unloading the balls.

[0047] Preferably, in order to make the ball move smoothly in the ball storage barrel 2, a pulley track is arranged on the inner wall of the barrel, including a guide rail 423 and a pulley (small roller), and a row of small rollers are installed on the guide rail 423 so that the ball will not get stuck during rolling. In order to prevent the ball from falling during transportation, the barrel mouth is in a closed state. When it is in the target position, the ball storage mechanism 4 extends the barrel mouth out of the limit part through the cylinder (rear ball pushing cylinder member) 422, and the fourth driving motor 421 drives the ball blocking plate 424 to rotate, opens the barrel mouth, and the cylinder 413 drives the ball pushing plate 411 to push the ball out. It is also possible to know the intention or instruction of the person through voice control, push forward and push the ball to the ball receiving part, so that the person can directly pick up the ball without going to the storage point to get it.

[0048] As mentioned above, the robot system generally uses CAN communication and UART communication to achieve stable control of the motor by the lower computer and decision-making control of the upper computer to control the operation of the lower computer, realize the movement of the chassis and the grabbing of the upper layer, and ensure the normal operation of the system. The IDE environment of the electronic control system uses STM32CUBEIDE, and the compiler uses CLION. STM32CubeIDE is an integrated development environment (IDE) provided by STMicroelectronics, which is specially used to develop embedded applications based on STM32 microcontrollers. It integrates STM32CubeMX (a configuration and initialization code generation tool) and STM32CubeHAL (hardware abstraction layer) and is built on Eclipse. STM32CubeIDE supports multiple debuggers and provides rich functions such as code editing, compilation, debugging, simulation, etc., making the development of STM32 microcontroller applications more efficient. Using the STM32F42.13.0 chip package from STMicroelectronics, equipped with the 10.0.1 version of FreeRTOS, accurate, fast and senseless scheduling at different task levels can be achieved.

[0049] NVIDIA JetsonNano is a small AI computer that can bring amazing new capabilities to millions of energy-efficient small AI systems. The JetsonNano module is only 70mm x 45mm in size, smaller than a credit card, but its performance is excellent and can play an important role in deploying AI at the edge. It has a 40PIN GPIO expansion port that can communicate with the lower computer through pins; a Micro USB interface that can communicate with the lower computer through anonymous data transmission; a Gigabit Ethernet interface that can be connected to the MID360 LiDAR to obtain real-time point cloud data of the venue; and a MIPI CSI camera interface that can be connected to the d435i depth camera to complete the recognition and coordinate acquisition of the volleyball.

[0050] Ubuntu is used as the operating system, which is known for its high stability and security. It can run for a long time without restarting, which has certain advantages for long-term navigation needs. It has lower requirements for hardware resources and can run smoothly on hardware with lower configurations, which also shows that it has certain control over costs. The robot operating system ROS1 is chosen as the framework. With its openness, flexibility and strong community support, it is an ideal choice for implementing complex robot systems. Through the precise state machine technology based on the ROS operating system and the message passing mechanism of ROS1, it can efficiently process the data streams from the visual sensor and lidar and analyze the robot's surrounding environment, realize path planning and obstacle avoidance selection in real time, and complete the accurate recovery of the volleyball.

[0051] The YOLOv5 model is used as the recognition framework, which is known for its high-speed operation. It can achieve extremely fast inference speed while maintaining high detection accuracy, which is crucial for real-time applications of autonomous driving and robot visual navigation.

[0052] The YOLOv5 algorithm is deployed in the ROS operating system, and a model training is created for the volleyball collection data set. After the model training is performed on the PC, it is deployed on the Jetson nano platform. The specific implementation process is as follows:

[0053] 1. Collect the dataset. In order to make the object detection as accurate as possible, we considered a variety of environments and tightened the dataset extensively. We used high-resolution cameras to ensure that the image quality of the training set was good enough; we ensured that the video contained balls from different angles, lighting conditions, and background environments, and that the color thresholds, sizes, and stacking of the balls were as diverse as possible to improve the generalization ability of the model; we also used static shooting and dynamic recording, including stillness, movement, and rotation, to simulate the different states of the ball in actual applications.

[0054] 2. Label the dataset. Select the appropriate dataset from all the collected datasets, and use labelme to label the targets in the dataset images. After labeling, convert the json format to YOLO format for subsequent model training.

[0055] 3. The first training obtains the initial weights. Divide the labeled data set into training set and test set, set the corresponding storage path and configuration, and run the train.py file in the YOLOv5 project. After the training is completed, select the best result best.pt as the initial training weight.

[0056] 4. Combine the training results with the stereo camera. Combine the YOLOv5 target detection model with the stereo camera (using the Intel series D435i). After identifying the image position of the target in the pixel coordinate system, the depth coordinates can be output through the stereo camera PNP solution. Then, the depth coordinates are converted into three-dimensional coordinates in the real world through solution. Not only can the distance relative to the target be obtained, but it can also be combined with the lidar.

[0057] 5. Evaluate the results and improve the model. Perform multiple tests to determine the direction based on the actual performance of the model. Improve the target detection model by adjusting the training set to achieve more accurate recognition accuracy.

[0058] By taking a variety of images of "obstacles" such as pedestrians, vehicles, buildings, etc. through cameras or network resources, the model can recognize the corresponding objects in real time through a certain amount of training.

[0059] SLAM technology refers to the robot creating a map in a completely unknown environment under the condition that its own position is uncertain, and using the map for autonomous positioning and navigation. During the movement, the robot locates its own position and posture through repeatedly observed environmental features, and then builds an incremental map of the surrounding environment based on its own position, thereby achieving the purpose of simultaneous positioning and map construction. SLAM mainly includes the following modules:

[0060] Data collection: Obtain environmental data.

[0061] Visual odometry: Obtain feature points of the surrounding environment through visual sensors and calculate the robot's posture and position.

[0062] Back-end optimization: By optimizing the algorithm, data from different sensors are integrated to improve the accuracy of positioning and map construction.

[0063] Mapping: Build a map of the environment based on the robot's position and posture.

[0064] Loop closure detection: Optimize and correct the map by identifying places the robot has visited before.

[0065] By using multi-sensor fusion SLAM technology, the data obtained from the LIDAR sensor in MID360 and the IMU odometer built into MID360 can be combined to obtain a more robust and accurate position of the robot on the volleyball court.

[0066] In the SLAM path planning system, the mature open source framework move_base is used for path planning and obstacle avoidance. The Gmapping algorithm is deployed to realize the construction of the indoor volleyball court, the positioning of the robot, and global path planning. The DWA algorithm is deployed and improved to realize the planning of the robot's local path. The two algorithms work together to enable the robot to navigate autonomously in the indoor volleyball court.

[0067] After starting the camera node, the node will intercept the video image stream captured by the current camera, continuously intercept and publish topics in the form of data frames; after subscribing to the image, the YOLOv5 model is used to identify the specific position of the object in the image, and the coordinates obtained by the binocular depth camera are published to the relevant topic. The state machine fits it with the point cloud coordinate system generated by the lidar to determine the exact position of the volleyball.

[0068] Since objects may be in motion, and objects with changing coordinates at any time may affect the operation of automatic planning, the extended Kalman filter is used to estimate the movement of the object and predict the position in advance. By measuring the object's motion state and position offset, correcting the timestamp of the gyroscope and lens data frame, and using the extended Kalman filter algorithm for state estimation and prediction, the prediction of the object state and distance compensation are achieved, and after analyzing the relevant data, the analysis information and the corresponding item information are delivered to the automatic planning.

[0069] In the ball-catching mechanism, when the robot moves to the front of the ball, the motor drives the gear to work, and the gear drives the aluminum square tube to rotate. The claw is connected to the aluminum square tube, so that the claw rotates to an angle that can place the ball in the claw, and then the cylinder is closed to speed up the retrieval of the ball. Finally, the motor rotates the claw back to the initial position, and the claw is opened to allow the ball to fall into the temporary storage area of ​​the placement mechanism.

[0070] In the lifting mechanism, the motor drives the take-up trough to rotate and retract the steel cable. The steel cable is connected to the ball storage mechanism and is lifted during the retraction process. During the lifting process, the slide rails and sliders on the four aluminum square tubes limit the displacement of its upward movement. The motor drives the take-up trough to reverse and release the steel cable. The elastic rope under the ball storage mechanism is connected to the chassis. The ball storage mechanism moves downward under the action of its own gravity and the elastic force of the elastic rope.

[0071] In the ball storage mechanism, the cylinder drives the ball pushing plate and the ball blocking plate to extend and retract, thereby controlling the balls to enter and exit the ball storage cylinder.

[0072] As mentioned above, the lifting of the gripper adopts the RoboMaster M2006 P36 DC brushless reduction motor. The M2006 P36 motor is a three-phase permanent magnet DC brushless motor with the characteristics of high speed, small size, high power and high reliability. It also uses PID position control angle and interpolation to limit the movement speed to achieve stable and precise control of the position.

[0073] The gripping of the gripper adopts solenoid valve to control the cylinder. The opening and closing of the gripper can be controlled by only giving high and low levels. It is simple and easy to use.

[0074] The lifting device also uses the M2006 P36 brushless DC motor and adopts the same control strategy as the lifting of the gripper. The launching device is also the same as the gripper opening and closing control, and uses a solenoid valve to control the cylinder, which has a better launching effect.

[0075] It should be pointed out that the examples of the above-mentioned embodiments can be preferably combined with one or more of them according to actual needs, and multiple examples use a set of drawings to illustrate the combined technical features, which will not be described one by one here.

[0076] It should be noted that the orientation or position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0077] The above descriptions are detailed descriptions and illustrations of the preferred embodiments of the present invention, but these descriptions are not intended to limit the scope of protection required by the present invention. All equivalent changes or modified modifications accomplished under the technical teachings suggested by the present invention should fall within the scope of patent protection covered by the present invention.

Claims

1. An intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition, comprising a steerable mobile chassis and a camera assembly arranged on the mobile chassis; characterized in that: Also included are the following: A ball-catching mechanism, the ball-catching mechanism comprising a flip frame, a flip motor, a ball-clamping frame and a ball-clamping assembly capable of clamping a ball, the flip frame being arranged at the front end of the mobile chassis, the ball-clamping frame being rotatably arranged on the flip frame, and the ball-clamping frame being capable of flipping on the front and rear vertical planes, the flip motor being transmission-connected to the ball-clamping frame, the ball-clamping assembly being arranged on the ball-clamping frame, and the ball-clamping assembly being capable of flipping between a ball-clamping position and a ball-releasing position driven by the flip motor, and the ball-clamping position being in front of the mobile chassis, and the ball-releasing position being above the mobile chassis; A lifting mechanism, the lifting mechanism comprising a lifting frame and a lifting assembly, the lifting frame is arranged at the rear end of the mobile chassis, and the lifting assembly is arranged on the lifting frame and the mobile chassis; The ball storage mechanism comprises a ball barrel member, a front plate member and a front ball pushing assembly, the ball barrel member is slidably connected to the lifting frame up and down, the ball barrel member is connected to a lifting assembly, and the ball barrel member can be lifted and lowered between a ball receiving position and a ball storing position under the drive of the lifting assembly, and the ball storing position is higher than the ball receiving position, and the ball receiving position is below the ball releasing position, the ball barrel member is extended and arranged along the front and rear directions, and a slide extending along the front and rear directions is provided at the bottom of the inner wall of the ball barrel member, and the slide protrudes from the front end of the ball barrel member by a certain length to form a ball receiving portion, the front plate member is slidably connected to the front end of the ball barrel member front and back, and the front plate member is extended and arranged in front of the ball receiving portion along the up and down directions, the front plate member is connected to a front pushing assembly, and the front plate member can be moved backward a certain distance and reset under the drive of the front pushing ball assembly.

2. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 1 is characterized in that: The ball clamping assembly includes an opening and closing driver, a claw arm and a surface claw. The two claw arms are respectively arranged on the left and right sides of the ball clamping frame through their rear ends. The two claw arms are connected to the opening and closing driver to perform opening and closing actions under the drive of the opening and closing driver. The front ends of the two claw arms are respectively provided with surface claws to clamp the ball; wherein the opening and closing driver is an opening and closing cylinder component.

3. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 2 is characterized in that: The front and rear ends of the bottom of the inner side of the noodle claw are respectively provided with lower rollers, and the lower rollers can be arranged to roll inwardly, and the front and rear ends of the top of the inner side of the noodle claw are respectively provided with upper rollers, and the upper rollers can be arranged to roll inwardly.

4. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 3 is characterized in that: The lower rollers are bull's eye wheels. When the face claws are in a clamping state at the ball clamping position, the four bull's eye wheels can hold up the ball so that there is a certain gap between the ball and the ground.

5. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 1, characterized in that: The ball storage mechanism also includes a rear plate and a switch driver, wherein the I end of the rear plate can be rotatably arranged at the side edge of the rear end of the ball cylinder, the I end of the rear plate is transmission-connected to the switch driver, and the II end of the rear plate can be rotated between the center and the outer edge of the ball cylinder under the drive of the switch driver; wherein the switch driver is a switch servo.

6. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 5 is characterized in that: The ball storage mechanism also includes a rear push frame and a rear push ball assembly. The rear push frame is slidably connected to the side edge of the rear end of the ball cylinder member. The I end of the rear plate member is rotatably arranged behind the rear push frame with the side edge of the rear end surface of the rear push frame as the center. The switch driver is arranged on the rear push frame. The rear push frame is connected to the rear push assembly, and the rear push frame can move forward a certain distance and reset under the drive of the rear push ball assembly; wherein the front push ball assembly and the rear push ball assembly are both ball push cylinder members.

7. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 6, characterized in that: The rear push frame is equipped with an L-shaped limiting part, the I end of the limiting part is arranged on the rear end surface of the rear push frame, the II end of the limiting part protrudes in the circumferential direction, the limiting part and the rear push frame form a U-shaped groove, and the rear plate is inserted into the U-shaped groove when its II end is rotated to the center position of the ball cylinder.

8. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 1, characterized in that: The ball storage mechanism also includes a pulley rail, two pulley rails are arranged relatively on the left and right sides of the slideway, the pulley rails include a plurality of pulleys, the plurality of pulleys are evenly arranged along the front-to-back direction, the pulleys can be arranged to roll forward and backward, and there is a certain smooth spacing between two adjacent pulleys so that the ball can stop between a pair of adjacent pulleys.

9. The intelligent omnidirectional ball recovery robot based on SLAM path planning and visual recognition according to claim 1, characterized in that: It also includes a power supply and a controller, and the controller is electrically connected to the mobile chassis, the camera assembly, the flip motor, the ball clamping assembly, the lifting assembly, and the front ball pushing assembly.

Images