Ball catching control method of tennis robot and tennis robot

Abstract

The invention is suitable for the technical field of tennis robots, and provides a ball receiving control method of a tennis robot and the tennis robot. Wherein the tennis robot comprises a tennis robot body, a moving device, a control device and a ball receiving assembly, the moving device is arranged at the bottom of the tennis robot body, and the control device is connected with the moving device; the ball receiving control method of the tennis robot comprises the steps that the control device controls the moving device to move the tennis robot to the ball receiving position of a to-be-received tennis ball, so that the tennis robot receives the to-be-received tennis ball through a ball receiving assembly of the tennis robot at the ball receiving position. According to the embodiment of the invention, the tennis robot can accurately receive the tennis ball, the cyclic utilization of the tennis ball in the playing process is realized, and the playing experience is improved.

Description

[0001] This application is a divisional application of the Chinese patent application submitted to the China Patent Office on December 13, 2024, with application number 202411836926.4 and application name "Tennis Robot and Control Method". Technical Field

[0002] The present application relates to the technical field of tennis robots, and in particular to a ball catching control method of a tennis robot and a tennis robot. Background Art

[0003] With the popularity of tennis, more and more tennis enthusiasts have emerged. Consequently, a tennis robot is provided to enable tennis enthusiasts to play tennis against each other. However, how to accurately catch the ball during the playing process is a difficult problem in this field.

[0004] Common tennis robots can only serve at a preset position on the court, but have difficulty catching the ball during the game. They can only pick up the tennis balls scattered on the court lights one by one after the game is over.

[0005] It is not conducive to the immediate recycling of tennis balls, resulting in a poor playing experience. Summary of the Invention

[0006] Based on this, it is necessary to provide a ball catching control method for a tennis robot and a tennis robot in response to the above technical problems.

[0007] In a first aspect, the present application provides a ball catching control method, wherein the tennis robot includes a tennis robot body, a moving device, a control device, and a ball catching assembly, wherein the moving device is arranged at the bottom of the tennis robot body, and the control device is connected to the moving device; the ball catching control method includes: the control device controls the moving device to move the tennis robot to a ball catching position for catching a tennis ball and catches the ball, so that the tennis robot receives the tennis ball to be caught through the ball catching assembly at the ball catching position.

[0008] In one embodiment, the control device controls the mobile device to move the tennis robot to a receiving position for receiving a tennis ball, including: based on the motion information of the tennis robot and an image containing the tennis ball to be received, the control device controls the mobile device to move the tennis robot to the receiving position.

[0009] In one of the embodiments, the control device controls the movement device to move the robot to the receiving position according to the motion information of the robot and the image containing the ball to be received, which comprises: receiving the motion information acquired by the robot at multiple time points and a set of images containing the ball to be received; obtaining a target predicted motion trajectory of the ball to be received according to the set of images and the motion information; and controlling the movement device to move the robot to the receiving position according to the target predicted motion trajectory.

[0010] In one of the embodiments, the set of images containing the ball comprises a first set of images containing the ball and a second set of images containing the ball; and the target predicted motion trajectory of the ball to be received is obtained according to the set of images and the motion information, which comprises: receiving the first set of images containing the ball acquired at multiple time points and the motion information acquired at corresponding time points; controlling the robot to move based on the first set of images and the corresponding motion information, so as to acquire the motion information and the second set of images containing the ball during the movement of the robot; and obtaining the target predicted motion trajectory according to the first set of images, the second set of images and the motion information.

[0011] In one of the embodiments, the robot is controlled to move based on the first set of images and the corresponding motion information, so as to acquire the motion information and the second set of images containing the ball during the movement of the robot, which comprises: obtaining first coordinates of the ball to be received in a first coordinate system at multiple time points according to the first set of images; the first coordinate system is a three-dimensional coordinate system with the robot as the origin; obtaining second coordinates of the ball to be received in a second coordinate system at multiple time points according to the first coordinates and the corresponding motion information; the second coordinate system is a three-dimensional coordinate system with any fixed point in the court as the origin; and controlling the robot to move according to the second coordinates corresponding to the ball to be received at multiple time points, so as to acquire the motion information and the second set of images containing the ball during the movement of the robot.

[0012] In one of the embodiments, the second tennis image set comprises images captured at multiple time points; and the obtaining the target predicted movement trajectory according to the first tennis image set, the second tennis image set and the movement information comprises: obtaining a first predicted movement trajectory according to multiple second coordinates corresponding to the first tennis image set; calibrating the first predicted movement trajectory according to an image captured at an initial time point and movement information at the initial time point to obtain a second predicted movement trajectory corresponding to the initial time point; the initial time point being the first time point in the multiple time points during the movement of the tennis robot; and successively calibrating a predicted movement trajectory corresponding to a previous time point according to an image captured at a remaining time point and movement information at the remaining time point to obtain a target predicted movement trajectory corresponding to a final time point; the remaining time point being a time point other than the initial time point and the final time point in the multiple time points during the movement of the tennis robot; and the final time point being the last time point in the multiple time points during the movement of the tennis robot.

[0013] In one of the embodiments, the tennis robot further comprises a ball serving assembly, and the control device is connected with the ball serving assembly; and the first tennis image set is captured in at least one of the following ways: the control device controls the ball serving assembly to serve the tennis ball, at this time the moving device does not move, and the first tennis image set is captured at a position where the tennis ball is served; or the control device controls the ball serving assembly to serve the tennis ball, and controls the moving device to move to a preset original position, and the first tennis image set is captured during the movement.

[0014] In one of the embodiments, the tennis robot further comprises a camera assembly, and the camera assembly is fixed to the main body of the tennis robot; and the camera assembly comprises two visible light cameras, and the two visible light cameras are used to capture images containing the tennis ball.

[0015] In one of the embodiments, the two visible light cameras are arranged at a distance greater than or equal to 12 cm in a width direction of the main body of the tennis robot, and the width direction is a dimension direction of the main body of the tennis robot in a direction parallel to the ground.

[0016] In one of the embodiments, an angle between an optical axis of each of the two visible light cameras and the ground is less than or equal to 30°.

[0017] In one of the embodiments, a longitudinal angle between the optical axes of the two visible light cameras in space is less than or equal to 15°.

[0018] In one of the embodiments, the tennis robot further comprises a ball serving assembly, and the control device is connected with the ball serving assembly;

[0019] In one of the embodiments, the tennis robot receives the to-be-received tennis ball at the receiving position by the receiving assembly of the tennis robot, including: while the tennis robot receives the to-be-received tennis ball at the receiving position by the receiving assembly of the tennis robot, the control device controls the serving assembly of the tennis robot to serve a second tennis ball.

[0020] In one of the embodiments, the control device controls the serving assembly of the tennis robot to serve a second tennis ball, including: the control device controls the orientation of the tennis robot facing the opponent of the tennis ball, and the second tennis ball is served to the position of the opponent on the court by the serving assembly.

[0021] In a second aspect, the application further provides a tennis robot, including a tennis robot body, a moving device, a control device and a receiving assembly, the moving device is arranged at the bottom of the tennis robot body, the control device is connected with the moving device, and the control device is used to implement the steps of the receiving control method according to any one of the first aspect.

[0022] In a third aspect, the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the receiving control method according to any one of the first aspect.

[0023] The above receiving control method and tennis robot can move the tennis robot to the receiving position of the to-be-received tennis ball by the control device controlling the moving device, so that the tennis robot receives the to-be-received tennis ball at the receiving position by the receiving assembly, so that the tennis robot can accurately receive the tennis ball, the tennis ball can be recycled in the playing process, and the playing experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0025] Figure 1 The structural schematic diagram of the tennis robot in an embodiment of the present application is shown in the figure.

[0026] Figure 2(a) is a schematic diagram of the longitudinal angle between the optical axes of two visible light cameras and the angle between the optical axes and the ground in an embodiment of the present application.

[0027] Figure 2(b) is a schematic diagram of the longitudinal angle range of two visible light cameras in space in an embodiment of the present application.

[0028] Figure 3 A schematic diagram of the lateral view angle range of two visible light cameras in an embodiment of the present application in space;

[0029] Figure 4 A schematic diagram of the angle between the view angle axes of two visible light cameras in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the flow of the control method in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of the flow of acquiring the target predicted motion trajectory of the to-be-received tennis ball according to the tennis ball image set and the motion information in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the flow of moving the moving device of the tennis robot based on the first tennis ball image set and the corresponding motion information to acquire the motion information in the moving process of the moving device and the second tennis ball image set in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of the flow of acquiring the target predicted motion trajectory of the to-be-received tennis ball according to the first tennis ball image set, the second tennis ball image set and the motion information in an embodiment of the present application;

[0034] Figure 9 A schematic diagram of the tennis robot in the moving process in an embodiment of the present application.

[0035] Explanation of the reference signs:

[0036] 110: tennis robot main body; 120: moving device; 131: one of the two visible light cameras; 132: the other of the two visible light cameras; 140: sensing component; 150: control device; 160: anti-shake firmware. DETAILED DESCRIPTION

[0037] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. In the following description, a large number of specific details are set forth in order to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the present application, so the present application is not limited to the specific embodiments disclosed below.

[0038] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the purpose of facilitating the description of the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0039] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified and limited.

[0040] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0041] During the process of creating the present application, the inventors also made the following research:

[0042] A plurality of visible light cameras are arranged at preset positions of the court to capture the trajectory of the tennis ball in real time during the playing process. The control device in the tennis robot controls the movement device to move to the predicted tennis ball catching position to catch the tennis ball based on the trajectory of the tennis ball.

[0043] However, the above scheme reduces the adaptability of the tennis robot. The tennis robot can only catch the ball at places where visible light cameras capable of data transmission with the control device in the tennis robot are provided.

[0044] Based on this, the inventors provide another solution, without changing the structure of the current tennis robot, only setting a visible light camera on the tennis robot, solving the limitation of the adaptability of the tennis robot caused by the setting position of the visible light camera. However, since the tennis robot needs to move constantly during the match to ensure timely reception of the tennis ball, that is, the position and orientation of the visible light camera on the tennis robot are changing at any time, even if the same position of the tennis ball is photographed, different images will be obtained, and thus the position of the tennis ball on the court cannot be obtained, thereby greatly increasing the difficulty of predicting the trajectory and landing point of the tennis ball.

[0045] Based on the above research, in one embodiment, the present application provides a tennis robot, referring to the accompanying Figure 1 , the accompanying Figure 1 structure schematic block diagram of the tennis robot in the embodiment is shown. The tennis robot in the embodiment includes a tennis robot body 110, a moving device 120, a camera assembly, a sensing assembly 140 and a control device 150.

[0046] The moving device 120 is arranged at the bottom of the tennis robot body 110; the camera assembly is fixed to the tennis robot body 110, and the camera assembly includes two visible light cameras (131 and 132); the sensing assembly 140 is used to obtain the motion information of the tennis robot; the control device 150 is connected with the moving device 120, the camera assembly and the sensing assembly 140, and is used to control the moving device 120 to move the tennis robot to a ball receiving position according to the motion information and the images containing the tennis ball to be received obtained by the two visible light cameras (131 and 132).

[0047] Among them, the visible light camera can also be understood as a monochrome camera, an R(Red, red) G(Green, green) B(Blue, blue) camera, without limitation.

[0048] The ball receiving position refers to the position in the court where the ball receiving assembly of the tennis robot can receive the tennis ball to be received.

[0049] In the embodiment, two visible light cameras (131 and 132) are arranged on the tennis robot, which can obtain images of the tennis ball observed at each position in the court under the driving of the moving device 120, and the real-time position of the tennis ball on the court can be estimated based on the images obtained by the two visible light cameras. The motion information of the tennis robot can be obtained based on the sensing assembly 140, and the position information of the tennis robot in the court can be obtained, therefore, the control device 150 can accurately control the moving device 120 to move to the ball receiving position of the tennis ball based on the images of the tennis ball and the motion information, so that the tennis robot can accurately receive the ball, realize the recycling of the tennis ball during the match, and improve the match experience.

[0050] In one embodiment, please refer to the accompanying drawings Figure 1 The distance d between the two visible light cameras (131 and 132) arranged in the width direction Y of the tennis robot body 110 is greater than or equal to 12 cm, and the width direction Y is the size direction of the tennis robot body 110 in the direction parallel to the ground.

[0051] Preferably, the distance d between the two visible light cameras (131 and 132) arranged in the width direction Y of the tennis robot can be greater than or equal to 20 cm, so as to make more accurate depth estimation of the ball to be received based on the images captured by the two visible light cameras (131 and 132).

[0052] In this embodiment, the distance d between the two visible light cameras (131 and 132) arranged in the width direction Y of the tennis robot is greater than or equal to 12 cm, which can ensure the field of view of the two visible light cameras and ensure that the images obtained by the two visible light cameras (131 and 132) can accurately estimate the real-time position of the ball on the court.

[0053] In one embodiment, please refer to FIG. 2(a), which shows the longitudinal angle between the optical axes of the two visible light cameras (131 and 132) and the angle between the optical axes and the ground; the angle between the optical axes of the two visible light cameras (131 and 132) and the ground is less than or equal to 30°.

[0054] For example, the angle between the optical axes of the two visible light cameras (131 and 132) and the ground can be 30°, 20°, 10°, 5°, etc., without being limited thereto.

[0055] In this embodiment, the angle between the optical axes of the two visible light cameras (131 and 132) and the ground is less than or equal to 30°, which can ensure that the two visible light cameras (131 and 132) can capture the ball to be received when the tennis robot is located at most positions on the court.

[0056] In one embodiment, the angle between the optical axes of the two visible light cameras in space is less than or equal to 30°.

[0057] Please refer to FIG. 2(b) and FIG. 2(c) Figure 3 and FIG. 2(d) Figure 4 FIG. 2(d) shows the schematic diagram of the transverse angle range of the two visible light cameras (131 and 132) in space in this embodiment; and Figure 3 FIG. 2(c) shows the schematic diagram of the angle between the optical axes of the two visible light cameras (131 and 132) in this embodiment. Among them, the transverse angle range refers to the angle range of the two visible light cameras (131 and 132) in the direction parallel to the ground. Figure 4 ​

[0058] Exemplarily, the angle between the axes of the two visible light cameras (131 and 132) in space can be 30°, 29°, 28°, 27°, 26°, 25°, 24°, 23°, 22°, 21°, 20°, 19°, 18°, 17°, 16°, 15°, 14°, 13°, 12°, 11°, 10°, 9°, 8°, 7°, 6°, 5°, 4°, 3°, 2°, 1°, etc., without being limited thereto.

[0059] In this embodiment, the angle between the axes of the two visible light cameras (131 and 132) in space is less than or equal to 30°, which can ensure that the two visible light cameras can simultaneously capture images of the tennis ball in the lateral direction, thereby improving the accuracy of the tennis robot in catching the ball.

[0060] In one embodiment, the longitudinal angle between the optical axes of the two visible light cameras in space is less than or equal to 15°.

[0061] Referring to FIG. 2(a) and FIG. 2(b), FIG. 2(b) shows a schematic diagram of the longitudinal angle range of the two visible light cameras (131 and 132) in space in this embodiment. The longitudinal angle between the optical axes of the two visible light cameras in space can be understood as the longitudinal angle range of the two visible light cameras (131 and 132) in space, which refers to the angle range of the two visible light cameras in the direction perpendicular to the ground.

[0062] Exemplarily, the longitudinal angle between the optical axes of the two visible light cameras (131 and 132) in space can be 15°, 14°, 13°, 12°, 11°, 10°, 9°, 8°, 7°, 6°, 5°, 4°, 3°, 2°, 1°, etc., without being limited thereto.

[0063] In this embodiment, the longitudinal angle between the optical axes of the two visible light cameras (131 and 132) in space is less than or equal to 15°, which can ensure that the two visible light cameras can simultaneously capture images of the tennis ball in the longitudinal direction, thereby improving the accuracy of the tennis robot in catching the ball.

[0064] In one embodiment, the field of view angle range of the two visible light cameras is 20°-100°.

[0065] Exemplarily, the field of view angle range of the two visible light cameras is 20°, 30°, 40°, 50°, 60°, 70°, 80°, 90°, 100°, etc., without being limited thereto.

[0066] When the field of view angle is too large, image distortion will occur, which makes the depth estimation of the tennis ball inaccurate, and the tennis ball in the distance is too small to be recognized. When the field of view angle is too small, the tennis ball cannot be shot in some areas. In the embodiment, the field of view angle of the two visible light cameras ranges from 20° to 100°, which can ensure that the visual angle blind area of the two visible light cameras is less than or equal to 500 mm, and the two visible light cameras can obtain more tennis ball images in more positions, realize more accurate prediction of the position of the tennis ball, and improve the ball catching accuracy.

[0067] In one embodiment, the field of view angle of the two visible light cameras ranges from 35° to 60°.

[0068] Exemplarily, the field of view angle of the two visible light cameras can range from 35°, 36°, 37°, 38°, 39°, 40°, 45°, 50°, 55°, 60°, etc., and is not limited thereto.

[0069] In the embodiment, the field of view angle of the two visible light cameras ranges from 35° to 60°, which can ensure that the two visible light cameras can obtain more tennis ball images in more positions, realize more accurate prediction of the position of the tennis ball, and improve the ball catching accuracy.

[0070] In one embodiment, referring to FIG. 1, the distance between the at least one visible light camera (131 or 132) and the ground is a preset height h; and the preset height h is greater than or equal to 20 cm. Figure 1

[0071] Exemplarily, the preset height h can be 20 cm, 25 cm, 30 cm, 35 cm, 40 cm, 45 cm, 50 cm, etc., and can be flexibly set according to actual needs, and is not limited thereto.

[0072] In the embodiment, the distance between the at least one visible light camera (131 or 132) and the ground is greater than or equal to 20 cm, which avoids that the visible light camera is too close to the ground, and the object unrelated to the motion trajectory of the tennis ball is shot, which affects the recognition efficiency and accuracy of the tennis ball.

[0073] In one embodiment, referring to FIG. 1, the camera assembly further comprises an anti-shake firmware 160, and the hardness of the anti-shake firmware 160 is greater than the hardness of the tennis robot main body 110, so as to fix the camera assembly on the preset position of the tennis robot main body 110. Figure 1

[0074] ​​In the embodiment, the two visible light cameras (131 and 132) are fixed on the tennis robot body 110 by the anti-shake fixture 160, so that the relative angle between the two visible light cameras (131 and 132) is unchanged. Meanwhile, the anti-shake fixture 160 also ensures that the relative positions of the two visible light cameras (131 and 132) and the mobile device and the tennis robot body 110 are as unchanged as possible, thereby improving the prediction accuracy of the ball catching position of the tennis ball based on the images acquired by the two visible light cameras (131 and 132), and improving the accuracy of the tennis robot catching the tennis ball. The relative angle of the camera is changed due to impact, vibration and other factors, thereby causing the position estimation error of the tennis ball to be large.

[0075] In one embodiment, the motion information includes acceleration information and angular velocity information; the sensing component includes an accelerometer and a gyroscope, both of which are electrically connected to the control device; the accelerometer is configured to acquire the acceleration information; and the gyroscope is configured to acquire the angular velocity information.

[0076] In the embodiment, the angular velocity information can be used to acquire the rotation angle of the first coordinate system relative to the second coordinate system. The first coordinate system is a three-dimensional coordinate system with the tennis robot as the origin, and the second coordinate system is a three-dimensional coordinate system with any fixed point in the tennis court as the origin. The acceleration information can be used to acquire the position of the tennis robot in the coordinate system established with any fixed point in the tennis court as the origin. In combination with the image, the angle information of the tennis robot relative to the reference origin of the tennis court, and the position of the tennis robot relative to the reference origin of the tennis court, the position of the tennis ball in the tennis court can be accurately acquired, thereby improving the ball catching accuracy of the tennis robot.

[0077] In one embodiment, referring to the accompanying Figure 5 , the accompanying Figure 5 flowchart of the control method in the embodiment is shown. The control method in the embodiment can be executed by the control device in any of the above embodiments, or can be executed by other terminals or processors capable of communicating with the control device in any of the above embodiments, without being limited thereto. In the embodiment, the control method is taken as an example of being executed by the control device in any of the above embodiments.

[0078] The control method includes the following steps S501 to S503.

[0079] Step S501, receiving motion information acquired by a sensing component in a tennis robot at multiple time points and a set of tennis ball images of a tennis ball to be caught collected by two visible light cameras at multiple time points.

[0080] The set of tennis ball images includes a plurality of sets of tennis ball images corresponding to the multiple time points, and each set of tennis ball images includes tennis ball images collected by the two visible light cameras respectively.

[0081] The time when the sensing component acquires the motion information corresponds to the time when the two visible light cameras acquire the set of tennis ball images. For example, the two visible light cameras acquire the set of tennis ball images at a first time, and the sensing component acquires the motion information at the first time.

[0082] At step S502, a target predicted motion trajectory of the to-be-received tennis ball is acquired according to the set of tennis ball images and the motion information.

[0083] According to the set of tennis ball images corresponding to one time and the motion information of the tennis robot at the same time, the position of the to-be-received tennis ball in the tennis court at the time can be determined. Based on the positions of the to-be-received tennis ball in the tennis court determined at multiple times, the target predicted motion trajectory of the to-be-received tennis ball can be predicted accurately.

[0084] The tennis court is the place where the tennis robot plays tennis.

[0085] At step S503, the moving device is controlled to move to the receiving position according to the target predicted motion trajectory.

[0086] In this embodiment, the set of tennis ball images of the to-be-received tennis ball collected by the two visible light cameras of the tennis robot at multiple times is received, and the motion trajectory of the to-be-received tennis ball can be predicted based on the set of tennis ball images and the motion information of the tennis robot, that is, the position of the tennis robot in the tennis court is considered through the motion information of the tennis robot, so that the target predicted motion trajectory of the to-be-received tennis ball can be acquired accurately. The endpoint of the to-be-received tennis ball in the target predicted motion trajectory is taken as the receiving position of the to-be-received tennis ball, so that the accuracy of the tennis robot in receiving the tennis ball can be improved.

[0087] In one embodiment, referring to the accompanying drawings Figure 6 , the accompanying Figure 6 Fig. 6 shows a flowchart of acquiring a target predicted motion trajectory of a to-be-received tennis ball according to a set of tennis ball images and motion information in this embodiment. In this embodiment, the set of tennis ball images includes a first set of tennis ball images and a second set of tennis ball images. Acquiring a target predicted motion trajectory of a to-be-received tennis ball according to a set of tennis ball images and motion information includes the following steps S601 to S603.

[0088] At step S601, a first set of tennis ball images containing a to-be-received tennis ball collected by two visible light cameras at multiple times and motion information acquired by a sensing component at corresponding multiple times are received.

[0089] The first set of tennis ball images includes a set of tennis ball images corresponding to multiple times, and a set of tennis ball images includes tennis ball images acquired by the two visible light cameras at the same time.

[0090] The first set of tennis images can be collected by the two visible light cameras at the same position of the court at multiple time points, or can be collected by the two visible light cameras at different positions of the court at multiple time points.

[0091] Exemplarily, the tennis robot further comprises a ball serving assembly, and the control device is connected with the ball serving assembly. The control device controls the ball serving assembly to serve a tennis ball, and the moving device is not moved at this time. The first set of tennis images is collected at the serving position.

[0092] Exemplarily, the tennis robot further comprises a ball serving assembly, and the control device is connected with the ball serving assembly. The control device controls the ball serving assembly to serve a tennis ball, and the control device controls the two visible light cameras to collect images of the served target tennis ball. The moving device is controlled to move to the preset origin position. In the moving process, steps S601 to S603 are performed to collect the first set of tennis images, to predict the movement trajectory of the tennis ball, and to control the moving device to move to the predicted position of the tennis ball if the predicted movement trajectory indicates that the receiving position of the tennis ball is inconsistent with the preset origin position.

[0093] In actual use, the collection time of the first set of tennis images can be flexibly set according to actual needs, and is not limited to the above examples.

[0094] In order to ensure that the collected image set (including the first set of tennis images and the second set of tennis images) can accurately represent the position of the tennis ball at the corresponding time, the two visible light cameras can be configured so that the time interval of the images collected by the two visible light cameras at the same time is as small as possible, for example, 1 ms, which is not limited thereto.

[0095] In step S602, the moving device of the tennis robot is controlled to move based on the first set of tennis images and the corresponding movement information, so as to respectively obtain the movement information and the second set of tennis images in the moving process of the moving device.

[0096] Similar to the first set of tennis images, the second set of tennis images also includes a plurality of tennis image groups corresponding to a plurality of time points, and each tennis image group includes tennis images collected by the two visible light cameras at the same time.

[0097] It can be understood that the tennis robot includes a moving device, which is driven to move by the moving device. The movement information of the moving device is the movement information of the tennis robot. Based on the movement information of the tennis robot, the position of the tennis robot on the court can be obtained. The two visible light cameras are arranged on the tennis robot, so that the positions of the two visible light cameras on the court can be obtained. In combination with the position of the to-be-received tennis ball relative to the two visible light cameras represented by the first set of tennis images, the position of the to-be-received tennis ball on the court can be obtained.

[0098] Therefore, the movement trajectory of the to-be-received tennis ball can be preliminarily predicted according to the first tennis ball image set and the movement information of the tennis robot, the movement device is controlled to move based on the preliminarily predicted movement trajectory, and in the moving process, the second tennis ball image set including more effective position information of the to-be-received tennis ball can also be acquired, so that the target predicted movement trajectory of the to-be-received tennis ball acquired based on the second tennis ball image set is more accurate.

[0099] In step S603, a target predicted movement trajectory of the to-be-received tennis ball is acquired according to the first tennis ball image set, the second tennis ball image set and the movement information.

[0100] For example, the initial movement trajectory preliminarily predicted according to the first tennis ball image set can be calibrated according to the second tennis ball image set, so that a more accurate target predicted movement trajectory is obtained.

[0101] The movement information in step S603 includes movement information corresponding to the first tennis ball image set and movement information in the moving process of the tennis robot.

[0102] In this embodiment, the first tennis ball image set of the to-be-received tennis ball collected by the two visible light cameras of the tennis robot at multiple time points is received, the movement trajectory of the to-be-received tennis ball can be preliminarily predicted based on the first tennis ball image set and corresponding movement information, the movement device of the tennis robot is controlled to move to the ball receiving position of the preliminarily predicted movement trajectory, the second tennis ball image set and movement information are acquired in the moving process, the movement trajectory preliminarily predicted based on the first tennis ball image set is calibrated according to the second tennis ball image set and the movement information in the moving process, so as to acquire the target predicted movement trajectory of the to-be-received tennis ball, thereby improving the ball receiving accuracy of the tennis robot.

[0103] In one embodiment, refer to the accompanying drawings Figure 7 , the accompanying drawings Figure 7 show a flowchart of controlling the movement device of the tennis robot to move based on the first tennis ball image set and corresponding movement information to respectively acquire the movement information in the moving process of the movement device and the second tennis ball image set in this embodiment.

[0104] In this embodiment, controlling the movement device of the tennis robot to move based on the first tennis ball image set and corresponding movement information to respectively acquire the movement information in the moving process of the movement device and the second tennis ball image set includes the following steps S701 to S703.

[0105] In step S701, first coordinates of the to-be-received tennis ball in a first coordinate system at multiple time points are acquired according to the first tennis ball image set.

[0106] The first coordinate system is a three-dimensional coordinate system with the tennis robot as the origin.

[0107] ​​​​​​​​​During the movement of the mobile device, the two visible light cameras are always directed towards the to-be-received tennis ball, so as to ensure that the two visible light cameras can acquire the tennis ball images during the movement.

[0108] For example, the pixel coordinates of the to-be-received tennis ball in the tennis ball images collected by the two visible light cameras at the i-th moment are identified respectively, and the corresponding first coordinates (xci, yci, zci) are acquired by using the triangulation method, where i is a positive number, and the y-axis represents the height of the observation point from the ground.

[0109] In step S702, the second coordinates of the to-be-received tennis ball in the second coordinate system at multiple moments are acquired according to the first coordinates at the multiple moments and corresponding motion information.

[0110] The second coordinate system is a three-dimensional coordinate system with any fixed point in the tennis court as the origin.

[0111] The tennis court refers to the place where the tennis robot plays tennis.

[0112] For example, the motion information includes the angular velocity and acceleration of the tennis robot, the integral of the angular velocity can acquire the orientation angle θi of the tennis robot relative to the origin of the second coordinate system at the i-th moment, and the coordinates (xri, yri, zri) of the tennis robot relative to the origin of the second coordinate system at the i-th moment can be acquired according to the acceleration. The second coordinates (xgi, ygi, zgi) can be acquired according to the first coordinates, (xri, yri, zri), and the orientation angle θi, since the tennis robot always moves on the same horizontal plane, i.e. the ground, so the value of the y-axis does not need to be rotated. The acquisition formula can be represented as: xgi = xcix cos(θi) - zci x sin(θi) + xri; ygi = yci; zgi = xcix sin(θi) + zci x cos(θi) + zri.

[0113] In step S703, the movement of the mobile device is controlled according to the second coordinates corresponding to the to-be-received tennis ball at multiple moments, so as to acquire the motion information of the mobile device during the movement and the second tennis ball image set collected by the two visible light cameras.

[0114] In this embodiment, the initial predicted motion trajectory of the to-be-received tennis ball can be acquired according to the second coordinates corresponding to the multiple moments, the mobile device is controlled to move towards the terminal position of the initial predicted motion trajectory, and the motion information and the second tennis ball image set are acquired at multiple moments during the movement, so that the position of the to-be-measured tennis ball in the tennis court during the movement is more accurate.

[0115] In one embodiment, the second tennis ball image set includes images collected by the two visible light cameras at multiple moments during the movement of the mobile device.

[0116] Reference is made to the accompanying drawingsFigure 8 , attached Figure 8 A flowchart of acquiring the target predicted motion trajectory of the to-be-accepted tennis ball according to the first tennis ball image set, the second tennis ball image set and the motion information in the embodiment is shown. In the embodiment, acquiring the target predicted motion trajectory of the to-be-accepted tennis ball according to the first tennis ball image set, the second tennis ball image set and the motion information includes the following steps S801 to S803.

[0117] In step S801, a first predicted motion trajectory is acquired according to the plurality of second coordinates corresponding to the first tennis ball image set.

[0118] That is, it can be understood that the tennis ball image in the first tennis ball image set is the earliest image acquired by the two visible light cameras, and the accuracy of the position of the to-be-accepted tennis ball represented by the image is the lowest, but the general motion direction of the to-be-accepted tennis ball can be determined based on the first tennis ball image set and the motion information, which provides a reference for the tennis robot to further calibrate the position of the to-be-accepted tennis ball.

[0119] In step S802, the first predicted motion trajectory is calibrated to acquire a second predicted motion trajectory corresponding to the initial time according to the image acquired by the two visible light cameras at the initial time and the motion information at the initial time.

[0120] The initial time is the earliest time in the plurality of times during the movement of the mobile device.

[0121] Compared with the position of the to-be-accepted tennis ball represented by the first predicted motion trajectory, the observation position of the to-be-accepted tennis ball is better after the movement of the tennis robot, so that the motion position of the to-be-accepted tennis ball at the next time can be more accurately predicted based on the tennis ball image collected after the movement, which is the image in the second tennis ball image set, and the position is more accurate than the position corresponding to the same time predicted by the first predicted motion trajectory. Therefore, the initial predicted motion trajectory is calibrated based on the position, and the corresponding predicted motion trajectory can more accurately predict the receiving position of the to-be-accepted tennis ball.

[0122] Exemplarily, refer to attached Figure 9 , attached Figure 9A schematic diagram of the tennis robot in the moving process is shown. It is assumed that time 1 precedes time 2. The predicted trajectory of the ball to be received obtained by the tennis robot at time 1 includes the predicted receiving point of the ball to be received, but the predicted receiving point predicted at time 1 may not be accurate because the image of the ball to be received obtained is earlier. Compared with time 1, the tennis robot has moved and collected more images containing position information of the ball to be received, so the predicted receiving point of the ball to be received at time 2 is more accurate. Therefore, the ball receiving position of the ball to be received can be more accurately predicted by calibrating the initial predicted motion trajectory according to the images obtained by the two visible light cameras at the initial time to obtain the predicted motion trajectory corresponding to the initial time.

[0123] At step S803, the predicted motion trajectory corresponding to the previous time is calibrated according to the images obtained by the two visible light cameras at the remaining times and the motion information at the remaining times to obtain the target predicted motion trajectory corresponding to the last time.

[0124] The remaining times are other times in the moving process of the moving device except the initial time and the last time; and the last time is the last time in the moving process of the moving device.

[0125] Exemplarily, in the moving process of the tennis ball, the two visible light cameras obtain images containing the ball to be received at the first time to the fifth time. The initial time is the first time, and the first predicted motion trajectory is calibrated according to the images obtained by the two visible light cameras and the motion information obtained by the sensing assembly at the first time to obtain the second predicted motion trajectory. The remaining times include the second time, the third time and the fourth time. The second predicted motion trajectory corresponding to the first time is calibrated according to the images obtained at the second time and the motion information obtained at the second time to obtain the third predicted motion trajectory; the third predicted motion trajectory corresponding to the second time is calibrated according to the images obtained at the third time and the motion information obtained at the third time to obtain the fourth predicted motion trajectory; and the fourth predicted motion trajectory corresponding to the third time is calibrated according to the images obtained at the fourth time and the motion information obtained at the fourth time to obtain the fifth predicted motion trajectory. The last time is the fifth time. The fifth predicted motion trajectory corresponding to the fourth time is calibrated according to the images obtained at the fifth time and the motion information obtained at the fifth time to obtain the target predicted motion trajectory.

[0126] In the moving process of the moving device, the predicted motion trajectory obtained at the previous time is calibrated based on the ball image and the motion information corresponding to each next time, so that the target predicted motion trajectory corresponding to the ball image obtained at the last time is the most accurate, and the ball receiving accuracy of the tennis robot is improved.

[0127] In one embodiment, the two visible light cameras in the tennis robot are configured with master-slave mode or hard trigger mode, so that the time interval for the control device in the tennis robot to receive the images output by the two visible light cameras is less than or equal to 1ms.

[0128] In this embodiment, the two visible light cameras are configured with master-slave mode or hard trigger mode, so that the time interval for the control device to receive the images obtained by the two visible light cameras is less than or equal to 1ms, which can ensure the accuracy of the depth estimation of the position of the ball to be received based on the images obtained by the two visible light cameras.

[0129] In one embodiment, the tennis robot includes a serving assembly connected to the control device, and the control device can be configured to calculate a serving instruction to the serving assembly at a preset time before reaching the receiving position according to the angular velocity information and acceleration information obtained by the sensing assembly in real time, so as to avoid the time difference between the start of the serving instruction and the formal serving of the serving assembly, which causes the second tennis ball to be unable to be served immediately when the first tennis ball cannot be received, so as to ensure that the receiving and serving are basically simultaneous, and improve the user experience of the tennis robot.

[0130] In one embodiment, the tennis robot includes a serving assembly connected to the control device, and the control device can further control the serving speed and angle of the serving assembly and the orientation of the tennis robot facing the tennis opponent according to the angular velocity information and acceleration information collected by the sensing assembly in real time, so as to accurately serve the second tennis ball to the position of the tennis opponent on the court.

[0131] It should be understood that although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0133] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0134] The above embodiments only express several implementation ways of the present application, and the description is more specific and detailed, but it should not be understood as a limitation to the scope of the application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A tennis robot ball catching control method, characterized in that: The tennis robot includes a tennis robot body, a moving device, a control device, and a ball catching assembly. The moving device is provided at the bottom of the tennis robot body, and the control device is connected to the moving device. The ball catching control method includes: The control device controls the moving device to move the tennis robot to a ball receiving position for receiving a tennis ball, so that the tennis robot receives the tennis ball to be received through the ball receiving assembly at the ball receiving position.

2. The tennis robot ball catching control method according to claim 1, wherein: The control device controls the moving device to move the tennis robot to a receiving position for receiving a tennis ball, including: According to the movement information of the tennis robot and the image containing the tennis ball to be caught, the control device controls the moving device to move the tennis robot to the ball catching position.

3. The tennis robot ball catching control method according to claim 2, wherein: The control device controls the moving device to move the tennis robot to the ball receiving position based on the motion information of the tennis robot and the image containing the tennis ball to be received, including: receiving the motion information and the tennis image set of the tennis ball to be received acquired by the tennis robot at multiple moments; Acquire a target predicted motion trajectory of the tennis ball to be received according to the tennis ball image set and the motion information; According to the target predicted motion trajectory, the control device controls the moving device to move the tennis robot to the ball receiving position.

4. The tennis robot ball catching control method according to claim 3, wherein: The tennis image set includes a first tennis image set and a second tennis image set; and obtaining a target predicted motion trajectory of the tennis ball to be received based on the tennis image set and the motion information includes: receiving the first set of tennis ball images including the tennis ball to be received, collected at multiple moments, and the motion information acquired at corresponding multiple moments; Based on the first tennis image set and the corresponding motion information, controlling the tennis robot to move, so as to respectively obtain the motion information and the second tennis image set during the movement of the tennis robot; The target predicted motion trajectory is obtained according to the first tennis image set, the second tennis image set, and the motion information.

5. The tennis robot ball catching control method according to claim 4, wherein: The controlling the movement of the tennis robot based on the first tennis image set and the corresponding motion information to respectively obtain the motion information and the second tennis image set during the movement of the tennis robot includes: Acquire first coordinates of the tennis ball to be received in a first coordinate system at multiple moments according to the first tennis ball image set; the first coordinate system is a three-dimensional coordinate system with the tennis robot as the origin; Obtaining second coordinates of the tennis ball to be received at multiple moments in a second coordinate system based on the first coordinates at multiple moments and the corresponding motion information; the second coordinate system is a three-dimensional coordinate system with any fixed point on the court as its origin; The movement of the tennis robot is controlled according to the second coordinates corresponding to the tennis balls to be received at multiple moments, so as to respectively obtain the motion information and the second tennis ball image set during the movement of the tennis robot.

6. The tennis robot ball catching control method according to claim 5, wherein: The second tennis image set includes images collected at multiple moments; and obtaining the target predicted motion trajectory based on the first tennis image set, the second tennis image set, and the motion information includes: Obtaining a first predicted motion trajectory according to a plurality of second coordinates corresponding to the first tennis ball image set; calibrating the first predicted motion trajectory based on the image acquired at an initial moment and the motion information at the initial moment to obtain a second predicted motion trajectory corresponding to the initial moment; the initial moment being the first moment among multiple moments in the movement process of the tennis robot; The predicted motion trajectory corresponding to the previous moment is calibrated successively according to the images obtained at the remaining moments and the motion information at the remaining moments to obtain the target predicted motion trajectory corresponding to the last moment; the remaining moments are the other moments in the multiple moments in the movement process of the tennis robot except the initial moment and the last moment; the last moment is the last moment in the multiple moments in the movement process of the tennis robot.

7. The ball catching control method of a tennis robot according to any one of claims 4 to 6, wherein: The tennis robot further includes a serving assembly, and the control device is connected to the serving assembly. The first tennis image set is collected in a manner including at least one of the following: The control device controls the serving assembly to send out the tennis ball to be received, while the moving device does not move and collects the first tennis ball image set at the position where the tennis ball to be received is sent out; The control device controls the serving assembly to send out the tennis ball to be received, controls the moving device to move to a preset origin position, and collects the first tennis ball image set during the movement.

8. The ball catching control method of a tennis robot according to any one of claims 2 to 6, wherein: The tennis robot further includes a camera assembly, which is fixed to the main body of the tennis robot. The camera assembly includes two visible light cameras, which are used to capture the image containing the tennis ball to be caught.

9. The tennis robot ball catching control method according to claim 8, characterized in that: The distance between the two visible light cameras in the width direction of the tennis robot body is greater than or equal to 12 cm, and the width direction is the dimension direction of the tennis robot body in a direction parallel to the ground.

10. The ball catching control method of a tennis robot according to claim 8, characterized in that: The angle between the optical axes of the two visible light cameras and the ground is less than or equal to 30°.

11. The tennis robot ball catching control method according to claim 8, characterized in that: The angle between the optical axes of the two visible light cameras in space is less than or equal to 30°.

12. The tennis robot ball catching control method according to claim 8, characterized in that: The longitudinal angles of the optical axes of the two visible light cameras in space are both less than or equal to 15°.

13. The ball catching control method of a tennis robot according to any one of claims 1 to 6, characterized in that: The tennis robot further includes a serving assembly, and the control device is connected to the serving assembly; the tennis robot receives the tennis ball to be received at the receiving position through the receiving assembly of the tennis robot, comprising: While the tennis robot receives the tennis ball to be received through the ball receiving assembly of the tennis robot at the ball receiving position, the control device controls the ball serving assembly of the tennis robot to launch a second tennis ball.

14. The tennis robot ball catching control method according to claim 13, wherein: The control device controls the serving assembly of the tennis robot to launch a second tennis ball, including: The control device controls the direction of the tennis robot facing the opponent of the tennis ball, and launches the second tennis ball to the position of the opponent on the court through the serving assembly.

15. A tennis robot, characterized in that: The invention comprises a tennis robot body, a moving device, a control device and a ball catching assembly, wherein the moving device is arranged at the bottom of the tennis robot body, the control device is connected to the moving device, and the control device is used to implement the steps of the ball catching control method according to any one of claims 1 to 14.

Images