Tennis robot ball receiving method, device, equipment and storage medium

Abstract

The application discloses a method, device and equipment for catching a ball by a tennis robot and a storage medium, relates to the technical field of robots, and the method comprises the following steps: collecting a color image and a depth image of a tennis court area; determining self-position information of the tennis robot and ball position information of a target tennis ball based on the color image and the depth image; predicting future position information of the target tennis ball according to a plurality of pieces of the ball position information acquired at continuous time points; and controlling the tennis robot to move to a ball catching position according to the self-position information of the tennis robot and the future position information. The application can control the tennis robot to actively catch a ball, thereby improving user experience.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a method, apparatus, device, and storage medium for a tennis robot to receive the ball. Background Technology

[0002] Tennis has gained immense popularity both domestically and internationally in recent years, with many enthusiasts having a significant training need. However, the high cost of coaches and the difficulty in finding suitable practice partners pose considerable challenges for those eager to receive tennis training. Tennis robots can effectively address this need. However, during practice sessions with tennis robots, common models can only perform fixed-point serves and ball retrieval after training, lacking precise ball receiving capabilities. Therefore, how to control tennis robots to actively receive balls and thus improve the user experience remains a problem that needs to be solved.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, and storage medium for catching tennis balls with a tennis robot, aiming to solve the technical problem of how to control a tennis robot to actively catch balls, thereby improving the user experience.

[0005] To achieve the above objectives, this application proposes a method for receiving balls using a tennis robot, the method comprising:

[0006] Acquire color and depth images of the tennis court area; Based on the color image and the depth image, the position information of the tennis robot and the position information of the target tennis ball are determined. Based on the acquired tennis ball position information at multiple consecutive moments, predict the future position information of the target tennis ball; Based on the tennis robot's own position information and its future position information, the tennis robot is controlled to move to the receiving position.

[0007] In one embodiment, the step of determining the tennis robot's own position information and the target tennis ball's position information based on the color image and the depth image includes: Identify fixed features of the tennis court in the color image, and calculate the tennis robot's own position information in the tennis court area based on the fixed features of the tennis court and the depth image; The tennis ball's position information within the tennis court area is calculated based on the color image, the depth image, and its own position information.

[0008] In one embodiment, the step of identifying fixed features of the tennis court in the color image and calculating the tennis robot's own position information in the tennis court area based on the fixed features of the tennis court and the depth image includes: The net features and court line features are segmented from the color image; The image coordinates of the center point of the court in the tennis robot coordinate system are determined based on the net features, the court line features, and the depth image. The position information of the tennis robot in the court coordinate system can be deduced from the image coordinates.

[0009] In one embodiment, the step of calculating the tennis ball's position information in the tennis court area based on the color image, the depth image, and its own position information includes: Tennis features were identified in the color image; The tennis ball image coordinates are obtained based on the tennis ball features and the depth image. The tennis ball's position information within the tennis court area is calculated based on the tennis ball image coordinates and its own position information.

[0010] In one embodiment, the step of predicting the future position information of the target tennis ball based on multiple tennis ball position information obtained at consecutive time points includes: A tennis motion model is obtained by curve fitting based on multiple tennis ball position information at consecutive time points; The future position information of the target tennis ball is calculated based on the tennis motion model.

[0011] In one embodiment, the step of controlling the tennis robot to move to the receiving position based on the tennis robot's own position information and the future position information includes: Determine whether the target tennis ball has bounced after hitting the ground; If the target tennis ball does not land, the tennis robot is controlled to align horizontally with the future position information based on its own position information and the future position information. If the target tennis ball has bounced after landing, then based on the horizontal alignment, the tennis robot is further controlled to move vertically to the receiving position according to its own position information and the future position information.

[0012] In one embodiment, after the step of acquiring color and depth images of the tennis court area, the method further includes: The movement direction of all tennis balls in the tennis court area is determined based on the color image and the depth image; Based on the direction of motion, tennis balls whose direction of motion is toward the tennis robot are selected as target tennis balls.

[0013] Furthermore, to achieve the above objectives, this application also proposes a ball-receiving device for a tennis robot, the ball-receiving device of which includes: The acquisition module is used to acquire color and depth images of the tennis court area; The determination module is used to determine the position information of the tennis robot itself and the position information of the target tennis ball based on the color image and the depth image. The prediction module is used to predict the future position information of the target tennis ball based on the tennis ball position information obtained at multiple consecutive moments. The control module is used to control the tennis robot to move to the receiving position based on the tennis robot's own position information and the future position information.

[0014] In addition, to achieve the above objectives, this application also proposes a ball-receiving device for a tennis robot, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the ball-receiving method for the tennis robot as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the tennis robot's ball-receiving method as described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the tennis robot's ball-receiving method as described above.

[0017] This application provides a method for a tennis robot to receive a ball. The method involves acquiring color and depth images of a tennis court area; determining the robot's own position and the target tennis ball's position based on the color and depth images; predicting the target tennis ball's future position based on multiple tennis ball position data acquired at consecutive moments; and controlling the robot to move to a receiving position based on its own position and the future position information. This application uses color and depth images of the tennis court area to perceive the environment and the tennis ball, determines the robot's real-time position relative to the ball based on the images, predicts the ball's future landing point, and finally controls the robot to autonomously move to a receiving position based on the position information. This method improves the user experience by enabling the tennis robot to actively receive the ball. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the ball-receiving method of the tennis robot in this application (Example 1). Figure 2 A schematic diagram of the tennis robot structure provided for Embodiment 1 of the tennis robot's ball-receiving method in this application; Figure 3 A flowchart illustrating the second embodiment of the ball-receiving method for the tennis robot of this application; Figure 4 This is a schematic diagram of the modular structure of the ball-receiving device of the tennis robot according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware operating environment involved in the ball-receiving method of the tennis robot in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] This application acquires color and depth images of a tennis court area; based on the color and depth images, it determines the position information of the tennis robot and the position information of the target tennis ball; based on the acquired tennis ball position information at multiple consecutive moments, it predicts the future position information of the target tennis ball; and based on the tennis robot's own position information and the future position information, it controls the tennis robot to move to the receiving position.

[0025] Tennis has gained immense popularity both domestically and internationally in recent years, with many enthusiasts having a significant training need. However, the high cost of coaches and the difficulty in finding suitable practice partners pose considerable challenges for those eager to receive tennis training. Tennis robots can effectively address this need. However, during practice sessions with tennis robots, common models can only perform fixed-point serves and ball retrieval after training, lacking precise ball receiving capabilities. Therefore, how to control tennis robots to actively receive balls and thus improve the user experience remains a problem that needs to be solved.

[0026] This application uses color and depth images of the tennis court area to perceive the environment and the tennis ball, determines the real-time position of the robot and the tennis ball based on the images, predicts the future landing point of the tennis ball, and finally controls the robot to move autonomously to the receiving position based on the position information, thereby improving the user experience by controlling the tennis robot to actively receive the ball.

[0027] Based on this, embodiments of this application provide a method for a tennis robot to receive the ball, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the ball-receiving method for the tennis robot of this application.

[0028] In this embodiment, the ball-receiving method of the tennis robot includes steps S10 to S40: Step S10: Acquire color and depth images of the tennis court area; It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a ball-receiving device for a tennis robot. The following description uses a ball-receiving device for a tennis robot as an example to illustrate this embodiment and the subsequent embodiments.

[0029] It should be noted that the structural diagram of the tennis robot can be referenced. Figure 2 , Figure 2 The tennis ball robot comprises a perception unit 1 consisting of multiple sensors, a motion unit 3 consisting of a mobile device, a central computing unit 2 consisting of a CPU / GPU, and a ball-catching device 4. The perception unit acquires multimodal data containing the tennis ball, such as color images, depth images, and point clouds. The central computing unit (CPU / GPU) calculates the real-time and predicted positions of the tennis ball. Based on the predicted position, the motion unit guides the robot to a suitable position so that the ball-catching device can catch the ball.

[0030] Understandably, the perception unit contains multiple sensors, which can be combined in various ways, as long as they can acquire color images and depth information. Examples include two color cameras, a color camera and a LiDAR, or a structured light camera. These sensors are fixed in position within the robot and their relative positions are pre-calibrated to align the acquired data to the same preset coordinate system. Each sensor then continuously acquires data containing the tennis ball at the same time and transmits it to the central computing unit.

[0031] In one feasible approach, after the step of acquiring color and depth images of the tennis court area, the method further includes: determining the motion direction of all tennis balls in the tennis court area based on the color and depth images; and selecting tennis balls whose motion direction is toward the tennis robot as target tennis balls based on the motion direction.

[0032] It should be noted that images captured on a tennis court may contain tennis balls hit by other athletes, which could interfere with the tennis robot. Therefore, the tennis balls can be filtered beforehand. Specifically, an object detection algorithm can be used to obtain the coordinate set P of the tennis ball in the sensor coordinate system at the current moment. t Based on the current coordinate set P of the tennis ball. t and the coordinate set P of the previous moment t 1. Calculate the direction of motion of the tennis ball, and eliminate tennis balls that are stationary or moving away from the robot based on the direction of motion, keeping only tennis balls that are moving towards the robot as the target tennis balls.

[0033] Step S20: Based on the color image and the depth image, determine the position information of the tennis robot and the position information of the target tennis ball; It should be noted that both the user's own position information and the tennis ball's position information are based on the coordinate system of the tennis court.

[0034] Step S30: Based on the acquired tennis ball position information at multiple consecutive moments, predict the future position information of the target tennis ball; It should be noted that by using multiple tennis ball position information at consecutive moments, a time-position motion model of the tennis ball can be constructed. By inputting the next moment into the motion model, the predicted position of the tennis ball at the next moment can be obtained.

[0035] In one feasible approach, the step of predicting the future position information of the target tennis ball based on the acquired tennis ball position information at multiple consecutive time points includes: performing curve fitting based on the tennis ball position information at multiple consecutive time points to obtain a tennis ball motion model; and calculating the future position information of the target tennis ball based on the tennis ball motion model.

[0036] It should be noted that the specific process of obtaining the tennis motion model by curve fitting based on multiple tennis ball position information at consecutive time points can be as follows: First, store the sequence of tennis ball positions (x, y, y) calculated at the most recent N time points in the court coordinate system. t y t , z t Let t be the time interval. Since the tennis ball's flight is primarily influenced by gravity, its motion can be approximated as a uniformly accelerated curve over a short period. The least squares method can be used to perform polynomial fitting on the coordinate values ​​in the X (horizontal), Y (vertical, perpendicular to the ground), and Z (horizontal, the direction the tennis ball is flying towards) directions over time t. The fitting yields three equations, which together constitute the tennis ball's motion model for the current time interval. Substituting the next moment into the motion model then yields the future position information of the target tennis ball.

[0037] Step S40: Based on the tennis robot's own position information and the future position information, control the tennis robot to move to the receiving position.

[0038] It should be noted that, based on the calculated future location information, the tennis robot can continuously move its position, gradually approaching the tennis ball until it finally catches it.

[0039] In one feasible approach, the step of controlling the tennis robot to move to the receiving position based on its own position information and the future position information includes: determining whether the target tennis ball has landed and bounced; if the target tennis ball has not landed, controlling the tennis robot to align horizontally with the future position information based on its own position information and the future position information; if the target tennis ball has landed and bounced, further controlling the tennis robot to move vertically to the receiving position based on the horizontal alignment and the future position information.

[0040] It should be noted that, for tennis, the ball should be caught only after it bounces once upon impact. Therefore, the tennis robot's movement is divided into two phases: before the ball bounces and after it bounces. Before the ball bounces after flying over the net, the robot's movement is determined by the sensor's current position in the court coordinate system (x...). t y t , z t ) and the position of the tennis ball in the court coordinate system at the next moment (x) t+1 y t+1 , z t+1The robot is guided to move towards the tennis ball in the x-direction, aligning them roughly. A landing and bounce detection is performed: before and after the tennis ball bounces, its speed and direction change abruptly, with the y-direction changing from negative to positive. Based on this characteristic, the landing point and time of the tennis ball are detected. After the tennis ball bounces, the alignment in the x-direction continues. When the ball's y-direction coordinate is greater than the y-coordinate of the lowest point of the receiving device at the next moment, the robot quickly approaches the ball in the z-direction, causing the tennis ball to fall into the receiving device. If the ball's y-direction coordinate is greater than the y-coordinate of the highest point of the receiving device at the next moment, the robot quickly moves away from the ball in the z-direction, ensuring it falls smoothly into the receiving device during its subsequent descent.

[0041] This embodiment acquires color and depth images of the tennis court area; based on the color and depth images, it determines the tennis robot's own position information and the target tennis ball's position information; based on multiple tennis ball position information obtained at consecutive moments, it predicts the future position information of the target tennis ball; and based on the tennis robot's own position information and the future position information, it controls the tennis robot to move to the receiving position. This embodiment uses color and depth images of the tennis court area to perceive the environment and the tennis ball, determines the real-time position of the robot and the tennis ball based on the images, predicts the future landing point of the tennis ball, and finally controls the robot to autonomously move to the receiving position based on the position information, thus improving the user experience by controlling the tennis robot to actively receive the ball.

[0042] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S20 also includes steps S201 to S202: Step S201: Identify the fixed features of the tennis court in the color image, and calculate the position information of the tennis robot in the tennis court area based on the fixed features of the tennis court and the depth image; It should be noted that the fixed features of a court include net features, court line features, etc.

[0043] In one feasible approach, the step of identifying fixed court features in the color image and calculating the tennis robot's own position information in the tennis court area based on the fixed court features and the depth image includes: segmenting net features and court line features from the color image; determining the image coordinates of the center point of the court in the tennis robot's coordinate system based on the net features, the court line features, and the depth image; and inferring the tennis robot's own position information in the court coordinate system based on the image coordinates.

[0044] It should be noted that deep learning computer vision algorithms, such as masked region convolutional neural networks, can be used to process color images and segment the net and court lines. Based on the net and court lines, the coordinates of the net's center in the color image are determined. By combining this location with the value in the depth image, the position of the net's center in the sensor coordinate system can be determined. If we establish a court coordinate system with the origin at the court and the three axes aligned with the robot's coordinate system, then the robot's position in the court coordinate system at time t is: .

[0045] Step S202: Calculate the tennis ball position information of the target tennis ball in the tennis court area based on the color image, the depth image, and the self-position information.

[0046] It should be noted that an object detection algorithm (such as YOLO) can be called to process the same frame of color image, identify the target tennis ball, obtain the pixel coordinates, and transform the relative coordinates of the tennis ball to the tennis court area coordinate system based on the depth value of the target tennis ball at the corresponding position in the depth image and the robot's own position information, so as to obtain the tennis ball's position information in the court.

[0047] In one feasible approach, the step of calculating the tennis ball's position information in the tennis court area based on the color image, the depth image, and its own position information includes: identifying tennis ball features in the color image; obtaining tennis ball image coordinates based on the tennis ball features and the depth image; and calculating the tennis ball's position information in the tennis court area based on the tennis ball image coordinates and its own position information.

[0048] It should be noted that, specifically, the color image acquired at time t can be input into the YOLO target detection algorithm to obtain the coordinate set of the target tennis ball in the image coordinate system. By combining the depth image at time t and the intrinsic parameters of the color camera, the coordinate set of the target tennis ball in the robot coordinate system at time t can be obtained. Then, based on the robot's own position information, the transformation relationship between the court coordinate system and the robot coordinate system can be determined, thereby converting the tennis ball's position into the tennis ball's position information in the tennis court area coordinate system.

[0049] This embodiment identifies fixed features of the tennis court in the color image and calculates the tennis robot's own position information within the tennis court area based on these features and the depth image. It then calculates the target tennis ball's position information within the tennis court area based on the color image, the depth image, and the robot's own position information. This embodiment decomposes the robot's self-localization and the tennis ball's localization into two independent calculation sub-steps, allowing each to use optimal algorithms to handle the static environment and dynamic target respectively. This effectively avoids mutual interference, thereby improving the overall system's localization accuracy and reliability.

[0050] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the ball-receiving method of the tennis robot of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0051] This application also provides a ball-receiving device for a tennis robot; please refer to [reference needed]. Figure 4 The ball-catching device of the tennis robot includes: Acquisition module 10 is used to acquire color and depth images of the tennis court area; The determining module 20 is used to determine the position information of the tennis robot itself and the position information of the target tennis ball based on the color image and the depth image. Prediction module 30 is used to predict the future position information of the target tennis ball based on multiple tennis ball position information obtained at consecutive times. The control module 40 is used to control the tennis robot to move to the receiving position based on the tennis robot's own position information and the future position information.

[0052] This embodiment acquires color and depth images of the tennis court area; based on the color and depth images, it determines the tennis robot's own position information and the target tennis ball's position information; based on multiple tennis ball position information obtained at consecutive moments, it predicts the future position information of the target tennis ball; and based on the tennis robot's own position information and the future position information, it controls the tennis robot to move to the receiving position. This embodiment uses color and depth images of the tennis court area to perceive the environment and the tennis ball, determines the real-time position of the robot and the tennis ball based on the images, predicts the future landing point of the tennis ball, and finally controls the robot to autonomously move to the receiving position based on the position information, thus improving the user experience by controlling the tennis robot to actively receive the ball.

[0053] In one embodiment, the determining module 20 is further configured to identify fixed features of the tennis court in the color image, and calculate the position information of the tennis robot in the tennis court area based on the fixed features of the tennis court and the depth image; and calculate the position information of the target tennis ball in the tennis court area based on the color image, the depth image and the position information of the robot.

[0054] In one embodiment, the determining module 20 is further configured to segment net features and court line features from the color image; determine the image coordinates of the center point of the court in the tennis robot coordinate system based on the net features, the court line features, and the depth image; and infer the position information of the tennis robot in the court coordinate system based on the image coordinates.

[0055] In one embodiment, the determining module 20 is further configured to identify tennis ball features in the color image; obtain tennis ball image coordinates based on the tennis ball features and the depth image; and calculate the tennis ball position information of the target tennis ball in the tennis court area based on the tennis ball image coordinates and its own position information.

[0056] In one embodiment, the prediction module 30 is further configured to perform curve fitting based on multiple tennis ball position information at consecutive times to obtain a tennis ball motion model; and calculate the future position information of the target tennis ball based on the tennis ball motion model.

[0057] In one embodiment, the control module 40 is further configured to determine whether the target tennis ball has landed and bounced; if the target tennis ball has not landed, the control module 40 controls the tennis robot to align with the future position information in the horizontal direction based on the tennis robot's own position information and the future position information; if the target tennis ball has landed and bounced, the control module further controls the tennis robot to move to the receiving position in the vertical direction based on the tennis robot's own position information and the future position information, in addition to the horizontal alignment.

[0058] In one embodiment, the acquisition module 10 is further configured to determine the movement direction of all tennis balls in the tennis court area based on the color image and the depth image; and based on the movement direction, select tennis balls whose movement direction is toward the tennis robot as target tennis balls.

[0059] The tennis robot receiving device provided in this application, employing the tennis robot receiving method described in the above embodiments, solves the technical problem of how to control the tennis robot to actively receive the ball, thereby improving the user experience. Compared with the prior art, the beneficial effects of the tennis robot receiving device provided in this application are the same as those of the tennis robot receiving method provided in the above embodiments, and other technical features in the tennis robot receiving device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0060] This application provides a ball-receiving device for a tennis robot, the ball-receiving device for the tennis robot including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the ball-receiving method of the tennis robot in the first embodiment described above.

[0061] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a ball-receiving device suitable for implementing the tennis robot embodiments of this application. The ball-receiving device for the tennis robot in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The ball-receiving device of the tennis robot shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0062] like Figure 5As shown, the ball-receiving device of the tennis robot may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the tennis robot's ball-receiving device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the tennis robot's receiving device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show tennis robot receiving devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0063] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0064] The tennis robot receiving device provided in this application, employing the tennis robot receiving method described in the above embodiments, solves the technical problem of how to control the tennis robot to actively receive the ball, thereby improving the user experience. Compared with the prior art, the beneficial effects of the tennis robot receiving device provided in this application are the same as those of the tennis robot receiving method provided in the above embodiments, and other technical features of the tennis robot receiving device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0065] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0067] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the ball-receiving method of the tennis robot in the above embodiments.

[0068] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0069] The aforementioned computer-readable storage medium may be included in the ball-receiving device of the tennis robot; or it may exist independently and not assembled into the ball-receiving device of the tennis robot.

[0070] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the receiving device of a tennis robot, cause the receiving device of the tennis robot to: acquire color and depth images of the tennis court area; determine the tennis robot's own position information and the target tennis ball's position information based on the color and depth images; predict the future position information of the target tennis ball based on multiple tennis ball position information acquired at consecutive times; and control the tennis robot to move to the receiving position based on the tennis robot's own position information and the future position information.

[0071] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0073] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0074] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the ball-receiving method of the tennis robot described above. This solves the technical problem of how to control the tennis robot to actively receive the ball, thereby improving the user experience. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the ball-receiving method of the tennis robot provided in the above embodiments, and will not be repeated here.

[0075] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the tennis robot's ball-receiving method as described above.

[0076] The computer program product provided in this application solves the technical problem of how to control a tennis robot to actively receive the ball, thereby improving the user experience. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the tennis robot ball-receiving method provided in the above embodiments, and will not be repeated here.

[0077] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for receiving balls using a tennis robot, characterized in that, The method includes: Acquire color and depth images of the tennis court area; Based on the color image and the depth image, the position information of the tennis robot and the position information of the target tennis ball are determined. Based on the acquired tennis ball position information at multiple consecutive moments, predict the future position information of the target tennis ball; Based on the tennis robot's own position information and its future position information, the tennis robot is controlled to move to the receiving position.

2. The method as described in claim 1, characterized in that, The step of determining the tennis robot's own position information and the target tennis ball's position information based on the color image and the depth image includes: Identify fixed features of the tennis court in the color image, and calculate the tennis robot's own position information in the tennis court area based on the fixed features of the tennis court and the depth image; The tennis ball's position information within the tennis court area is calculated based on the color image, the depth image, and its own position information.

3. The method as described in claim 2, characterized in that, The step of identifying fixed features of the tennis court in the color image and calculating the tennis robot's own position information in the tennis court area based on the fixed features of the tennis court and the depth image includes: The net features and court line features are segmented from the color image; The image coordinates of the center point of the court in the tennis robot coordinate system are determined based on the net features, the court line features, and the depth image. The position information of the tennis robot in the court coordinate system can be deduced from the image coordinates.

4. The method as described in claim 2, characterized in that, The step of calculating the tennis ball's position information in the tennis court area based on the color image, the depth image, and its own position information includes: Tennis features were identified in the color image; The tennis image coordinates are obtained based on the tennis ball features and the depth image; The tennis ball's position information within the tennis court area is calculated based on the tennis ball image coordinates and its own position information.

5. The method as described in claim 1, characterized in that, The step of predicting the future position information of the target tennis ball based on multiple tennis ball position information obtained at consecutive time points includes: A tennis motion model is obtained by curve fitting based on multiple tennis ball position information at consecutive time points; The future position information of the target tennis ball is calculated based on the tennis motion model.

6. The method as described in claim 1, characterized in that, The step of controlling the tennis robot to move to the receiving position based on its own position information and the future position information includes: Determine whether the target tennis ball has bounced after hitting the ground; If the target tennis ball does not land, the tennis robot is controlled to align horizontally with the future position information based on its own position information and the future position information. If the target tennis ball has bounced after landing, then based on the horizontal alignment, the tennis robot is further controlled to move vertically to the receiving position according to its own position information and the future position information.

7. The method as described in claim 1, characterized in that, Following the steps of acquiring color and depth images of the tennis court area, the method further includes: The movement direction of all tennis balls in the tennis court area is determined based on the color image and the depth image; Based on the direction of motion, tennis balls whose direction of motion is toward the tennis robot are selected as target tennis balls.

8. A ball-receiving device for a tennis robot, characterized in that, The device includes: The acquisition module is used to acquire color and depth images of the tennis court area; The determination module is used to determine the position information of the tennis robot itself and the position information of the target tennis ball based on the color image and the depth image. The prediction module is used to predict the future position information of the target tennis ball based on the tennis ball position information obtained at multiple consecutive moments. The control module is used to control the tennis robot to move to the receiving position based on the tennis robot's own position information and the future position information.

9. A ball-receiving device for a tennis robot, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the ball-receiving method of the tennis robot as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the ball-receiving method of the tennis robot as described in any one of claims 1 to 7.

Images