Tennis training virtual-real fusion multi-device cooperative control system and method

Abstract

The invention discloses a tennis training virtual-real fusion multi-device cooperative control system and method.The system comprises training equipment and a controller, the training equipment is connected with the controller, and the training equipment comprises a virtual building assembly, an image collecting device, a ball serving robot and a ball picking robot; the virtual construction component is used for constructing a half virtual scene of the opposite side on the training field; the image acquisition device is used for acquiring motion data of a player in a training field, and the motion data comprises relevant information of a ball; the serving robot is used for serving and serving towards the direction of the player. Through the system, linkage among equipment such as a virtual construction assembly, an image acquisition device, a ball serving robot and a ball picking robot in training equipment is realized, and the training system integrating intelligent ball serving, analysis and ball picking is formed. In addition, the system also has the advantages of reducing the demand for the size of a site, reducing the human capital investment, improving the training effect and the like.

Description

Technical Field

[0001] This invention relates to the field of sports equipment, specifically to a multi-device collaborative control system and method for tennis training that integrates virtual and real elements. Background Technology

[0002] Currently, tennis training courts generally adopt a decentralized control mode with independent equipment. The serve robot system controls the serve trajectory based on a preset program, but lacks the function of automatically selecting the serve mode based on the athlete's level; the ball retrieval device: is mostly a manually operated auxiliary ball retrieval device; the training net or training wall: is mostly a simple screen or wall, without real-time scene display and various training data. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-device collaborative control system and method for virtual and real integration in tennis training, which solves one or more of the problems in the prior art.

[0004] This invention proposes a virtual-real fusion multi-device collaborative control system for tennis training, including training equipment and a controller. The training equipment is connected to the controller, and the training equipment includes a virtual construction component, an image acquisition device, and a serving robot.

[0005] The virtual construction component is used to construct a virtual scene of the opponent's half of the court on the training field;

[0006] The image acquisition device is used to collect the player's movement data in the virtual training field. The movement data includes ball-related information, including the ball's trajectory.

[0007] The serving robot is used to perform serving operations, and the serving robot serves the ball towards the player's location;

[0008] The controller controls the operation of the training equipment connected to it.

[0009] In some implementations...

[0010] A multi-device collaborative control system for virtual and real-world tennis training is installed on the training field;

[0011] The virtual-real integrated multi-device collaborative control system for tennis training also includes a mobile terminal, which is connected to the controller and / or training equipment.

[0012] In some implementations...

[0013] The mobile terminal is connected to the controller and the virtual building components;

[0014] The training ground is divided into a training area and a work area, with players training in the training area.

[0015] The virtual construction component includes a projector and a screen. The projector is connected to the mobile terminal. The projector and the screen are suspended above the training area. The projector projects virtual images onto the screen to form a virtual scene of the opponent's half of the field. The virtual scene of the opponent's half of the field includes virtual meshes. The screen is set in the work area.

[0016] The image acquisition device includes a camera system, which includes at least one set of cameras mounted on the top of the screen to capture and record the user hitting the ball and the ball's movement to form a corresponding video, and transmit the video to the controller.

[0017] The serving robot is positioned on the same side as the curtain, and below the curtain.

[0018] In some implementations...

[0019] The camera system also includes a set of cameras positioned on the front side of the screen for capturing and recording information displayed on the screen;

[0020] A set of cameras mounted on top of the screen includes two cameras;

[0021] The training equipment also includes a ball retriever, which is connected to the controller and / or mobile terminal.

[0022] A method for collaborative control of multiple devices integrating virtual and real-world elements in tennis training, comprising the following steps:

[0023] Obtain shot data;

[0024] Predict the trajectory of the sphere;

[0025] Virtual display;

[0026] Strategy optimization;

[0027] Adjust the serve.

[0028] In some implementations...

[0029] The process of acquiring ball-hitting data is as follows: acquire the ball-hitting video captured by the camera system, identify the ball moving in the video, capture the key frame of the swing hitting the ball, acquire several frames of the ball's flight after the key frame, and combine the key frame with the ball-hitting data to form the ball-hitting data for that instance.

[0030] The process of predicting the trajectory of the ball is as follows: calculate the position of the ball in each frame of the ball hitting data, fit the data, and calculate the motion trend of the ball in the physical coordinate system after hitting the screen. The motion trend includes the ball's flight direction, flight speed, subsequent flight trajectory and landing point.

[0031] In some implementations...

[0032] The hitting video includes the video from the moment the racket hits the ball until the ball leaves the camera's field of view, and at least 3 frames are included in the several frames of the ball's flight after the keyframe starts.

[0033] During the prediction of the sphere's trajectory, dynamic coordinate conversion is performed to establish a mapping relationship between physical coordinates and the virtual scene, thereby obtaining the sphere's position coordinates in the virtual coordinate system.

[0034] In some implementations, during the virtual display process, the ball's flight trajectory and landing point are displayed in the projected virtual scene of the opponent's half of the court based on the ball's position coordinates in the virtual coordinate system.

[0035] In some implementations, the player's skill level is determined based on real-time data of the trainee's shots, including ball speed, ball landing point on the virtual court, shot success rate, and number of consecutive shots, and a training mode that matches the player's skill level is matched (multiple mode algorithms are pre-stored and can be iteratively upgraded online).

[0036] In some implementations, the process of adjusting the serve is as follows: based on the deviation of the virtual landing point in the virtual scene, the serving robot is dynamically adjusted using a pre-defined algorithm to dynamically adjust the difficulty of the serve, thereby achieving the goal of adjusting the training difficulty in real time according to the player's level. The algorithm formula for dynamically adjusting the difficulty of the serve is as follows:

[0037]

[0038] In the formula: V new To adjust the speed of the serve, V base The base serve speed, 'a' is the ball speed adjustment gain coefficient, and ΔP virtual S represents the virtual landing point deviation in the virtual scene. difficulty k represents the dynamic difficulty coefficient. v t is the Sigmoid steepness coefficient for ball speed adjustment. 0v The ball speed difficulty activation time constant, t is the duration of existence, e is the natural constant, and w new To adjust the serve spin speed, w base The base rotational speed, β is the rotational adjustment gain coefficient, and k w t is the rotationally adjusted Sigmoid steepness coefficient. 0w Adjust the time constant to suit the difficulty level.

[0039] The advantages of the tennis training virtual-real integrated multi-device collaborative control system and method described in this invention are: it effectively reduces the need for court size, and at the same time, it reduces the investment of human capital through the system settings, forming a training system that integrates intelligent serving, analysis, and ball retrieval, thereby improving the training effect. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the framework of a multi-device collaborative control system for virtual and real tennis training, as described in some embodiments of the present invention.

[0041] Figure 2 This is a schematic diagram of the training ground structure in some embodiments of the present invention.

[0042] Figure 3 This is a schematic diagram showing the relative positions of two cameras and a sphere in some embodiments of the present invention.

[0043] Figure 4 This is a schematic diagram of the imaging of a sphere in a camera system according to some embodiments of the present invention. Detailed Implementation

[0044] A player is someone who trains with a ball, such as someone who trains in tennis.

[0045] Combination Figure 1 The embodiment presented here is a multi-device collaborative control system for tennis training that integrates virtual and real elements. The system includes a controller and training equipment. The training equipment comprises a virtual construction component, an image acquisition device, and a serving robot. The training equipment is connected to the controller. This multi-device collaborative control system for tennis training is deployed on the training field as needed, and combines... Figure 2 As shown, the training ground can be divided into a training area and a work area. Usually, players perform training exercises in the training area.

[0046] The virtual-real integrated multi-device collaborative control system for tennis training includes a mobile terminal, which can be a smart terminal with a screen display, such as a tablet or mobile phone.

[0047] The aforementioned controller includes the main control board.

[0048] The virtual construction components include a projector and a screen. The projector is suspended above the training area, and the screen is set up in the work area. The projector is connected to a mobile device and / or controller. The projector is used to project AR rendering data acquired from the mobile device (including the interior and exterior of the opponent's half of the virtual scene, virtual humans, and the flight trajectory and landing point of the tennis ball in the virtual environment) onto the screen to form the opponent's half of the virtual scene. The opponent's half of the virtual scene may include virtual netgear, virtual target area, etc.

[0049] The image acquisition device is a camera system (specifically, a high-speed camera system). The camera in this system (which can be a binocular global shutter camera) can be mounted on top of the screen (when mounted on top of the screen, the camera is generally shot from a slightly downward angle, starting at eye level, with a downward tilt angle typically between 0-90°, primarily to ensure a clear view of the training area; the angle can be adjusted manually). This is used to capture and record the user's shot and the ball's trajectory, forming corresponding video. If there is display information on the screen that needs to be captured, at least one camera device (e.g., a webcam) connected to the shooting system can be added to the front of the screen to capture and record the information displayed on the screen.

[0050] The serving robot and the screen are located on the same side of the training field (working area). The starting point of the serving robot is set on the ground below the center of the screen, and the launching direction of the serving robot is set towards the training area where the player is located. This setup breaks the traditional opposing layout of training fields, saving 50% of the field space and reducing the requirements for the size of the training field.

[0051] The main control board is equipped with a communication module, which can connect to a mobile terminal via wired and / or wireless (such as WiFi, Bluetooth) means, allowing users to interact with the controller and other devices through the mobile terminal.

[0052] The control method for a multi-device collaborative control system that integrates virtual and real-device interaction in tennis training includes the following steps:

[0053] Obtain shot data;

[0054] Trajectory prediction;

[0055] Virtual display;

[0056] Strategy optimization: The system intelligently determines the trainee's skill level based on real-time data of the trainee's shots, including ball speed, ball landing point on the virtual court (within the boundaries, within the service area, etc.), shot success rate (forehand, backhand, volley, etc.), and number of consecutive shots. For example, according to the NTRP grading standard, the above data is automatically compared with the corresponding grading data of the standard to determine the trainee's level. The system also intelligently matches a training mode that is equivalent to the player's skill level. (Multiple training mode algorithms are pre-stored, and the algorithm is set to correspond to the skill level. The system automatically selects the algorithm corresponding to the skill level. Each algorithm includes the ball machine's footwork and serving speed, the corresponding screen and data display in the virtual construction components, etc.)

[0057] Adjust your serve.

[0058] Taking a camera system mounted on top of a screen as an example, which includes two cameras, namely the left camera and the right camera.

[0059] The specific process for obtaining shot data can be as follows:

[0060] Acquire the video of the ball being hit captured by the camera system. Generally, the video of the ball being hit includes the video from the moment the racket hits the ball until the ball leaves the camera's field of view.

[0061] Identify a moving sphere;

[0062] Capture keyframes of the swing and the impact of the ball;

[0063] Acquire several frames (at least 3 frames, and at most all frames from the start of the shot until the ball leaves the camera's field of view) of the ball's flight after the keyframe, and combine them with the keyframe to form the shot data.

[0064] The specific process of trajectory prediction can be as follows: calculate the position (e.g., three-dimensional coordinates) of the ball in each frame of the shot data, fit the data, and calculate the motion trend of the ball in the three-dimensional coordinate system after hitting the screen. The motion trend includes the ball's flight direction, flight speed, subsequent flight trajectory, and landing point.

[0065] The controller can train a model of a moving ball using PyTorch, capture the pixel position of the ball in each frame of the hitting video, and capture keyframes of the user's swing hitting the ball.

[0066] Acquire and store multiple shot videos. A portion of these videos will be used as a training video set, and the remaining portion as a test video set. The number of shot videos in each set can be randomly assigned. The model training process includes the following steps:

[0067] S2.1 Convert the hitting videos in the training video set frame by frame into images;

[0068] S2.2. Label the pixel positions corresponding to the target (e.g., sphere) in the image to provide the model with the pixel positions of the target image in subsequent model training. The labeled images form the training set (generally, the training set includes at least 2000 labeled images).

[0069] S2.3 Training: Build and apply the training set to train the model;

[0070] S2.4 Test: Import the videos from the test video set into the model for testing. If the model can identify the target (i.e., the sphere) in the video frame by frame and the recognition accuracy is greater than the preset threshold (e.g., 90%), it can be judged as accurate recognition and the model training is completed; otherwise, it is judged as inaccurate recognition and proceeds to steps S2 and 5.

[0071] S2.5 Add new hitting videos to the training video set, then re-enter S2.1.

[0072] The model training steps may also include: S2.6, saving the images in S2.1, randomly adjusting the brightness of the images in S2.1 and saving them again to obtain more materials with different brightness, and then proceeding to S2.2.

[0073] The object whose brightness is adjusted and saved in S2.6 can also be adjusted to the labeled image described in S2.2.

[0074] During the execution of S2.3, the gradient descent optimization algorithm can be used to calculate the gradient of the loss function with respect to the weights, and then update the weights in the opposite direction of the gradient, so that the loss function gradually decreases, thereby obtaining a more effective training effect.

[0075] The controller calculates the straight-line distance between the sphere and the cameras based on the positional difference between the two input cameras. Using this straight-line distance and the angle formed by the camera's center of view and the sphere, it calculates the sphere's three-dimensional coordinates in different frames. By fitting the three-dimensional coordinates of all spheres in the current hitting data, the controller can calculate the sphere's motion trend in the three-dimensional coordinate system after hitting the screen. This is achieved by combining... Figure 3 The calculation process for the straight-line distance between the sphere and the camera, as shown, is as follows:

[0076] Let b be the distance between the lines connecting the projection centers of the two cameras, P be a point in the three-dimensional space of the sphere, and P be the imaging point of the left camera. L The imaging point of the right camera is P. R In three-dimensional space, point P (i.e., the position of the tennis ball in three-dimensional space) is the intersection of the lines connecting the projection centers of the two cameras and the imaging point. X L and X R Let be the distances from the imaging points of the left and right cameras to their respective imaging surfaces, respectively. Then, the parallax d of a point P in three-dimensional space between the two cameras is:

[0077] d=|X L -X R |,

[0078] Two imaging points P L To P R Distance P between L P R for

[0079]

[0080] Based on the theory of similar triangles, we can conclude that...

[0081]

[0082] In the above formula, f represents the focal length of the camera.

[0083] Therefore, the distance Z from point P in three-dimensional space to the projection plane is...

[0084]

[0085] This distance Z can be used as the Z-axis coordinate value of the sphere in the camera coordinate system. c .

[0086] The image of the sphere in the camera is as follows: Figure 4 As shown, the coordinates X and Y of the sphere in the camera coordinate system are... c and Y c They are respectively:

[0087]

[0088] A dynamic coordinate transformation algorithm is used between trajectory prediction and virtual display to establish a mapping relationship between physical coordinates and the virtual scene. Dynamic projection calibration is achieved through a checkerboard calibration and compensation algorithm. The mapping relationship is as follows:

[0089]

[0090] In the formula: X v Y v Y v X represents the position coordinates of the sphere in the virtual coordinate system. c Y c Z c R represents the position coordinates of the sphere in the physical coordinate system. calib The rotation matrix is ​​used for calibration (which can be obtained through checkerboard calibration), and k is the velocity attenuation coefficient (to compensate for visual measurement errors, typically ranging from 0.95 to 1.05). Let t be the initial velocity vector of the ball (captured and calculated by the image acquisition device), and t be the motion time (the flight time of the tennis ball from its launch to this coordinate system, calculated from the moment of impact, captured and calculated by the image acquisition device). C is the gravitational acceleration vector. d ρ is the air drag coefficient (using a typical value for a tennis ball, which can range from 0.5 to 0.6; a typical value of 0.5 generally indicates a worn tennis ball, while a value of 0.6 generally indicates a new tennis ball. The specific value depends on factors such as the surface texture of the tennis ball and can be obtained through wind tunnel experiments or other existing techniques such as fluid dynamics testing), ρ is the air density, A is the cross-sectional area of ​​the ball facing the wind, v is the instantaneous velocity scalar of the ball, and m is the mass of the ball. T is a unit vector in the direction of velocity. 0ffset The coordinate translation compensation amount (representing the offset between the origin of the physical coordinate system and the origin of the virtual scene coordinate system, which can be achieved using existing precise measurement or calibration methods) is calculated. The precise measurement method uses a total station, laser rangefinder, etc., to accurately measure the coordinates of physical and virtual reference points, determining the offset between them to obtain Toffset, ensuring precise alignment between physical and virtual coordinates. The calibration method uses multiple known marker points, through image capture and calculation, to find the optimal translation relationship between the physical and virtual coordinate systems, obtaining Toffset, achieving precise matching between the physical and virtual scenes. This conversion algorithm effectively controls the conversion error within 3mm, effectively solving the projection misalignment problem caused by viewing angle deviation in traditional systems.

[0091] In the specific process of virtual display, the sphere's flight trajectory and landing point are displayed in the projected virtual scene of the opposite half of the field based on the sphere's position coordinates in the virtual coordinate system. After receiving the data transmitted from the vision, Unity initializes the physical state of a sphere object. Based on Unity's physics engine, the parsed velocity vector is assigned to the sphere's initial velocity, and a rotational torque or custom force field is constructed based on the spin parameters to simulate the offset effect of the sphere rotating and flying in the air.

[0092] During the simulation, Unity utilizes its built-in physics calculation system to simulate the sphere's trajectory in real time, including the effects of gravity, rotational offset, air resistance, and collision responses. To visualize the trajectory, the system adds trajectory rendering components (such as TrailRenderer or LineRenderer) to the sphere and can overlay particle effects to enhance the visual presentation.

[0093] When the sphere collides physically with the ground, the coordinates of the landing point are captured through the event and visually labeled in the virtual scene.

[0094] The Unity system supports multi-view switching and trajectory replay functions. Users can switch between multiple preset views such as player view and top-down view as needed to observe the tennis ball's flight process from different angles.

[0095] In the specific process of serving adjustment: based on the virtual landing point deviation in the virtual scene (the horizontal distance between the target landing point and the actual landing point; for example, if a player hits the virtual target area displayed on the virtual net 5 times consecutively: the serving speed increases by 10%, and the virtual target area shrinks by 20%; if there are 3 consecutive errors, the serving spin decreases by 15%, and the virtual target area remains unchanged), the serving robot can be dynamically adjusted through a set algorithm to achieve the goal of adjusting the training difficulty in real time according to the player's level. The algorithm formula for dynamically adjusting the serving difficulty is as follows:

[0096]

[0097] In the formula: V new To adjust the speed of the serve, V base The base serve speed (preset according to the training mode), 'a' is the ball speed adjustment gain coefficient (to control the influence of virtual landing point deviation on ball speed; its value range can be set to 0.1-0.5, specifically obtained through system calibration experiments such as regression analysis of 100 hits), ΔP virtual S represents the virtual landing point deviation in the virtual scene (the horizontal distance between the virtual landing point and the virtual target area, which can be automatically calculated directly from the deviation between the virtual landing point and the virtual target area). difficulty The dynamic difficulty coefficient (reflecting the current training difficulty level, 0.5 for beginner to 1.5 for professional, based on the success rate of trainees; it can be automatically rated and matched by the algorithm, such as according to the NTRP rating method, or the parameter can be manually selected on the mobile device), k v This is the Sigmoid steepness coefficient for ball speed adjustment (to control the smoothness of the adjustment rate; the value range of this Sigmoid steepness coefficient for ball speed adjustment can be 0.05-0.2, with a default of 0.1. The lower range is suitable for beginners and can be manually adjusted on the mobile device). 0v t is the ball speed difficulty activation time constant (representing the time required to reach 50% difficulty adjustment, default is 300s, can be set on mobile devices), t is the duration of the connection (starting from the current training phase, its value ranges from 0-3600s), e is the natural constant (approximately 2.71828), and w is the ball speed difficulty activation time constant. new To adjust the serve spin speed, w base The base spin speed (preset according to the training mode), β is the spin adjustment gain coefficient (controlling the influence of virtual landing point deviation on spin speed, its value range can be 0.3-0.8, specifically obtained through system calibration experiments such as regression analysis of 100 shots), k w This is the Sigmoid steepness factor for rotational adjustment (to control the smoothness of the adjustment rate; the Sigmoid steepness factor for rotational adjustment can range from 0.05 to 0.2, with a default of 0.1; the lower range is suitable for beginners), t 0w The time constant for difficulty adjustment (representing the time required to reach 50% difficulty adjustment; the default value is 300s, and it can be manually set directly on the mobile device; maximum difficulty refers to the maximum serving speed and maximum spin angle of the serving robot). The system has initial values ​​for the above parameters, which can be manually adjusted by selecting the difficulty mode on the app installed on the mobile device.

[0098] The training modes described above may include the first mode and the second mode as follows. These training modes can be manually selected via an app installed on a mobile device.

[0099] The first mode is the dynamic difficulty training mode. Under the dynamic difficulty training mode:

[0100] Initialization: Players select "Precision Landing Point Training".

[0101] Controller settings:

[0102] Serving robot: Initial speed 50 mph, topspin 2500 rpm;

[0103] Virtual projection: Displays a 1m x 1m virtual target area on a screen (size can be selected on mobile devices);

[0104] Adaptive adjustment: If 5 balls hit the target area displayed on the screen in a row (error <10cm), the target area is reduced to 0.8m×0.8m and the ball speed is increased to 55mph; if 3 balls miss the target in a row (error >30cm), the ball speed is reduced to 45mph.

[0105] The second mode is the tactical combat mode, in which:

[0106] Virtual players are displayed in the opponent's half of the virtual arena on the screen. Players can manually or automatically select and load opponents of different levels (based on the assessment of the player's training level). The opponent's pre-movement trend is displayed on the screen using bright lines and other methods.

[0107] The multi-device coordination and control system for the intelligent tennis training court based on virtual fusion also has a real-time feedback function. This real-time feedback function projects data such as ball speed, hit success rate, and landing accuracy (calculated by the vision processing system and Unity engine) onto the screen and / or transmits them to the mobile terminal in real time.

[0108] The training equipment may also include a ball-collecting robot, which can be selectively connected to a controller and / or a mobile terminal. When it is necessary to link with a ball-serving robot, the ball-collecting robot connects to the controller.

[0109] This structural design enables the formation of a dynamic collaborative system between the virtual environment and physical devices, centered on a controller and built upon a communication architecture via wireless networks (Wi-Fi / Bluetooth). In the tennis training virtual-real fusion multi-device collaborative control system, the data flow can complete a full chain under the collaborative control of the controller: generating a start signal → image acquisition device capturing actions → virtual construction components generating virtual data and projecting it → serving robot serving → ball-collecting robot collecting the ball. The entire closed-loop collaborative process achieves millisecond-level latency from end to end.

[0110] The control method based on the virtual-real fusion multi-device collaborative control system may also include the collaborative control of the ball-collecting robot. The controller will coordinate the working rhythm of the ball-serving robot and the ball-collecting robot according to the training progress and the player's needs.

[0111] If the training intensity is high, the serving robot will serve at a higher frequency, and the ball-collecting robot will increase the frequency of ball collection.

[0112] Conversely, when training intensity is low, the ball-collecting robot can appropriately reduce its working frequency to decrease energy consumption. The controller monitors the status of the serving robot and the ball-collecting robot in real time, including information such as the number, position, and working status of the balls, and controls the ball-collecting robot to set off to collect balls when the serving robot is in a safe state (resting or not moving), ensuring safety.

[0113] The above training modes can also be adjusted as needed.

[0114] In other embodiments, the ball-collecting robot, the ball-serving robot, and the image acquisition device may also be directly connected to a mobile terminal, and the mobile terminal may directly interact with one or more of the ball-collecting robot, the ball-serving robot, and the image acquisition device.

[0115] The corresponding app for the control method of the above-mentioned tennis training virtual-real fusion multi-device collaborative control system is installed on the mobile terminal. Through the program settings in the app, all or part of the above data can be viewed and analyzed on the mobile terminal. The data includes video captured by the image acquisition device, real-time data of the trainee's ball hitting, the ball's flight trajectory, landing point, etc.

[0116] All of the above-mentioned undisclosed matters can be directly implemented using existing technologies, so they will not be elaborated here.

[0117] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several similar modifications and improvements can be made without departing from the inventive concept of the present invention, and these should also be considered within the scope of protection of the present invention.

Claims

1. A multi-device collaborative control system for virtual and real-world tennis training, characterized in that, It includes training equipment and a controller, with the training equipment connected to the controller. The training equipment includes a virtual construction component, an image acquisition device, and a serving robot. The virtual construction component is used to construct a virtual scene of the opponent's half of the court on the training field; The image acquisition device is used to collect the player's movement data in the virtual training field. The movement data includes ball-related information, including the ball's trajectory. The serving robot is used to perform serving operations, and the serving robot serves the ball towards the player's location; The controller controls the operation of the training equipment connected to it.

2. The tennis training virtual-real fusion multi-device collaborative control system according to claim 1, wherein, A multi-device collaborative control system for virtual and real-world tennis training is installed on the training field; The virtual-real integrated multi-device collaborative control system for tennis training also includes a mobile terminal, which is connected to the controller and / or training equipment.

3. The tennis training virtual-real fusion multi-device collaborative control system according to claim 2, wherein, The mobile terminal is connected to the controller and the virtual building components; The training ground is divided into a training area and a work area, with players training in the training area. The virtual construction component includes a projector and a screen. The projector is connected to the mobile terminal. The projector and the screen are suspended above the training area. The projector projects virtual images onto the screen to form a virtual scene of the opponent's half of the field. The virtual scene of the opponent's half of the field includes virtual meshes. The screen is set in the work area. The image acquisition device includes a camera system, which includes at least one set of cameras mounted on the top of the screen to capture and record the user hitting the ball and the ball's movement to form a corresponding video, and transmit the video to the controller. The serving robot is positioned on the same side as the curtain, and below the curtain.

4. The tennis training virtual-real fusion multi-device collaborative control system according to claim 3, wherein, The camera system also includes a set of cameras positioned on the front side of the screen for capturing and recording information displayed on the screen; A set of cameras mounted on top of a screen consists of two or more cameras; The training equipment also includes a ball retriever, which is connected to the controller and / or mobile terminal.

5. A method for collaborative control of multiple devices integrating virtual and real technologies in tennis training, characterized in that, The method for performing collaborative control of multiple devices in tennis training using the virtual-real fusion multi-device collaborative control system described in claim 4 includes the following steps: Obtain shot data; Predict the trajectory of the sphere; Virtual display; Strategy optimization; Adjust the serve.

6. The method for collaborative control of multiple devices integrating virtual and real elements in tennis training according to claim 5, wherein, The process of acquiring ball-hitting data is as follows: acquire the ball-hitting video captured by the camera system, identify the ball moving in the video, capture the key frame of the swing hitting the ball, acquire several frames of the ball's flight after the key frame, and combine the key frame with the ball-hitting data to form the ball-hitting data for that instance. The process of predicting the trajectory of the ball is as follows: calculate the position of the ball in each frame of the ball hitting data, fit the data, and calculate the motion trend of the ball in the physical coordinate system after hitting the screen. The motion trend includes the ball's flight direction, flight speed, subsequent flight trajectory and landing point.

7. The method for collaborative control of multiple devices integrating virtual and real elements in tennis training according to claim 6, wherein, The hitting video includes the video from the moment the racket hits the ball until the ball leaves the camera's field of view, and at least 3 frames are included in the several frames of the ball's flight after the keyframe starts. During the prediction of the sphere's trajectory, dynamic coordinate conversion is performed to establish a mapping relationship between physical coordinates and the virtual scene, thereby obtaining the sphere's position coordinates in the virtual coordinate system.

8. The method for collaborative control of multiple devices integrating virtual and real elements in tennis training according to claim 7, wherein, In the specific process of virtual display, the flight trajectory and landing point of the sphere are displayed in the virtual scene of the opponent's half of the field based on the position coordinates of the sphere in the virtual coordinate system.

9. The method for collaborative control of multiple devices integrating virtual and real elements in tennis training according to claim 8, wherein, The strategy optimization process involves: based on the real-time data of the trainees' shots, including ball speed, ball landing point on the virtual court, shot success rate, and number of consecutive shots, determining the player's level and matching a training mode that is equivalent to the player's level.

10. The method for collaborative control of multiple devices integrating virtual and real elements in tennis training according to claim 9, wherein, The process of adjusting the serve is as follows: Based on the deviation of the virtual landing point in the virtual scene, the serving robot is dynamically adjusted using a pre-set algorithm to dynamically adjust the difficulty of the serve. This achieves the goal of adjusting the training difficulty in real time according to the player's level. The algorithm formula for dynamically adjusting the difficulty of the serve is: In the formula: V new To adjust the speed of the serve, V base The base serve speed, 'a' is the ball speed adjustment gain coefficient, and ΔP virtual S represents the virtual landing point deviation in the virtual scene. difficulty k represents the dynamic difficulty coefficient. v t is the Sigmoid steepness coefficient for ball speed adjustment. 0v The ball speed difficulty activation time constant, t is the duration of existence, e is the natural constant, and w new To adjust the serve spin speed, w base The base rotational speed, β is the rotational adjustment gain coefficient, and k w t is the rotationally adjusted Sigmoid steepness coefficient. 0w Adjust the time constant to suit the difficulty level.

Images