Tennis training data processing method, system and equipment and storage medium

Abstract

The invention discloses a tennis training data processing method, system and device and a storage medium, and belongs to the technical field of tennis teaching. The tennis training data processing method comprises the steps that a serving instruction is transmitted to the tennis serving machine, so that the tennis serving machine serves a user training area in front of the curtain; obtaining an ith frame of curtain image shot by the first camera; converting the ith frame of curtain image into an HSV color space to obtain a to-be-processed image, and determining a tennis ball contour according to an area with a color parameter within a preset range in the to-be-processed image; judging whether the circularity of the tennis ball contour is greater than a preset value or not; if yes, determining a ball falling position according to the position of the tennis ball contour in the ith frame of curtain image; and if not, adding 1 to the numerical value of i, and entering the step of obtaining the ith frame of curtain image shot by the first camera. The method can automatically detect the ball falling position in the tennis training process.

Description

Technical Field

[0001] The present application relates to the technical field of tennis teaching, and in particular to a method, system, device and storage medium for processing tennis training data. Background Art

[0002] At present, professional tennis courts mostly use intelligent tennis training robots to assist users in completing training tasks. Specifically, the intelligent tennis training robot can launch the tennis ball at a set speed, angle and rotation speed, helping users to practice serving efficiently and improve their skills.

[0003] Accurately analyzing the ball drop position is a key factor in evaluating the user's technical level. However, the above-mentioned related technical solutions can only rely on the coaches in the venue to record the ball drop position, which has low accuracy and efficiency.

[0004] Therefore, how to automatically detect the ball landing position during tennis training is a technical problem that technicians in this field currently need to solve. Summary of the invention

[0005] The purpose of this application is to provide a method, system, device and storage medium for processing tennis training data, which can automatically detect the ball landing position during tennis training.

[0006] In order to solve the above technical problems, the present application provides a method for processing tennis training data, which is applied to a central controller of a tennis training device, wherein the tennis training device also includes a tennis serving machine and a first camera, wherein the first camera is used to shoot an area where the screen is located, and the screen is used to receive a tennis ball hit back by a user, and the method for processing tennis training data includes:

[0007] Transmitting a serving instruction to the tennis serving machine so that the tennis serving machine serves the ball to the user training area in front of the curtain;

[0008] Obtaining the i-th frame of the curtain image captured by the first camera;

[0009] Convert the i-th frame of the curtain image into the HSV color space to obtain an image to be processed, and determine the outline of the tennis ball according to an area in the image to be processed where the color parameter is within a preset range; wherein the color parameter includes any one or a combination of any several of hue, saturation and brightness;

[0010] Determining whether the roundness of the tennis ball outline is greater than a preset value;

[0011] If yes, then determine the ball drop position according to the position of the tennis ball outline in the i-th frame of the screen image; wherein the ball drop position is the position where the tennis ball hit back by the user contacts the screen;

[0012] If not, the value of i is increased by 1, and the process proceeds to the step of obtaining the i-th frame of the curtain image taken by the first camera.

[0013] Optionally, determining the outline of the tennis ball according to an area in the image to be processed whose color parameters are within a preset range includes:

[0014] Creating a mask on the image to be processed so as to retain a target area; wherein the target area is an area where the color parameter is within the preset range;

[0015] Gaussian blur processing is performed on the target area, and contour detection and interference removal operations are performed on the Gaussian blur processing result to obtain the tennis ball contour.

[0016] Optionally, the tennis training device further comprises a Bluetooth gateway, wherein the Bluetooth gateway is connected to a plurality of Bluetooth Mesh lights, wherein the Bluetooth Mesh lights are arranged behind the curtain, and a plurality of scoring areas are arranged on the curtain, and the arrangement positions of the Bluetooth Mesh lights correspond to the scoring areas;

[0017] Accordingly, it also includes:

[0018] determining a current training mode of the tennis training device;

[0019] If the current training mode is the first mode, when the ball landing position is in the scoring area, the Bluetooth Mesh light corresponding to the scoring area where the ball landing position is located is controlled by the Bluetooth gateway to light up;

[0020] Delay for a preset time length to control all the Bluetooth Mesh lights to turn off; wherein the preset time length is less than the serving interval length of the tennis serving machine.

[0021] Optionally, after determining the current training mode of the tennis training device, the method further includes:

[0022] If the current training mode is the second mode, before the tennis serving machine performs a serving operation, a target scoring area is selected from all the scoring areas, and a Bluetooth Mesh light corresponding to the target scoring area is controlled to light up;

[0023] After determining the ball landing position, determining whether the ball landing position is within the target scoring area;

[0024] If so, the Bluetooth Mesh light corresponding to the target scoring area is controlled to turn off, and the step of selecting the target scoring area from all the scoring areas is entered.

[0025] Optionally, the tennis training device further includes a second camera, and the second camera is used to photograph the user training area;

[0026] Accordingly, it also includes:

[0027] Acquire a user posture image captured by the second camera;

[0028] Performing posture estimation based on the user posture image to obtain human body feature points and racket feature points;

[0029] If the number of users in the user posture image is greater than 1, determining a target user according to the distance between the racket feature point and the human body feature point; wherein the target user is a user holding a racket;

[0030] The posture information is generated according to the human body feature points of the target user; wherein the posture information includes any one or a combination of any several of the left and right hand grip information, the forehand and backhand grip information, the squatting angle and the swinging angle.

[0031] Optionally, determining the target user according to the distance between the racket feature point and the human body feature point includes:

[0032] Determining wrist feature points of each of the users from the body feature points;

[0033] The user whose wrist feature point is closest to the racket feature point is set as the target user.

[0034] Optionally, transmitting a serving instruction to the tennis serving machine includes:

[0035] The serving parameters are set according to the ball landing position and ball speed of the user's last shot, and a serving instruction containing the serving parameters is transmitted to the tennis serving machine; wherein the serving parameters include ball delivery speed, ball delivery angle and ball delivery rotation speed.

[0036] The present application also provides a tennis training data processing system, which is applied to a central controller of a tennis training device, wherein the tennis training device further includes a tennis serving machine and a first camera, wherein the first camera is used to shoot an area where the screen is located, and the screen is used to receive a tennis ball hit back by a user, and the tennis training data processing system includes:

[0037] A serving control module, used for transmitting a serving instruction to the tennis serving machine, so that the tennis serving machine serves the ball to the user training area in front of the curtain;

[0038] An image acquisition module, used to acquire the i-th frame of the curtain image taken by the first camera;

[0039] a contour determination module, configured to convert the i-th frame of the curtain image into an HSV color space to obtain an image to be processed, and determine the tennis ball contour according to an area in the image to be processed where the color parameter is within a preset range; wherein the color parameter includes any one or a combination of any several of hue, saturation and brightness;

[0040] A judging module, used to judge whether the roundness of the tennis ball outline is greater than a preset value;

[0041] a ball drop position determination module, configured to determine the ball drop position according to the position of the tennis ball outline in the i-th frame of the screen image if the roundness of the tennis ball outline is greater than the preset value; wherein the ball drop position is the position where the tennis ball hit back by the user contacts the screen;

[0042] A loop module is used to add 1 to the value of i if the circularity of the tennis ball outline is less than or equal to the preset value, and enter the step of obtaining the i-th frame of the curtain image taken by the first camera.

[0043] The present application also provides a storage medium on which a computer program is stored. When the computer program is executed, the steps of the method for processing tennis training data are implemented.

[0044] The present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned tennis training data processing method when calling the computer program in the memory.

[0045] The present application provides a method for processing tennis training data. After a tennis ball serving machine serves a ball to a user's training area, the i-th frame of the screen image is obtained, the i-th frame of the screen image is converted into the HSV color space to obtain the image to be processed, and the area in the image to be processed where the color parameter is within a preset range determines the outline of the tennis ball. A tennis ball in flight has a significant trailing image, and a tennis ball in contact with the screen will appear as a perfect circle. The present application determines whether the tennis ball is in contact with the screen in the i-th frame of the screen image according to the circularity of the tennis ball outline. When the circularity of the tennis ball outline in the i-th frame of the screen image is greater than a preset value, the ball drop position can be determined according to the position of the tennis ball outline in the i-th frame of the screen image; when the circularity of the tennis ball outline in the i-th frame of the screen image is not greater than a preset value, the value of i is added by 1 to determine whether there is a ball drop position in the next frame of the screen image. Therefore, the present application can automatically detect the ball drop position during tennis training. The present application also provides a system for processing tennis training data, a storage medium, and a storage medium, which have the above-mentioned beneficial effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a method for processing tennis training data provided by an embodiment of the present application;

[0048] Figure 2 A top view of a tennis training venue provided in an embodiment of the present application;

[0049] Figure 3 A functional block diagram of a central controller provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of a tennis training provided by an embodiment of the present application;

[0051] Figure 5 A flowchart of serving processing in a single scoring statistics processing process provided by an embodiment of the present application;

[0052] Figure 6 A flowchart of serving processing in a game points statistics processing process provided by an embodiment of the present application;

[0053] Figure 7 A schematic diagram of the appearance of a screen provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of coordinates of key points of a curtain provided in an embodiment of the present application;

[0055] Fig. 9 A flow chart of gesture image recognition provided by an embodiment of the present application;

[0056] Fig.10 A schematic diagram of a key feature point extraction structure provided in an embodiment of the present application;

[0057] Fig.11 A schematic diagram of a key feature point extraction structure provided in an embodiment of the present application;

[0058] Fig.12 A schematic diagram of a system including multiple tennis training devices provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] See below Figure 1 , Figure 1 A flowchart of a method for processing tennis training data provided by an embodiment of the present application, the specific steps may include:

[0061] S101: transmitting a serving instruction to the tennis ball serving machine, so that the tennis ball serving machine serves the ball to the user training area in front of the screen;

[0062] Among them, this embodiment can be applied to the central controller of a tennis training device, and the tennis training device also includes a tennis ball serving machine and a first camera, the first camera is used to shoot the area where the screen is located, and the screen is used to receive the tennis ball hit back by the user. The color of the above-mentioned screen is different from the color of the tennis ball. As a feasible way, the color of the screen can include red, blue, white, etc. A ball catching cloth table can be set under the above-mentioned curtain. Since the curtain is very soft, when the tennis ball hits the curtain, most of its kinetic energy is absorbed, and the tennis ball can fall into the cloth table; a hole is set in the center of the cloth table, and the hole is connected to the ball serving machine so that the tennis ball can roll back into the tennis ball serving machine.

[0063] See also Figure 2 , Figure 2 An overhead view of a tennis training venue provided in an embodiment of the present application shows the relative position relationship of the Bluetooth Mesh light, the screen, the tennis serving machine and the user training area. The Bluetooth Mesh light is set behind the screen, the tennis serving machine is used to serve in front of the screen, and the user training area is set in front of the screen.

[0064] The central controller transmits the serve instruction to the tennis serve machine through a preset communication protocol (such as serial communication or network protocol). The serve instruction contains the specific parameters of the serve, such as the speed, angle and rotation of the serve. After receiving the serve instruction, the tennis serve machine adjusts its internal settings according to these parameters and automatically launches the tennis ball into the user's training area in front of the curtain.

[0065] In order to effectively simulate a tennis game scene, a serving instruction can be transmitted to the tennis serving machine in the following manner, including: setting serving parameters according to the ball landing position and ball speed of the user's last hit, and transmitting a serving instruction containing the serving parameters to the tennis serving machine; wherein the serving parameters include ball delivery speed, ball delivery angle, and ball delivery rotation speed.

[0066] S102: Acquire the i-th frame of the curtain image captured by the first camera;

[0067] In the tennis training process, in order to analyze the user's hitting situation, this embodiment obtains real-time image data from the first camera. Specifically, this embodiment can read the images captured by the camera frame by frame according to the shooting order, wherein each frame of the curtain image contains information about the curtain area. When S102 is executed for the first time, i is equal to 1.

[0068] S103: converting the i-th frame of the curtain image into an HSV color space to obtain an image to be processed, and determining the outline of the tennis ball according to an area in the image to be processed where the color parameters are within a preset range;

[0069] Among them, the above-mentioned i-th frame curtain image is an RGB (red, green, blue) image, and the i-th frame curtain image can be converted into an HSV (Hue, Saturation, Value) color space to obtain a to-be-processed image, where H in HSV represents hue, S represents saturation, and V represents brightness.

[0070] In this embodiment, a preset range of color parameters can be determined according to the color of the tennis ball, and the color parameters include any one or a combination of hue, saturation and brightness; for example, the hue is between 20°-45°, the saturation is between 60%-100%, and the brightness is between 70%-100%. In this embodiment, the area where the color parameters are within the preset range can be used as the area where the tennis ball outline is located.

[0071] S104: Determine whether the roundness of the tennis ball outline is greater than a preset value; if so, proceed to S105; if not, add 1 to the value of i and proceed to S102.

[0072] S105: determining a ball drop position according to the position of the tennis ball outline in the i-th frame of the screen image; wherein the ball drop position is a position where the tennis ball hit back by the user contacts the screen;

[0073] After the tennis ball outline is determined, the circularity of the tennis ball outline (i.e., the degree to which the outline is close to a circle) can be calculated. If the circularity is greater than a preset threshold (e.g., 88%), the outline is considered to be a tennis ball and the tennis ball has contacted the curtain.

[0074] If the circularity of the tennis ball outline is greater than a preset value, the exact position where the tennis ball hit back by the user touches the screen can be determined based on the position of the tennis ball outline in the i-th frame of the screen image as the ball landing position. If the circularity of the tennis ball outline is not greater than the preset value, it is considered that the contact between the tennis ball and the screen is not accurately captured in the current frame; at this time, the value of i is increased by 1, and the step of obtaining the next frame of the screen image taken by the first camera is entered to continue looking for a tennis ball outline that meets the conditions.

[0075] When processing tennis training data, there may be multiple frames of screen images in which the tennis ball is in contact with the screen. In order to ensure accuracy and timely feedback, this embodiment can determine the ball drop position based on the position of the first contour detected in the screen image that meets the circularity condition. That is, this embodiment can set the position of the first detected tennis ball contour with a circularity greater than a preset value in the corresponding screen image after serving as the ball drop position.

[0076] When the ball hits the screen, it rises continuously, and then drops after reaching the landing point. Therefore, when the ball rises, the outline of the tennis ball presents a trailing shape (i.e., an ellipse), and when it hits the screen, the outline of the tennis ball presents a round shape. The most round ball landing point can be identified as the final landing position. When the ball falls, the outline of the tennis ball presents a trailing shape (ellipse). Therefore, in this embodiment, when the roundness is greater than a preset value, the position of the tennis ball outline with the highest roundness in the corresponding screen image can be set as the landing position.

[0077] The present embodiment provides a method for processing tennis training data. After a tennis ball serving machine serves a ball to a user's training area, the i-th frame of the screen image is obtained, the i-th frame of the screen image is converted into the HSV color space to obtain an image to be processed, and the outline of the tennis ball is determined in the area of ​​the image to be processed where the color parameters are within a preset range. A tennis ball in flight has a significant trailing image, and a tennis ball in contact with the screen will appear as a perfect circle. The present embodiment determines whether the tennis ball is in contact with the screen in the i-th frame of the screen image based on the circularity of the tennis ball outline. When the circularity of the tennis ball outline in the i-th frame of the screen image is greater than a preset value, the ball drop position can be determined based on the position of the tennis ball outline in the i-th frame of the screen image; when the circularity of the tennis ball outline in the i-th frame of the screen image is not greater than a preset value, the value of i is increased by 1 to determine whether there is a ball drop position in the next frame of the screen image.

[0078] As for Figure 1 In a further description of the embodiment, the tennis ball outline can be determined from the image to be processed in the following manner: a mask is created on the image to be processed so as to retain a target area; Gaussian blur processing is performed on the target area, and contour detection and interference removal operations are performed on the Gaussian blur processing result to obtain the tennis ball outline. The target area is an area where the color parameter is within the preset range. The interference removal operation is used to remove color difference, shadows and stains.

[0079] This embodiment can create a binary mask on the image to be processed according to the preset range of color parameters (such as hue, saturation and brightness), and only retain the target area that meets the color conditions. This embodiment can perform contour detection on the image after Gaussian blur processing to identify all possible contours; remove small interfering contours through morphological operations (such as corrosion and expansion) and other filtering methods to ensure that only the contour that is most likely to be a tennis ball is retained, that is, the tennis ball contour is obtained.

[0080] As for Figure 1 In a further description of the embodiment, the tennis training device further includes a Bluetooth gateway, the Bluetooth gateway is connected to a plurality of Bluetooth Mesh lights, the Bluetooth Mesh lights are arranged behind the curtain, a plurality of scoring areas are arranged on the curtain, and the arrangement positions of the Bluetooth Mesh lights correspond to the scoring areas. The light of the Bluetooth Mesh lights can pass through the curtain. The Bluetooth Mesh lights can be installed on a board or bracket fixed behind the curtain.

[0081] Based on the setting of the above-mentioned Bluetooth Mesh light, this embodiment can control the tennis serving machine and the Bluetooth Mesh light through multiple modes.

[0082] Specifically, this embodiment can determine the current training mode of the tennis training device; if the current training mode is the first mode, when the ball landing position is in the scoring area, the Bluetooth Mesh light corresponding to the scoring area where the ball landing position is located is controlled by the Bluetooth gateway to light up; delay for a preset time length, and control all the Bluetooth Mesh lights to turn off; wherein the preset time length is less than the serving interval length of the tennis serving machine. In the above manner, the scoring area where the tennis ball has been hit can be displayed by controlling the Bluetooth Mesh light.

[0083] If the current training mode is the second mode, before the tennis serving machine performs the serving operation, a target scoring area is selected from all the scoring areas, and the Bluetooth Mesh light corresponding to the target scoring area is controlled to light up; after the ball landing position is determined, it is determined whether the ball landing position is within the target scoring area; if so, the Bluetooth Mesh light corresponding to the target scoring area is controlled to turn off, and the step of selecting the target scoring area from all the scoring areas is entered. In the above manner, the scoring area that the user needs to hit can be displayed by controlling the Bluetooth Mesh light.

[0084] As for Figure 1 According to a further description of the embodiment, the tennis training device further includes a second camera, and the second camera is used to photograph the user training area.

[0085] Based on the existence of the above-mentioned second camera, the user's posture analysis can be performed in the following manner: obtaining a user posture image taken by the second camera; performing posture estimation based on the user posture image to obtain human body feature points and racket feature points; if the number of users in the user posture image is greater than 1, determining the target user based on the distance between the racket feature points and the human body feature points; wherein the target user is a user holding a racket; generating posture information based on the human body feature points of the target user; wherein the posture information includes any one or a combination of any several of the left and right hand grip information, forehand and backhand grip information, squatting angle and swinging angle.

[0086] Furthermore, in this embodiment, the wrist feature point of each user can be determined from the human body feature points; and the user whose wrist feature point is closest to the racket feature point is set as the target user.

[0087] The process described in the above embodiment is explained below through an embodiment in actual application.

[0088] At present, professional tennis courts mostly use intelligent tennis training robots to assist athletes in completing training tasks. Specific technical solutions: Based on the automatic serve type, image recognition systems, sensors, and intelligent control systems are added. The camera captures the movement trajectory of the tennis ball and the athlete's hitting action, the sensor provides real-time feedback, and the intelligent control system analyzes and judges based on these data, thereby realizing functions such as automatic movement, simulated return and battle.

[0089] The above-mentioned related technical solutions have the following disadvantages:

[0090] (1) The technology is not mature and stable enough: The accuracy of image recognition may be affected by factors such as light and environment, resulting in deviations in the judgment of the trajectory of the tennis ball and the hitting action.

[0091] (2) High cost: Due to its complex technology and high-end configuration, the price of intelligent tennis training robots is usually very expensive, and generally only professional tennis training institutions or high-end users can afford it.

[0092] (3) Complex maintenance: It involves a variety of high-tech components and complex electronic systems. Once a failure occurs, its repair and maintenance is difficult and costly, and requires professional technicians to handle it.

[0093] (4) High requirements for the use environment: In order to ensure the normal operation of image recognition and sensors, it is necessary to use it in a relatively flat area with good lighting conditions, and the requirements for the use environment are relatively stringent.

[0094] In order to solve the defects of the above-mentioned related technologies, this embodiment provides a tennis training solution. This solution adopts the Bluetooth Mesh light networking control method, which can flexibly expand the scoring application mode and create more training methods; this solution can support the traditional fixed-point scoring method, and can also support the fun game interactive method of scoring by hitting the target light to optimize the user experience; this solution uses the front camera to capture the dynamic landing point, combined with the image recognition method, to realize the intelligent visual AI scoring method of landing point scoring; this solution uses the rear camera to capture the athlete's posture, combined with the human body modeling algorithm, to realize the AI ​​posture recognition statistics of receiving the ball; this solution adopts the fusion central controller device to realize the efficient collaboration and accurate recognition of the functional modules at each end, and other workflow coordination; this solution opens up the data communication with the tennis automatic serving machine, and adaptively adjusts the system working sequence according to the different serving modes selected by the user. This solution can collect and share interactive game data (such as training data playback, sharing, and ranking).

[0095] See also Figure 3 , Figure 3 This is a functional block diagram of a central controller provided in an embodiment of the present application. The background program of the central controller supports the following modules: automatic serving function module, landing point detection function module, posture detection function module, scoring and speed function module, learning and training data statistical analysis function module, configuration module, display module, and lighting function module. The automatic serving function module is connected to the automatic serving machine through a universal asynchronous receiver / transmitter (UART), the landing point detection function module is connected to the forward camera through a wireless local area network WiFi, the posture detection function module is connected to the rear camera through WiFi, the scoring and speed function module is connected to the LED display screen through WiFi, the central controller is connected to the router through WiFi, the learning and training data statistical analysis function module is connected to the Pad tablet (Web end), the configuration module is connected to the Pad tablet (Web end) through WiFi, the display module is connected to the display screen through HDMI (High Definition Multimedia Interface), the lighting function module is connected to the Bluetooth gateway through Ethernet ethernet, and the Bluetooth gateway is connected to the Mesh light (i.e., Bluetooth Mesh background light).

[0096] The function of the automatic serving function module is to control the serving mode of the automatic serving machine by communicating with the serial port of the automatic serving machine.

[0097] The function of the lighting function module is to control the lighting and off status of 5 Bluetooth Mesh lights by communicating with the Bluetooth gateway Ethernet.

[0098] The function of the drop point detection module is to capture the falling ball image and identify and analyze the falling ball result through wireless Wi-Fi communication with the forward-facing camera.

[0099] The function of the posture detection module is to capture posture images and obtain posture data through wireless Wi-Fi communication with the rear-facing camera and recognition and analysis.

[0100] The function of the display module is to push the video streams of the two cameras to the large screen in real time for display through HDMI communication with the large screen.

[0101] The function of the configuration module is to configure the learning and training parameters through the web page by communicating with the Pad tablet's Wi-Fi.

[0102] The function of the training data statistics and analysis function is to store historical training data, perform statistics and analysis on the training data and present it on the Web, so as to provide solutions for athletes and coaches to adjust their training. Among them, each Wi-Fi communication module is connected to the same router. The Wi-Fi communication module refers to a module with the function of network communication.

[0103] This embodiment adopts a central controller to achieve efficient collaboration, accurate identification and other workflow coordination of functional modules at each end; this embodiment supports completely offline work and can also be connected to the Internet for remote control.

[0104] See also Figure 4 , Figure 4 A flowchart of a tennis training provided in an embodiment of the present application, the process is as follows: configure the current round training mode; determine whether the current round training mode is a scoring mode (i.e., the first mode); if it is a scoring mode, read the current round configuration (number of serves, time interval between serves, serving angle, etc.), initialize the number of serves to 0 and send a serving signal, so as to start the current round of serving and enter the single scoring statistics processing process until the current round is finished. If it is not a scoring mode, determine whether it is a game mode (i.e., the second mode); if it is a game mode, read the current round configuration (number of serves, time interval between serves, serving angle, etc.), initialize the number of serves to 0 and send a serving signal, so as to start the current round of serving and enter the game points statistics processing process until the current round is finished.

[0105] See also Figure 5 , Figure 5A flow chart of serving processing in a single scoring and statistical processing process provided by an embodiment of the present application, the process is as follows: after starting a single serve, capture the posture image to determine whether the posture capture is complete, if not, recapture the posture image; if captured, capture the landing point image to determine whether the landing point capture is complete, if not, recapture the landing point image, if captured, cyclically determine whether the serving time interval has been reached. If the serving time interval has been reached, add 1 to the number of serves and determine whether the number of serves in this round has been reached. If so, end this round, otherwise start a single serve. After the posture capture is completed, the posture information (such as left-hand grip, right-hand grip, forehand grip, backhand grip, squatting angle, swinging angle, etc.) can be extracted, the data can be stored and sent to the large screen for display, and finally determine whether the storage operation and display operation are completed. After the landing point is captured, the landing point information is extracted (such as whether it falls into the target area, the landing point coordinates, the time and space position of the return ball, etc.). According to the landing point coordinates, if the white circle is hit, the scoring Mesh light is turned on, the score and ball speed are calculated, the data is stored and sent to the large screen to display the real-time scene data, and the data can also be sent to the LED display to display the current score and ball speed. If the storage and display operations are completed, the Bluetooth Mesh light is turned off. The time and space position of the return ball refers to the coordinates of the tennis ball landing point.

[0106] See also Figure 6 , Figure 6 A serving process flow chart in a game points statistics processing process provided by an embodiment of the present application, the process is as follows: start lighting up the Mesh light, start a single serve, capture the posture image, and determine whether the posture capture is complete. If not, recapture the posture image; if the capture is complete, capture the landing point image to determine whether the landing point capture is complete. If not, recapture the landing point image. If the capture is complete, cyclically determine whether the serving time interval has been reached. If the serving time interval has been reached, add 1 to the number of serves and determine whether the number of serves in this round has been reached. If so, end this round. Otherwise, start lighting up the Mesh light. After the posture capture is completed, the posture information (such as left-hand grip, right-hand grip, forehand grip, backhand grip, squat angle, swing angle, etc.) can be extracted, the data can be stored and sent to the large screen for display, and finally determine whether the storage and display operations are completed. After the landing point is captured, the landing point information is extracted (such as whether it falls into the target area, the landing point coordinates, the time and space position of the return ball, etc.), and the score and ball speed are calculated based on whether the landing point hits the lit white circle range of the Mesh light. The data is stored and sent to the large screen to display the real-time scene data. The data can also be sent to the LED display to display the current number of hits and ball speed. If the storage and display operations are completed, the Bluetooth Mesh light is turned off.

[0107] See also Figure 7 , Figure 7This is a schematic diagram of the appearance of a screen provided in an embodiment of the present application. If a tennis ball lands at the diagonal position, no points will be scored. If a tennis ball lands in the area corresponding to 5, 5 points will be scored. If a tennis ball lands in the area corresponding to 10, 10 points will be scored. If a tennis ball lands in the area corresponding to 20, 20 points will be scored. If a tennis ball lands in the area corresponding to 30, 30 points will be scored. If a tennis ball lands in the area corresponding to 50, 50 points will be scored.

[0108] Take 20 points as an example. When the traditional scoring training mode is configured, when the ball falls into the 20-point white circle area, the Bluetooth Mesh light at that location will light up, and 20 points will be scored. When the game interactive training mode is configured, if the 20-point white circle area light is on, the return ball hits the white circle area, the Bluetooth Mesh light will go out, and 20 points will be scored.

[0109] The following is the ball landing point image recognition solution:

[0110] When the camera captures the image, the shape of the ball in flight and the ball after it slows down after contacting the screen are significantly different. When the camera is at 25-30FPS, the ball in flight has a significant trailing image, and when the ball touches the screen, it will appear as a perfect circle due to the slowdown of the ball speed, and the green color of the tennis ball will also be clearer.

[0111] The ball landing point capture strategy process is as follows: obtain the image data after streaming through the camera RTSP (Real Time Streaming Protocol), and convert the image into HSV color space. Create a mask to retain only the green part, use Gaussian blur to reduce noise, and then search for a circular contour. Remove interference from the contour being found, and then judge the circularity of all contours. When it is greater than 88% (adjustable), it is considered that the tennis ball is in a characteristic state of contact with the curtain. In this embodiment, the coordinates of the center point of the contour in the image can be the landing point coordinates, and the landing point coordinates can be brought into the data of the field calibration configuration to determine the score.

[0112] In this embodiment, the coordinates of the upper left corner of the entire image taken by the forward camera can be set to (0,0). The camera pixel is 1280 x 720, the radius of the 20-point white circle is 20cm, the radius of the 30-point white circle is 15cm, and the radius of the 50-point white circle is 10cm. This embodiment can determine the coordinate position of each scoring area on the screen. When the camera is used for the first time, the coordinate calibration is required (the horizontal distance between the camera and the screen is a fixed value). Please refer to Figure 8 , Figure 8This is a schematic diagram of the coordinates of key points of a screen provided in an embodiment of the present application, in which (0,0), (318,68), (962,68), (456,204), (716,204), (256,340), (1024,340), (409,524), (586,474), (885,524), (128,652), and (1152,652) represent the coordinate values ​​of each key point.

[0113] When the ball landing point image is captured, the ball's coordinate position is obtained according to the calibrated coordinate position, and then the scoring area of ​​the ball landing point is determined according to the screen coordinate distribution. Taking the landing point (751,387) in the above picture as an example, first determine whether it is in the scoring area, then determine whether it is in the upper half or the lower half, and finally determine whether it is in the circle of the half area.

[0114] The posture image recognition scheme in this embodiment is as follows:

[0115] This embodiment can detect and analyze whether the student is performing a forehand or backhand action through the image captured by the front camera. The specific process is as follows:

[0116] Step 1: At the beginning of this round, the front camera starts to capture pictures of the students' preparation postures, 50 times per second, with an average of about 20ms to take a picture.

[0117] Step 2: Get the first configuration picture.

[0118] Step 3: Detect the acquired images.

[0119] Specifically, in this step, the acquired picture can be first input into the detection object determination module to determine the subject of the detected student object. This step is to solve the problem of multiple people in the stadium and prepare for the next step of accurate posture recognition. In this embodiment, the picture can also be input into the student left hand posture and right hand posture determination module to determine the direction of the student's holding hand, in preparation for the next step of accurate posture recognition.

[0120] After completing the above preparations, the system starts serving, obtains pictures, and inputs them into the forehand and backhand judgment model function module. In this embodiment, a deep learning model (such as a convolutional neural network CNN) can be used to recognize and analyze the posture in the image or video.

[0121] Step 4: Determine whether it is a forehand grip based on the detection result; if so, determine that the user is holding the forehand, and store the relevant data in the forehand and backhand summary table; if not, determine that the user is holding the backhand, and store the relevant data in the forehand and backhand summary table; this embodiment can determine whether the end time of this round has ended. If the time has not ended, continue to capture pictures and repeat the above steps.

[0122] See also Fig. 9 , Fig. 9 A posture image recognition flow chart provided for the embodiment of the present application includes the following process: at the beginning of this round of training, the front camera takes the first posture picture of the trainee, and gives it to the object determination module to determine the trainee object, and inputs it to the left and right determination module to determine whether the trainee holds the racket with the left hand or the right hand. After the above preparation analysis system is completed, it starts to serve and capture the Nth picture; inputs it to the forehand and backhand determination module to obtain the forehand and backhand results, and determine whether it is a forehand. If it is a forehand, the forehand return result summary table information is stored; if it is a backhand, the backhand return result summary table information is stored. Determine whether the end time of this round has been reached. If so, determine that this round of training is over. If not, add 1 to the value of N and capture the Nth picture.

[0123] The contents of the forehand return result summary table and the backhand return result summary table may include ball number, left or right hand grip information (left or right hand) and picture frame number. The ball number refers to the number of returned balls, and the picture frame number refers to the number of picture frames obtained in a round of practice.

[0124] Regarding the recognition of human posture, the basic principles and training algorithms of related models are described as follows:

[0125] First, divide the human body into 17 key feature points. Here, we can remove the four irrelevant feature points 14, 15, 16, and 17 (i.e., eyes and ears), and there are 13 feature points left. Fig.10 , Fig.10 A schematic diagram of a key feature point extraction structure provided in an embodiment of the present application, 1 to 17 are key feature points. 30 forehand and backhand photos are taken by a camera, and the coordinate value of each photo is obtained according to the position of the human body feature point. The training coordinate data of the batting action may include the position of each key point.

[0126] Through the neural network CNN convolution algorithm, a neural network model for forehand and backhand judgment is generated.

[0127] Then the coordinates of the 13 key feature points are input into the large model, and the forehand and backhand results are output.

[0128] Regarding the student object determination module, the implementation method is as follows: obtain a picture of the student and another person, capture the key feature points of the two bodies and the racket feature points; analyze the position distance between the racket feature points and the wrist feature points of the two people, and determine that the object with the closest distance is the student; use the determined student as the object for subsequent detection and determination. Please refer to Fig.11 , Fig.11A schematic diagram of a key feature point extraction structure provided in an embodiment of the present application, 1 to 18 are key feature points. By detecting user 1 and user 2, user 1 is closer to racket feature point 18, so user 1 on the left is the user participating in the training. Regarding the student left-hand and right-hand judgment module, the function implementation method is as follows: select the feature points of the racket and include them in the key feature point set. In the stage of preparing to serve, determine the distance between the left and right wrist feature points of the student and the racket, and determine whether the racket is held with the left hand or the right hand based on the closest one. That is: the relative distance between feature point 4 and 7 and 18 here.

[0129] In this embodiment, multiple tennis training devices can be combined into an overall system. Fig.12 , Fig.12 A system schematic diagram including multiple tennis training devices provided in an embodiment of the present application shows a WiFi router, a multi-port PoE router, a gateway, a central controller, a display screen, camera 1, camera 2, a tablet computer, a Bluetooth Mesh lamp and a tennis serving machine.

[0130] This embodiment adopts the Bluetooth Mesh light networking control method, which can flexibly expand the scoring application mode and create more learning and training methods; this embodiment can support the traditional fixed-point scoring method, and can also support the fun game interaction method of hitting the target light to score, optimizing the user experience; this embodiment uses the front camera to capture the dynamic landing point, combined with the image recognition method, to achieve the landing point scoring intelligent visual AI scoring method; this embodiment uses the rear camera to capture the athlete's posture, combined with the human body modeling algorithm, to achieve the ball receiving posture recognition statistics; this embodiment adopts the fusion central controller device to achieve efficient collaboration and accurate recognition of the functional modules at each end, and other workflow coordination; it supports completely offline work, and can also be connected to the Internet for remote control. This embodiment 1 opens up data communication with the tennis automatic serving machine, and adaptively adjusts the system working sequence according to the different serving modes selected by the user.

[0131] The embodiment of the present application provides a system for processing tennis training data, which is applied to a central controller of a tennis training device, wherein the tennis training device further includes a tennis serving machine and a first camera, wherein the first camera is used to shoot an area where the screen is located, and the screen is used to receive a tennis ball hit back by a user, and the system for processing tennis training data includes:

[0132] A serving control module, used for transmitting a serving instruction to the tennis serving machine, so that the tennis serving machine serves the ball to the user training area in front of the curtain;

[0133] An image acquisition module, used to acquire the i-th frame of the curtain image taken by the first camera;

[0134] a contour determination module, configured to convert the i-th frame of the curtain image into an HSV color space to obtain an image to be processed, and determine the tennis ball contour according to an area in the image to be processed where the color parameter is within a preset range; wherein the color parameter includes any one or a combination of any several of hue, saturation and brightness;

[0135] A judging module, used to judge whether the roundness of the tennis ball outline is greater than a preset value;

[0136] a ball drop position determination module, configured to determine the ball drop position according to the position of the tennis ball outline in the i-th frame of the screen image if the roundness of the tennis ball outline is greater than the preset value; wherein the ball drop position is the position where the tennis ball hit back by the user contacts the screen;

[0137] A loop module is used to add 1 to the value of i if the circularity of the tennis ball outline is less than or equal to the preset value, and enter the step of obtaining the i-th frame of the curtain image taken by the first camera.

[0138] The present embodiment provides a method for processing tennis training data. After a tennis ball serving machine serves a ball to a user's training area, the i-th frame of the screen image is obtained, the i-th frame of the screen image is converted into the HSV color space to obtain an image to be processed, and the outline of the tennis ball is determined in the area of ​​the image to be processed where the color parameters are within a preset range. A tennis ball in flight has a significant trailing image, and a tennis ball in contact with the screen will appear as a perfect circle. The present embodiment determines whether the tennis ball is in contact with the screen in the i-th frame of the screen image based on the circularity of the tennis ball outline. When the circularity of the tennis ball outline in the i-th frame of the screen image is greater than a preset value, the ball drop position can be determined based on the position of the tennis ball outline in the i-th frame of the screen image; when the circularity of the tennis ball outline in the i-th frame of the screen image is not greater than a preset value, the value of i is increased by 1 to determine whether there is a ball drop position in the next frame of the screen image.

[0139] Furthermore, the process of determining the outline of the tennis ball by the outline determination module according to the area in the image to be processed where the color parameters are within the preset range includes: creating a mask on the image to be processed so as to retain the target area; wherein the target area is the area where the color parameters are within the preset range; performing Gaussian blur processing on the target area, and performing outline detection and interference removal operations on the Gaussian blur processing result to obtain the outline of the tennis ball.

[0140] Furthermore, the tennis training device further comprises a Bluetooth gateway, the Bluetooth gateway is connected to a plurality of Bluetooth Mesh lights, the Bluetooth Mesh lights are arranged behind the curtain, a plurality of scoring areas are arranged on the curtain, and the arrangement positions of the Bluetooth Mesh lights correspond to the scoring areas;

[0141] Accordingly, it also includes:

[0142] A mode determination module, used to determine the current training mode of the tennis training device;

[0143] The first control module is used for controlling the Bluetooth Mesh light corresponding to the scoring area where the ball falls to light up through the Bluetooth gateway if the current training mode is the first mode and when the ball falls to the scoring area; and is also used for delaying for a preset time to control all the Bluetooth Mesh lights to turn off; wherein the preset time is less than the serving interval of the tennis serving machine.

[0144] Furthermore, it also includes:

[0145] The second control module is used for, after determining the current training mode of the tennis training device, if the current training mode is the second mode, selecting a target scoring area from all the scoring areas before the tennis serving machine performs a serving operation, and controlling the Bluetooth Mesh light corresponding to the target scoring area to light up; and is also used for, after determining the ball landing position, judging whether the ball landing position is within the target scoring area; if so, controlling the Bluetooth Mesh light corresponding to the target scoring area to turn off, and entering the step of selecting the target scoring area from all the scoring areas.

[0146] Furthermore, the tennis training device further includes a second camera, and the second camera is used to photograph the user training area;

[0147] Accordingly, it also includes:

[0148] A posture analysis module is used to obtain a user posture image taken by the second camera; it is also used to perform posture estimation based on the user posture image to obtain human body feature points and racket feature points; it is also used to determine the target user according to the distance between the racket feature points and the human body feature points if the number of users in the user posture image is greater than 1; wherein the target user is a user holding a racket; it is also used to generate posture information based on the human body feature points of the target user; wherein the posture information includes any one or a combination of any several of the left and right hand grip information, forehand and backhand grip information, squatting angle and swinging angle.

[0149] Furthermore, the process of determining the target user according to the distance between the racket feature points and the human body feature points by the posture analysis module includes: determining the wrist feature points of each user from the human body feature points; and setting the user whose wrist feature point is closest to the racket feature points as the target user.

[0150] Furthermore, the process of transmitting the serving instruction from the serving control module to the tennis serving machine includes: setting serving parameters according to the ball landing position and ball speed of the user's last hit, and transmitting the serving instruction including the serving parameters to the tennis serving machine; wherein the serving parameters include ball delivery speed, ball delivery angle and ball delivery rotation speed.

[0151] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, which will not be repeated here.

[0152] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0153] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0154] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of this application.

[0155] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for processing tennis training data, characterized in that: A central controller applied to a tennis training device, the tennis training device further comprising a tennis ball serving machine and a first camera, the first camera being used to photograph an area where a screen is located, the screen being used to receive a tennis ball hit back by a user, the method for processing tennis training data comprising: Transmitting a serving instruction to the tennis serving machine so that the tennis serving machine serves the ball to the user training area in front of the curtain; Obtaining the i-th frame of the curtain image captured by the first camera; Convert the i-th frame of the curtain image into the HSV color space to obtain an image to be processed, and determine the outline of the tennis ball according to an area in the image to be processed where the color parameter is within a preset range; wherein the color parameter includes any one or a combination of any several of hue, saturation and brightness; Determining whether the roundness of the tennis ball outline is greater than a preset value; If yes, then determine the ball drop position according to the position of the tennis ball outline in the i-th frame of the screen image; wherein the ball drop position is the position where the tennis ball hit back by the user contacts the screen; If not, the value of i is increased by 1, and the process proceeds to the step of obtaining the i-th frame of the curtain image taken by the first camera.

2. The method for processing tennis training data according to claim 1, characterized in that: Determining the outline of the tennis ball according to an area in the image to be processed where the color parameter is within a preset range includes: Creating a mask on the image to be processed so as to retain a target area; wherein the target area is an area where the color parameter is within the preset range; Gaussian blur processing is performed on the target area, and contour detection and interference removal operations are performed on the Gaussian blur processing result to obtain the tennis ball contour.

3. The method for processing tennis training data according to claim 1, characterized in that: The tennis training device further includes a Bluetooth gateway, the Bluetooth gateway is connected to a plurality of Bluetooth Mesh lights, the Bluetooth Mesh lights are arranged behind the curtain, a plurality of scoring areas are arranged on the curtain, and the arrangement positions of the Bluetooth Mesh lights correspond to the scoring areas; Correspondingly, it also includes: determining a current training mode of the tennis training device; If the current training mode is the first mode, when the ball landing position is in the scoring area, the Bluetooth Mesh light corresponding to the scoring area where the ball landing position is located is controlled by the Bluetooth gateway to light up; Delay for a preset time length to control all the Bluetooth Mesh lights to turn off; wherein the preset time length is less than the serving interval length of the tennis serving machine.

4. The method for processing tennis training data according to claim 3, characterized in that: After determining the current training mode of the tennis training device, the method further includes: If the current training mode is the second mode, before the tennis serving machine performs a serving operation, a target scoring area is selected from all the scoring areas, and a Bluetooth Mesh light corresponding to the target scoring area is controlled to light up; After determining the ball landing position, determining whether the ball landing position is within the target scoring area; If so, the Bluetooth Mesh light corresponding to the target scoring area is controlled to turn off, and the step of selecting the target scoring area from all the scoring areas is entered.

5. The method for processing tennis training data according to claim 1, characterized in that: The tennis training device further includes a second camera, which is used to photograph the user training area; Correspondingly, it also includes: Acquire a user posture image captured by the second camera; Performing posture estimation based on the user posture image to obtain human body feature points and racket feature points; If the number of users in the user posture image is greater than 1, determining a target user according to the distance between the racket feature point and the human body feature point; wherein the target user is a user holding a racket; The posture information is generated according to the human body feature points of the target user; wherein the posture information includes any one or a combination of any several of the left and right hand grip information, the forehand and backhand grip information, the squatting angle and the swinging angle.

6. The method for processing tennis training data according to claim 5, characterized in that: Determining a target user according to the distance between the racket feature point and the human body feature point includes: Determining wrist feature points of each of the users from the body feature points; The user whose wrist feature point is closest to the racket feature point is set as the target user.

7. The method for processing tennis training data according to claim 1, characterized in that: Transmitting a serving instruction to the tennis serving machine includes: The serving parameters are set according to the ball landing position and ball speed of the user's last shot, and a serving instruction containing the serving parameters is transmitted to the tennis serving machine; wherein the serving parameters include ball delivery speed, ball delivery angle and ball delivery rotation speed.

8. A system for processing tennis training data, characterized in that: A central controller applied to a tennis training device, the tennis training device further comprising a tennis ball serving machine and a first camera, the first camera being used to photograph an area where a screen is located, the screen being used to receive a tennis ball hit back by a user, the tennis training data processing system comprising: A serving control module, used for transmitting a serving instruction to the tennis serving machine, so that the tennis serving machine serves the ball to the user training area in front of the curtain; An image acquisition module, used to acquire the i-th frame of the curtain image taken by the first camera; a contour determination module, configured to convert the i-th frame of the curtain image into an HSV color space to obtain an image to be processed, and determine the tennis ball contour according to an area in the image to be processed where the color parameter is within a preset range; wherein the color parameter includes any one or a combination of any several of hue, saturation and brightness; A judging module, used to judge whether the roundness of the tennis ball outline is greater than a preset value; a ball drop position determination module, configured to determine the ball drop position according to the position of the tennis ball contour in the i-th frame of the screen image if the roundness of the tennis ball contour is greater than the preset value; wherein the ball drop position is the position where the tennis ball hit back by the user contacts the screen; A loop module is used to add 1 to the value of i if the circularity of the tennis ball outline is less than or equal to the preset value, and enter the step of obtaining the i-th frame of the curtain image taken by the first camera.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method for processing tennis training data according to any one of claims 1 to 7 when calling the computer program in the memory.

10. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the method for processing tennis training data according to any one of claims 1 to 7 are implemented.

Images