Training robot control method, system, device and storage medium

Abstract

The present invention provides a control method, system, device and storage medium for a sparring robot, the method comprising: collecting user data and inputting a rating model to obtain a sparring level; acquiring data collected by a data acquisition module, analyzing to obtain user motion data and ball motion trajectory; selecting a hitting strategy according to the user motion data, ball motion trajectory and the sparring level; generating a robot control instruction according to the hitting strategy, and controlling the robot to hit based on the robot control instruction. The present invention assesses the sparring level of users and selects different hitting strategies according to different sparring levels, thereby meeting the sparring needs of different users.

Description

Background Art

[0002] Currently, several ball sports practice robots are available on the market, such as badminton and table tennis practice robots. However, existing practice robots operate in a relatively simple manner, limited to simply serving the ball and providing simple feedback on the user's strikes. This traditional practice model no longer meets the diverse needs of users, and the results are less than ideal. Summary of the Invention

[0003] In response to the problems in the prior art, the purpose of the present invention is to provide a sparring robot control method, system, device and storage medium, which can assess the sparring level of users and select different striking strategies according to different sparring levels to meet the sparring needs of different users.

[0004] An embodiment of the present invention provides a training robot control method, comprising the following steps:

[0005] Collect user data and input it into the rating model to obtain the training level;

[0006] Obtain the data collected by the data acquisition module and analyze it to obtain the user's motion data and the ball's motion trajectory;

[0007] Selecting a hitting strategy based on the user's motion data, the ball's motion trajectory, and the sparring partner's level;

[0008] A robot control instruction is generated according to the striking strategy, and the robot is controlled to strike based on the robot control instruction.

[0009] In some embodiments, after collecting user data and inputting the rating model to obtain the training level, the method further includes obtaining a training mode, wherein the training mode includes a practice mode and a battle mode;

[0010] The selecting of the hitting strategy includes selecting the hitting strategy according to the user motion data, the ball motion trajectory, the sparring level and the sparring mode.

[0011] In some embodiments, obtaining the sparring mode includes selecting the sparring mode corresponding to the user's sparring level according to the correspondence between the sparring levels and the sparring modes, or obtaining the sparring mode selected by the user.

[0012] In some embodiments, when the sparring mode is the practice mode, selecting a striking strategy includes the following steps:

[0013] Select the corresponding hitting difficulty coefficient according to the user's sparring level;

[0014] determining a hitting strategy of the robot based on the user's motion data, the ball's motion trajectory, and the selected hitting difficulty coefficient;

[0015] Collecting user's hitting feedback data;

[0016] Determine whether to trigger a change in the hitting difficulty coefficient based on the hitting feedback data.

[0017] In some embodiments, determining the robot's hitting strategy based on the user's motion data, the ball's motion trajectory, and the selected hitting difficulty coefficient comprises the following steps:

[0018] Selecting a corresponding striking range, striking speed, and setting frequency of accidental stimulation according to the striking difficulty coefficient;

[0019] The robot's hitting strategy is determined according to the user's motion data, the ball's motion trajectory, the hitting range, the hitting speed, and the set frequency of the accidental stimulation.

[0020] In some embodiments, when the sparring mode is a battle mode, selecting a striking strategy includes the following steps:

[0021] The user motion data and the ball motion trajectory are used as input data and input into a trained tactic selection model to obtain a selected hitting tactic;

[0022] The hitting strategy of the training robot is determined according to the user's motion data, the ball's motion trajectory and the selected hitting tactics.

[0023] In some embodiments, the input data of the tactical selection model further includes a sparring partner level;

[0024] When the sparring mode is a battle mode, the selecting of the striking strategy further comprises the following steps:

[0025] Collecting user's hitting feedback data;

[0026] Determine whether to trigger a change in the sparring level based on the hitting feedback data.

[0027] In some embodiments, the method further comprises training the tactic selection model by:

[0028] Collecting battle data, wherein the battle data includes player movement data and ball movement trajectory;

[0029] For each round in the game data, using the motion data of the first player and the motion trajectory of the ball as sample data, analyzing the hitting tactic type of the other player based on the motion data of the second player and the motion trajectory of the ball, and adding a hitting tactic type label to the sample data;

[0030] The tactic selection model is trained based on the sample data and the striking tactic type labels.

[0031] In some embodiments, the sample data further includes an athlete's sparring level, where the sparring level is obtained by inputting the user data of the first athlete into the rating model.

[0032] In some embodiments, the method further comprises the steps of:

[0033] Collecting the hitting feedback data of a battle between the user and the robot;

[0034] Score the user's battle according to the preset scoring algorithm;

[0035] Determine whether the score is greater than a preset score threshold. If so, collect the battle data between the user and the robot, and optimize and train the tactical selection model based on the battle data.

[0036] In some embodiments, collecting the battle data between the user and the robot, and optimizing and training the tactical selection model based on the battle data, includes the following steps:

[0037] Obtain the battle video between the user and the robot;

[0038] Extracting, from the battle video, the robot's motion data during a round in which the user successfully hits the ball and the ball's motion data after the robot hits the ball as optimization sample data;

[0039] Extract the user's hitting tactic type in the round in which the user successfully hits, and add a hitting tactic label to the corresponding optimized sample data;

[0040] The tactic selection model is optimized and trained based on the optimized sample data and the corresponding striking tactic labels.

[0041] In some embodiments, collecting the user's hitting feedback data includes the following steps:

[0042] Collect the user's physical condition data and determine the user's fatigue status;

[0043] Identify the position of the ball in the user's image and obtain the ball's trajectory;

[0044] Counting the user's hitting stability in multiple rounds based on the ball movement trajectory;

[0045] The fatigue state and striking stability are used as striking feedback data of the user.

[0046] In some embodiments, collecting the user's physical condition data and determining the user's fatigue state includes the following steps:

[0047] Collect multiple frames of user images and / or radar detection data to identify the positions of key points on the user's body;

[0048] The user's fatigue state is judged by combining the position changes of key points of the body at multiple moments.

[0049] In some embodiments, the user data includes user identity data and user training data;

[0050] After determining whether to trigger the change of the sparring level according to the hitting feedback data, the method further includes the following steps:

[0051] If the change of the sparring level is triggered, the training data of the user in this training and the user identity data are collected and input into the level assessment model to obtain the sparring level.

[0052] In some embodiments, the hitting strategy includes specifying a position, a hitting posture of the racket, a hitting speed of the racket, and a control of the landing point of the ball executed by the racket;

[0053] The robot is controlled to hit based on the robot control instruction, including controlling the robot to move to a specified position based on the robot control instruction, and controlling the robot to adjust the racket hitting posture and hit at the racket hitting speed.

[0054] An embodiment of the present invention further provides a training robot control system for implementing the training robot control method, the system comprising:

[0055] A rating module, configured to collect the user's first data and input it into a rating model to obtain a training sparring partner rating;

[0056] The data analysis module is used to obtain the data collected by the data acquisition module and analyze it to obtain the user's motion data and the ball's motion trajectory;

[0057] A strategy selection module, configured to select a hitting strategy based on the user's motion data, the ball's motion trajectory, and the sparring partner's level;

[0058] The striking control module is used to generate a robot control instruction according to the striking strategy, and control the robot to strike based on the robot control instruction.

[0059] An embodiment of the present invention further provides a training robot control device, comprising:

[0060] processor;

[0061] a memory storing executable instructions for the processor;

[0062] Wherein, the processor is configured to execute the steps of the training robot control method by executing the executable instructions.

[0063] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, which, when executed by a processor, implements the steps of the training robot control method.

[0064] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0065] The training robot control method, system, device, and storage medium of the present invention have the following beneficial effects:

[0066] The present invention assesses the sparring levels of users and selects different hitting strategies according to different sparring levels, thereby meeting the sparring needs of different users. In addition, the present invention automatically collects the user's motion data and ball motion trajectory during the sparring process, automatically determines the hitting strategy, and controls the robot in real time to operate according to the robot control instructions containing the hitting strategy, so that the robot responds to the user's hitting action in real time to achieve the best sparring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Other features, objects and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0068] Figure 1 is a flow chart of a training robot control method according to an embodiment of the present invention;

[0069] Figure 2 2. It is a schematic diagram of an application scenario of a training robot control method according to an embodiment of the present invention;

[0070] Figure 3 is a schematic diagram of a training robot according to an embodiment of the present invention;

[0071] Figure 4 is a schematic diagram of ball motion trajectory collection according to an embodiment of the present invention;

[0072] Figure 5 is a flow chart of determining a hitting strategy in a practice mode according to an embodiment of the present invention;

[0073] Figure 6 This is a flow chart of determining a striking strategy in a battle mode according to an embodiment of the present invention;

[0074] Figure 7 This is a schematic diagram of a site after adding a radar according to an embodiment of the present invention;

[0075] Figure 8 1 is a schematic structural diagram of a training robot control system according to an embodiment of the present invention;

[0076] Figure 9 1 is a schematic structural diagram of a training robot control device according to an embodiment of the present invention;

[0077] Figure 10 It is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0078] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0079] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0080] The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may change according to actual circumstances.

[0081] like Figure 1 As shown, an embodiment of the present invention provides a training robot control method, comprising the following steps:

[0082] S100: Collect user data and input it into the rating model to obtain the training sparring level;

[0083] S200: Collecting data collected by the data acquisition module and analyzing it to obtain user motion data and ball motion trajectory;

[0084] S300: Selecting a hitting strategy based on the user's motion data, the ball's motion trajectory, and the training partner's level;

[0085] S400: Generate a robot control instruction according to the striking strategy, and control the robot to strike based on the robot control instruction.

[0086] The present invention evaluates the sparring level of the user through step S100, selects different hitting strategies according to different sparring levels, thereby meeting the sparring needs of different users, and automatically collects the user's motion data and ball motion trajectory through step S200 during the sparring process, automatically determines the hitting strategy through step S300, and controls the robot in real time through step S400 to operate according to the robot control instruction containing the hitting strategy, so that the robot responds to the user's hitting action in real time to achieve the best sparring effect.

[0087] The training robot of the present invention can be used in various ball sports, such as badminton, table tennis, tennis, volleyball, and soccer. The term "user" as used herein refers to a training partner, i.e., an athlete who needs to train with the training robot, and can be either an amateur or a professional athlete. The training robot control method of the present invention can be applied to the training robot's controller or to a separate controller that can communicate with the training robot's controller to obtain motion data from the training robot and send control commands to the training robot.

[0088] In this embodiment, the user data includes user identity data and user training data. The user identity data may include, for example, the user's age, gender, height and weight, and the user's experience in the ball game. The user training data may include, for example, the user's response speed, success rate, hitting speed, single training session duration, and other data related to the training process.

[0089] The user identity data may be identity data collected from user input. For example, a user may communicate with a controller implementing the control method for a training robot through their mobile terminal, and transmit the identity data entered on the mobile terminal to the controller implementing the control method for a training robot. The controller implementing the control method for a training robot may also be connected to an input device such as a keyboard or touch screen, through which the user may enter the identity data. In other embodiments, the user's identity data may be read from a database storing user data.

[0090] Step S100: User data is collected and input into a rating model to obtain a training partner rating. If historical user training data is stored or can be collected, the user identity data and the user training data can be input into the machine learning-based rating model together. If historical user training data cannot be collected, the various parameters in the user training data can be set to default values ​​and then input into the rating model together with the user identity data to obtain an initial training partner rating. After the user subsequently trains with the training partner robot, the user's training data is collected again and the user's training partner rating is reassessed.

[0091] The grading model may adopt a machine learning model based on deep learning, such as a convolutional neural network, or a decision tree structure, such as a lasso regression algorithm, a random forest, an extreme gradient boosting algorithm, etc.

[0092] The control method for the training robot further includes the step of training the grading model. Training the grading model includes:

[0093] collecting athlete identity data and athlete training data as sample data;

[0094] Adding a training level label to the sample data;

[0095] The grading model is iteratively trained based on the sample data and the corresponding sparring grade labels until the grading model converges, thereby obtaining a model that can be used for sparring grade assessment of users.

[0096] like Figure 2 FIG2 is a schematic diagram of an application scenario of the control method for a training robot according to an embodiment of the present invention. Here, a training robot J400 is used in a badminton court J100. The user J300 and the training robot J400 are located on either side of the badminton court J100. Figure 2 The dotted line in the figure represents the motion trajectory of the badminton J500. The data acquisition module includes a camera J200, which is installed on at least one side of the badminton court J100. The camera J200 can capture images of the court, including the user J300 and the badminton J500, and can further include a robot J400. The motion data of the user J300 and the motion trajectory of the badminton J500 can be obtained by analyzing the images captured by the camera J200.

[0097] like Figure 3The figure shows the structure of the training robot of this embodiment. The training robot J400 includes a base J410, a mechanical arm J420 and a racket J430. The racket J430 includes a racket face J431 and a racket shaft J432. The training robot J400 is provided with a robot controller inside and a moving mechanism (at the bottom) Figure 3 The robot controller can control the robot to move in the field by controlling the mobile mechanism, and can control the racket posture and hitting speed of the racket J430 and realize the control of the ball landing point by the racket J430 by controlling the robot arm J420. Here, only the structure of the training robot is illustrated. In other embodiments, the training robot can also adopt other structural arrangements, not limited to Figure 3 The structures shown in the figure all fall within the protection scope of the present invention.

[0098] In step S200, after reading the captured multiple frames of images from the camera, the user and the ball in the images can be identified and analyzed to obtain the user's motion data and the ball's motion trajectory. The user's motion data may include the user's motion speed, motion direction, swing height, hitting speed, hitting posture and other data, and the ball's motion trajectory may include the three-dimensional coordinates of the ball at each moment. Figure 4 Figure 1 shows a schematic diagram of acquiring a ball's trajectory. After user J300 hits ball J500, the ball's trajectory will vary depending on user J300's hitting posture and speed. For example, the ball may land at one of L1, L2, or L3. The dotted lines in the figure represent possible ball trajectories. After capturing the ball's trajectory, the ball's subsequent trajectory can be predicted, including its likely landing point. The closer the ball is to robot J400, the more complete the capture of the ball's trajectory.

[0099] In this embodiment, after step S100, the process further includes obtaining a sparring mode, where the sparring mode includes a practice mode and a battle mode. In step S300, selecting a hitting strategy includes selecting a hitting strategy based on the user's motion data, the ball's trajectory, the sparring level, and the sparring mode. Different sparring modes will require different hitting strategies. Therefore, the present invention can further implement sparring control for different sparring modes, which, combined with the sparring level, can meet the training needs of different users in different scenarios.

[0100] Furthermore, the sparring mode can be obtained based on the correspondence between the sparring level and the sparring mode, and the sparring mode corresponding to the user's sparring level can be selected. For example, a level limit is pre-set. If the user's sparring level is higher than the level limit, it means that the user's level is relatively high and can be applied to the battle mode. The battle mode is also called the battle-to-win mode, which refers to the user and the robot fighting against each other. For the user, it is more challenging, and when selecting a hitting strategy, a hitting strategy that is more difficult for the user to handle will also be selected. If the user's sparring level is lower than or equal to the level limit, it means that the user's level is still relatively low, and it is suitable to first adopt the practice mode, that is, select the practice mode. The practice mode is less challenging for the user and focuses more on the entertainment of the user during the training process. When selecting a hitting strategy, some relatively easy-to-handle hitting strategies are selected to maintain the user's physical and mental pleasure during training.

[0101] In another embodiment, the acquisition of the sparring mode may also be the acquisition of the sparring mode selected by the user. For example, the user can independently select the sparring mode they desire. Thus, users with higher sparring levels can choose the practice mode to relax, while users with lower sparring levels can choose the battle mode to challenge themselves. The robot will then increase the difficulty of the competition to match the user's own sparring level.

[0102] The user can select the training mode by, for example, sending a selection instruction to the controller that applies the training robot control method through a mobile terminal, or by directly inputting a selection instruction on an input device connected to the controller of the training robot control method to select the training mode he needs.

[0103] The following introduces different ways to choose striking strategies in practice mode and battle mode respectively.

[0104] like Figure 5 As shown, in this embodiment, when the sparring mode is the practice mode, the step S300 of selecting a striking strategy includes the following steps:

[0105] S311: Determine that the current sparring mode is the practice mode;

[0106] S312: Selecting a corresponding hitting difficulty coefficient based on the user's sparring level. Different hitting difficulty coefficients correspond to different hitting parameters. A mapping relationship between the sparring level and the hitting difficulty coefficient can be preset. The current hitting difficulty coefficient can be obtained by searching the mapping relationship.

[0107] S313: Determining a hitting strategy of the robot based on the user's motion data, the ball's motion trajectory, and the selected hitting difficulty coefficient;

[0108] S314: Collecting the user's hitting feedback data;

[0109] S315: determining whether to trigger a change in the hitting difficulty coefficient based on the hitting feedback data;

[0110] If yes, proceed to step S316: adjust the difficulty coefficient. For example, when the hitting feedback data meets the requirement of increasing the sparring level, increase the difficulty coefficient; when the hitting feedback data meets the requirement of decreasing the sparring level, decrease the difficulty coefficient. Thus, the difficulty of the practice mode can be dynamically adjusted according to the user's current hitting feedback, and then proceed to step S313.

[0111] If not, proceed to step S317: maintain the current difficulty coefficient.

[0112] In this embodiment, different hitting difficulty coefficients correspond to different hitting ranges, hitting speeds, and accidental stimulation settings. Step S313: Determining the robot's hitting strategy based on the user's motion data, the ball's trajectory, and the selected hitting difficulty coefficient includes the following steps:

[0113] The setting method of the corresponding hitting range, hitting speed and accidental stimulation is selected according to the hitting difficulty coefficient. Here, accidental stimulation refers to a sudden change in the hitting strategy while maintaining the hitting range and hitting speed corresponding to the current difficulty coefficient, so as to stimulate the user's sparring process and increase the fun of the sparring process. Here, the sudden change can refer to a sudden increase in difficulty in a round, such as increasing the hitting speed or increasing the hitting range, etc., and the sudden change can also refer to a sudden decrease in difficulty in a round, such as decreasing the hitting speed or decreasing the hitting range, etc., so that the opponent has an opportunity to take advantage. Here, the setting method of accidental stimulation can include the hitting data of the accidental stimulation (including the hitting range, hitting speed, etc.), and can also include the setting time of the accidental stimulation, such as setting the setting time of the accidental stimulation to once every 5 rounds, once every 10 rounds, etc., or setting the first accidental stimulation in the 5th round and the second accidental stimulation in the 8th round in an irregular arrangement, or setting the user's hitting feedback data to determine whether to increase the accidental stimulation.

[0114] The robot's hitting strategy is determined based on the user motion data, ball motion trajectory, hitting range, hitting speed, and the set frequency of accidental stimulation. Specifically, the hitting strategy may include a designated position, a racket hitting posture, and a racket hitting speed. The robot's hitting strategy may be determined based on a preset hitting parameter calculation formula or a machine learning model, with the user motion data, ball motion trajectory, hitting range, hitting speed, and the set frequency of accidental stimulation as inputs to the calculation formula or the machine learning model, and the output is the value of each corresponding hitting parameter, such as the two-dimensional or three-dimensional coordinates of the designated position, the racket hitting posture (racket shaft tilt angle, racket face tilt angle, racket height, etc.), the racket hitting speed, etc. In the step S400, the robot is controlled to hit based on the robot control instruction, including controlling the robot to move to the designated position based on the robot control instruction, and controlling the robot to adjust the racket hitting posture and hit at the racket hitting speed.

[0115] Furthermore, in the practice mode, the user can further select a multi-ball practice method. When the user selects multi-ball practice, the selection of the hitting strategy includes the following steps:

[0116] Selecting serving data based on the user's motion data, the ball's motion trajectory, and the sparring level, the serving data including at least one of a serving range, a serving speed, and a serving frequency; for example, the serving range may be selected based on the user's position and motion speed, and the serving speed and serving frequency may be selected based on the user's sparring level. For users with a high sparring level, the serving speed may be set to be faster, and the serving frequency may be set to be more frequent;

[0117] Collecting user's hitting feedback data;

[0118] Determine whether to adjust the serving data based on the hitting feedback data. For example, if it is determined based on the user's hitting feedback data that the user's hitting action is not standard or the hitting success rate is relatively low, the serving data can be adjusted to reduce the hitting difficulty. If it is determined based on the user's hitting feedback data that the user's hitting action is standard and the hitting success rate is relatively high, the serving data can be adjusted to increase the hitting difficulty.

[0119] The multi-ball mode can also be applied to continuous training of multiple users. That is, after a user arrives at a designated location, the serve is started when the camera image data determines that there is a user. After the user changes, the serve data can be adjusted according to the relevant data of the new user and the serve is continued.

[0120] like Figure 6 As shown, in this embodiment, when the sparring mode is the battle mode, the step S300 of selecting a striking strategy includes the following steps:

[0121] S321: Determine that the current sparring mode is a battle mode;

[0122] S322: The user's motion data and the ball's motion trajectory are used as input data and input into a trained tactic selection model to obtain a selected hitting tactic;

[0123] S323: Determine the hitting strategy of the training robot according to the user's motion data, the ball's motion trajectory and the selected hitting tactics.

[0124] In this embodiment, the input data of the tactic selection model also includes the sparring partner level. That is, in step S322, the user motion data, ball motion trajectory, and the sparring partner level are input into the trained tactic selection model to obtain the selected hitting tactic.

[0125] When the sparring mode is a battle mode, the selecting of the striking strategy further comprises the following steps:

[0126] S324: Collecting the user's hitting feedback data;

[0127] S325: Determining whether to trigger a change in the sparring level based on the hitting feedback data;

[0128] If yes, proceed to step S326: adjust the sparring level. For example, when the hitting feedback data meets the requirement of raising the sparring level, increase the sparring level. When the hitting feedback data meets the requirement of lowering the sparring level, lower the sparring level. This can achieve dynamic adjustment of the sparring level based on the hitting feedback data in the battle mode, and then proceed to step S322.

[0129] If not, proceed to step S327: maintain the current sparring level.

[0130] The tactical selection model can adopt a machine learning model, such as a model based on deep learning or other types of classification models. Different tactical types can include pull-and-lob attacks, defensive counterattacks, downward pressure control, weakness restriction, serve-and-attack, serve-and-attack, baseline control, net control, repeated routes, fast and slow combinations, and other types. In this embodiment, the sparring robot control method also includes training the tactical selection model using the following steps:

[0131] Collecting battle data, the battle data including player motion data and ball motion trajectory. The battle data may be collected, for example, from a recorded video of a game, where the two players are the first player and the second player, or may be pre-stored battle data between the robot and other users, where the robot is also considered the first player and the user is considered the second player.

[0132] For each round in the battle data, the motion data and ball motion trajectory of the first athlete are used as sample data, and the hitting tactic type of the other athlete is analyzed based on the motion data and ball motion trajectory of the second athlete, and a hitting tactic type label is added to the sample data. Here, adding the hitting tactic type label to the sample data can be manually added, or it can be pre-trained by a deep learning-based tactic recognition model for label prediction. The input data of the tactic recognition model includes the motion data and ball motion trajectory of the athlete, and the output data is the predicted tactics adopted by the athlete. The tactic recognition model is different from the tactic selection model. When the inputs of the tactic recognition model and the tactic selection model are both the motion data and ball motion trajectory of athlete A, the tactic recognition model predicts the tactics adopted by athlete A, while the tactic selection model predicts the tactics of athlete B playing against athlete A;

[0133] The tactic selection model is trained based on the sample data and the striking tactic type labels. For example, when a deep learning model is used, a loss function is constructed based on the predicted tactics of the tactic selection model and the striking tactic type labels, and the tactic selection model is iteratively trained until the loss function is less than a preset loss function threshold.

[0134] Furthermore, the input data of the tactical selection model may also include the user's sparring level, so the sample data also includes the athlete's sparring level, and the sparring level is obtained by inputting the user data of the first athlete into the level assessment model.

[0135] In this embodiment, the training robot control method may further include a step of further optimizing the tactical selection model. Specifically, the optimization training includes the following steps:

[0136] Collecting the hitting feedback data of a battle between the user and the robot;

[0137] Scoring the user's match according to a preset scoring algorithm. For example, the scoring algorithm may be set to calculate indicators such as the user's success rate, interception rate, response speed, and number of out-of-bounds calls, and then taking the weighted sum or difference of each indicator to obtain the result as the match score, i.e., the score of the user's performance in the match;

[0138] Determine whether the score is greater than a preset score threshold. If so, it indicates that the user's performance in this battle is relatively superior. Collect the battle data between the user and the robot, and optimize and train the tactical selection model based on this battle data, so that the tactical selection model can continuously learn new battle data and tactical selection strategies, which is conducive to selecting the most appropriate tactics in subsequent tactical predictions.

[0139] Furthermore, the collecting of the battle data between the user and the robot, and optimizing and training the tactical selection model based on the battle data, includes the following steps:

[0140] Obtain the battle video between the user and the robot;

[0141] Extracting, from the battle video, the robot's motion data during a round in which the user successfully hits the ball and the ball's motion data after the robot hits the ball as optimization sample data;

[0142] Extract the user's hitting tactic type in the rounds in which the user successfully hits, and add a hitting tactic label to the corresponding optimization sample data; that is, in the optimization training, the robot's hitting data is used as sample data, and the user's hitting data is used as the label, so that when the tactic selection model is learned, it learns the user's tactic selection;

[0143] Here, the user's hitting tactic type is extracted. This can be done manually by determining the hitting display based on the user's video and then manually adding hitting tactic labels. Alternatively, the user's motion data and ball trajectory can be input into the above-mentioned tactic recognition model, and hitting tactic labels can be added based on the output data of the tactic recognition model.

[0144] The tactic selection model is optimized and trained based on the optimized sample data and the corresponding striking tactic labels. For example, when a deep learning model is used, a loss function is constructed based on the predicted tactics of the tactic selection model and the striking tactic type labels, and the tactic selection model is iteratively trained.

[0145] In this embodiment, the hitting feedback data may include the user's fatigue state and hitting stability. The step S314 and the step S324 of collecting the user's hitting feedback data may be performed by the following steps respectively:

[0146] Collecting the user's physical condition data and determining the user's fatigue state. For example, the user's fatigue state may be classified into fatigue levels, and the fatigue level corresponding to the physical condition data may be determined based on the range of physical condition parameters corresponding to different fatigue states.

[0147] Identify the position of the ball in the user's image and obtain the ball's trajectory;

[0148] The hitting stability of the user in multiple rounds is calculated based on the ball movement trajectory. The hitting stability can be represented by a single indicator, such as the number of user mistakes, the stability of the user's return ball landing point, the number of user successes, etc., or can be calculated by combining multiple indicators;

[0149] The fatigue state and striking stability are used as striking feedback data of the user.

[0150] Furthermore, the collecting of the user's physical condition data and determining the user's fatigue state includes the following steps:

[0151] Collect detection data from multiple frames of user images and identify the positions of key points on the user's body. For example, multiple frames of images captured by a camera can be obtained, and a neural network can be used to identify key points on the user's body in the images to obtain key point position information in each frame of the image.

[0152] In another embodiment, the data acquisition module may also include a radar. Figure 7 As shown, at least one radar J600 can be further installed on the side of the badminton court J100. The radar can be a laser radar, infrared radar, or the like. The radar can capture the ball's trajectory and the user's motion data. The user's motion data and the ball's trajectory can also be analyzed by combining the detection data from the camera J200 and the radar J600.

[0153] Furthermore, radar can be used to detect the positions of key points of the user's body, such as Figure 7 As shown, at least one radar J600 may be further provided on the side of the badminton court J100;

[0154] The user's fatigue state is judged by combining the position changes of key points of the body at multiple moments.

[0155] For example, the user's chest can be used as a key point on the body. By detecting changes in the position of the chest, the chest's rise and fall state can be determined, and the user's fatigue level can be determined. If the user's chest rises and falls significantly and frequently, it indicates that the user's current fatigue level is relatively high.

[0156] In other alternative embodiments, the data acquisition module may also include other types of data acquisition devices, such as a wearable device worn by the user to collect the user's movement speed, etc., and may also detect the user's fatigue state in other ways, for example, through a wearable device worn by the user to obtain the user's pulse, heart rate, body temperature and other data, thereby judging the user's fatigue state.

[0157] Furthermore, after collecting the user's hitting feedback data, the method further includes the following steps:

[0158] Determining the degree of standardization of the user's movements based on the user's hitting feedback data. Determining the degree of standardization of the user's movements may include determining whether the user's movements are accurate and whether they meet technical specifications. For example, the positions of the user's key points may be determined based on images captured by a camera or detection data from a radar to determine the user's posture when hitting the ball and compare it with standard standard movements.

[0159] For users who do not meet the preset action specifications, correction information is generated and pushed to the user. Pushing the correction information here can be pushed to the user's mobile terminal, or a display screen can be set up on the badminton court to display the correction information on the display screen for user reference.

[0160] In practice mode, after obtaining the user's striking feedback data, the difficulty coefficient can be adjusted based on the user's striking feedback data. To maintain the user's physical and mental state during practice, the difficulty coefficient is lowered when the user's fatigue level is above a first fatigue threshold and / or their stability is below a first stability threshold. The difficulty coefficient is raised when the user's fatigue level is below a second fatigue threshold and / or their stability is above a second stability threshold. The first fatigue threshold is greater than the second fatigue threshold, and the first stability threshold is less than the second stability threshold.

[0161] In battle mode, after receiving the user's hitting feedback data, the system can determine whether to trigger an adjustment of the sparring level based on the user's hitting feedback data. For example, if the user's fatigue level is below a third fatigue threshold and / or their stability is above a third stability threshold, the sparring level can be increased to increase the intensity of the confrontation with the user.

[0162] In this embodiment, after determining whether to trigger a change in the sparring level based on the striking feedback data in step S324, the user's sparring level may be re-evaluated using the sparring model. Specifically, if a change in the sparring level is triggered, the user's training data from the current training session and the user's identity data are collected and input into the sparring model to obtain a new sparring level. In subsequent matches with the user, striking strategies corresponding to the new sparring level are then selected.

[0163] like Figure 8 As shown, an embodiment of the present invention further provides a training robot control system for implementing the training robot control method, the system comprising:

[0164] The rating module M100 is used to collect the user's first data and input it into the rating model to obtain the training sparring level;

[0165] The data analysis module M200 is used to obtain the data collected by the data acquisition module and analyze it to obtain the user's motion data and the ball's motion trajectory;

[0166] A strategy selection module M300 is configured to select a hitting strategy based on the user's motion data, the ball's motion trajectory, and the sparring partner's level;

[0167] The striking control module M400 is configured to generate a robot control instruction according to the striking strategy, and control the robot to strike based on the robot control instruction.

[0168] The present invention uses a level assessment module M100 to assess the sparring level of users, selects different hitting strategies according to different sparring levels, and thus adapts to the sparring needs of different users. During the sparring process, the data analysis module M200 automatically collects the user's motion data and ball motion trajectory, and the strategy selection module M300 automatically determines the hitting strategy. The hitting control module M400 controls the robot in real time to operate according to the robot control instructions containing the hitting strategy, so that the robot responds to the user's hitting action in real time to achieve the best sparring effect.

[0169] The training robot control system of the present invention can be deployed in the controller of the training robot, or it can be deployed in a separate controller that can communicate with the controller of the training robot. The functions of each module in the training robot control system of the present invention can be implemented by adopting the specific implementation methods of each step in the above-mentioned training robot control method. For example, the level assessment module M100 can adopt the implementation method of the above-mentioned step S100 to implement the user's training level assessment, the data analysis module M200 can adopt the implementation method of the above-mentioned step S200 to extract the user's motion data and the ball motion trajectory, the strategy selection module M300 can adopt the implementation method of the above-mentioned step S300 to select the hitting strategy, and the hitting control module M400 can adopt the implementation method of the above-mentioned step S400 to implement the hitting control of the training robot.

[0170] An embodiment of the present invention also provides a training robot control device, comprising a processor; a memory in which executable instructions of the processor are stored; wherein the processor is configured to execute the steps of the training robot control method by executing the executable instructions.

[0171] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Accordingly, various aspects of the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as "circuits," "modules," or "platforms."

[0172] Refer to the following Figure 9 An electronic device 600 according to this embodiment of the present invention will be described. Figure 9 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0173] like Figure 9 As shown, electronic device 600 is implemented as a general-purpose computing device. Components of electronic device 600 may include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting various system components (including storage unit 620 and processing unit 610), a display unit 640, and the like.

[0174] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned training robot control method section of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0175] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0176] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0177] Bus 630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0178] The electronic device 600 can also communicate with one or more external devices 700 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 650. Furthermore, the electronic device 600 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 via the bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device 600, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0179] In the training robot control device, the program in the memory implements the steps of the training robot control method when the processor executes it. Therefore, the computer storage medium can also obtain the technical effects of the above-mentioned training robot control method.

[0180] An embodiment of the present invention further provides a computer-readable storage medium for storing a program that, when executed by a processor, implements the steps of the training robot control method. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the training robot control method section above.

[0181] refer to Figure 10 , a program product 800 for implementing the above method according to an embodiment of the present invention is described. The program product 800 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be executed on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0182] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0183] The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. The data signal propagated may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0184] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] When the program in the computer storage medium is executed by the processor, the steps of the training robot control method are implemented. Therefore, the computer storage medium can also obtain the technical effects of the above-mentioned training robot control method.

[0186] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A training robot control method, characterized in that: The steps include: Collecting user data and inputting it into a rating model to obtain a training sparring level, wherein the user data includes user identity data and user training data; Obtain the data collected by the data acquisition module, and analyze the user's motion data and the ball's motion trajectory; Selecting a hitting strategy according to the user's motion data, the ball's motion trajectory and the sparring level; generating a robot control instruction according to the striking strategy, and controlling the robot to strike based on the robot control instruction; Wherein, after collecting user data and inputting the level assessment model to obtain the sparring level, the method further includes obtaining the sparring mode; when the sparring mode is the battle mode, the method further includes selecting the striking strategy, including the following steps: The user motion data and the ball motion trajectory are used as input data and input into a trained tactic selection model to obtain a selected hitting tactic, wherein the tactic selection type is a classification model; Determining the hitting strategy of the training robot according to the user's motion data, the ball's motion trajectory and the selected hitting tactics; The method further comprises the steps of: Counting the hitting feedback data in a battle between the user and the robot; Score the user's battle according to the preset scoring algorithm; Determine whether the score is greater than a preset score threshold, and if so, collect the battle data between the user and the robot, and optimize and train the tactical selection model based on the battle data; When optimizing and training the tactical selection model, the robot's hitting data is used as sample data, the user's motion data and the ball's motion trajectory are input into the tactical recognition model, and a hitting tactical label is added according to the output data of the tactical recognition model.

2. The training robot control method according to claim 1, characterized in that: The sparring mode also includes a battle mode; The selecting of the hitting strategy includes selecting the hitting strategy according to the user motion data, the ball motion trajectory, the sparring level and the sparring mode.

3. The training robot control method according to claim 2, characterized in that: The acquiring of the sparring mode includes selecting the sparring mode corresponding to the sparring level of the user according to the corresponding relationship between the sparring level and the sparring mode, or acquiring the sparring mode selected by the user.

4. The training robot control method according to claim 2, characterized in that: When the sparring mode is the practice mode, the step of selecting a striking strategy comprises the following steps: Select the corresponding hitting difficulty coefficient according to the user's sparring level; Determining the robot's hitting strategy according to the user's motion data, the ball's motion trajectory, and the selected hitting difficulty coefficient; Collecting the user's hitting feedback data; Determine whether to trigger a change in the hitting difficulty coefficient based on the hitting feedback data.

5. The training robot control method according to claim 4, characterized in that: The step of determining the robot's hitting strategy according to the user's motion data, the ball's motion trajectory and the selected hitting difficulty coefficient comprises the following steps: Selecting a corresponding striking range, striking speed and setting frequency of accidental stimulation according to the striking difficulty coefficient; The robot's hitting strategy is determined according to the user's motion data, the ball's motion trajectory, the hitting range, the hitting speed, and the setting frequency of the accidental stimulation.

6. The training robot control method according to claim 2, characterized in that: When the sparring mode is the practice mode, the step of selecting a striking strategy comprises the following steps: Selecting serving data according to the user motion data, the ball motion trajectory and the sparring level, the serving data including at least one of serving range, serving speed and serving frequency; Collecting the user's hitting feedback data; Determine whether to adjust the serving data according to the hitting feedback data.

7. The training robot control method according to claim 1, characterized in that: The input data of the tactical selection model also includes the sparring level; When the sparring mode is a battle mode, the selecting of the striking strategy further comprises the following steps: Collecting the user's hitting feedback data; Determine whether to trigger a change in the sparring level based on the striking feedback data.

8. The training robot control method according to claim 1, characterized in that: The method further comprises training the tactical selection model by: Collecting battle data, wherein the battle data includes player movement data and ball movement trajectory; For each round in the battle data, the motion data of the first player and the motion trajectory of the ball are used as sample data, the hitting tactic type of the other player is analyzed according to the motion data of the second player and the motion trajectory of the ball, and a hitting tactic type label is added to the sample data; The tactic selection model is trained based on the sample data and the striking tactic type labels.

9. The training robot control method according to claim 8, characterized in that: The sample data also includes an athlete's sparring level, and the sparring level is obtained by inputting the user data of the first athlete into the level assessment model.

10. The training robot control method according to claim 1, characterized in that: The collecting of the battle data between the user and the robot, and optimizing and training the tactical selection model based on the battle data, comprises the following steps: Obtain the battle video between the user and the robot; Extracting the motion data of the robot in the round in which the user successfully hits the ball and the motion data of the ball after the robot hits the ball from the battle video as optimization sample data; Extract the hitting tactic type of the user in the round in which the user hits successfully, and add a hitting tactic label to the corresponding optimized sample data; The tactic selection model is optimized and trained based on the optimized sample data and the corresponding striking tactic labels.

11. The training robot control method according to claim 4 or 7, characterized in that: The step of collecting the hitting feedback data of the user comprises the following steps: Collect the user's physical condition data and determine the user's fatigue status; Identify the position of the ball in the user's image and obtain the ball's motion trajectory; Counting the hitting stability of the user in multiple rounds according to the ball movement trajectory; The fatigue state and striking stability are used as striking feedback data of the user.

12. The training robot control method according to claim 11, characterized in that: The collecting of the user's physical condition data and determining the user's fatigue state comprises the following steps: Collect multiple frames of user images and / or radar detection data to identify the positions of key points on the user's body; The user's fatigue state is judged by combining the position changes of key points of the body at multiple moments.

13. The training robot control method according to claim 4 or 7, characterized in that: After collecting the hitting feedback data of the user, the following steps are also included: Judging the degree of user's action standardization based on the user's hitting feedback data; For users who do not meet the preset action specification requirements, correction information is generated and pushed to the users.

14. The training robot control method according to claim 7, characterized in that: After determining whether to trigger the change of the sparring level according to the hitting feedback data, the method further includes the following steps: If the change of the sparring level is triggered, the training data of the user in this training and the user identity data are collected and input into the level assessment model to obtain the sparring level.

15. The training robot control method according to claim 1, characterized in that: The hitting strategy includes the designated position, the hitting posture of the racket, the hitting speed of the racket and the control of the landing point of the ball executed by the racket; The robot is controlled to hit based on the robot control instruction, including controlling the robot to move to a specified position based on the robot control instruction, and controlling the robot to adjust the racket hitting posture and hit at the racket hitting speed.

16. The training robot control method according to claim 1, characterized in that: The data acquisition module includes a camera and / or a radar, and the analysis to obtain the user motion data and the ball motion trajectory includes analyzing the images acquired by the camera and / or the data acquired by the radar to obtain the user motion data and the ball motion trajectory.

17. A training robot control system, used to implement the training robot control method according to any one of claims 1 to 16, characterized in that: The system comprises: A level assessment module, used to collect the first data of the user and input it into the level assessment model to obtain the training level; A data analysis module is used to obtain the data collected by the data collection module and analyze the user's motion data and the ball's motion trajectory; A strategy selection module, for selecting a hitting strategy according to the user's motion data, the ball's motion trajectory and the sparring level; The striking control module is used to generate a robot control instruction according to the striking strategy, and control the robot to strike based on the robot control instruction.

18. A training robot control device, characterized in that: include: processor; a memory storing executable instructions of the processor; Wherein, the processor is configured to execute the steps of the training robot control method described in any one of claims 1 to 16 by executing the executable instructions.

19. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the training robot control method described in any one of claims 1 to 16 are implemented.

Images