A reinforcement learning-based rebound falling point prediction method and system

Abstract

The application provides a rebounding point prediction method and system based on reinforcement learning, relates to the technical field of reinforcement learning, adopts an inertial sensor and machine vision to acquire dynamic information of an initial position of a basketball during fixed-point shooting, constructs a data pair by using a three-dimensional position relationship between the dynamic information of the initial position and a backboard, establishes a reinforcement learning model, trains and iterates the data until convergence, wherein a reward function is set based on a distance difference between a predicted landing point coordinate and an actual landing point coordinate, and the reinforcement learning model is used for rebounding point prediction; without fitting an entire movement track of the basketball, only the dynamic information of the initial point of the basketball and the landing point position are used, the prediction capability is automatically learned and optimized through the reward function, data utilization is improved, and calculation cost is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of reinforcement learning, and particularly relates to a rebound falling point prediction method and system based on reinforcement learning. BACKGROUND

[0002] The statements in this section merely provide background information related to the application and do not necessarily constitute prior art.

[0003] Rebound falling point prediction is an important and practical reinforcement learning task, which is a fixed-point shooting falling point prediction method for visual scenes, and has a very broad development prospect.

[0004] Existing rebound prediction methods mainly analyze video sequences through computer vision, collect data through trajectory tracking, and then construct a motion trajectory equation or use a deep neural network for learning and prediction, but there are generally problems of part of the data being unusable for fitting the motion trajectory due to occlusion and excessive overall calculation overhead, and therefore, a rebound falling point prediction method that does not need to fit the entire motion trajectory and is efficient is needed. SUMMARY

[0005] To overcome the deficiencies of the prior art, the application provides a rebound falling point prediction method and system based on reinforcement learning, which uses an inertial sensor and a machine vision scheme to obtain dynamic information of the initial position of the basketball during fixed-point shooting, uses the three-dimensional position relationship between the dynamic information of the initial position and the rebound to construct a data pair, establishes a reinforcement learning model, uses these data for training iteration until convergence, wherein the reward function is set based on the distance difference between the predicted falling point coordinates and the actual falling point coordinates, and the reinforcement learning model is used for rebound falling point prediction; without fitting the entire motion trajectory of the basketball, only the dynamic information of the initial point of the basketball and the falling point position are used, the prediction ability is automatically learned and optimized through the reward function, the data utilization rate is improved, and the calculation overhead is reduced.

[0006] To achieve the above object, one or more embodiments of the application provide the following technical solutions:

[0007] The first aspect of the application provides a rebound falling point prediction method based on reinforcement learning;

[0008] A rebound falling point prediction method based on reinforcement learning comprises:

[0009] The process data of fixed-point shooting is obtained through an inertial sensor scheme and a machine vision scheme respectively, and the basketball dynamic information, including the initial point basketball coordinates, velocity, acceleration and falling point position coordinates, is obtained after merging;

[0010] The basketball dynamic information is filtered and converted in coordinates using the three-dimensional position relationship between the basketball dynamic information and the rebound, and sample data sets for model training are obtained.

[0011] Based on the sample data set, the established landing point prediction reinforcement learning model is trained;

[0012] The trained landing point prediction reinforcement learning model is used to predict the rebound landing point of the shot, and the shot prediction result is obtained.

[0013] Further, the inertial sensor scheme is to obtain the current state of the basketball speed, acceleration and direction by using the inertial sensing device;

[0014] The machine vision scheme is to obtain the three-dimensional coordinates of the basketball at different times by using the target detection algorithm, and to calculate the speed, acceleration and direction of the basketball.

[0015] Further, in the process of fixed-point shooting, two groups of basketball speed, acceleration and direction sequences are obtained by the inertial sensor scheme and the machine vision scheme;

[0016] The data between the start time and the end time of the fixed-point shooting is intercepted, and the data is denoised;

[0017] The pose description method is used to represent the spatial attitude in the process of basketball movement, the conversion between the basketball coordinate system and the basketball court coordinate system is completed, and the basketball dynamic information is obtained.

[0018] Further, the three-dimensional position relationship between the basketball dynamic information and the backboard is compared, if the predicted landing point coordinate exceeds the backboard range, it is determined as invalid ball.

[0019] Further, the basketball dynamic information is transformed into the coordinate system of the reinforcement learning environment, the initial point basketball coordinate is transformed into (x nr ,y nr ,z nr ), the actual landing point position coordinate is transformed into (x nt ,y nt ,z nt ), and the predicted landing point position coordinate is marked as (x np ,y np ,z np ), the distance between the predicted landing point position coordinate and the actual landing point position coordinate is taken as the reward function index.

[0020] Further, the landing point prediction reinforcement learning model is based on a deep reinforcement learning network, and the three-dimensional vector of the initial point basketball coordinate, speed and acceleration is taken as the input, and the three-dimensional coordinate of the predicted landing point position is output.

[0021] Further, the sample data set is divided into a training set and a validation set, and the landing point prediction reinforcement learning model is trained and validated under the guidance of the reward function.

[0022] The second aspect of the present application provides a rebound landing point prediction system based on reinforcement learning.

[0023] A rebound landing point prediction system based on reinforcement learning comprises a data acquisition module, a sample construction module, a model construction module and a landing point prediction module.

[0024] The data acquisition module is configured to acquire fixed-point shooting process data through an inertial sensor scheme and a machine vision scheme respectively, and obtain basketball dynamic information including initial point basketball coordinates, speed, acceleration and landing point position coordinates after merging.

[0025] The sample construction module is configured to filter and convert coordinates of the basketball dynamic information by using the three-dimensional positional relationship between the basketball dynamic information and the rebound, to obtain a sample data set for model training.

[0026] The model construction module is configured to train the established landing point prediction reinforcement learning model based on the sample data set.

[0027] The landing point prediction module is configured to use the trained landing point prediction reinforcement learning model to predict the rebound landing point of the shot, to obtain a shot prediction result.

[0028] The third aspect of the present application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a rebound landing point prediction method based on reinforcement learning according to the first aspect of the present application.

[0029] The fourth aspect of the present application provides an electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor implements the steps of a rebound landing point prediction method based on reinforcement learning according to the first aspect of the present application when executing the program.

[0030] The above one or more technical solutions have the following beneficial effects:

[0031] The present application uses an inertial sensor and a machine vision scheme to acquire dynamic information of the initial position of the basketball during fixed-point shooting, only retains the dynamic information of the initial point of the basketball and the data pair of the landing point position coordinates, does not need to fit the entire motion trajectory, avoids the problems of excessive cumulative error when using the inertial sensor scheme or unusable data due to other objects blocking or too dark light affecting the recognition effect when using the vision scheme, improves the data utilization rate, and combines reinforcement learning to automatically optimize the prediction ability through a reward function, while the calculation overhead is small and the landing point prediction result is relatively accurate.

[0032] The application utilizes the dynamic information of the initial position and the three-dimensional position relationship of the backboard to construct data pairs, establishes a reinforcement learning model, and uses these data to train iterations to convergence, wherein the reward function is set based on the distance difference between the predicted landing point coordinates and the actual landing point coordinates, and the reinforcement learning model is used for backboard landing point prediction; the application does not need to fit the whole movement trajectory of the basketball, only uses the dynamic information of the initial point of the basketball and the landing point position, automatically learns and optimizes the prediction ability through the reward function, the proposed reinforcement learning model does not need additional supervision, can independently learn in the training stage and is guided to optimize through the reward function, improves the data utilization rate while reducing the calculation overhead, so as to provide the most accurate prediction in the deployment stage.

[0033] Advantages of the additional aspects of the application will be partially given in the following description, partially will become obvious from the following description, or will be understood by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated herein by reference. The illustrations are shown for the purpose of enabling those skilled in the art to implement the application and not intended to limit the scope of the application.

[0035] Figure 1 The method flowchart of the first embodiment.

[0036] Figure 2 The structure diagram of the model in the first embodiment.

[0037] Figure 3 The system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0038] The application will be further described below in conjunction with the drawings and embodiments.

[0039] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the application; unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the application belongs.

[0040] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments and are not intended to limit the exemplary embodiments according to the application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and in addition, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of a feature, step, operation, device, component and / or combination thereof.

[0041] Embodiment one

[0042] The embodiment discloses a rebounding point prediction method based on reinforcement learning.

[0043] As shown in the figure, a rebounding point prediction method based on reinforcement learning comprises: Figure 1

[0044] Step S1: Obtain fixed-point shooting process data through an inertial sensor scheme and a machine vision scheme respectively, and obtain basketball dynamic information after merging, including initial point basketball coordinates, speed, acceleration and landing point position coordinates, and the specific steps are as follows:

[0045] Step S101: Obtain data through the inertial sensor scheme

[0046] First, the preparation work of the inertial sensor scheme is carried out, and it is required to fix the inertial sensor assembly inside the basketball, and the assembly is required to maintain a relatively static positional relationship with the basketball; the inertial sensor assembly comprises an inertial sensor module, a calculation module and a communication module, wherein the inertial sensor module comprises an accelerometer, a gyroscope and a magnetometer, which are used to measure acceleration, angular velocity and magnetic field strength data in real time as inertial sensor data; the calculation module is used to process the original data of the inertial sensor, and calculate the speed, acceleration and direction of the basketball under the current state; the communication module is used to interact with a terminal computer, and periodically send the data obtained by the inertial sensor; in order to ensure the accuracy of the sensor, it needs to be calibrated before use.

[0047] After the preparation work is completed, data acquisition and initialization of the inertial sensor scheme are carried out, and the specific steps are as follows:

[0048] (1) Use the inertial sensor to collect the whole-process dynamic information of the basketball during the fixed-point shooting process;

[0049] (2) Based on the whole-process dynamic information of the basketball, the starting time t0 and the ending time t of the fixed-point shooting are analyzed through a sliding window method s Only the data in this time period is intercepted for subsequent calculation, and the data is denoised through a smoothing filter algorithm;

[0050] In fact, the inertial sensor can obtain the whole-process dynamic information of the basketball, and the information of the current state (especially the coordinates) needs to be compared and calculated with the information of the previous state, which is a continuous process, so the information of the initial point cannot be used to obtain the landing point information; the starting time point of the fixed-point shooting can be analyzed through the sliding window method; since the target is the landing point prediction, only the dynamic information in this time period (i.e. the shooting process) is effective for the landing point prediction.

[0051] Only the data in this time period is retained, which has two effects:

[0052] ​1) Determine the time point of the basketball plate, which corresponds to the spatial coordinates of the drop point coordinates;

[0053] 2) The data outside this time period is invalid information for drop point prediction, which can be filtered out in the preprocessing stage to reduce the subsequent computational overhead.

[0054] In general, the input required by the reinforcement learning prediction algorithm is only the dynamic information of the initial time point and the drop point coordinates, i.e. the information of two time points, but the drop point coordinates need to be calculated based on the dynamic information of the entire shooting event segment. Therefore, the time period is intercepted during the analysis process in this stage, and only the information of two time points is retained when the model is input in the prediction stage.

[0055] (3) Use the pose description method to represent the spatial pose in the basketball movement process, use the unit quaternion method to complete the conversion between the basketball coordinate system and the basketball court coordinate system, and use the extended Kalman filter method to realize the data fusion of angular velocity, acceleration and magnetic field intensity. The unit quaternion is modified to make the data estimate value closest to the true value, and finally the dynamic information of the basketball under the inertial sensor scheme is obtained, including the initial point basketball coordinates (x n0 ,y n0 ,z n0 ), velocity (v nx ,v ny ,v ns ), acceleration (a nx ,a ny ,a ns ) and drop point position coordinates (x ns ,y ns ,z ns ).

[0056] The pose description method is to decompose the pose change of a rigid body in three-dimensional space into position change and attitude change, which is realized by solving the translation and rotation motion of the rigid body.

[0057] The motion data collected by the inertial sensor is based on the basketball coordinate system, which needs to be converted to the basketball court coordinate system. The main methods of coordinate system conversion are: Euler angle, rotation matrix, unit quaternion, direction cosine, etc. Among them, the unit quaternion method has more advantages and is widely used, so this method is selected.

[0058] The basketball coordinate system is the carrier coordinate system of the inertial sensor. Its origin coincides with the carrier's center of gravity. All inertial data output by the sensor is based on this carrier coordinate system, which is used to describe and process the physical quantities directly measured by the sensor. The basketball court coordinate system, on the other hand, is the navigation coordinate system, also known as the geographic coordinate system or the real-world coordinate system. It describes the actual motion state of the carrier relative to its real environment. In short, while sensor data output is based on the carrier coordinate system, it needs to be mapped and transformed to the real-world coordinate system to describe the carrier's motion parameters relative to real space.

[0059] The purpose of the extended Kalman filter is simply this: if an accurate angular velocity value can be obtained, the current accurate attitude quaternion can be calculated by combining it with the attitude quaternion of the previous moment; however, in practice, the angular velocity collected by the gyroscope has errors, and after the errors accumulate, angular velocity drift will occur; in order to correct the errors, the Kalman gain can be calculated using acceleration and magnetic field strength, thereby correcting the predicted quaternion and improving the accuracy of attitude description.

[0060] Step S102: Machine vision solution acquires data

[0061] First, we prepared the machine vision solution. To improve data accuracy, we adopted a binocular stereo vision solution, setting up two cameras in fixed positions and using a calibration board to capture image pairs. We then calibrated each camera individually to determine the positional relationship between them, thereby determining the world coordinate system used by the binocular vision solution. To facilitate the reconstruction of the basketball's three-dimensional motion posture, we needed to extend the scope of the world coordinate system to the entire basketball court.

[0062] After preparation, data acquisition and initialization under the machine vision solution are performed. First, the image sequence captured by the binocular camera is denoised, and background subtraction is used to perform motion detection and target recognition on the basketball. Then, feature points corresponding to the basketball are found in image pairs captured at different positions, and the three-dimensional coordinates of the basketball in the court coordinate system are calculated based on the parallax principle. Dynamic information is calculated based on the three-dimensional coordinates of the basketball's trajectory points at different times, and finally, the dynamic information of the basketball under the machine vision solution is obtained, including the initial basketball coordinates (x, y, y). n0 ,y n0 ,z n0 ), velocity (v) nx ,v ny ,v ns ), acceleration (a nx ,a ny ,a ns ) and landing point coordinates (x ns ,y ns ,z ns ).

[0063] Step S103: merging data

[0064] The basketball dynamic information under the inertial sensor scheme and the basketball dynamic information under the machine vision scheme are merged and put into a data set, and the items in the data set are four-dimensional vectors composed of initial point basketball coordinates, speed, acceleration and landing point position coordinates.

[0065] Step S2: using the three-dimensional position relationship between the basketball dynamic information and the backboard to filter and convert the coordinates of the basketball dynamic information, and obtaining a sample data set for model training;

[0066] Considering that there are invalid balls in actual operation, i.e. non-touching board balls or out-of-bound balls, the landing point may be out of the range of the basketball court coordinate system, so the prediction environment needs to be expanded when predicting, and 1.5 times the entire basketball court is taken as the landing point prediction range of the intelligent agent, and the prediction result is compared with the backboard range. If the predicted landing point coordinates are out of the backboard range, it is determined as an invalid ball.

[0067] The obtained basketball dynamic information is transformed into the coordinate system of the reinforcement learning environment, i.e. the initial point basketball coordinates are transformed into (x nr ,y nr ,z nr ), the actual landing point position coordinates are transformed into (x nt ,y nt ,z nt ), and the landing point position coordinates predicted by reinforcement learning are marked as (x np ,y np ,z np ), i.e. the corresponding data of the shooting point basketball dynamic information and the backboard landing point position, which is taken as the sample data set for model training, and the sample data set is divided into a training set R and a verification set T.

[0068] The coordinate system of the reinforcement learning environment is the extended 1.5 times basketball court coordinate system, and the basketball court coordinate system refers to the coordinate system of the real world, the origin and range of which are based on the basketball court.

[0069] Step S3: based on the sample data set, the established landing point prediction reinforcement learning model is trained, specifically:

[0070] Step S301: establishing a landing point prediction reinforcement learning model

[0071] Based on the value-based DQN (deep Q-network) algorithm, a deep reinforcement learning network is adopted to build a landing point prediction reinforcement learning model, such as Figure 2As shown, the drop point prediction reinforcement learning model includes two neural networks, namely an estimation network and a target network. The estimation network is used to predict the estimated Q value and has the latest neural network parameters. The target network is used to predict the target Q value and does not update the parameters in time. The estimation network and the target network have the same structure, and their parameters can be different and automatically updated by the algorithm during training. The input vector of the drop point prediction reinforcement learning model is the state space, and the output is the action space. The state space is set as the initial position coordinate information, a three-dimensional vector composed of acceleration and speed, and a state matrix S=(x nr ,y nr ,z nr ,v nx ,v ny ,v ns a nx ,a ny ,a ns ). The action space is set as the three-dimensional coordinates of the predicted drop point position A=(x np ,y np ,z np ).

[0072] The distance between the predicted drop point position coordinates (x np ,y np ,z np ) and the actual drop point position coordinates (x nt ,y nt ,z nt ) is taken as the reward function index, and the reward function is constructed as:

[0073] Rew=-k[(x np -x nt ) 2 +(y bp -y nt ) 2 +(z np -z nt ) 2 ]

[0074] where k is a proportionality factor, which is a small value such as 0.001, limiting the range of the reward function value in the prediction process to obtain better training effect. The closer the predicted drop point and the actual drop point, the greater the reward obtained.

[0075] The structure of the two networks is specifically set as follows: the observed state space S is taken as the input vector, which is output to the next activation layer through a first fully connected layer with 125 neurons, and then is transmitted to a second fully connected layer with 1 neuron after being activated by a Sigmoid function. The output of the second fully connected layer is the Q value, and the action with the maximum Q value is selected as the next action.

[0076] The Q value is the value of the "state-action function", which is the output of the DQN network, and refers to the expected value Q(s, a) of taking an action a in a state s.

[0077] Step S302: Model training

[0078] The landing point prediction reinforcement learning model is trained under the guidance of the reward function using the training set data R. To improve the learning of the reinforcement learning agent, n=20 data are randomly selected from the training set R for each Episode and trained as a group. A total of 100,000 training is performed until the landing point prediction algorithm converges or the training number is reached. The model weight is saved for verification testing and deployment.

[0079] Each Episode refers to each round of training. Here, "continuous prediction of 20 data" is regarded as a round. The prediction effect is related to each result in the entire prediction process, avoiding the influence of individual data on the overall effect.

[0080] The trained landing point prediction reinforcement learning model is verified and tested. The validation set data T is input into the landing point prediction reinforcement learning model to predict the rebound point position coordinates. If the predicted landing point coordinates exceed the rebound range, it is determined as an invalid ball. If the distance between the predicted landing point coordinates and the actual landing point position coordinates does not exceed the set threshold, the model training is completed. Otherwise, the data is reorganized for training until the training stopping condition is met.

[0081] Step S4: The trained landing point prediction reinforcement learning model is used to predict the rebound point of the shot, and the shot prediction result is obtained.

[0082] The embodiment adopts two schemes to obtain the dynamic information of the initial position of the basketball during the fixed-point shooting, including the current state of the speed, acceleration and direction, and adopts a reinforcement learning model to predict the rebounding point. The two schemes for collecting the dynamic information of the initial position of the basketball are an inertial sensor scheme and a machine vision scheme. The inertial sensor scheme is to obtain the dynamic information of the initial position of the basketball in real time through the inertial sensor element fixed in the basketball. The machine vision scheme is to obtain the three-dimensional coordinates of the basketball at different times through the basketball detection algorithm under the binocular camera to calculate the dynamic information of the initial position of the basketball. Through collecting a large amount of measured data, the data pairs are constructed by using the relationship between the dynamic information of the initial position and the three-dimensional position of the rebounding point, the reinforcement learning model is established, the data is used for training iteration until convergence, and the setting of the reward function is based on the distance difference between the predicted landing point coordinates and the actual landing point coordinates. The model does not need to fit the whole motion trajectory of the basketball, only uses the dynamic information of the initial point of the basketball and the landing point position, automatically learns and optimizes the prediction ability through the reward function, improves the data utilization rate, and reduces the calculation overhead. Finally, in the actual deployment or use stage, the method of the embodiment is used to accurately predict the rebounding point during the fixed-point shooting.

[0083] Embodiment Two

[0084] The embodiment discloses a rebounding point prediction system based on reinforcement learning.

[0085] As shown in Figure 3 , a rebounding point prediction system based on reinforcement learning includes a data acquisition module, a sample construction module, a model construction module, and a landing point prediction module:

[0086] The data acquisition module is configured to obtain fixed-point shooting process data through the inertial sensor scheme and the machine vision scheme respectively, and obtain the dynamic information of the basketball after merging, including the initial point basketball coordinates, the speed, the acceleration, and the landing point position coordinates.

[0087] The sample construction module is configured to filter and convert the coordinates of the basketball dynamic information by using the three-dimensional position relationship between the basketball dynamic information and the rebounding point, to obtain a sample data set for model training.

[0088] The model construction module is configured to train the established landing point prediction reinforcement learning model based on the sample data set.

[0089] The landing point prediction module is configured to use the trained landing point prediction reinforcement learning model to predict the rebounding point of the shot, to obtain a shot prediction result.

[0090] Embodiment Three

[0091] The purpose of the embodiment is to provide a computer-readable storage medium.

[0092] A computer readable storage medium, having stored thereon a computer program, which when executed by a processor implements the steps of the method for predicting a rebound landing point based on reinforcement learning according to any one of the preceding embodiments.

[0093] Embodiment Four

[0094] An object of the present embodiment is to provide an electronic device.

[0095] An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for predicting a rebound landing point based on reinforcement learning according to any one of the preceding embodiments when executing the program.

[0096] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting the landing point of a backboard based on reinforcement learning, characterized in that, include: Data on the fixed-point shooting process is acquired using both inertial sensor and machine vision methods, and then merged to obtain dynamic information of the basketball, including the initial basketball coordinates, velocity, acceleration, and landing point coordinates. The inertial sensor solution uses inertial sensing devices to obtain the current state of the basketball's speed, acceleration, and direction. The machine vision solution obtains the three-dimensional coordinates of the basketball at different times through a target detection algorithm, and calculates the basketball's speed, acceleration, and direction. During the fixed-point shooting process, two sets of basketball speed, acceleration, and direction sequences are obtained through an inertial sensor solution and a machine vision solution; Extract the data between the start and end times of the fixed-point shot, and then perform noise reduction on the data; The pose description method is used to represent the spatial posture during basketball movement, and the transformation between the basketball coordinate system and the basketball court coordinate system is completed to obtain the dynamic information of the basketball. By utilizing the dynamic information of the basketball and the three-dimensional positional relationship of the backboard, the dynamic information of the basketball is filtered and its coordinates are transformed to obtain a sample dataset for model training. The established landing point prediction reinforcement learning model is trained based on the sample dataset. The trained landing point prediction reinforcement learning model is used to predict the landing point of the basketball shot on the backboard, and the shooting prediction result is obtained.

2. The reinforcement learning-based method for predicting the landing point of a backboard as described in claim 1, characterized in that, The relationship between the basketball's dynamic information and the three-dimensional position of the backboard is achieved by comparing the basketball's dynamic information with the backboard's range. If the predicted landing point coordinates exceed the backboard's range, the ball is deemed invalid.

3. The backboard landing point prediction method based on reinforcement learning as described in claim 1, characterized in that, The dynamic information of the basketball is transformed into the coordinate system of the reinforcement learning environment. The initial coordinates of the basketball are transformed as follows: The actual landing point coordinates are transformed as follows: The predicted landing point coordinates are as follows: The distance between the predicted landing point coordinates and the actual landing point coordinates is used as the reward function metric.

4. The backboard landing point prediction method based on reinforcement learning as described in claim 1, characterized in that, The landing point prediction reinforcement learning model is based on a deep reinforcement learning network. It takes the three-dimensional vectors of the basketball's initial coordinates, velocity, and acceleration as input and outputs the three-dimensional coordinates of the predicted landing point.

5. The reinforcement learning-based method for predicting the landing point of a backboard as described in claim 3, characterized in that, The sample dataset is divided into a training set and a validation set. Under the guidance of the reward function, the landing point prediction reinforcement learning model is trained and validated.

6. A backboard landing point prediction system based on reinforcement learning, characterized in that, It includes a data acquisition module, a sample construction module, a model construction module, and an impact prediction module: The data acquisition module is configured to acquire data of the fixed-point shooting process through inertial sensor and machine vision solutions respectively, and merge them to obtain basketball dynamic information, including the initial basketball coordinates, velocity, acceleration and landing point coordinates. The inertial sensor solution uses inertial sensing devices to obtain the current state of the basketball's speed, acceleration, and direction. The machine vision solution obtains the three-dimensional coordinates of the basketball at different times through a target detection algorithm, and calculates the basketball's speed, acceleration, and direction. During the fixed-point shooting process, two sets of basketball speed, acceleration, and direction sequences are obtained through an inertial sensor solution and a machine vision solution; Extract the data between the start and end times of the fixed-point shot, and then perform noise reduction on the data; The pose description method is used to represent the spatial posture during basketball movement, and the transformation between the basketball coordinate system and the basketball court coordinate system is completed to obtain the dynamic information of the basketball. The sample construction module is configured to: use the dynamic information of the basketball and the three-dimensional positional relationship of the backboard to filter and transform the dynamic information of the basketball to obtain a sample dataset for model training; The model building module is configured to train the established landing point prediction reinforcement learning model based on the sample dataset. The landing point prediction module is configured to use a trained landing point prediction reinforcement learning model to predict the landing point of the backboard during a shot, and obtain the shot prediction result.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the reinforcement learning-based rebound landing point prediction method as described in any one of claims 1-5.

8. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the reinforcement learning-based backboard landing point prediction method as described in any one of claims 1-5.

Images