Basketball projection detection method, equipment system and storage medium

Abstract

The invention relates to a vision measurement technology, and discloses a basketball projection detection method, equipment system and storage medium, and the method comprises the steps: obtaining an RGB video stream and a depth map in a process that a basketball flies towards a basket based on a depth camera; carrying out basketball target detection according to the RGB video stream, and aligning the depth map with a corresponding image frame in the RGB video stream; based on the depth map, obtaining point cloud data of a basketball target in each image frame; fitting the point cloud data of the basketball target in each image frame to generate a three-dimensional model of a sphere, and calculating the coordinates of the center of sphere; fitting the center coordinates of the continuous image frames to generate a basketball flight path; and based on the basketball flight path after coordinate conversion, combining polynomial regression and extended Kalman filtering to predict a projection result. According to the method, the falling point coordinates and the incident angle of the basketball are accurately calculated, more detailed and accurate shooting result information is provided, and the requirement for accurate analysis of basketball shooting is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual measurement, in particular to a basketball shooting detection method, device system and computer readable storage medium. BACKGROUND

[0002] In the traditional way of basketball shooting detection, the devices used mostly rely on infrared sensors or pressure sensing devices installed on the basketball hoop. Such traditional devices have significant limitations, with relatively single function, only being able to simply identify whether the basketball has successfully entered the basketball hoop, that is, only being able to judge the two basic results of "in" or "not in"; and the misjudgment rate of such devices is high, which is difficult to provide accurate and reliable detection information, and is difficult to meet the demand for accurate analysis of basketball shooting.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a basketball shooting detection method, device system and computer readable storage medium, which aims to accurately calculate the landing point coordinates and the incidence angle of the basketball, and provide more detailed and accurate shooting result information, so as to meet the demand for accurate analysis of basketball shooting.

[0005] To achieve the above purpose, the present application provides a basketball shooting detection method, comprising the following steps: Based on a depth camera, an RGB video stream and a depth map in the flying process of the basketball towards the basket are obtained; According to the RGB video stream, basketball target detection is performed, and the depth map is aligned with the corresponding image frame in the RGB video stream; Based on the depth map, point cloud data of the basketball target in each image frame is obtained; The point cloud data of the basketball target in each image frame is fitted respectively, a spherical three-dimensional model is generated, and the center coordinates are calculated; The center coordinates of the continuous image frames are fitted to generate the flight trajectory of the basketball; According to the relative position relationship between the depth camera and the center of the basket, the camera coordinates corresponding to the flight trajectory of the basketball are converted into the world coordinates of the basketball court; Based on the flight trajectory of the basketball after coordinate conversion, the shooting result of the basketball relative to the center of the basket is jointly predicted by using polynomial regression and extended Kalman filtering; wherein, the shooting result includes the landing point coordinates and the incidence angle.

[0006] Optionally, the step of detecting the basketball target according to the RGB video stream comprises: For the image frames of the RGB video stream, a YOLOv11 architecture is adopted for basketball target detection. The YOLOv11 architecture is based on an attention mechanism to enhance the target recognition capability.

[0007] Optionally, the step of aligning the depth map with the corresponding image frame in the RGB video stream comprises: The depth map is optimized by bilateral filtering and morphological operation, and the optimized depth map is aligned with the corresponding image frame in the RGB video stream by guided filtering.

[0008] Optionally, before the step of fitting the ball center coordinates of the continuous image frames to generate the basketball flight trajectory, the method further comprises: The ball center coordinates of the continuous image frames are time-optimized based on Kalman filtering and smoothing spline interpolation.

[0009] Optionally, the step of fitting the ball center coordinates of the continuous image frames to generate the basketball flight trajectory comprises: Detecting whether the ball center coordinates of the continuous image frames meet the curvature detection condition; If yes, fitting the ball center coordinates of the continuous image frames to generate the basketball flight trajectory by parabolic fitting; If no, fitting the ball center coordinates of the continuous image frames to generate the basketball flight trajectory by linear fitting.

[0010] Optionally, after the step of jointly predicting the shooting result of the basketball relative to the center of the basket based on the basketball flight trajectory after coordinate conversion, using polynomial regression and extended Kalman filtering, the method further comprises: Outputting the shooting result to a display device on the basketball court.

[0011] Optionally, the depth camera is a first depth camera arranged above the basket; after the step of jointly predicting the shooting result of the basketball relative to the center of the basket based on the basketball flight trajectory after coordinate conversion, using polynomial regression and extended Kalman filtering, the method further comprises: Adjusting the shooting parameters of the shooting machine according to the shooting result and the shooting action score; Wherein, based on the second depth camera, the shooting action data when the basketball is shot is obtained, and the shooting action score is generated based on the shooting action data; the second depth camera is arranged on the side of the basketball court.

[0012] Optionally, the shooting action data includes the RGB video stream and depth information when the basketball is shot; the basketball shooting detection method further comprises: Based on the shooting action data, the three-dimensional coordinates of multiple human joint nodes are extracted to generate a dynamic sequence of human joint nodes. Calculate the biomechanical characteristics according to the dynamic sequence; Input the biomechanical characteristics into the action score model to generate the shot action score; wherein the action score model has previously learned the correlation between the biomechanical characteristics and the action score.

[0013] To achieve the above-mentioned purpose, the present application also provides a device system, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program realizes the steps of the above-mentioned basketball shooting detection method when executed by the processor.

[0014] To achieve the above-mentioned purpose, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program realizes the steps of the above-mentioned basketball shooting detection method when executed by a processor.

[0015] The basketball shooting detection method, device system and computer readable storage medium provided by the present application can capture more basketball flight information based on the RGB video stream and depth map obtained by the depth camera during the flight of the basketball, and can comprehensively and meticulously master the dynamic process of the basketball shooting through a series of operations such as target detection, point cloud data fitting, generation of a spherical three-dimensional model and flight trajectory. Finally, the shooting result can be accurately calculated by combining polynomial regression and extended Kalman filter prediction, and more detailed and accurate shooting result information is provided, thereby meeting the demand for accurate analysis of the basketball shooting. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The figure is a step schematic diagram of the basketball shooting detection method in an embodiment of the present application; Figure 2 The figure is a schematic diagram of the internal architecture of the device system in an embodiment of the present application.

[0017] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0018] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] In addition, if the description in the present application involves "first", "second", etc., it is only for the purpose of description (such as for distinguishing the same or similar features), and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0020] Reference Figure 1 In an embodiment, the basketball shooting detection method comprises: Step S10, based on the depth camera, acquiring the RGB video stream and the depth map in the process of the basketball flying towards the basket; Step S20, performing basketball target detection according to the RGB video stream, and aligning the depth map with the corresponding image frame in the RGB video stream; Step S30, based on the depth map, acquiring the point cloud data of the basketball target in each image frame; Step S40, fitting the point cloud data of the basketball target in each image frame respectively, generating a spherical three-dimensional model, and calculating the center coordinates of the sphere; Step S50, fitting the center coordinates of the sphere of the continuous image frames to generate the flight trajectory of the basketball; Step S60, according to the relative position relationship between the depth camera and the center of the basket, converting the camera coordinates corresponding to the flight trajectory of the basketball into the world coordinates of the basketball court; Step S70, based on the flight trajectory of the basketball after coordinate conversion, using polynomial regression and extended Kalman filter to jointly predict the shooting result of the basketball relative to the center of the basket; wherein the shooting result includes the landing point coordinates and the incidence angle.

[0021] In the embodiment, the execution terminal of the embodiment can be a device system, or other devices or apparatuses (such as control apparatuses) that control the device system.

[0022] As described in step S10, the depth camera is a device that can acquire color image (RGB) and depth information of a scene at the same time, which measures the distance from each point in the scene to the camera through different technologies (such as structured light, time-of-flight method, etc.), thereby generating a depth map.

[0023] In the basketball shooting scenario, the depth camera is placed in a suitable position to ensure that the entire process of the basketball flying towards the basket can be captured completely. During the process of the basketball being shot and flying towards the basket, the RGB video stream and the corresponding depth map are continuously recorded by the depth camera. The RGB video stream is composed of a series of consecutive color image frames, while the depth map provides distance information between each pixel point and the camera.

[0024] Optionally, the depth camera can be placed above the basket, preferably at a top-down angle of 2m±0.3m or clamped on the backboard at 1m±0.3m above the backboard.

[0025] Optionally, the depth camera is a binocular stereo depth camera.

[0026] Optionally, before performing step S10, load the pre-defined camera calibration parameters, turn off the automatic exposure and adjust the exposure time, and input the parameters through the advanced API mode to exclude the influence of most environmental changes. The depth camera usually provides an advanced API mode, which allows users to directly input various parameters to control the working mode and performance of the camera. Through the advanced API mode, various parameters of the camera can be finely adjusted to adapt to different environmental conditions.

[0027] Optionally, according to the actual environmental changes, input the corresponding parameters to exclude the influence of environmental changes on image acquisition. For example, the gain, contrast, white balance and other parameters of the camera can be adjusted to improve the quality and stability of the image.

[0028] Optionally, during the execution of step S10, distortion can be eliminated by camera calibration, and pixel-level depth information can be calculated.

[0029] Optionally, after obtaining the depth map, an interpolation algorithm (such as bilinear interpolation, nearest neighbor interpolation, etc.) is used to process the depth map to calculate the accurate depth information of each pixel point. By calculating the pixel-level depth information, the resolution and accuracy of the depth map can be improved, providing a more accurate basis for subsequent point cloud data generation and analysis.

[0030] As described in step S20, a target detection algorithm (such as YOLO, Faster R-CNN, etc.) is used to process each frame of image in the RGB video stream to identify the position and bounding box of the basketball. The target detection algorithm will find the area matching the characteristics of the basketball in the image and output the coordinate information of the basketball in the image.

[0031] Since the RGB video stream and the depth map are acquired simultaneously by the depth camera, there may be some spatial and temporal differences between them. Therefore, an alignment operation is needed to ensure that each pixel point in the depth map corresponds to the same physical location in space as the corresponding pixel point in the RGB image frame. This can be achieved through the camera's internal parameters (such as focal length, principal point coordinates, etc.) and external parameters (such as the camera's rotation and translation matrix).

[0032] As described in step S30, the point cloud is a dataset composed of a large number of three-dimensional points, each representing a location in space. According to the position of the basketball target detected in the RGB image frame in step S20, the depth information of the basketball region is extracted from the corresponding depth map.

[0033] Optionally, the depth values in the depth map are combined with the pixel coordinates in the RGB image frame to convert the two-dimensional pixel coordinates into three-dimensional space coordinates through the camera's projection model (such as the pinhole camera model), thereby obtaining the point cloud data of the basketball target. The coordinates of each point represent the location of a point on the basketball surface in the camera coordinate system.

[0034] Optionally, a multi-modal fusion network (MMF-Net) is constructed to extract the spherical point cloud data of the corresponding region in the depth map guided by the basketball detection box in the RGB image frame. The multi-modal fusion network aims to effectively fuse the data of the two different modalities of RGB image and depth map, and utilize the rich semantic information of the RGB image frame and the three-dimensional spatial information of the depth map to achieve more accurate extraction of basketball-related data.

[0035] Within the corresponding region of the depth map, the depth value of each pixel is converted into a coordinate in three-dimensional space. This is usually based on the camera's projection model, which calculates the three-dimensional coordinates of each point in the camera coordinate system through the depth value and pixel coordinates, and then combines the converted three-dimensional coordinate points into point cloud data.

[0036] To improve the quality of the point cloud data, the three-dimensional coordinates can be screened and processed, such as removing outliers and noise points. This can use statistical filtering, radius filtering, etc. to preprocess the point cloud data, making the extracted spherical point cloud data more accurate and reliable.

[0037] To enable the multi-modal fusion network to better complete the task, a suitable loss function can be designed. The loss function can consider factors such as the accuracy of basketball detection in the RGB image frame and the precision of spherical point cloud data extraction in the depth map. For example, cross-entropy loss can be used to measure the accuracy of basketball detection, and mean square error loss can be used to measure the precision of point cloud data extraction.

[0038] In the model training process, the MMF-Net is trained using a training dataset containing multiple RGB image frames and depth maps in time series, and the parameters of the network are continuously adjusted through the back propagation algorithm, so that the value of the loss function gradually decreases, thereby improving the performance of the network. The trained model is evaluated using a test dataset, and the model is further optimized based on the evaluation results. The performance and generalization ability of the model can be improved by adjusting the network structure, hyperparameters, etc.

[0039] As described in step S40, since the shape of the basketball is approximately spherical, a sphere fitting algorithm (such as least squares) can be used to fit the point cloud data of the basketball target in each image frame. The goal of sphere fitting is to find a best sphere model that minimizes the sum of distances from points in the point cloud to the surface of the sphere.

[0040] After completing the sphere fitting, the center coordinates of the sphere are the center coordinates of the sphere. The parameters of the sphere model obtained by fitting (such as the center coordinates and the radius) can be used to directly calculate the three-dimensional coordinates of the center in the camera coordinate system.

[0041] Optionally, an improved random sample consensus (RANSAC) algorithm is used to fit the three-dimensional model of the sphere, and the center coordinates are iteratively calculated by the least squares method, as follows: ; where (x c ,y c ,z c ) is the center coordinates to be solved, r is the calibration radius of the basketball, N is the number of points in the point cloud, and (x i ,y i ,z i ) is the three-dimensional coordinates of the i-th point in the point cloud.

[0042] As described in step S50, the center coordinates of the continuous image frames calculated in step S40 are considered as the positions of the basketball at different times. A curve fitting algorithm (such as polynomial fitting) can be used to fit these center coordinates to obtain a curve that describes the flight trajectory of the basketball. The order of the polynomial fitting can be selected according to the actual situation, and an appropriate order can be selected to ensure that the fitted curve accurately reflects the flight trajectory of the basketball.

[0043] The fitted curve is the flight trajectory of the basketball, which represents the motion path of the basketball in the camera coordinate system.

[0044] As described in step S60, before performing coordinate transformation, the relative position relationship between the depth camera and the center of the basket in the basketball court world coordinate system is input. This can be determined by measuring the actual positions of the camera and the basket, and combining the installation parameters of the camera (such as height, angle, etc.).

[0045] The coordinate transformation matrix (including rotation and translation matrices) is used to convert the coordinates of the basketball's flight trajectory in the camera coordinate system to the coordinates in the basketball court world coordinate system. The coordinate transformation matrix can be calculated based on the relative positions of the camera and the basket. By multiplying the coordinates in the camera coordinate system by the coordinate transformation matrix, the corresponding basketball court world coordinates can be obtained.

[0046] As described in step S70, a multinomial regression analysis is performed on the basketball's trajectory after coordinate transformation to further fit the basketball's motion trajectory. Multinomial regression can predict the basketball's position at future moments based on known basketball position data. Based on the results of multinomial regression and extended Kalman filtering, the projection result of the basketball relative to the center of the basket is predicted. Specifically, the landing coordinates of the basketball (i.e., the final position of the basketball on the ground) and the angle of incidence (i.e., the angle at which the basketball enters the basket) are predicted. These projection results can be used to evaluate the accuracy and success rate of the basketball shot.

[0047] Optionally, the accuracy of predictions can be improved by choosing an appropriate polynomial order. Generally, the trajectory of a basketball can be approximated as a quadratic or cubic polynomial. The optimal order can be determined by trying different orders and comparing the fitting errors (such as mean squared error).

[0048] A multinomial regression model is constructed with time *t* as the independent variable and the three-dimensional coordinates of the basketball as the dependent variable. A separate multinomial regression equation is established for each coordinate component. The coefficients in the multinomial regression equation are estimated using methods such as least squares, and the optimal coefficient estimate is obtained by minimizing the sum of squared errors between the actual coordinate values ​​and the model's predicted values.

[0049] Extended Kalman Filter (EPF) is a filtering algorithm used for state estimation of nonlinear systems. In this step, the motion of the basketball is treated as a nonlinear system, and EPF is used to estimate and predict the basketball's position and velocity. EPF can combine the results of multinomial regression to update and correct the basketball's trajectory in real time, improving the stability and accuracy of the prediction.

[0050] Optionally, define the state vector and observation vector of the basketball, and establish a state transition model and an observation model. Initialize the state vector and covariance matrix (the state vector can be initialized based on the initial prediction results of the multinomial regression model), and determine the process noise covariance matrix and observation noise covariance matrix (the values ​​of these matrices need to be adjusted according to the actual situation to balance the accuracy and stability of the prediction). Then, at each time step, perform the following operations: predict the state vector and covariance matrix of the next time step according to the state transition model, and update the state vector and covariance matrix according to the observation model and the actual observations.

[0051] Based on the basketball state vector predicted by the extended Kalman filter, the landing point coordinates of the basketball when reaching the basket plane are continuously predicted, and at the landing point, the incidence angle of the basketball relative to the basket plane is calculated according to the velocity vector of the basketball (the incidence angle can be calculated using the dot product and cross product operations of vectors).

[0052] In this way, combined with polynomial regression and extended Kalman filtering, the historical data of the basketball flight trajectory can be fully utilized to accurately jointly predict the projection result (landing point coordinates and incidence angle) of the basketball relative to the center of the basket. This method combines the fitting ability of polynomial regression and the dynamic prediction ability of extended Kalman filtering, improving the accuracy and reliability of the prediction.

[0053] By fusing RGB images and depth map data, high-precision three-dimensional trajectory reconstruction and landing point prediction of the basketball center before and after landing the frame are realized. The system can combine target detection driven by deep learning, multi-modal feature fusion, and spatio-temporal trajectory optimization algorithm to ultimately achieve an accuracy of ≤1 cm in landing point position calculation error and ≤1° in angle deviation, providing an academic-level solution for sports scientific analysis.

[0054] In an embodiment, based on the depth camera, the RGB video stream and the depth map during the flight of the basketball are obtained, more basketball flight information can be captured, and through a series of operations such as target detection, point cloud data fitting, generating a three-dimensional model of the ball, and flight trajectory, the dynamic process of the basketball projection can be comprehensively and meticulously mastered. Finally, combined with polynomial regression and extended Kalman filter prediction of the projection result, the landing point coordinates and incidence angle of the basketball can be accurately calculated, providing more detailed and accurate shooting result information, thereby meeting the demand for accurate analysis of the basketball projection.

[0055] In an embodiment, based on the above embodiment, the step of detecting the basketball target according to the RGB video stream comprises: For the image frames of the RGB video stream, a YOLOv11 architecture is used for basketball target detection. Among them, the YOLOv11 architecture enhances the target recognition ability based on the attention mechanism.

[0056] In this embodiment, the YOLOv11 architecture is used for basketball target detection of the image frames of the RGB video stream, and the YOLOv11 architecture introduces an attention mechanism to enhance its target recognition ability, so as to more accurately locate and identify the small target of the basketball in the image frames.

[0057] First, the RGB video stream is decomposed into a series of RGB images frame by frame. These image frames contain visual information of the basketball at different moments, but due to the high-speed motion of the basketball, there may be motion blur phenomena, and in complex game scenes, the basketball may be partially occluded.

[0058] The YOLO (You Only Look Once) series algorithm is a classic real-time target detection algorithm, and YOLOv11, as its subsequent version, inherits the characteristics of efficient detection. YOLOv11 converts the target detection task into a regression problem, directly predicting the bounding box and class probability on the image through a single neural network. Specifically, YOLOv11 divides the input RGB image into multiple grids, and each grid is responsible for predicting multiple bounding boxes and their corresponding class probabilities.

[0059] Optionally, the attention mechanism uses the SE module. The SE module is a channel attention mechanism, and its core idea is to learn the importance of each channel and adaptively adjust the channels of the feature map, thereby enhancing the network's focus on important features.

[0060] In order to enhance the recognition ability of YOLOv11 for small targets and solve the problems of motion blur and occlusion, the SE module is integrated into the feature extraction part of YOLOv11. The specific method is to add an SE module after the convolution layer of YOLOv11, and perform attention weighting on the convolution output feature map. The following are the detailed integration steps: Use the convolution network of YOLOv11 to extract features from the input RGB image, and get a series of feature maps of different scales. These feature maps contain different levels of feature information of the image, but may be affected by motion blur and occlusion, resulting in unclear feature expression.

[0061] The extracted feature map is input into the SE module and processed according to the compression, excitation and re-scaling steps. Through the SE module, the network can adaptively adjust the feature response of each channel, enhance the important features related to the basketball target, and suppress the influence of irrelevant background and noise.

[0062] The feature map weighted by the SE module is input into the subsequent detection layer of YOLOv11 for bounding box prediction and class classification. At this time, due to the effect of the SE module, the network can focus more on the key features of the basketball target, improving the recognition ability for small targets such as distant or partially occluded basketballs.

[0063] Motion blur can cause the target edge in the image to be blurred, and the feature information to be lost. The SE module can enhance the channel information related to the target feature through the channel attention mechanism, so that the network can extract more effective features from the blurred image to some extent, thereby improving the recognition ability of the basketball in motion blur.

[0064] When the basketball is partially occluded, its complete feature information cannot be fully presented. The SE module can adaptively adjust the channel weight according to the global feature information, highlight the important features of the unoccluded part, and help the network better recognize the occluded basketball target.

[0065] After the YOLOv11 architecture integrated with the SE module is processed, the model will output the detection results of the basketball target in each RGB image, including the bounding box information (such as the coordinates of the upper left corner and the lower right corner) of the basketball target and the confidence of the target being a basketball. By setting a suitable confidence threshold, reliable basketball target detection results can be filtered out to provide a basis for subsequent point cloud data acquisition and flight trajectory analysis.

[0066] In an embodiment, by integrating the SE module into the YOLOv11 architecture, the recognition ability of the small target such as basketball can be effectively enhanced, the problems of motion blur and occlusion in high-speed motion can be solved to some extent, and the accuracy and reliability of basketball target detection can be improved.

[0067] In an embodiment, on the basis of the above-mentioned embodiment, the step of aligning the depth map with the corresponding image frame in the RGB video stream comprises: The depth map is subjected to bilateral filtering and morphological operation optimization, and the optimized depth map is aligned with the corresponding image frame in the RGB video stream through guided filtering.

[0068] In this embodiment, when acquiring the basketball shooting related data based on the depth camera, the depth map needs to be aligned with the corresponding image frame in the RGB video stream to realize effective fusion and utilization of multi-modal data. The depth map contains the depth information of the scene, but may have noise and void phenomenon, which affects the subsequent target detection and analysis. Therefore, the depth map can be preprocessed first, and the accurate alignment with the RGB image frame can be realized through a suitable filtering method.

[0069] Bilateral filtering is a nonlinear filtering method that combines the spatial proximity and pixel value similarity of the image. In the filtering process, not only the spatial position relationship of the pixels is considered, but also the difference in pixel value. For each pixel in the depth map, bilateral filtering calculates the weighted average value according to the spatial distance and depth value difference of the surrounding pixels, so as to smooth the image while preserving the edge information.

[0070] In the depth map, noise can manifest as random fluctuations in depth values, and bilateral filtering can effectively remove these noises while avoiding smoothing out edges and details in the depth map, such as the boundary between the basketball and the background. This can make the depth map clearer and provide a better foundation for subsequent processing.

[0071] Optionally, appropriate bilateral filtering parameters are selected, including spatial domain standard deviation and value domain standard deviation. The spatial domain standard deviation controls the spatial range of filtering, and the value domain standard deviation controls the degree of influence of pixel value difference on the filtering result.

[0072] Optionally, for each pixel in the depth map, the weighted average value of its surrounding pixels is calculated as the filtering result of the pixel.

[0073] Morphological operations are methods of processing based on the shape of images, mainly including dilation, erosion, opening operation, and closing operation. Dilation operation will make the object boundary in the image expand outward, and erosion operation will make the object boundary shrink inward. Opening operation first performs erosion and then performs dilation, which can remove small noise points and isolated regions in the image; closing operation first performs dilation and then performs erosion, which can fill small holes in the image.

[0074] The hole phenomenon in the depth map may be caused by measurement errors of the sensor or occlusion, etc. Morphological operations can fill these holes through dilation and closing operations, making the depth map more complete. At the same time, opening operation can remove some small noise points, further improving the quality of the depth map.

[0075] Optionally, according to the situation of holes and noise in the depth map, appropriate morphological operations are selected. For example, if there are many holes, dilation operation can be performed first, and then closing operation can be performed; if there are small noise points, opening operation can be performed first.

[0076] Optionally, appropriate structural elements, such as circles, squares, etc., are selected for morphological operations. The size of the structural element will affect the effect of the operation, which needs to be adjusted according to the specific situation.

[0077] Guided filtering is a filtering method based on guide images, which uses the structural information of guide images to guide the filtering process. In the application of aligning depth maps with RGB image frames, the RGB image is used as the guide image, and the depth map is used as the image to be filtered. Guided filtering establishes a linear relationship between the depth map and the RGB image, so that the filtered depth map retains edge information while being more consistent with the structure of the RGB image.

[0078] By guiding filtering, the optimized depth map can be accurately aligned with the corresponding image frame in the RGB video stream. Since the RGB image contains rich color and texture information, the guiding filtering can use this information to adjust the pixel values of the depth map, so that the depth map and the RGB image are matched at the pixel level, thereby realizing the fusion of multi-modal data.

[0079] Optionally, the RGB image frame is selected as the guide image, and the optimized depth map is selected as the image to be filtered. The local linear model of the guide image and the image to be filtered is calculated, and the filtering coefficient is determined by minimizing the error of the local linear model. The depth map is filtered according to the calculated filtering coefficient to obtain the depth map aligned with the RGB image frame.

[0080] Optionally, the alignment of the depth map and the RGB image frame realized by the guiding filtering makes the depth map and the RGB image have a one-to-one corresponding relationship at the pixel level. This means that each RGB image pixel has a corresponding depth value, thereby realizing the fusion of multi-modal data. In subsequent processing, both the color and texture information of the RGB image and the depth information of the depth map can be used to improve the accuracy of target detection and analysis.

[0081] Pixel-level registration provides more accurate data for subsequent basketball target detection and trajectory analysis. For example, when performing basketball target detection, the color features of the RGB image and the depth information of the depth map can be combined to more accurately determine the position and contour of the basketball. When generating the flight trajectory of the basketball, the aligned depth information can also be used to improve the accuracy of the trajectory.

[0082] In an embodiment, by performing bilateral filtering and morphological operation on the depth map, the depth noise and hole phenomenon can be effectively reduced, and the quality of the depth map can be improved. Then, the guiding filtering is used to realize the alignment of the depth map and the corresponding image frame in the RGB video stream, and the pixel-level registration of multi-modal data is realized. This can provide more accurate and reliable data for subsequent basketball shooting detection and analysis, and improve the performance of the entire detection system.

[0083] In an embodiment, on the basis of the above-mentioned embodiment, before the step of fitting the ball center coordinates of the continuous image frames to generate the flight trajectory of the basketball, the method further comprises: Based on Kalman filtering and smoothing spline interpolation, the ball center coordinates of the continuous image frames are time-optimized.

[0084] In this embodiment, when obtaining the center coordinates of the basketball in the continuous image frames, due to factors such as camera jitter, light changes, instability of the detection algorithm, and the like, the obtained center coordinates of the basketball may have jitter errors. These errors will make the center coordinate sequence discontinuous and irregular, and if directly used for fitting the flight trajectory of the basketball, the trajectory will be inaccurate and cannot truly reflect the actual flight path of the basketball. Therefore, the center coordinates of the continuous image frames can be first time-optimized to eliminate these jitter errors.

[0085] Kalman filtering is a recursive optimal estimation algorithm, which is based on the state equation of a linear system and uses system input and output observation data to perform optimal estimation on the state of the system. When processing the center coordinates of the continuous image frames, the position and velocity of the center can be taken as the state variables of the system. By establishing a state transition model and an observation model, Kalman filtering can predict the state at the current time according to the state estimation value at the previous time and the observation value at the current time, and correct the predicted value to obtain the optimal state estimation.

[0086] Smooth spline interpolation is a method for fitting discrete data points, which determines the interpolation curve by minimizing an objective function containing data fitting error and smoothness. When processing the center coordinates after Kalman filtering, smooth spline interpolation can further eliminate possible small fluctuations, making the center coordinate sequence smoother.

[0087] In an embodiment, after time optimization by Kalman filtering and smooth spline interpolation, the center coordinates of the continuous image frames are smoother and more accurate, and the jitter errors are eliminated. This makes the flight trajectory of the basketball generated by subsequent fitting of the center coordinates more truly reflect the actual flight path of the basketball, and improves the accuracy of trajectory fitting.

[0088] At the same time, the uncertainty caused by jitter errors is reduced, making the entire basketball shooting detection system more stable and reliable. Under different environmental conditions and detection scenarios, the flight information of the basketball can be more accurately obtained, providing a more reliable basis for subsequent shooting result prediction.

[0089] In an embodiment, on the basis of the above embodiment, the step of fitting the center coordinates of the continuous image frames to generate the flight trajectory of the basketball comprises: detecting whether the center coordinates of the continuous image frames satisfy the curvature detection condition; if yes, performing parabolic fitting on the center coordinates of the continuous image frames to generate the flight trajectory of the basketball; if no, performing linear fitting on the center coordinates of the continuous image frames to generate the flight trajectory of the basketball.

[0090] In this embodiment, in a sequence of continuous image frames, the position of the basketball is recognized by a target detection algorithm (such as a deep learning-based target detection model), and the coordinates of the ball center are extracted. These ball center coordinates are recorded in the time sequence of the image frames to form a sequence of coordinate points.

[0091] Optionally, the curvature detection condition includes that the number of ball center coordinate points is greater than or equal to a preset number.

[0092] The number of ball center coordinate points obtained by statistics is compared with the preset number. The setting of the preset number needs to consider various factors, such as the accuracy requirement of curvature calculation, the stability of the fitting method, etc. Generally speaking, if the number of points is too small, the curvature calculation may not be accurate, and the fitting result will also be unstable; if the number of points is too large, the calculation amount will increase. For example, the value of the preset number can be 5.

[0093] Optionally, if the curvature detection condition is met, a parabolic fitting is performed to generate the flight trajectory of the basketball; if the curvature detection condition is not met, a linear fitting is performed to generate the flight trajectory of the basketball.

[0094] Further, if the number of ball center coordinate points is greater than or equal to the preset number, a quadratic polynomial fitting can be performed first, and then it is detected whether the curvature of the fitting curve is less than -1e-6, if yes, a complete parabolic prediction and fitting are performed (that is, the curvature detection condition can also include that the curvature of the initial fitting curve is less than -1e-6); otherwise, a terminal linear extrapolation processing is performed.

[0095] Optionally, the parabolic prediction and fitting can use a parabolic equation, and the least squares method is used again to more accurately fit the ball center coordinate data, to determine the parameters of the parabolic equation, thereby generating a complete basketball flight trajectory that can better reflect the curved flight path of the basketball under the action of gravity and other factors.

[0096] Optionally, the terminal linear extrapolation processing can determine the related parameters of the linear equation by the least squares method, and then extrapolate the flight trajectory of the basketball according to the linear equation to predict the subsequent position.

[0097] In an embodiment, by setting the preset number condition of the number of ball center coordinate points, it is ensured that there is enough data for curvature calculation and fitting, which improves the accuracy of curvature calculation and the stability of the fitting result. Further, according to the curvature characteristics of the quadratic polynomial fitting curve, the fitting method can be further subdivided, which can more accurately adapt to different flight states of the basketball, so that the generated flight trajectory is more in line with the actual situation, and provides a more reliable basis for subsequent projection result analysis.

[0098] In an embodiment, on the basis of the above-mentioned embodiment, after the step of jointly predicting the projection result of the basketball relative to the center of the basket based on the converted trajectory of the basketball flight by using polynomial regression and extended Kalman filter, the method further comprises: outputting the projection result to a display device on the basketball court.

[0099] In this embodiment, after predicting the projection result of the basketball relative to the center of the basket (including the landing point coordinates and the incident angle), the important projection result information can be timely and intuitively displayed to the user, the analyst and other related personnel. This process can be realized by outputting the projection result to a specific display device. The display device can be installed above the backboard, or can be movably arranged at the side of the basketball court, or both.

[0100] In this way, the basketball trainer can easily understand his own shooting data. After each shot, the trainer can immediately see the landing point coordinates of the basketball from the display device. If the landing point deviates greatly from the center of the basket, the trainer can intuitively understand the deviation degree and direction of this shot. For example, if the displayed landing point always deviates to the left side of the basket, the trainer can realize that there may be a problem of excessive force to the left or incorrect hand posture when shooting, so as to make targeted adjustments.

[0101] The display of the incident angle can help the trainer clearly know the angle of the basketball entering the basket when he shoots. Different shooting methods and distances correspond to a relatively appropriate incident angle range. For example, medium and long distance shots usually require a large vertical incident angle to increase the probability of entering the basket. The trainer can judge whether his shooting angle is reasonable by observing the displayed incident angle, and then improve the shooting height and force control.

[0102] As the training continues, the trainer can accumulate the projection result data of multiple shots. By comparing the shooting data of different time periods, the trainer can analyze whether his training effect has improved. For example, whether the distribution of the landing point coordinates gradually converges to the center of the basket, or whether the incident angle is getting closer to the ideal range. This trend analysis helps the trainer understand his progress in shooting technology, and also helps him find possible bottleneck problems.

[0103] In basketball training, the trainer will practice shooting at different positions, such as different areas outside the three-point line, near the free throw line, etc. The projection result provided by the display device can help the trainer compare his shooting data at different positions. The trainer may find that his shooting accuracy is higher at a certain position, but lower at other positions, so as to focus on training the weak positions and improve the overall shooting ability.

[0104] According to the shooting data presented by the display device, the trainer can accurately find out the weak links in his own shooting technique. For example, if it is found that the landing point deviation is large or the incident angle is unstable when shooting a jump shot, the jump shot can be taken as a key training project, and a special training plan can be developed to strengthen this aspect of the technique.

[0105] Based on the understanding of the shooting data of oneself, the trainer can design a more personalized training program. For example, if the data shows that the incident angle is too small when shooting at a long distance, some training actions specifically aimed at improving the height and angle of long-distance shooting can be added, such as high-arc shooting practice using auxiliary equipment, etc.

[0106] In an embodiment, on the basis of the above-mentioned embodiment, the depth camera is a first depth camera arranged above the basket; after the step of jointly predicting the shooting result of the basketball relative to the center of the basket based on the coordinate-converted flight trajectory of the basketball using polynomial regression and extended Kalman filtering, the method further comprises: adjusting the shooting parameters of the shooting machine according to the shooting result and the shooting action score; Wherein, based on the second depth camera, shooting action data of the basketball shooting is obtained, and the shooting action score is generated based on the shooting action data; the second depth camera is arranged on the side of the basketball court.

[0107] In this embodiment, on the basis of the original basketball shooting detection method, a second depth camera is introduced to obtain shooting action data and generate a shooting action score. At the same time, the shooting parameters of the shooting machine are adjusted in combination with the previously predicted shooting result of the basketball relative to the center of the basket, forming a feedback regulation system to optimize the subsequent shooting training effect.

[0108] Optionally, a second depth camera is installed at a suitable position on the basketball court, and its position and angle should be able to capture the whole body action of the player when shooting, for example, it can be installed on the side of the basketball court.

[0109] When the player performs a shooting action, the second depth camera starts working and obtains the RGB video stream and depth map of the player in real time during the shooting process. These data contain the position and posture information of each part of the player's body in three-dimensional space, such as the stretching angle of the arm, the bending degree of the leg, the transfer of the center of gravity of the body, etc.

[0110] Optionally, key features related to the shooting motion are extracted from the RGB video stream and depth map obtained from the second depth camera. For example, through a skeletal key point detection algorithm, the positions of various joints of the player's body (such as shoulders, elbows, wrists, knees, etc.) are identified, and features such as the angles, distances, and speeds between these joints are calculated. Deep learning models such as OpenPose can be used to achieve accurate detection of skeletal key points.

[0111] Optionally, a shooting motion scoring model is established, which can be based on machine learning or deep learning methods. For example, machine learning algorithms such as support vector machines, random forests, etc. can be used, or deep learning models such as convolutional neural networks or recurrent neural networks can be constructed. In the model training phase, a large amount of shooting motion data of players of different levels is collected, and the corresponding scores given by professional coaches are used as labels to train the model.

[0112] The extracted shooting motion features are input into the trained scoring model, and the model calculates the shooting motion score according to the pre-set rules and weights. The score can be in percentage, and the higher the score, the more standard and normative the shooting motion.

[0113] The previously predicted projection result of the basketball relative to the center of the basket (including the landing point coordinates and the incident angle) obtained through the first depth camera and the currently generated shooting motion score are integrated. These two data respectively reflect the flight trajectory of the basketball and the quality of the player's shooting motion.

[0114] Optionally, an adjustment strategy for the shooting parameters of the shooting machine is formulated according to the projection result and the shooting motion score. For example, if the projection result shows that the basketball landing point deviates far from the basket, and the shooting motion score is low, it indicates that the player's shooting technique has a big problem, at this time the shooting speed and height of the shooting machine can be reduced, and the shooting angle can be adjusted to make the basketball more easily caught by the player and performed a shot; if the projection result is close to the basket, but the shooting motion score is not high, the shooting rhythm of the shooting machine may need to be adjusted to give the player more preparation time to adjust the shooting motion.

[0115] According to the formulated adjustment strategy, control instructions are sent to the ball launching machine to adjust the ball launching parameters of the ball launching machine, such as the ball launching speed, the ball launching height, the ball launching angle, and the ball launching interval time, etc. After receiving the instructions, the ball launching machine launches the ball according to the new parameters, and provides more suitable shooting training conditions for the player. For example, if the user's shooting action score shows that the user's jumping height is not high enough, resulting in a lower shooting point, and the shooting result shows that the basketball often cannot hit the basket due to insufficient height, the system can adjust the ball launching machine to increase the ball launching height, so that the user can shoot at a more suitable height, thereby having the opportunity to improve the shooting action and increase the hit rate. If the user's shooting action score shows that the user's reaction after receiving the ball is slow, resulting in poor shooting rhythm, the system can reduce the ball launching frequency to give the user more preparation time.

[0116] In an embodiment, by adjusting the ball launching parameters of the ball launching machine in combination with the shooting result and the shooting action score, personalized shooting training programs can be provided for players of different levels, and the training effect can be improved; at the same time, a closed-loop feedback regulation system is formed, which continuously adjusts the training conditions according to the performance of the player, which helps the player to gradually improve the shooting technique and increase the shooting hit rate; the large amount of data obtained by the depth camera is analyzed and used for decision-making, so that the parameter adjustment of the ball launching machine is more scientific and accurate.

[0117] Further, the shooting result, the shooting action score, and the analysis data are integrated to form high-level player performance data. These data not only contain the flight result of the basketball, but also reflect the quality and efficiency of the player's shooting action, and comprehensively show the player's performance in the shooting process.

[0118] Optionally, the high-level player performance data is visualized, such as generating a shooting heat map and an action trajectory graph, etc. Through the analysis of these visualized data, the coach and the player can intuitively understand the advantages and disadvantages of the player, and provide targeted suggestions for subsequent training.

[0119] According to the high-level player performance data, an adjustment strategy for the ball launching parameters of the ball launching machine is formulated. For example, if the shooting result shows that the landing point of the basketball deviates to the right side of the basket, and the shooting action analysis shows that the player is used to exerting force to the right side, the ball launching position and angle of the ball launching machine can be adjusted to make the basketball more biased to the left side, helping the player to correct the action. At the same time, the ball launching speed and rhythm are adjusted according to the shooting action score. For players who are not familiar with the action, the ball launching speed can be appropriately reduced and the ball launching interval time can be increased.

[0120] The adjusted ball launching parameters are converted into control instructions and sent to the ball launching machine. After receiving the instructions, the ball launching machine launches the ball according to the new parameters, and provides more suitable shooting opportunities for the player according to the training needs.

[0121] Players begin a new round of shooting practice based on the adjusted serves from the ball-serving machine. The depth cameras collect data again, and the data processing, analysis, and parameter adjustment process is repeated. As training continues, data is continuously collected and analyzed to optimize the shooting motion scoring model and serve parameter adjustment strategies. This allows the training system to better adapt to player progress and changes, continuously improving training effectiveness.

[0122] In this way, the entire training system forms a closed loop, continuously adjusting the ball-serving parameters of the ball-serving machine based on the player's performance, thereby achieving personalized and efficient shooting training.

[0123] In one embodiment, based on the above embodiments, the shooting action data includes an RGB video stream and depth information during the basketball shot; the basketball shot detection method further includes: Based on the shooting motion data, the three-dimensional coordinates of multiple human joints are extracted to generate a dynamic sequence of human joints. Biomechanical characteristics were calculated based on dynamic sequences; Biomechanical features are input into a motion scoring model to generate the shooting motion score; wherein the motion scoring model has been pre-learned to have a correlation between biomechanical features and motion scores.

[0124] In this embodiment, during the user's shooting process, the second depth camera continuously acquires RGB video streams and corresponding depth information at a certain frame rate. The RGB video stream records a sequence of color images of the shooting scene, while the depth information provides the distance from each point in the scene to the camera. The combination of the two can provide rich visual and spatial information for subsequent analysis.

[0125] The location and extent of the human body are detected from each frame of RGB images acquired using a second depth camera, and the human body is segmented from the background. Human body detection can be performed using deep learning-based object detection algorithms (such as YOLO, Faster R-CNN, etc.), and then the human body region can be separated from the background using image segmentation algorithms (such as Mask R-CNN, etc.).

[0126] By combining RGB images and depth information, a human pose estimation algorithm is used to extract the two-dimensional coordinates of multiple human joints. Optional human pose estimation algorithms include OpenPose and AlphaPose, which can identify key joints of the human body, such as the head, shoulders, elbows, wrists, hips, knees, and ankles. For example, 33 key joints distributed across the head, shoulders, elbows, wrists, hips, knees, and ankles are extracted.

[0127] The two-dimensional coordinates of the extracted human joint points are converted into three-dimensional coordinates using depth information. By combining the two-dimensional coordinates and the corresponding depth values, the position of each joint point in three-dimensional space can be calculated using the intrinsic and extrinsic parameters of the depth camera.

[0128] In consecutive video frames, data association is performed on the same human joint points to ensure the correct correspondence of each joint point between different frames. Data association can be achieved using methods based on feature matching, motion tracking, etc. For example, a Kalman filter can be used to track the motion trajectory of the joint points.

[0129] Optionally, the skeletal point data is synchronized with the video frames using time series alignment technology to generate a skeletal point dynamic sequence (SSD) as the dynamic sequence of the human joint points. In this way, 30 frames of data per second can be processed, with a coordinate error controlled within ±1.5 cm.

[0130] In which the three-dimensional coordinates of each joint point in consecutive video frames are arranged in chronological order to generate a dynamic sequence of human joint points. Each dynamic sequence contains the position change information of the joint point during the entire shooting process, which can reflect the motion posture and action process of the human body.

[0131] Kinematic features such as joint displacement, velocity, and acceleration are calculated from the dynamic sequence of human joint points. For example, the three-dimensional coordinate difference between adjacent frames can be used to obtain the joint displacement, and the time derivative of the displacement can be used to obtain the velocity and acceleration.

[0132] Combining kinematic features and physical parameters of the human body (such as mass, moment of inertia, etc.), dynamic features such as joint torque and power are calculated. Dynamic features can reflect the muscle force and energy consumption of the human body during the shooting process.

[0133] The calculated kinematic and dynamic features are fused to form comprehensive biomechanical features. Then, according to specific analysis requirements and feature correlations, features that are important for shooting motion analysis are selected, and redundant and irrelevant features are removed.

[0134] Optionally, the biomechanical features include joint angles (such as shoulder flexion angle, elbow extension angle, and knee flexion angle), kinematic parameters (such as shooting speed, force chain timing synchronization, and body stability index), and spatiotemporal features (such as shooting preparation time and shooting instant height).

[0135] Wherein, shoulder flexion refers to the movement of the arm in the sagittal plane towards the upper front, and the three-dimensional coordinates of the human body joint nodes can be determined to calculate the limb vector related to the shoulder joint, and then the shoulder flexion angle can be calculated. Elbow extension refers to the movement of the forearm away from the upper arm in the sagittal plane, and the three-dimensional coordinates of the relevant joint nodes can be determined to construct a vector to calculate the angle. Knee flexion refers to the movement of the lower leg towards the thigh in the sagittal plane, and the three-dimensional coordinates of the hip joint, knee joint and ankle joint can be determined to calculate the angle.

[0136] Wherein, the shooting speed refers to the speed at the moment of shooting the basketball, which can be calculated by tracking the position change of the basketball or the hand joint nodes before and after shooting. The body stability index reflects the balance and stability of the body during shooting, which can be calculated by analyzing the displacement and shaking of the body center of gravity.

[0137] Wherein, the force chain refers to the sequence of body parts that exert force in a certain order during shooting to achieve the best shooting effect. The hip joint, shoulder joint, elbow joint, etc. can be selected as the key force joints; the time when each joint starts to exert force can be determined by calculating the acceleration or moment change of each joint; the time difference between the adjacent joint force exertion time can be calculated; the time sequence synchronization of the force chain can be evaluated according to the comparison of these time differences with the standard time difference. Correlation coefficient or error index can be used to quantify the synchronization.

[0138] Wherein, the shooting preparation stage refers to the time period from the user starting to prepare for shooting to the moment before shooting, and the duration of this stage can be determined by analyzing the shooting video and the dynamic sequence of the joint nodes. The shooting moment height refers to the vertical height at the moment of shooting the basketball, which can be calculated by tracking the three-dimensional coordinates of the basketball or the hand joint nodes at the shooting moment.

[0139] A plurality of data sets are collected in advance, each data set including biomechanical features, and shooting videos and shooting motion scores associated with the biomechanical features. These data sets can be obtained by collecting and labeling the shooting motions of a large number of different users.

[0140] Optionally, a professional basketball coaching team (with at least 10 years of coaching experience) can be invited to manually score tens of thousands of shooting videos, and the scoring dimensions include the timing relationship of each stage of shooting and the angle relationship of each stage of shooting.

[0141] A ternary training data set is constructed, including biomechanical features, and shooting videos and shooting motion scores associated with the biomechanical features. Among them, the label is standardized as a standardized score (Z-score normalization).

[0142] An initial decision tree is constructed using a CART (Classification and Regression Trees) algorithm, with the shooting motion score as the target variable and the biomechanical features as the input variables. The decision tree model is iteratively trained using the collected data set. During the training process, the decision tree model selects the splitting features by information gain ratio, i.e., determines which features to use for node splitting based on the classification ability of each feature on the data set, to improve the classification accuracy of the model.

[0143] Optionally, by calculating the information gain ratio of each feature, the feature with the maximum information gain ratio is selected as the splitting feature of the current node. This ensures that the selected feature can most effectively divide the data set, while avoiding the selection of features with too many values but little actual help for classification or regression, thereby reducing the risk of overfitting.

[0144] Optionally, the selected key splitting features include the shoulder joint angle (at the moment of shooting), the elbow joint extension speed, and the lower limb force timing difference (the time offset of the angle changes of the knee joint and the hip joint).

[0145] The decision tree model is optimized using the cross-validation method to adjust parameters such as tree depth and node splitting threshold to avoid overfitting and underfitting problems and improve the generalization ability of the model.

[0146] In the practical application of the decision tree model, the biomechanical features calculated from the field measurement are input into the trained decision tree model, and the model classifies and judges according to the feature values and the tree structure to finally generate the shooting motion score. This score can reflect the standardization, accuracy, and efficiency of the user's shooting motion.

[0147] In an embodiment, the RGB video stream and depth information during the user's shooting process are obtained, the three-dimensional coordinates of multiple human body joints can be extracted, and the player's motion posture and position change in three-dimensional space can be accurately obtained, avoiding errors caused by shooting angles and occlusions when judging key actions. At the same time, by generating a dynamic sequence of human body joints and calculating biomechanical features, a more comprehensive and accurate data basis is provided for shooting analysis. Furthermore, by inputting the biomechanical features into the decision tree model trained by multiple data sets to generate the shooting motion score, the analysis efficiency is improved, the human and time costs are reduced, the influence of subjective judgment differences is reduced, the accuracy and consistency of the analysis results are guaranteed.

[0148] In an embodiment, based on the above embodiment, the decision tree model uses 10-fold cross-validation to optimize the tree depth and node splitting threshold.

[0149] In this embodiment, 10-fold cross-validation is a model evaluation and parameter tuning technique that randomly and uniformly divides the entire dataset into 10 non-overlapping subsets (folds). In each iteration, 9 subsets are selected as the training set to train the decision tree model, and the remaining 1 subset is used as the validation set to evaluate the performance of the model. In this way, a total of 10 iterations will be performed, each using a different validation set. Finally, the evaluation results of the 10 iterations are averaged to obtain a comprehensive performance indicator.

[0150] In an embodiment, through 10-fold cross-validation, the performance of the model on different data subsets can be more comprehensively evaluated, avoiding evaluation bias caused by randomness of data division, so as to find a more general parameter combination; and selecting the optimal tree depth and node split threshold can make the decision tree model better fit the data, improve the accuracy of the model in predicting the shooting action score; and by reasonably adjusting the tree depth and node split threshold, the problem of overfitting or underfitting of the model can be effectively avoided.

[0151] In an embodiment, on the basis of the above embodiment, in the training process of the decision tree model, the decision tree split rule is output to an artificial review interface; Based on the correction information fed back by the artificial review interface, the decision tree split rule is updated.

[0152] In this embodiment, in the shooting analysis method based on the decision tree model, the decision tree model is the core analysis tool. However, the split rules generated in the model training process may not conform to the logic of sports physiology. In order to improve the accuracy and reliability of the decision tree model and make the rules generated by it more consistent with the actual shooting movement rules, an artificial review mechanism is introduced. This mechanism allows sports biomechanics experts and coaching teams to review and correct the decision tree split rules to ensure that the results output by the model have practical application value.

[0153] In the training process of the decision tree model, the generated decision tree split rules are output to a specially designed artificial review interface. This interface is a visual interactive platform that can display the split rules of the decision tree in a clear and understandable way to the reviewers. For example, the rules may be presented in the form of "if the shooting height is greater than X centimeters, the score tends to Y", which facilitates reviewers to intuitively understand the meaning of each rule.

[0154] An audit team composed of sports biomechanics experts and coaching teams carefully reviews the decision tree split rules displayed on the artificial review interface. These experts and coaches have rich professional knowledge and practical experience, and can judge from the perspective of sports physiology whether each rule is reasonable.

[0155] During the review process, special attention will be paid to branches that may not conform to the logic of sports physiology. For example, in a normal shooting motion, there is usually a positive correlation between shooting height and scoring, as a higher shooting height can increase the hit rate of the shot. If the decision tree model generates an abnormal rule such as "shooting height is negatively correlated with scoring", the reviewer will identify it.

[0156] Once the rule that does not conform to the logic of sports physiology is found, the reviewer will modify it on the manual review interface. The way to modify it can be to delete the unreasonable rule, adjust the conditions or conclusions of the rule, or add new rules according to professional knowledge. For example, for the abnormal rule "shooting height is negatively correlated with scoring", the reviewer can directly delete it to ensure that the rules of the decision tree model are more in line with the actual situation.

[0157] After the manual review is completed, the manual review interface will feed back the modification information of the reviewer to the decision tree model. The model will update its splitting rules according to these feedback information. During the update process, the model will re-adjust the structure of the decision tree to ensure that the modified rules are correctly applied to the model. The updated decision tree model will more accurately reflect the actual rules of the shooting motion, thus improving the accuracy and reliability of the shooting motion evaluation.

[0158] In an embodiment, by introducing the manual review mechanism, the rules in the decision tree model that do not conform to the logic of sports physiology can be found and corrected in a timely manner, making the shooting motion evaluation generated by the model more accurate. The review and correction of professionals ensure that the rules of the decision tree model have practical application value, improving the reliability and stability of the model.

[0159] In this way, the combination of the professional knowledge of sports biomechanics experts and the coaching team with the data analysis capabilities of the decision tree model fully utilizes the advantages of both, making the shooting analysis method more scientific and reasonable.

[0160] Taking "the shoulder joint angle at the moment of shooting is higher than 140° as the shooting angle being too large" as an example, the optimization process of the model rules is described: (1) Data-driven rule discovery The initial decision tree found that the shoulder joint angle (α) was significantly negatively correlated with the coach's score (correlation coefficient r=-0.82) during splitting. When α>140°, 73% of the shots in the sample received a coach's score lower than 6 points (out of 10), and were often accompanied by the comment "the shot was too high, causing the trajectory to be unstable".

[0161] (2) Biomechanical interpretation Over-extended shoulder joint (a>140°) will destroy the force chain transmission efficiency, resulting in: upper limb compensation force ratio more than 45% (standard value is 30%); the proportion of sphere side rotation increases (>30%), which reduces the accuracy of the basket angle; the ideal shooting shoulder angle should be kept between 120°-135° (based on NBA player database statistics).

[0162] (3) Expert rule revision The initial rule "a>140°, then deduct points" is revised by the coach team to a conditional rule: "IF a>140° AND core stability index<0.6 (detecting trunk shaking); THEN shooting angle score=original score×0.7; ELSE IF a>140° BUT shooting distance≥7.25 meters (outside the three-point line); THEN shooting angle score=original score×0.9 (super long shot allows a slightly larger angle); ".

[0163] (4) Model verification results The performance of this rule in the test set (2000 groups of samples): accuracy is 88.7% (differences within ±1 points from the coach's score are considered correct), precision is 91.2% (for "shooting angle too large" class samples), recall is 85.4%.

[0164] In an embodiment, on the basis of the above embodiment, a fuzzy logic compensation mechanism is introduced in the training process of the decision tree model, expert adjustment scores and corresponding expert weights are added to the scores of the terminal nodes of the decision tree, and the decision tree prediction scores and the expert adjustment scores of the decision tree model are weighted and summed to generate the shooting motion score.

[0165] In this embodiment, when shooting analysis is performed based on the decision tree model, although the decision tree model can derive a prediction score of the shooting motion according to the training data, due to the complexity, diversity and some difficult-to-quantify factors of the shooting motion, the prediction result of the decision tree model may have certain limitations. The introduction of the fuzzy logic compensation mechanism, combined with the professional judgment of experts, can comprehensively consider more actual situations, further optimize the shooting motion score, make the scoring result more consistent with the real shooting performance, and improve the accuracy and reliability of the score.

[0166] Fuzzy logic is a mathematical method for handling uncertainty and fuzziness. In the shooting analysis scenario, there is a certain fuzzy boundary for the goodness of the shooting motion, which is difficult to completely define with precise rules. Through the fuzzy logic compensation mechanism, the adjustment opinions given by experts based on rich experience and professional knowledge can be considered to supplement and correct the scoring result of the decision tree model.

[0167] In the decision tree model training process, when the model analyzes the biomechanical characteristics of a shooting action and obtains the prediction score of the terminal node of the decision tree, the related information of the shooting action (such as shooting video, biomechanical characteristic data, etc.) is provided to the expert team. The expert team evaluates the shooting action based on their professional knowledge and experience, and gives a corresponding adjustment score for each shooting action. The adjustment score can be positive, indicating an increase in the score; or negative, indicating a decrease in the score.

[0168] The decision tree prediction score of the decision tree model and the adjustment score given by each expert are weighted and summed to generate the final shooting action score.

[0169] Select experts with deep professional knowledge and rich practical experience in related fields such as shooting technology and sports biomechanics to form an expert team. These experts can use their professional knowledge to observe and analyze the shooting action in detail and give reasonable adjustment scores.

[0170] The decision tree prediction score of the decision tree model and the adjustment score given by the expert are weighted and summed to generate the final shooting action score. The specific calculation formula is as follows: F = A x Tree + B x Expert; Where Tree is the decision tree prediction score, A is the decision tree weight, Expert is the expert adjustment score, and B is the expert weight.

[0171] Optionally, the decision tree weight corresponding to the decision tree prediction score of the decision tree model is greater than the expert weight. For example, the decision tree weight is 0.7 and the expert weight is 0.3.

[0172] In an embodiment, by combining the professional opinions of experts, the shortcomings of the decision tree model in handling complex shooting actions can be compensated for, making the scoring results more accurately reflect the true level of the shooting action. The fuzzy logic compensation mechanism allows for flexible adjustments based on different shooting scenarios and specific circumstances, making the scoring more targeted. Combining the subjective judgments of experts with the objective analysis of the decision tree model fully utilizes the advantages of both, improving the scientificity and reliability of shooting analysis.

[0173] In an embodiment, on the basis of the above embodiment, the shooting action score is output to a display device on the basketball court.

[0174] In this embodiment, after the shot motion score is completed based on the decision tree model, the system outputs the generated shot motion score to the display device on the basketball court in real time to enable the user to intuitively understand the shot performance. The display device can be installed above the backboard, or can be movably arranged on the side of the basketball court, or both.

[0175] Optionally, the display device displays the shot motion score in real time in the form of a number, enabling the user to intuitively understand the score of each shot. The score number can be in a prominent color, such as red or green, to distinguish between high and low scores, so that the audience and the coach can quickly judge the shot performance of the player.

[0176] In addition to displaying the current shot motion score in real time, the display device can also statistically display the historical scores of the player. For example, the average score, the highest score, and the lowest score of the player in the current game or training can be displayed. Through these statistical data, the user can more comprehensively understand the trend of the shot level.

[0177] The shot motion scores of different players can be compared and displayed. In team training or competition, this comparison method enables the player to intuitively see the gap between himself and his teammates, stimulates the competitive consciousness, and also helps the coach to find the advantages and disadvantages of the player and to develop more targeted training plans.

[0178] This not only enhances the user experience, but also provides valuable reference information for coaches, audiences, and the like, and enriches the interactivity and professionalism of basketball activities.

[0179] In addition, the device system provided in the embodiment of the present application has an internal architecture as shown in Figure 2 The device system includes a processor, a memory, a communication interface, and an input interface connected through a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store data called by the computer program. The communication interface is used for data communication with an external terminal. The input interface is used to receive signals input by an external device. The computer program is executed by the processor to implement a basketball shooting detection method as described in the above embodiment.

[0180] Those skilled in the art can understand that Figure 2 the structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the device system to which the scheme of the present application is applied.

[0181] In addition, the application further provides a computer readable storage medium including a computer program, which, when executed by a processor, implements the steps of the basketball shooting detection method according to the above embodiments. It can be understood that the computer readable storage medium in the embodiments can be a volatile readable storage medium or a non-volatile readable storage medium.

[0182] In summary, the basketball shooting detection method, device system and computer readable storage medium provided in the embodiments of the application can acquire an RGB video stream and a depth map in the basketball flying process based on a depth camera, can capture more basketball flying information, and can comprehensively and meticulously master the dynamic process of basketball shooting through a series of operations such as target detection, point cloud data fitting, generation of a spherical three-dimensional model and flight trajectory. Finally, the shooting result is predicted by combining polynomial regression and extended Kalman filtering, the landing point coordinates and the incidence angle of the basketball can be accurately calculated, more detailed and accurate shooting result information is provided, and the demand for accurate analysis of basketball shooting is met.

[0183] It can be understood by those skilled in the art that all or part of the processes in the above embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above embodiments can be included. Any reference to a memory, storage, database or other medium provided by the application and used in the embodiments can include non-volatile and / or volatile memory. The non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0184] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or "comprises" does not, without further restriction, exclude the existence of additional elements of the process, method, article, or apparatus that comprises the element.

[0185] The preferred embodiments of the present application have been described above with the intent to enable those skilled in the art to make and use it. Various modifications to these embodiments will occur to those skilled in the art and are intended to be encompassed by the present application. Therefore, it is to be understood that, within the scope of the present application, the application can be practiced otherwise than as specifically described. For example, the order of steps can be varied, or some steps can be omitted or adapted; the use of some reagents can be tailored or made redundant, and other suitable endpoints and parameters can be employed.

Claims

1. A method for detecting basketball shooting, characterized in that, include: Based on a depth camera, acquire RGB video streams and depth maps of the basketball as it flies toward the basket; Basketball target detection is performed based on the RGB video stream, and the depth map is aligned with the corresponding image frames in the RGB video stream. Based on the depth map, point cloud data of the basketball target in each image frame is obtained; The point cloud data of the basketball target in each image frame are fitted to generate a 3D model of the sphere, and the coordinates of the center of the sphere are calculated. The ball's center coordinates are fitted to consecutive image frames to generate the basketball's flight trajectory; Based on the relative positional relationship between the depth camera and the center of the basketball hoop, the camera coordinates corresponding to the basketball's flight trajectory are converted into basketball court world coordinates; Based on the basketball's flight trajectory after coordinate transformation, multinomial regression and extended Kalman filtering are used to jointly predict the basketball's projection result relative to the center of the basket; wherein, the projection result includes the landing point coordinates and the incident angle.

2. The basketball shooting detection method as described in claim 1, characterized in that, The step of detecting a basketball target based on the RGB video stream includes: For image frames in RGB video streams, basketball target detection is performed using the YOLOv11 architecture; Among them, the YOLOv11 architecture enhances target recognition capabilities based on an attention mechanism.

3. The basketball shooting detection method as described in claim 1, characterized in that, The step of aligning the depth map with the corresponding image frames in the RGB video stream includes: The depth map is optimized by bilateral filtering and morphological operations, and guided filtering is used to align the optimized depth map with the corresponding image frames in the RGB video stream.

4. The basketball shooting detection method as described in claim 1, characterized in that, Before the step of fitting the center coordinates of the ball to generate the basketball's flight trajectory from consecutive image frames, the method further includes: Based on Kalman filtering and smooth spline interpolation, the sphere center coordinates of consecutive image frames are optimized in a temporal manner.

5. The basketball shooting detection method as described in claim 1, characterized in that, The step of fitting the center coordinates of the ball to generate the basketball's flight trajectory from consecutive image frames includes: Detect whether the coordinates of the center of the sphere in consecutive image frames satisfy the curvature detection condition; If so, then parabolic fitting is performed on the center coordinates of the ball in consecutive image frames to generate the basketball's flight trajectory; If not, then the coordinates of the center of the ball in consecutive image frames are linearly fitted to generate the basketball's flight trajectory.

6. The basketball shooting detection method as described in claim 1, characterized in that, After the step of jointly predicting the basketball's projection result relative to the center of the basket using multinomial regression and extended Kalman filtering based on the basketball's flight trajectory after coordinate transformation, the method further includes: The projection results are then output to the display device on the basketball court.

7. The basketball shooting detection method as described in any one of claims 1-6, characterized in that, The depth camera is a first depth camera, positioned above the basketball hoop; after the step of jointly predicting the basketball's projection result relative to the center of the hoop using multinomial regression and extended Kalman filtering based on the basketball's flight trajectory after coordinate transformation, the method further includes: Adjust the ball-serving parameters of the ball-serving machine based on the shooting results and shooting motion scores; Specifically, the second depth camera is used to acquire shooting motion data when a basketball is thrown, and a shooting motion score is generated based on the shooting motion data; the second depth camera is placed on the side of the basketball court.

8. The basketball shooting detection method as described in claim 7, characterized in that, The shooting motion data includes an RGB video stream and depth information during the basketball shot; the basketball shot detection method further includes: Based on the shooting motion data, the three-dimensional coordinates of multiple human joints are extracted to generate a dynamic sequence of human joints. Biomechanical characteristics were calculated based on dynamic sequences; Biomechanical features are input into a motion scoring model to generate the shooting motion score; wherein the motion scoring model has been pre-learned to have a correlation between biomechanical features and motion scores.

9. A device system, characterized in that, The device system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the basketball shot detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the basketball shot detection method as described in any one of claims 1 to 8.

Images