Basketball motion detection method and system based on deep learning

Abstract

The invention relates to a basketball motion detection method and system based on deep learning. The method comprises the following steps: acquiring a motion real-time video image of a ball holder; according to the motion real-time video image, extracting a hand feature position and a sphere feature position of the ball holder to obtain a corresponding hand coordinate sequence and a sphere coordinate sequence, and fusing the hand coordinate sequence and the sphere coordinate sequence by adopting an attention mechanism to obtain a person-sphere relation feature vector; calculating the ball holding probability and the ball passing probability of the ball holder through a classifier according to the person-ball relation feature vector, and obtaining a corresponding ball holding classification result and a ball passing classification result; and dynamically fusing the ball holding classification result and the ball passing classification result according to the person-ball relation feature vector to obtain a ball holding player action intention prediction result. By adopting the method, the prediction accuracy of the action and the ball holding intention can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of basketball motion prediction, and in particular relates to a basketball motion detection method and system based on deep learning. Background Art

[0002] With the development of basketball motion prediction technology, a basketball action detection method based on information processing has emerged. This method combines past historical game videos to summarize the laws of basketball motion and is applied to actual basketball games. Based on real-time action images in basketball games, it identifies ball-holding actions (such as dribbling, passing, and shooting) and tactical intentions (such as pick-and-rolls and open cuts).

[0003] The information processing-based basketball motion detection method collects basketball player sports videos in advance and identifies the original basketball motion detection data. In actual basketball games, the optical flow method is used to capture local details in basketball movements, and the three-dimensional convolutional neural network combined with the spatial attention mechanism is used to detect incorrect motion movements, providing an important foundation for the quality analysis and training of basketball movements.

[0004] However, current basketball motion detection methods based on information processing rely on artificial feature design (such as optical flow and trajectory tracking), which are difficult to handle complex occlusion and multi-person interaction scenarios, resulting in reduced accuracy in predicting motion and ball-holding intentions. Summary of the Invention

[0005] Based on this, it is necessary to address the above technical problems and provide a basketball motion detection method and system based on deep learning that can handle complex occlusion and multi-person interaction scenarios and improve the accuracy of action and ball-holding intention prediction.

[0006] In a first aspect, the present application provides a basketball motion detection method based on deep learning, comprising:

[0007] Acquire real-time video images of the player holding the ball;

[0008] Based on the real-time video images of the movement, the hand feature positions of the player holding the ball and the ball feature positions are extracted to obtain the corresponding hand coordinate sequence and ball coordinate sequence. The hand coordinate sequence and the ball coordinate sequence are then fused using the attention mechanism to obtain the human-ball relationship feature vector.

[0009] According to the human-ball relationship feature vector, the classifier is used to calculate the ball holding probability and passing probability of the ball holding player, and the corresponding ball holding classification results and passing classification results are obtained;

[0010] According to the human-ball relationship feature vector, the ball holding classification results and the pass classification results are dynamically fused to obtain the prediction result of the ball holding player's action intention.

[0011] Furthermore, based on the real-time video image of the movement, the hand feature positions of the player holding the ball and the ball feature positions are extracted to obtain the corresponding hand coordinate sequence and ball coordinate sequence. The hand coordinate sequence and the ball coordinate sequence are then fused using the attention mechanism to obtain the human-ball relationship feature vector, including:

[0012] A convolutional neural network is used to detect the coordinates of the preset hand feature points of the ball-holding player and the center coordinates of the sphere feature in the real-time motion video image, and the corresponding hand coordinate sequence and sphere coordinate sequence are obtained;

[0013] The long short-term memory network is used to analyze the hand coordinate sequence and the sphere coordinate sequence to obtain the handball motion feature data;

[0014] Use target detection algorithms to identify the nodes and motion trajectories of all players in real-time sports video images, build a relationship graph between players, and generate a global scene feature map;

[0015] According to the handball action feature data and combined with the global scene feature map, the attention mechanism is used for embedding fusion to obtain the human-ball relationship feature vector.

[0016] Furthermore, a convolutional neural network is used to detect the coordinates of the preset hand feature points of the ball-holding player and the center coordinates of the sphere feature in the real-time motion video image, and obtain the corresponding hand coordinate sequence and sphere coordinate sequence, including:

[0017] A convolutional neural network is used to extract multi-scale features of real-time motion video images by setting multi-branch convolution to obtain multi-branch video feature data.

[0018] The multi-branch video feature data is layered and differentially fused, and a convolutional layer is set to map the multi-branch video feature data to the heat map space to obtain the heat map of the key points of the hand;

[0019] Use the following formula to perform non-maximum suppression on the hand key point heat map, extract the peak point coordinates, and obtain the hand coordinate sequence:

[0020]

[0021] in, is the hand coordinate of the tth frame of the real-time motion video image, W is the horizontal resolution of the real-time motion video image, is the heat map of the key points of the hand in the tth frame of the real-time motion video image, and H is the vertical resolution of the real-time motion video image;

[0022] The color threshold segmentation technology is used to convert the real-time motion video image into color segmentation image data. The following formula is used to extract the sphere features in the color segmentation image data to obtain the sphere feature area data:

[0023]

[0024] Among them, M(x,y) is the basketball area data, I s (x,y) is the data with coordinates (x,y) in the color segmentation image data, and n is the basketball threshold;

[0025] Use the following formula to calculate the center coordinates of the sphere feature area data and obtain the sphere center coordinate data:

[0026]

[0027] Among them, (x b,t ,y b,t ) is the coordinate data of the center of the basketball in the tth frame of the real-time video image of the sports, and M(x, y) is the basketball area data;

[0028] Kalman filtering is used to smooth the trajectory and remove noise from the sphere center coordinate data to obtain the sphere coordinate sequence.

[0029] Furthermore, we use the target detection algorithm to identify the nodes and motion trajectories of all players in the real-time video images of sports, build a relationship graph between players, and generate a global scene feature map, including:

[0030] The DETR algorithm is used to extract the bounding boxes and player features of players in real-time sports video images to obtain player feature sequence data;

[0031] Use the following formula to calculate the cross-frame feature similarity based on the player feature sequence data to obtain the similarity matrix:

[0032]

[0033] Among them, S kl is the similarity matrix between player k in frame t and player l in frame t-1, is the player feature data of player k in frame t, is the player feature data of player l in frame t-1;

[0034] According to the similarity matrix, the Kalman filter equation is used to update the player's motion trajectory, and the player's feature sequence data is recombined to obtain the updated player's feature sequence data;

[0035] Use the following formula to perform geometric relationship encoding on the updated player feature sequence data to obtain the player direction detail features:

[0036] G=(D,θ)

[0037]

[0038] Among them, G is the direction detail feature, D is the distance feature, θ is the direction angle feature, d kl is the distance between player k and player l, is the horizontal coordinate of player k, is the ordinate of player k, is the horizontal coordinate of player l, is the vertical coordinate of player l, θ kl is the angle between the directions of player k and player l, v (k) is the velocity vector of player k, v (l) is the player l is the velocity vector;

[0039] Using the following formula, based on the self-attention weight, the graph structure features are constructed according to the updated player feature sequence data and the player direction detail features to obtain the global scene feature map:

[0040]

[0041] in, is the global scene feature map, is the player feature vector, ε t is the relative motion weight, v i is the player feature vector for player i, x i is the horizontal coordinate of player i, y i is the vertical coordinate of player i, is the lateral velocity of player i, is the longitudinal velocity of player i, cosθ i is the angle θ from the direction of player i i The cosine value of sinθ i is the angle θ from the direction of player i i The sine value of A ij is the relative motion weight of player i and player j, d ij is the Euclidean distance between player i and player j, σ d is the distance attenuation coefficient, v j is the player feature vector for player j.

[0042] Furthermore, based on the handball action feature data and combined with the global scene feature map, an attention mechanism is used for embedding fusion to obtain the human-ball relationship feature vector, including:

[0043] Extract node data from the global scene feature graph, encode the node data based on the graph attention mechanism, and obtain global encoding features;

[0044] A multi-head attention mechanism is used to align and fuse the handball action feature data and the global encoding features to obtain the human-ball relationship feature vector.

[0045] Furthermore, based on the player-ball relationship feature vector, the classifier is used to calculate the ball holding probability and pass probability of the ball holding player, and the corresponding ball holding classification results and pass classification results are obtained, including:

[0046] Extract the sphere position features from the human-sphere relationship feature vector to obtain the sphere position sequence. Based on the sphere position sequence, calculate the sphere instantaneous velocity:

[0047] Based on the instantaneous speed of the ball and the preset speed threshold, the ball holding classification result is calculated through logistic regression using the following formula:

[0048]

[0049] Among them, P h is the ball holding classification result, σ is the activation function, w h is the logistic regression weight, T is the time series length, v t is the instantaneous velocity of the sphere, b h is the logistic regression bias;

[0050] Extract the edge weight feature of the ball-holding player from the ball-holding relationship feature vector to obtain the ball-holding edge weight feature data;

[0051] According to the ball possession classification results and the ball possession edge weight feature data, based on the fully connected layer and the Softmax classifier, the pass classification result is calculated using the following formula:

[0052]

[0053] Where s is the classification score vector of the ball-carrying player, W p is the classification weight matrix, F fusion is the fusion vector of the ball holding classification result and the ball holding node feature data, b p is the classification bias term, s1 is the passing classification score in the ball-holding player classification score vector, s2 is the holding classification score in the ball-holding player classification score vector, P p is the pass classification result.

[0054] Furthermore, based on the player-ball relationship feature vector, the ball holding classification results and the pass classification results are dynamically fused to obtain the prediction results of the ball holding player's action intention, including:

[0055] Extract the node feature part of the ball-holding player from the human-ball relationship feature vector to obtain the ball-holding node feature data;

[0056] Extract the key points of the ball holding node feature data, detect the confidence sequence, and obtain the average feature confidence;

[0057] The occlusion coefficient is calculated from the feature average confidence using the following formula:

[0058]

[0059] Where μ is the occlusion coefficient, T is the length of the time series, and c t is the average confidence of the feature, τ is the confidence threshold;

[0060] The following formula is used to calculate the prediction result of the ball-holding player's action intention based on the ball-holding classification result, the pass classification result, and the occlusion coefficient:

[0061] P f =(1-μ)P h +μP s

[0062] Among them, P f is the prediction result of the ball player’s action intention, μ is the occlusion coefficient, P h is the ball holding classification result, P s is the pass classification result.

[0063] In a second aspect, the present application also provides a basketball motion detection system based on deep learning, comprising:

[0064] The ball holding video acquisition module is used to obtain real-time video images of the ball holding player;

[0065] The feature vector calculation module is used to extract the hand feature positions of the ball holder and the ball feature positions based on the real-time motion video image, obtain the corresponding hand coordinate sequence and ball coordinate sequence, and use the attention mechanism to fuse the hand coordinate sequence and the ball coordinate sequence to obtain the human-ball relationship feature vector;

[0066] The global probability calculation module is used to calculate the ball holding probability and passing probability of the ball holding player through the classifier based on the ball-handling relationship feature vector, and obtain the corresponding ball holding classification results and passing classification results;

[0067] The ball holding result prediction module is used to dynamically fuse the ball holding classification results and the pass classification results according to the human-ball relationship feature vector to obtain the prediction result of the ball holding player's action intention.

[0068] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any one of the deep learning-based basketball motion detection methods described in the first aspect of the present application.

[0069] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the deep learning-based basketball motion detection methods described in the first aspect of the present application.

[0070] The above-mentioned basketball motion detection method and system based on deep learning obtains real-time video images of the ball-holding player; according to the real-time video images, the hand feature positions and sphere feature positions of the ball-holding player are extracted to obtain corresponding hand coordinate sequences and sphere coordinate sequences, and the hand coordinate sequences and sphere coordinate sequences are fused by using an attention mechanism to obtain a human-ball relationship feature vector; according to the human-ball relationship feature vector, the ball-holding probability and the passing probability of the ball-holding player are calculated by a classifier to obtain corresponding ball-holding classification results and passing classification results; according to the human-ball relationship feature vector, the ball-holding classification results and the passing classification results are dynamically fused to obtain a prediction result of the ball-holding player's action intention, and the action characteristics of the ball-holding player, the position characteristics of the basketball and the position characteristics of other players are combined to realize the processing of multi-person interaction scenes and complex occlusions, thereby improving the accuracy of the prediction of actions and ball-holding intentions. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 A flowchart of a basketball motion detection method based on deep learning is provided as an exemplary embodiment of the present application;

[0073] Figure 2 A structural block diagram of a basketball motion detection system based on deep learning provided as an exemplary embodiment of the present application;

[0074] Figure 3 A structural block diagram of a computer device for a basketball motion detection method based on deep learning provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0076] In one embodiment, Figure 1As shown, a basketball motion detection method based on deep learning is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following S101-S104:

[0077] S101, obtaining a real-time video image of the movement of the ball-holding player.

[0078] Specifically, the terminal captures real-time video images of the player holding the ball. This video image can be captured in real time using a high-frame-rate camera deployed on the court to ensure synchronized timestamps. The real-time video image includes, but is not limited to, an image of the player holding the ball, an image of the basketball, images of several players surrounding the player holding the ball, and court border information. The real-time video image is illustratively in RGB format, and can also be converted to RGB format based on the user's actual camera usage.

[0079] S102: Extract the hand feature positions of the ball-holding player and the ball feature positions based on the real-time motion video image to obtain the corresponding hand coordinate sequence and ball coordinate sequence, and fuse the hand coordinate sequence and the ball coordinate sequence using the attention mechanism to obtain the human-ball relationship feature vector.

[0080] Specifically, the terminal extracts the hand features of the player holding the ball based on the real-time video image of the movement, records the position information of the hand feature points in combination with the number of video frames to obtain the corresponding hand coordinate sequence, and extracts the basketball feature information, calculates the spherical coordinates through the feature area, records the spherical coordinates in combination with the number of video frames to obtain the corresponding spherical coordinate sequence, and uses the attention mechanism to fuse the hand coordinate sequence and the spherical coordinate sequence to obtain the human-ball relationship feature vector. Schematically, the geometric center point of the basketball feature area is defined as the spherical coordinate, which can also be adjusted according to the user's ball-holding habits. Optionally, the terminal also extracts the features of all players in the image based on the real-time video image of the movement, records the changes and mutual relationships of the players' feature coordinates in combination with the number of video frames, uses the attention mechanism to embed them, and fuses them with the hand coordinate sequence and the spherical coordinate sequence to obtain the human-ball relationship feature vector containing the feature coordinate change information of the player holding the ball, the spherical coordinate information, and the coordinate change information of other players.

[0081] S103, calculating the ball holding probability and the passing probability of the ball holding player through a classifier according to the player-ball relationship feature vector, and obtaining corresponding ball holding classification results and passing classification results.

[0082] Specifically, based on the player-ball relationship feature vector, the terminal constructs a fully connected layer and classifier to extract and calculate the ball-holding probability and pass probability of the player holding the ball, thereby obtaining the corresponding ball-holding and pass classification results. Illustratively, the terminal calculates the ball-holding classification result based on the sphere coordinate information in the player-ball relationship feature vector. Optionally, the terminal calculates the pass classification result based on the coordinate changes of other players in the player-ball relationship feature vector.

[0083] S104: Dynamically fuse the ball holding classification result and the pass classification result based on the player-ball relationship feature vector to obtain a prediction result of the ball holding player's action intention.

[0084] Specifically, the terminal dynamically fuses the ball-handling classification results and the pass classification results based on the player-ball relationship feature vector to predict the player's action intention. Schematically, the terminal determines the occlusion situation based on the changes in the player's characteristic coordinates in the player-ball relationship feature vector. Based on the occlusion situation, the terminal dynamically fuses the ball-handling classification results with the pass classification results to predict the player's action intention.

[0085] This embodiment obtains the real-time video stream of the player holding the ball, extracts the hand feature point coordinate sequence and the ball trajectory sequence, and fuses the two using the attention mechanism to obtain the human-ball interaction feature vector. The ball holding classification result and the pass classification result are calculated by a fully connected classifier. The classification results are dynamically fused based on the occlusion situation of the ball holding player, and finally the action intention prediction is output. The hand position, ball trajectory, and spatial relationship are effectively combined to suppress instantaneous misjudgment, enhance the robustness of complex scenes (such as occlusion and fast pass), realize the processing of multi-person interaction scenes and complex occlusion, and improve the accuracy of the prediction of action and ball holding intention.

[0086] In one embodiment, based on real-time motion video images, the hand feature positions of the ball-holding player and the ball feature positions are extracted to obtain corresponding hand coordinate sequences and ball coordinate sequences. The hand coordinate sequences and ball coordinate sequences are then fused using an attention mechanism to obtain a human-ball relationship feature vector, including:

[0087] S201, using a convolutional neural network, detects the preset hand feature point coordinates of the ball-holding player and the center coordinates of the sphere feature in the real-time motion video image, and obtains a corresponding hand coordinate sequence and sphere coordinate sequence.

[0088] Specifically, the terminal uses a lightweight convolutional neural network, combined with preset trained hand feature points, to extract the preset hand feature point coordinates of the ball-holding player from the real-time motion video image to obtain the corresponding hand coordinate sequence; and extracts the center coordinates of the spherical features from the real-time motion video image to obtain the spherical coordinate sequence.

[0089] S202: Analyze the hand coordinate sequence and the sphere coordinate sequence using a long short-term memory network to obtain handball motion feature data.

[0090] Specifically, the terminal uses a bidirectional long short-term memory network to capture historical and future information about the hand and ball coordinate sequences, and then concatenates the two-way hidden states to generate handball motion feature data. Schematically, the hidden layer dimensions and time window length of the bidirectional long short-term memory network can be set based on the resolution and frame rate of the real-time video image.

[0091] S203, using a target detection algorithm to identify the nodes and motion trajectories of all players in the real-time motion video image, constructing a relationship graph between players, and generating a global scene feature graph.

[0092] Specifically, the terminal uses an object detection algorithm to identify all players in a real-time motion video image, detects the player's bounding box, and calculates motion vectors based on the video timing to obtain player node data. Based on this node data, edge weights are calculated to construct a player relationship graph and generate a global scene feature map. Schematically, the terminal uses DETR to detect player bounding boxes and output player node data.

[0093] In step S204, based on the handball action feature data and combined with the global scene feature map, an attention mechanism is used for embedding fusion to obtain a human-ball relationship feature vector.

[0094] Specifically, the terminal adopts the head count attention mechanism, combined with the handball action feature data, the global scene feature map, and the human-ball relationship feature vector.

[0095] This embodiment uses a convolutional neural network to accurately detect the coordinates of the key points of the hand and the center of the sphere, generating a temporal coordinate sequence of the hand and the sphere; uses a long short-term memory network to analyze the sequence, extracts the spatiotemporal dependence characteristics of the handball action, and obtains handball action feature data; uses a target detection algorithm to identify the feature points of all players in real-time motion video images, and constructs a dynamic player relationship graph based on the video time sequence; uses a multi-head attention mechanism to deeply fuse the handball action features with the global scene feature graph to generate a feature vector representing the human-ball interaction relationship, achieve accurate capture of key interaction areas, and effectively combine the action characteristics of the ball-holding player, the position characteristics of the basketball, and the position characteristics of other players to reduce the misjudgment rate in complex occlusion scenes.

[0096] In one embodiment, a convolutional neural network is used to detect the coordinates of the hand feature points of the ball-holding player and the center coordinates of the sphere feature in a real-time motion video image, and obtain corresponding hand coordinate sequences and sphere coordinate sequences, including:

[0097] S301, using a convolutional neural network, by setting multi-branch convolution, extracting multi-scale features of the real-time motion video image, and obtaining multi-branch video feature data.

[0098] Specifically, the terminal uses a multi-branch convolutional neural network to extract spatial features at different scales from real-time motion video images by setting convolution kernels of different scales, thereby obtaining multi-branch video feature data. In an illustrative example, the MobileNetV2 model can be used to extract multi-scale features. Optionally, three different convolutional layers can be set to extract features at different scales, and the step size can be controlled to control the size of the output multi-branch video feature data. The default maximum step size is set to 8.

[0099] S302: Perform hierarchical differential fusion on the multi-branch video feature data, and set a convolution layer to map the multi-branch video feature data to the heat map space to obtain a heat map of the key points of the hand.

[0100] Specifically, the terminal performs channel-by-channel differentiation and concatenation of the feature maps from different branches to obtain fused multi-branch video feature data. This fused multi-branch video feature data is then mapped to a heatmap space using a 1×1 convolution to obtain a heatmap of hand keypoints. Illustratively, a sigmoid activation function is used to map the fused multi-branch video feature data to the heatmap space. Optionally, the hand keypoint heatmap represents the confidence level of the presence of a hand keypoint at each pixel coordinate in the image.

[0101] S303: Use the following formula to perform non-maximum suppression on the heat map of the key points of the hand, extract the coordinates of the peak points, and obtain a hand coordinate sequence:

[0102]

[0103] in, is the hand coordinate of the tth frame of the real-time motion video image, W is the horizontal resolution of the real-time motion video image, is the heat map of the key points of the hand in the tth frame of the real-time motion video image, and H is the vertical resolution of the real-time motion video image.

[0104] Specifically, according to the formula, the non-maximum suppression is performed on the hand key point heat map, combined with the maximum step size of 8 to compress the motion real-time video image, and the horizontal and vertical coordinates of the point with the highest thermal confidence in the hand key point heat map are extracted to obtain the hand key point coordinates of the frame, and the hand coordinate sequence is formed with the hand key point coordinates of other frames. Schematically, the hand coordinates of the tth frame of the motion real-time video image are is the coordinate of the point with the highest confidence level of the hand key point in the heat map of the hand key point of the motion real-time video image at frame t. Optionally, the horizontal resolution W of the motion real-time video image is the horizontal resolution of the motion real-time video image input at S101. Schematically, the heat map of the hand key point of the motion real-time video image at frame t is is the heat map of the hand key points obtained in S302, indicating the confidence that the pixel at coordinate (i, j) has a hand key point. and and y represent the horizontal coordinate and vertical coordinate of the pixel point, respectively. Optionally, the vertical resolution H of the real-time motion video image is the vertical resolution of the input real-time motion video image.

[0105] S304, using color threshold segmentation technology to convert the real-time motion video image into color segmentation image data, and using the following formula to extract the sphere features in the color segmentation image data to obtain sphere feature area data:

[0106]

[0107] Among them, M(x,y) is the basketball area data, I s (x, y) is the data with coordinates (x, y) in the color segmentation image data, and n is the basketball threshold.

[0108] Specifically, the terminal converts the real-time video image of the motion from RGB format to obtain color segmentation image data in HSV format, sets the hue, saturation and brightness of the detected basketball to set the basketball threshold, extracts the sphere features from the color segmentation image data, and obtains sphere feature area data. Schematically, the basketball area data M(x,y) is the extracted basketball sphere area. Optionally, the data I with coordinates (x,y) in the color segmentation image data is s (x, y) is the data of pixel coordinates (x, y) in the color segmentation image data in the HSV format after conversion. Schematically, the basketball threshold n can be set according to the color characteristics of the basketball used.

[0109] S305: Calculate the center coordinates of the sphere feature area data using the following formula to obtain the sphere center coordinate data:

[0110]

[0111] Among them, (x b,t ,y b,t ) is the coordinate data of the center of the basketball in the tth frame of the real-time video image of the motion, and M(x, y) is the basketball area data.

[0112] Specifically, the terminal determines the center of the sphere by calculating the geometric centroid of the sphere feature area data to obtain the sphere center coordinate data. Schematically, the center coordinate data (x b,t ,y b,t ) is the geometric centroid of the sphere feature region data of the tth frame of the real-time motion video image calculated by the formula. Optionally, the basketball region data M(x,y) is the sphere feature region data obtained in S304, ∑M(x,y) is used to calculate the total number of pixels within the basketball region data, ∑x·M(x,y) is the weighted sum of the x coordinates within the basketball region data, and ∑y·M(x,y) is the weighted sum of the y coordinates within the basketball region data.

[0113] S306 , using Kalman filtering to perform trajectory smoothing and noise removal on the sphere center coordinate data to obtain a sphere coordinate sequence.

[0114] Specifically, the terminal uses Kalman filtering to smooth the trajectory and remove noise from the sphere center coordinate data, and predicts the position of the sphere during short-term occlusion to obtain a sphere coordinate sequence.

[0115] This embodiment uses a multi-branch convolutional neural network to extract multi-scale features and layered differential fusion to generate a heat map of hand key points, and uses non-maximum suppression to accurately locate the hand coordinate sequence. It also extracts sphere area data in HSV space through color threshold segmentation, calculates the sphere center coordinates using the center of mass formula, and uses Kalman filtering to smooth and suppress the trajectory of the sphere center coordinates. It outputs a stable hand and sphere coordinate sequence, achieving robust detection of hands and spheres, and providing reliable data support for the probability analysis of ball holding and passing.

[0116] In one embodiment, a target detection algorithm is used to identify the nodes and motion trajectories of all players in a real-time sports video image, construct a relationship graph between players, and generate a global scene feature graph, including:

[0117] S401, using the DETR algorithm, extracting the bounding box and player features of the players in the real-time sports video image to obtain player feature sequence data.

[0118] Specifically, the terminal uses the DETR model, which uses a CNN backbone network to extract image feature maps from real-time motion video images. Using a Transformer encoder-decoder, the feature maps are converted into player bounding boxes and feature vectors, which are then output as a sequence of player feature data. For example, the player's bounding box data includes the extreme values ​​of the two-dimensional coordinates, and the player's feature vector includes, but is not limited to, the player's speed.

[0119] S402, using the following formula, calculate the cross-frame feature similarity based on the player feature sequence data to obtain a similarity matrix:

[0120]

[0121] Among them, S kl is the similarity matrix between player k in frame t and player l in frame t-1, is the player feature data of player k in frame t, is the player feature data of player l in the t-1th frame.

[0122] Specifically, the terminal performs cross-frame trajectory matching calculation on the player feature sequence data to obtain the cross-frame player similarity matrix. Schematically, the similarity matrix S between player k in the tth frame and player l in the t-1th frame is kl It is used to measure the similarity between player k in the tth frame and player l in the t-1th frame. The higher the similarity, the greater the possibility that the two recognized players are the same person. Optionally, the player feature data of player k in the tth frame S401 obtains the player feature data of any player k in the t-th frame. Schematically, the player feature data of player l in the t-1th frame S401 obtains the player feature data of any player l in the t-1th frame.

[0123] S403 , based on the similarity matrix, the Kalman filter equation is used to update the player's motion trajectory, and the player's feature sequence data is recombined to obtain updated player feature sequence data.

[0124] Specifically, based on the similarity matrix obtained in S402, the terminal uses the Kalman filter equation and combines the similarity matrix to match players in adjacent frames, performs state integration and update on the player feature sequence data, and obtains updated player feature sequence data.

[0125] S404: Use the following formula to perform geometric relationship encoding on the updated player feature sequence data to obtain player direction detail features:

[0126] G=(D,θ)

[0127]

[0128]

[0129] Among them, G is the direction detail feature, D is the distance feature, θ is the direction angle feature, d kl is the distance between player k and player l, is the horizontal coordinate of player k, is the ordinate of player k, is the horizontal coordinate of player l, is the vertical coordinate of player l, θ kl is the angle between the directions of player k and player l, v(k) is the velocity vector of player k, v (l) is the player and l is the velocity vector.

[0130] Specifically, based on the updated player feature sequence data, the player's movement direction and speed information are calculated to obtain the player's direction detail feature. Schematically, the direction detail feature G includes the direction angle feature θ and the distance feature D, which are used to describe the cooperative or antagonistic relationship between players. Optionally, the distance feature D includes the Euclidean distance between each player, which is used to reflect the spatial proximity of each player. Schematically, the direction angle feature θ is used to measure the consistency of the movement direction. Optionally, the horizontal coordinate of player k is is the horizontal coordinate of any player k in the updated player feature sequence data obtained in S403. is the ordinate of any player k in the updated player feature sequence data obtained in S403. Alternatively, the abscissa of player l is the horizontal coordinate of any player 1 in the updated player feature sequence data obtained in S403. is the vertical coordinate of any player l in the updated player feature sequence data obtained in S403. Optionally, the angle θ between the direction of player k and player l is kl It is used to measure the consistency of the movement direction of player k and player l. Schematically, player k is the velocity vector v (k) is the velocity vector of any player k in the updated player feature sequence data obtained in S403. Optionally, player l is the velocity vector v (l) is the velocity vector in the feature vector of any player 1 in the updated player feature sequence data obtained in S403.

[0131] S405: Using the following formula, based on the self-attention weight, and according to the updated player feature sequence data and player direction detail features, a graph structure feature is constructed to obtain a global scene feature map:

[0132]

[0133] in, is the global scene feature map, is the player feature vector, ε t is the relative motion weight, v i is the player feature vector for player i, x i is the horizontal coordinate of player i, y i is the vertical coordinate of player i, is the lateral velocity of player i, is the longitudinal velocity of player i, cosθ i is the angle θ from the direction of player i i The cosine value of sinθi is the angle θ from the direction of player i i The sine value of A ij is the relative motion weight of player i and player j, d ij is the Euclidean distance between player i and player j, σ d is the distance attenuation coefficient, v j is the player feature vector for player j.

[0134] Specifically, based on the self-attention weight, according to the updated player feature sequence data and player direction detail features, the spatiotemporal relationship between players is encoded, the graph structure features are constructed, and the global scene feature map under the same video frame is obtained. Schematically, the global scene feature map Including player feature vectors and the relative motion weight ε t , used to characterize the interaction between player features and players. Optionally, the player feature vector Used to represent the set of all player states. Schematically, the relative motion weight ε t It is used to represent the set of interaction strengths between all players. Optionally, the player feature vector v of player i i is the player feature vector The player feature vector of any player i in . Schematically, the horizontal coordinate x of player i is i is the horizontal coordinate of player i in the updated player feature sequence data obtained in S403 under the video frame. i is the vertical coordinate of player i in the updated player feature sequence data obtained in S403 under the video frame. is the horizontal component of the velocity vector in any player i's feature vector in the updated player feature sequence data obtained in S403. Optionally, the longitudinal velocity of player i is the longitudinal component of the velocity vector in the feature vector of any player i in the updated player feature sequence data obtained in S403. i The cosine value cosθ i is the cosine value of the direction angle obtained in S404. Optionally, the direction angle θ of player i i The sine value of sinθ i is the cosine value of the direction angle obtained in S404. ij Used to represent the fusion distance attenuation and direction consistency. Optionally, the Euclidean distance d between player i and player j ij is the distance feature obtained in S404. Schematically, the distance attenuation coefficient σ dIt is used to control the influence range of neighboring players. It can be set according to the player hijacking situation and is set to 5 by default. Optionally, the player feature vector v of player j j is the player feature vector The player feature vector for any player j in .

[0135] This embodiment uses DETR to detect player features and bounding boxes in real time and generate a player feature sequence; combines cross-frame cosine similarity to associate player identities and uses Kalman filtering to optimize trajectory continuity; extracts speed and direction details, and constructs a global scene feature map that comprehensively represents the dynamics of the court through a spatial attention mechanism. This achieves accurate matching of player identities under complex occlusions, enhances global scene representation capabilities, and provides reliable data support for ball possession and pass probability calculations.

[0136] In one embodiment, based on the handball action feature data and combined with the global scene feature map, an attention mechanism is used for embedding fusion to obtain a human-ball relationship feature vector, including:

[0137] S501, extract the node data of the global scene feature graph, encode the node data based on the graph attention mechanism, and obtain the global encoding feature.

[0138] Specifically, the terminal extracts node features and edge weights from the global scene feature graph and calculates feature aggregation weights based on the graph attention mechanism. This aggregation combines the node features and edge weights of the nodes to create a global encoding feature. In principle, the dimensionality of the global encoding feature is determined by the number of players and the feature dimensionality.

[0139] S502: Using a multi-head attention mechanism, the handball action feature data and the global encoding features are aligned and fused to obtain a human-ball relationship feature vector.

[0140] Specifically, the terminal projects the handball action feature data and the global coding features to the same dimension, and fuses the handball action feature data and the global coding features based on the multi-head attention mechanism to obtain the human-ball relationship feature vector.

[0141] This embodiment uses this scheme to achieve collaborative analysis of hand movements and court environment through graph attention encoding of global scenes and multimodal fusion of local actions, thereby enhancing the robustness of intention recognition.

[0142] In one embodiment, based on the player-ball relationship feature vector, a classifier is used to calculate the ball holding probability and the passing probability of the ball holding player, and obtain corresponding ball holding classification results and passing classification results, including:

[0143] S601: extracting the ball position features from the human-ball relationship feature vector to obtain a ball position sequence, and calculating the ball instantaneous velocity based on the ball position sequence.

[0144] Specifically, the terminal uses the ball's position features in the human-ball relationship feature vector to derive a ball position sequence encompassing a preset time window length. Based on this ball position sequence and the preset time window length, the ball's instantaneous velocity is calculated. For example, the preset time window length can be set based on the user's desired accuracy in predicting the ball's motion. The time window length needs to balance computational load and velocity stability.

[0145] S602: Based on the instantaneous speed of the ball and a preset speed threshold, the following formula is used to perform a logistic regression calculation to obtain a ball holding classification result:

[0146]

[0147] Among them, P h is the ball holding classification result, σ is the activation function, w h is the logistic regression weight, T is the time series length, v t is the instantaneous velocity of the sphere, b h is the logistic regression bias.

[0148] Specifically, the terminal calculates the ball holding classification result based on the instantaneous speed of the ball and the preset speed threshold through logistic regression, and according to the basic principle that the higher the mean speed of the ball, the lower the probability of holding the ball. h Used to characterize the probability that the player holding the ball continues to hold the ball. Optionally, the activation function σ is a Sigmoid activation function. Schematically, the logistic regression weight w h is obtained by cross entropy loss training. Optionally, the time series length T is used to calculate the time series mean of the ball speed to characterize the ball holding stability. Schematically, the instantaneous speed of the ball v t is the instantaneous velocity of the sphere obtained in S601. Optionally, the logistic regression bias b h It is obtained through cross entropy loss training. Schematically, the preset speed threshold can be set according to the user's requirements for the passing speed.

[0149] S603: Extract the edge weight feature of the player holding the ball from the player-ball relationship feature vector to obtain ball holding edge weight feature data.

[0150] Specifically, the terminal extracts the edge weight feature of the ball-holding player from the player-ball relationship feature vector to obtain ball-holding edge weight feature data. In an exemplary embodiment, the ball-holding edge weight feature data may also include interaction relationship features of other players in the real-time video image to further predict the direction of the pass.

[0151] S604: Based on the ball possession classification result and the ball possession edge weight feature data, the pass classification result is calculated using the following formula using the fully connected layer and the Softmax classifier:

[0152]

[0153] Where s is the classification score vector of the ball-carrying player, W p is the classification weight matrix, F fusion is the fusion vector of the ball holding classification result and the ball holding node feature data, b p is the classification bias term, s1 is the passing classification score in the ball-holding player classification score vector, s2 is the holding classification score in the ball-holding player classification score vector, P p is the pass classification result.

[0154] Specifically, the terminal fuses the ball-holding classification results and the ball-holding edge weight feature data based on the fully connected layer, and calculates the classification probability of the fused data based on the Softmax classifier to obtain the pass classification result. Schematically, the ball-holding player classification score vector s is the ball-holding player classification score vector obtained by fusing the ball-holding classification results and the ball-holding edge weight feature data through the Softmax classifier. Optionally, the classification weight matrix W p is the classifier weight, which is learned through training. Schematically, the fusion vector F of the ball holding classification result and the ball holding node feature data fusion It is the fusion vector obtained by fusing the ball holding classification result and the ball holding edge weight feature data through the fully connected layer. Optionally, the classification bias term b p Is the classifier bias term, learned through training. Schematically, the pass classification score s1 in the ball-holding player classification score vector is the pass component in the ball-holding player classification score vector. Optionally, the holding classification score s2 in the ball-holding player classification score vector is the holding component in the ball-holding player classification score vector. Schematically, the pass classification result P p Used to represent the probability of the player with the ball passing the ball.

[0155] This embodiment extracts the historical position of the ball, calculates the average instantaneous speed, and obtains the ball holding probability distribution based on the logistic regression model. Combined with the edge weight characteristics of the ball-holding player, the passing probability is calculated through Softmax multi-classification, thereby improving the accuracy of identifying ball holding actions and passing intentions.

[0156] In one embodiment, based on the player-ball relationship feature vector, the ball holding classification results and the pass classification results are dynamically fused to obtain the prediction result of the ball holding player's action intention, including:

[0157] S701, extracting the node feature part of the ball-holding player from the player-ball relationship feature vector to obtain ball-holding node feature data.

[0158] Specifically, the terminal extracts the node feature portion of the ball-holding player from the player-ball relationship feature vector to obtain ball-holding node feature data. Schematically, the node feature data includes but is not limited to speed and motion direction features.

[0159] S702, extracting key points of the ball holding node feature data, detecting the confidence sequence, and obtaining the feature average confidence.

[0160] Specifically, the terminal extracts key points from the ball-holding node feature data, detects the confidence of the key points, and obtains the corresponding feature average confidence. Schematically, the feature average confidence is used to represent the probability of the feature point existing.

[0161] S703: Calculate the occlusion coefficient using the feature average confidence using the following formula:

[0162]

[0163] Where μ is the occlusion coefficient, T is the length of the time series, and c t is the feature average confidence, and τ is the confidence threshold.

[0164] Specifically, according to the formula, the occlusion coefficient is calculated by the feature average confidence. Schematically, the occlusion coefficient μ is used to characterize the coefficient of occlusion of the ball-carrying player in the video frame. The occlusion coefficient is used to balance the fusion of the ball-carrying classification results and the pass classification results. Optionally, the time series length T comes from the time window length contained in the input human-ball relationship feature vector. Schematically, the feature average confidence c t is the feature average confidence obtained in S702. Optionally, the confidence threshold τ is used to determine whether the feature average confidence meets the requirements, and can be set according to the user's judgment of the player occlusion feature.

[0165] S704: Using the following formula, the prediction result of the ball-holding player's action intention is calculated based on the ball-holding classification result, the pass classification result, and the occlusion coefficient:

[0166] P f =(1-μ)P h +μP s

[0167] Among them, P f is the prediction result of the ball player’s action intention, μ is the occlusion coefficient, P h is the ball holding classification result, P s is the pass classification result.

[0168] Specifically, the terminal occlusion coefficient is used to perform a weighted combination of the ball holding classification result and the pass classification result to obtain the prediction result of the ball holding player's action intention.f It is used to characterize the sum of the probabilities of the ball-holding player passing and holding the ball. Optionally, the occlusion coefficient μ is the occlusion coefficient obtained in S703. h is the ball holding classification result obtained in S602. Optionally, the passing classification result P s It is the pass classification result obtained in S604.

[0169] This embodiment extracts the node features of the ball-holding player from the human-ball relationship feature vector, detects the confidence sequence of its key points, calculates the average feature confidence within the time window, and generates the occlusion coefficient based on the feature average confidence and a preset threshold using a nonlinear weighted formula. The ball-holding classification results and the pass classification results are weightedly fused in combination with the occlusion coefficient to obtain the prediction result of the ball-holding player's action intention, achieves an adaptive balance between the probability of holding the ball and passing, and improves the robustness of the action intention prediction.

[0170] In the above-mentioned basketball motion detection method based on deep learning, by acquiring real-time video image data of the court, a convolutional neural network is used to extract the hand coordinate sequence of the hand feature points of the ball-holding player and the spherical coordinate sequence of the basketball, and DETR is used to construct a global scene feature map. The hand coordinate sequence, the spherical coordinate sequence and the global scene feature map are fused based on the attention mechanism to obtain a human-ball relationship feature vector. According to the human-ball relationship feature vector, the ball holding distribution result is obtained by logistic regression, and the pass distribution result is obtained based on the edge weight prediction; based on the node features of the ball-holding player, the ball holding distribution result and the pass distribution result are dynamically fused to obtain the action intention prediction result of the ball-holding player. Combining the action features of the ball-holding player, the position features of the basketball and the position features of other players, the processing of multi-person interaction scenes and complex occlusions is realized, thereby improving the accuracy of the prediction of actions and ball holding intentions.

[0171] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0172] Based on the same inventive concept, the embodiments of the present application also provide a deep learning-based basketball motion detection system for implementing the aforementioned deep learning-based basketball motion detection method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the deep learning-based basketball motion detection system can be found in the above-mentioned limitations of the deep learning-based basketball motion detection method, and will not be repeated here.

[0173] In an exemplary embodiment, Figure 2 As shown, a basketball motion detection system 200 based on deep learning is provided, comprising:

[0174] The ball holding video acquisition module 201 is used to acquire a real-time video image of the movement of the ball holding player;

[0175] The feature vector calculation module 202 is used to extract the hand feature positions of the ball-holding player and the ball feature positions based on the real-time motion video image, obtain the corresponding hand coordinate sequence and ball coordinate sequence, and fuse the hand coordinate sequence and ball coordinate sequence using the attention mechanism to obtain the human-ball relationship feature vector;

[0176] A global probability calculation module 203 is used to calculate the ball holding probability and the passing probability of the ball holding player through a classifier based on the player-ball relationship feature vector, and obtain corresponding ball holding classification results and passing classification results;

[0177] The ball holding result prediction module 204 is used to dynamically fuse the ball holding classification result and the pass classification result according to the player-ball relationship feature vector to obtain the prediction result of the ball holding player's action intention.

[0178] Furthermore, the feature vector calculation module also includes:

[0179] A coordinate sequence extraction unit is used to detect the coordinates of the hand feature points preset by the ball-holding player and the center coordinates of the sphere feature in the real-time motion video image using a convolutional neural network, and obtain corresponding hand coordinate sequences and sphere coordinate sequences;

[0180] A temporal feature extraction unit is used to analyze the hand coordinate sequence and the sphere coordinate sequence using a long short-term memory network to obtain handball action feature data;

[0181] A global feature extraction unit is used to identify the nodes and motion trajectories of all players in the real-time sports video image using a target detection algorithm, construct a relationship graph between players, and generate a global scene feature graph;

[0182] The feature data fusion unit is used to embed and fuse the handball action feature data with the global scene feature map using the attention mechanism to obtain the human-ball relationship feature vector.

[0183] Furthermore, the coordinate sequence extraction unit is further configured to:

[0184] A convolutional neural network is used to extract multi-scale features of real-time motion video images by setting multi-branch convolution to obtain multi-branch video feature data.

[0185] The multi-branch video feature data is layered and differentially fused, and a convolutional layer is set to map the multi-branch video feature data to the heat map space to obtain the heat map of the key points of the hand;

[0186] Use the following formula to perform non-maximum suppression on the hand key point heat map, extract the peak point coordinates, and obtain the hand coordinate sequence:

[0187]

[0188] in, is the hand coordinate of the tth frame of the real-time motion video image, W is the horizontal resolution of the real-time motion video image, is the heat map of the key points of the hand in the tth frame of the real-time motion video image, and H is the vertical resolution of the real-time motion video image;

[0189] The color threshold segmentation technology is used to convert the real-time motion video image into color segmentation image data. The following formula is used to extract the sphere features in the color segmentation image data to obtain the sphere feature area data:

[0190]

[0191] Among them, M(x,y) is the basketball area data, I s (x,y) is the data with coordinates (x,y) in the color segmentation image data, and n is the basketball threshold;

[0192] Use the following formula to calculate the center coordinates of the sphere feature area data and obtain the sphere center coordinate data:

[0193]

[0194] Among them, (x b,t ,y b,t ) is the coordinate data of the center of the basketball in the tth frame of the real-time video image of the sports, and M(x, y) is the basketball area data;

[0195] Kalman filtering is used to smooth the trajectory and remove noise from the sphere center coordinate data to obtain the sphere coordinate sequence.

[0196] Furthermore, the global feature extraction unit is further used to:

[0197] The DETR algorithm is used to extract the bounding boxes and player features of players in real-time sports video images to obtain player feature sequence data;

[0198] Use the following formula to calculate the cross-frame feature similarity based on the player feature sequence data to obtain the similarity matrix:

[0199]

[0200] Among them, S kl is the similarity matrix between player k in frame t and player l in frame t-1, is the player feature data of player k in frame t, is the player feature data of player l in frame t-1;

[0201] According to the similarity matrix, the Kalman filter equation is used to update the player's motion trajectory, and the player's feature sequence data is recombined to obtain the updated player's feature sequence data;

[0202] Use the following formula to perform geometric relationship encoding on the updated player feature sequence data to obtain the player direction detail features:

[0203] G=(D,θ)

[0204]

[0205] Among them, G is the direction detail feature, D is the distance feature, θ is the direction angle feature, d kl is the distance between player k and player l, is the horizontal coordinate of player k, is the ordinate of player k, is the horizontal coordinate of player l, is the vertical coordinate of player l, θ kl is the angle between the directions of player k and player l, v (k) is the velocity vector of player k, v (l) is the player l is the velocity vector;

[0206] Using the following formula, based on the self-attention weight, the graph structure features are constructed according to the updated player feature sequence data and the player direction detail features to obtain the global scene feature map:

[0207]

[0208] in, is the global scene feature map, is the player feature vector, ε t is the relative motion weight, v iis the player feature vector for player i, x i is the horizontal coordinate of player i, y i is the vertical coordinate of player i, is the lateral velocity of player i, is the longitudinal velocity of player i, cosθ i is the angle θ from the direction of player i i The cosine value of sinθ i is the angle θ from the direction of player i i The sine value of A ij is the relative motion weight of player i and player j, d ij is the Euclidean distance between player i and player j, σ d is the distance attenuation coefficient, v j is the player feature vector for player j.

[0209] Furthermore, the feature data fusion unit is also used for:

[0210] Extract node data from the global scene feature graph, encode the node data based on the graph attention mechanism, and obtain global encoding features;

[0211] A multi-head attention mechanism is used to align and fuse the handball action feature data and the global encoding features to obtain the human-ball relationship feature vector.

[0212] Furthermore, the global probability calculation module is also used to:

[0213] Extract the sphere position features from the human-sphere relationship feature vector to obtain the sphere position sequence. Based on the sphere position sequence, calculate the sphere instantaneous velocity:

[0214] Based on the instantaneous speed of the ball and the preset speed threshold, the ball holding classification result is calculated through logistic regression using the following formula:

[0215]

[0216] Among them, P h is the ball holding classification result, σ is the activation function, w h is the logistic regression weight, T is the time series length, v t is the instantaneous velocity of the sphere, b h is the logistic regression bias;

[0217] Extract the edge weight feature of the ball-holding player from the ball-holding relationship feature vector to obtain the ball-holding edge weight feature data;

[0218] According to the ball possession classification results and the ball possession edge weight feature data, based on the fully connected layer and the Softmax classifier, the pass classification result is calculated using the following formula:

[0219] s=W p F fusion +b p =[s1,s2]

[0220]

[0221] Where s is the classification score vector of the ball-carrying player, W p is the classification weight matrix, F fusion is the fusion vector of the ball holding classification result and the ball holding node feature data, b p is the classification bias term, s1 is the passing classification score in the ball-holding player classification score vector, s2 is the holding classification score in the ball-holding player classification score vector, P p is the pass classification result.

[0222] Furthermore, the ball possession result prediction module is also used to:

[0223] Extract the node feature part of the ball-holding player from the human-ball relationship feature vector to obtain the ball-holding node feature data;

[0224] Extract the key points of the ball holding node feature data, detect the confidence sequence, and obtain the average feature confidence;

[0225] The occlusion coefficient is calculated from the feature average confidence using the following formula:

[0226]

[0227] Where μ is the occlusion coefficient, T is the length of the time series, and c t is the average confidence of the feature, τ is the confidence threshold;

[0228] The following formula is used to calculate the prediction result of the ball-holding player's action intention based on the ball-holding classification result, the pass classification result, and the occlusion coefficient:

[0229] P f =(1-μ)P h +μP s

[0230] Among them, P f is the prediction result of the ball player’s action intention, μ is the occlusion coefficient, P h is the ball holding classification result, P s is the pass classification result.

[0231] In one embodiment, Figure 3 A computer device is provided, comprising:

[0232] At least one processor 301, and a memory 302 communicatively connected to the at least one processor 301: the memory stores application code that can be executed by the at least one processor, and the application code is executed by the at least one processor to enable the at least one processor to execute the basketball motion detection method based on deep learning as described above.

[0233] The computer device may further include: sensor 303.

[0234] The processor 301 , the memory 302 and the sensor 303 may be connected via a bus or other means, with the bus being used as an example in the figure.

[0235] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0236] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0237] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A basketball motion detection method based on deep learning, characterized in that: The method comprises: Acquire real-time video images of the player holding the ball; Extracting the hand feature positions of the ball-holding player and the ball feature positions from the real-time motion video image to obtain corresponding hand coordinate sequences and ball coordinate sequences, and fusing the hand coordinate sequences and the ball coordinate sequences using an attention mechanism to obtain a human-ball relationship feature vector; According to the player-ball relationship feature vector, a classifier is used to calculate the ball holding probability and the passing probability of the ball holding player, and obtain corresponding ball holding classification results and passing classification results; According to the human-ball relationship feature vector, the ball holding classification result and the pass classification result are dynamically fused to obtain a prediction result of the ball holding player's action intention.

2. The method according to claim 1, characterized in that The method further comprises extracting the hand feature positions and the sphere feature positions of the ball-holding player according to the real-time motion video image, obtaining a corresponding hand coordinate sequence and a sphere coordinate sequence, and fusing the hand coordinate sequence and the sphere coordinate sequence using an attention mechanism to obtain a human-ball relationship feature vector. Using a convolutional neural network, the coordinates of the hand feature points preset by the player holding the ball and the center coordinates of the sphere feature in the real-time motion video image are detected to obtain a corresponding hand coordinate sequence and sphere coordinate sequence; Analyzing the hand coordinate sequence and the sphere coordinate sequence using a long short-term memory network to obtain handball motion feature data; Using a target detection algorithm to identify the nodes and motion trajectories of all players in the real-time sports video image, constructing a relationship graph between players, and generating a global scene feature graph; According to the handball action feature data and combined with the global scene feature map, an attention mechanism is used for embedding fusion to obtain a human-ball relationship feature vector.

3. The method according to claim 2, characterized in that The convolutional neural network is used to detect the preset hand feature point coordinates of the ball-holding player and the center coordinates of the sphere feature in the real-time motion video image to obtain the corresponding hand coordinate sequence and sphere coordinate sequence, including: A convolutional neural network is used to extract multi-scale features of the real-time motion video image by setting a multi-branch convolution to obtain multi-branch video feature data; Performing hierarchical differential fusion on the multi-branch video feature data, and setting a convolution layer to map the multi-branch video feature data to a heat map space to obtain a heat map of hand key points; Use the following formula to perform non-maximum suppression on the hand key point heat map, extract the peak point coordinates, and obtain the hand coordinate sequence: in, is the hand coordinate of the tth frame of the real-time motion video image, W is the horizontal resolution of the real-time motion video image, is the heat map of the key points of the hand in the tth frame of the real-time motion video image, and H is the vertical resolution of the real-time motion video image; The color threshold segmentation technology is used to convert the real-time motion video image into color segmentation image data, and the sphere features in the color segmentation image data are extracted using the following formula to obtain sphere feature area data: Among them, M(x,y) is the basketball area data, I s (x,y) is the data with coordinates (x,y) in the color segmentation image data, and n is the basketball threshold; The center coordinates of the sphere feature area data are calculated using the following formula to obtain the sphere center coordinate data: Among them, (x b,t ,y b,t ) is the coordinate data of the center of the basketball in the tth frame of the real-time video image of the sports, and M(x, y) is the basketball area data; Kalman filtering is used to perform trajectory smoothing and noise removal on the spherical center coordinate data to obtain the spherical coordinate sequence.

4. The method according to claim 2, characterized in that The target detection algorithm is used to identify the nodes and motion trajectories of all players in the real-time sports video image, construct a relationship graph between players, and generate a global scene feature graph, including: Using a DETR algorithm, extracting bounding boxes and player features of players in the real-time sports video image to obtain player feature sequence data; The following formula is used to calculate the cross-frame feature similarity based on the player feature sequence data to obtain a similarity matrix: Among them, S kl is the similarity matrix between player k in frame t and player l in frame t-1, is the player feature data of player k in frame t, is the player feature data of player l in frame t-1; updating the player's motion trajectory using a Kalman filter equation according to the similarity matrix, and recombining the player's feature sequence data to obtain updated player's feature sequence data; Use the following formula to perform geometric relationship encoding on the updated player feature sequence data to obtain player direction detail features: G=(D,θ) Among them, G is the direction detail feature, D is the distance feature, θ is the direction angle feature, d kl is the distance between player k and player l, is the horizontal coordinate of player k, is the ordinate of player k, is the horizontal coordinate of player l, is the vertical coordinate of player l, θ kl is the angle between the directions of player k and player l, v (k) is the velocity vector of player k, v (l) is the player l is the velocity vector; The following formula is used to construct graph structure features based on the self-attention weight, the updated player feature sequence data and the player direction detail features, and a global scene feature map is obtained: in, is the global scene feature map, is the player feature vector, ε t is the relative motion weight, v i is the player feature vector for player i, x i is the horizontal coordinate of player i, y i is the vertical coordinate of player i, is the lateral velocity of player i, is the longitudinal velocity of player i, cosθ i is the angle θ from the direction of player i i The cosine value of sinθ i is the angle θ from the direction of player i i The sine value of A ij is the relative motion weight of player i and player j, d ij is the Euclidean distance between player i and player j, σ d is the distance attenuation coefficient, v j is the player feature vector for player j.

5. The method according to claim 2, characterized in that The method of embedding and fusing the handball action feature data and the global scene feature map using an attention mechanism to obtain a human-ball relationship feature vector includes: Extracting node data of the global scene feature graph, encoding the node data based on a graph attention mechanism to obtain a global encoding feature; A multi-head attention mechanism is used to align and fuse the handball action feature data and the global encoding features to obtain the human-ball relationship feature vector.

6. The method according to claim 1, wherein The method of calculating the holding probability and passing probability of the ball-holding player by a classifier based on the player-ball relationship feature vector to obtain corresponding holding classification results and passing classification results includes: The ball position features in the human-ball relationship feature vector are extracted to obtain a ball position sequence. Based on the ball position sequence, the instantaneous speed of the ball is calculated: According to the instantaneous speed of the ball and the preset speed threshold, the ball holding classification result is obtained by logistic regression calculation using the following formula: Among them, P h is the ball holding classification result, σ is the activation function, w h is the logistic regression weight, T is the time series length, v t is the instantaneous velocity of the sphere, b h is the logistic regression bias; Extracting edge weight features of the ball-holding player from the player-ball relationship feature vector to obtain ball-holding edge weight feature data; According to the ball holding classification result and the ball holding edge weight feature data, based on the fully connected layer and the Softmax classifier, the pass classification result is calculated using the following formula: s=W p Fusion+b p =[s1,s2] Where s is the classification score vector of the ball-carrying player, W p is the classification weight matrix, F fusion is the fusion vector of the ball holding classification result and the ball holding node feature data, b p is the classification bias term, s1 is the passing classification score in the ball-holding player classification score vector, s2 is the holding classification score in the ball-holding player classification score vector, P p is the pass classification result.

7. The method according to claim 1, characterized in that The method of dynamically fusing the ball holding classification result and the pass classification result based on the player-ball relationship feature vector to obtain a prediction result of the ball holding player's action intention includes: Extracting the node feature portion of the ball-holding player from the player-ball relationship feature vector to obtain ball-holding node feature data; Extracting key points of the ball-holding node feature data, detecting a confidence sequence, and obtaining an average feature confidence; The occlusion coefficient is calculated by the average confidence of the features using the following formula: Where μ is the occlusion coefficient, T is the length of the time series, and c t is the average confidence of the feature, τ is the confidence threshold; The following formula is used to calculate the prediction result of the ball-holding player's action intention based on the ball-holding classification result, the pass classification result, and the occlusion coefficient: P f =(1-μ)P h +μP s Among them, P f is the prediction result of the ball player’s action intention, μ is the occlusion coefficient, P h is the ball holding classification result, P s is the pass classification result.

8. A basketball motion detection system based on deep learning, characterized in that: The system comprises: The ball holding video acquisition module is used to obtain real-time video images of the ball holding player; a feature vector calculation module, configured to extract the hand feature positions of the ball-holding player and the ball feature positions based on the real-time motion video image, obtain corresponding hand coordinate sequences and ball coordinate sequences, and fuse the hand coordinate sequences and the ball coordinate sequences using an attention mechanism to obtain a human-ball relationship feature vector; A global probability calculation module is used to calculate the ball holding probability and the passing probability of the ball holding player through a classifier based on the ball-handling relationship feature vector, and obtain corresponding ball holding classification results and passing classification results; The ball holding result prediction module is used to dynamically fuse the ball holding classification result and the pass classification result according to the human-ball relationship feature vector to obtain a prediction result of the ball holding player's action intention.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images