Pick ball athlete action analysis method based on computer vision

Abstract

The invention discloses a Pick ball athlete action analysis method based on computer vision, and belongs to the technical field of data processing. Standard action videos corresponding to action types of Pick ball athletes are collected by arranging a camera, and the standard action videos are analyzed and learned by utilizing a computer vision technology; according to the technical scheme, the motion analysis method and the motion analysis device, the model capable of analyzing the motion of the Pick ball athlete is formed, then the to-be-analyzed video data of the to-be-analyzed Pick ball athlete is collected, the to-be-analyzed video data is analyzed through the obtained model, and finally the motion analysis result is obtained, so that the normalization of the technical motion can be effectively evaluated; scientific training suggestions are provided for athletes, and athletic performance is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and in particular relates to a computer vision method for analyzing the movements of pickleball players. Background Art

[0002] Pickleball is an indoor or outdoor racket / racquet sport in which two players (singles) or four players (doubles) use solid rackets to hit a perforated hollow polymer ball on a 36-inch high (0.91 meter) net. Opponents on both sides of the net hit the ball back and forth until one side commits a foul. In the prior art, the coach often analyzes and guides students' pickleball movements with the naked eye, resulting in inconsistent guidance from different coaches and prone to errors. In addition, there is a general problem of insufficient professional teaching staff and backward teaching methods in the current pickleball teaching in my country, resulting in students not being able to receive correct guidance and pickleball courses not being able to achieve the desired teaching results. Summary of the invention

[0003] The present invention provides a computer vision method for analyzing the movements of pickleball players, which is used to solve the problem in the prior art that trainees cannot receive correct and standardized guidance.

[0004] A computer vision method for analyzing pickleball player movements, comprising: For any action type of a pickleball player, a standard action video corresponding to the action type is collected, and frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to each action type; Constructing a portrait positioning detection model and a human body posture estimation model, and training the portrait positioning detection model and the human body posture estimation model to obtain the trained portrait positioning detection model and the human body posture estimation model; For any action type of a pickleball player, the trained portrait positioning detection model is used to detect the standard action frame set corresponding to the action type, and the portrait area corresponding to the standard action frame corresponding to each action type is obtained; For the portrait area corresponding to the standard action frame corresponding to any action type, the trained human posture estimation model is used to detect the portrait area to obtain the human posture key point data corresponding to the standard action frame corresponding to each action type; Reconstruct the coordinates of the key point data of the human body posture corresponding to the standard action frame corresponding to each action type, and determine the coordinate value corresponding to each key point data of the human body posture; According to the coordinate values ​​corresponding to each human posture key point data, determine the action start frame, action end frame and common frame between the action start frame and the action end frame corresponding to each action type; For any action type, a motion trajectory data matrix is ​​constructed according to the action start frame, action end frame and common frame, and the action trajectory data matrix is ​​used to train the action analysis model to obtain the trained action analysis model; The video data of the pickleball player to be analyzed is obtained, and the trained portrait positioning detection model, the trained human posture estimation model and the trained motion analysis model are scheduled to perform motion analysis of the pickleball player and determine the motion analysis result.

[0005] Furthermore, for any action type of a pickleball player, a standard action video corresponding to the action type is collected, and frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to each action type, including: The camera is placed at the boundary of the pickleball net with the central axis of the camera lens in a horizontal state, and the camera is rotated counterclockwise until the angle between the central axis of the camera lens and the pickleball net is 45 degrees, and the standard action video corresponding to each action type of the pickleball player is collected; wherein the number of standard action videos corresponding to each action type is at least one; For any standard action video, based on a preset data sampling frequency, frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to the standard action video corresponding to each action type.

[0006] Furthermore, a portrait positioning detection model and a human body posture estimation model are constructed, and after training the portrait positioning detection model and the human body posture estimation model, the trained portrait positioning detection model and the human body posture estimation model are obtained, including: The YOLOv3 neural network is used to build a portrait positioning detection model, and the ResNet50 is used to build a human posture estimation model; Use a portrait positioning detection model or a human body posture estimation model as a model to be trained; For the hyperparameters of the model to be trained, the random initialization method is used to randomly initialize the hyperparameters, and the initialized hyperparameters are encoded into vectors to obtain parameter vectors, and multiple different parameter vectors are obtained; For any parameter vector, a neighborhood search is performed on the parameter vector using the cosine pairing collaborative search method to obtain the parameter vector after the neighborhood search; For any parameter vector after neighborhood search, a gravitational search method is used to perform a secondary neighborhood search on the parameter vector to obtain the parameter vector after the secondary neighborhood search; For any parameter vector after the secondary neighborhood search, an adaptive random search method is used to perform a global search on the parameter vector to obtain the parameter vector after the global search; Determine whether the current number of training times is greater than or equal to the maximum number of training times. If so, determine the final hyperparameters of the model to be trained based on the parameter vector after the global search, and obtain the model to be trained after training. Otherwise, return to the neighborhood search step once.

[0007] Furthermore, for any action type of a pickleball player, the trained portrait positioning detection model is used to detect the standard action frame set corresponding to the action type, and the portrait area corresponding to the standard action frame corresponding to each action type is obtained, including: For any action type of a pickleball player, a standard action frame in any standard action frame set corresponding to the action type is input into the trained portrait positioning detection model, and the area where the portrait detection frame is located is cropped to obtain the portrait area corresponding to the standard action frame corresponding to each action type.

[0008] Furthermore, the human body posture key point data corresponding to the standard action frame include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0009] Furthermore, coordinate reconstruction is performed on the human body posture key point data corresponding to the standard action frame corresponding to each action type to determine the coordinate value corresponding to each human body posture key point data, including: Determine the left shoulder and the right shoulder according to the human body posture key point data corresponding to the standard action frame corresponding to each action type; The center point of the left shoulder and the right shoulder is used as the origin to construct a coordinate system, and based on the coordinate system, the coordinate value corresponding to each key point data of the human body posture is determined.

[0010] Further, according to the coordinate values ​​corresponding to each human posture key point data, the action start frame, the action end frame and the common frame between the action start frame and the action end frame corresponding to each action type are determined, including: For any action type, determine the action type start judgment condition and the action type end condition; For any standard action frame, determine the key angle of the action according to the coordinate value corresponding to each key point data of the human body posture; According to the action key angle, the action type start determination condition and the action type end condition, the action start frame, the action end frame and the common frame between the action start frame and the action end frame corresponding to each action type are determined.

[0011] Furthermore, for any action type, an action trajectory data matrix is ​​constructed according to the action start frame, the action end frame and the common frame, and the action trajectory data matrix is ​​used to train the action analysis model to obtain the trained action analysis model, including: For any action type, the key point data of the human body posture of the action start frame, action end frame and ordinary frame are constructed into a matrix to obtain the trajectory data matrix; A motion analysis model is constructed, and the trajectory data matrix is ​​used as the input of the motion analysis model. The motion type corresponding to the trajectory data matrix is ​​used as the expected output of the motion analysis model. The motion analysis model is trained to obtain the trained motion analysis model.

[0012] Further, the video data of the pickleball player to be analyzed is obtained, and the trained portrait positioning detection model, the trained human posture estimation model and the trained motion analysis model are scheduled to perform motion analysis of the pickleball player, and the motion analysis results are determined, including: Obtain the video data to be analyzed of the pickleball player to be analyzed, and extract the video frames to be analyzed one by one according to time; The trained portrait positioning detection model is scheduled to analyze the video frames to be analyzed, and the portrait area corresponding to each video frame to be analyzed is determined; The motion analysis model after scheduling training analyzes the portrait area corresponding to the video frame to be analyzed, and determines the key point data of the human body posture corresponding to the video frame to be analyzed; According to the action type start judgment condition and the key data of human posture corresponding to the video frame to be analyzed, determine the start frame to be analyzed, the end frame to be analyzed, and the common frame to be analyzed between the start frame to be analyzed and the end frame to be analyzed; According to the start frame to be analyzed, the end frame to be analyzed, and the common frame structure to be analyzed between the start frame to be analyzed and the end frame to be analyzed, a trajectory data matrix to be analyzed is obtained; The motion analysis model after the scheduling training performs pickleball player motion analysis on the trajectory data matrix to be analyzed and determines the motion analysis results.

[0013] Furthermore, the method also includes: determining the standard degree of the pickleball player's movements according to the movement analysis results.

[0014] The present invention provides a computer vision method for analyzing the actions of pickleball players. A camera is set to collect standard action videos corresponding to the action types of pickleball players, and computer vision technology is used to analyze and learn the standard action videos, thereby forming a model that can analyze the actions of pickleball players. Then, video data to be analyzed of the pickleball players to be analyzed is collected, and the video data to be analyzed is analyzed by the obtained model, and finally the action analysis results are obtained, which can effectively evaluate the standardization of technical actions, provide scientific training suggestions for athletes, and improve sports performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 A flowchart of a method for analyzing pickleball player movements using computer vision provided by an embodiment of the present invention.

[0017] Figure 2 A schematic diagram of camera shooting angles provided in an embodiment of the present invention.

[0018] Figure 3 A flow chart of the training steps for a portrait positioning detection model and a human body posture estimation model provided in an embodiment of the present invention.

[0019] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0021] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0022] like Figure 1 As shown, an embodiment of the present invention provides a computer vision method for analyzing the motion of a pickleball player, comprising: S101, for any action type of a pickleball player, collecting a standard action video corresponding to the action type, and performing frame extraction processing on the standard action video to obtain a standard action frame set corresponding to each action type; The action types of pickleball players can be: serve, forehand, backhand, high-pressure ball, slice, and net volley. Then for each action type, professional athletes can practice, and then collect the standard action video corresponding to each action type. Since the standard action video is long, the factor can first extract the frame of the standard action video to reduce the amount of data recognition and improve the data recognition ability and speed while ensuring correct data learning.

[0023] S102, constructing a portrait positioning detection model and a human body posture estimation model, and training the portrait positioning detection model and the human body posture estimation model to obtain the trained portrait positioning detection model and the human body posture estimation model; The YOLOv3 neural network can be used to build a portrait positioning detection model and the ResNet50 can be used to build a human posture estimation model. Then, the intelligent optimization algorithm can be used to train the portrait positioning detection model and the human posture estimation model, so as to realize portrait positioning and recognition of key point data of human posture.

[0024] S103, for any action type of a pickleball player, using the trained portrait positioning detection model to detect a standard action frame set corresponding to the action type, and obtaining a portrait area corresponding to the standard action frame corresponding to each action type; The trained portrait positioning detection model has the ability to recognize portraits. After the standard action frame is input into the trained portrait positioning detection model, the prediction frame of the portrait area can be obtained. According to the prediction frame of the portrait area, the standard action frame can be cropped and scaled to obtain a portrait area of ​​uniform size.

[0025] S104, for a portrait area corresponding to a standard action frame corresponding to any action type, using a trained human posture estimation model to detect the portrait area, and obtain human posture key point data corresponding to the standard action frame corresponding to each action type; Similarly, the trained human posture estimation model has the ability to detect key point data of human posture. The portrait area can be input into the trained human posture estimation model to obtain the key point data of human posture.

[0026] S105, reconstructing the coordinates of the human body posture key point data corresponding to the standard action frame corresponding to each action type, and determining the coordinate value corresponding to each human body posture key point data; By reading and studying the pickleball key point data set, the confidence of the left shoulder and the right shoulder in the data set is often the highest, so the midpoint of the left shoulder and the right shoulder is selected as the origin of the new coordinate system.

[0027] S106, determining the action start frame, action end frame, and common frame between the action start frame and the action end frame corresponding to each action type according to the coordinate values ​​corresponding to each human posture key point data; The relative positions of the key points of human body posture are all different for different action types. Therefore, based on this, the action start frame, action end frame and the distance between the action start frame and the action end frame corresponding to each action type can be determined.

[0028] S107, for any action type, constructing an action trajectory data matrix according to the action start frame, the action end frame and the common frame, and using the action trajectory data matrix to train an action analysis model to obtain a trained action analysis model; For the action start frame, action end frame and ordinary frame corresponding to each action type, the human body posture key point data corresponding to any frame data is taken as a row of data to construct a motion trajectory data matrix. Then, the motion analysis model can be trained according to the motion trajectory data matrix to obtain the trained motion analysis model.

[0029] Optionally, after the motion trajectory data matrix is ​​normalized, it is input into the motion analysis model, thereby reducing the complexity of the data and improving the accuracy of data recognition.

[0030] S108, obtaining the video data of the pickleball player to be analyzed, and scheduling the trained portrait positioning detection model, the trained human posture estimation model and the trained motion analysis model to perform motion analysis of the pickleball player and determine the motion analysis result.

[0031] The present invention provides a computer vision method for analyzing the actions of pickleball players. A camera is set to collect standard action videos corresponding to the action types of pickleball players, and computer vision technology is used to analyze and learn the standard action videos, thereby forming a model that can analyze the actions of pickleball players. Then, video data to be analyzed of the pickleball players to be analyzed is collected, and the video data to be analyzed is analyzed by the obtained model, and finally the action analysis results are obtained, which can effectively evaluate the standardization of technical actions, provide scientific training suggestions for athletes, and improve sports performance.

[0032] In the embodiment of the present invention, for any action type of a pickleball player, a standard action video corresponding to the action type is collected, and frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to each action type, including: The camera is placed at the boundary of the pickleball net with the central axis of the camera lens in a horizontal state, and the camera is rotated counterclockwise until the angle between the central axis of the camera lens and the pickleball net is 45 degrees, and the standard action video corresponding to each action type of the pickleball player is collected; wherein the number of standard action videos corresponding to each action type is at least one; For any standard action video, based on a preset data sampling frequency, frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to the standard action video corresponding to each action type.

[0033] like Figure 2 As shown, by setting the camera acquisition angle in this way, the frontal image of the pickleball player during the movement can be collected as much as possible, thereby improving data recognition efficiency.

[0034] like Figure 3 As shown, a portrait positioning detection model and a human body posture estimation model are constructed, and after the portrait positioning detection model and the human body posture estimation model are trained, the trained portrait positioning detection model and the human body posture estimation model are obtained, including: S301, use YOLOv3 neural network to build a portrait positioning detection model and use ResNet50 to build a human posture estimation model; It is worth noting that the above two models are only preferred methods of the embodiments of the present invention, and other models can also be used to construct a portrait positioning detection model and a human body posture estimation model.

[0035] S302, using a portrait positioning detection model or a human body posture estimation model as a model to be trained; S303, for the hyper parameters of the model to be trained, randomly initialize the hyper parameters using a random initialization method, encode the initialized hyper parameters into a vector, obtain a parameter vector, and obtain multiple different parameter vectors; For example, when training a YOLOv3 neural network, the connection weights between network layers can be used as the basis, random numbers can be initialized between the upper and lower limits of the connection weights, and then encoded into a vector, so that a parameter vector can be obtained.

[0036] It is worth noting that, during the training process of the embodiment of the present invention, some hyperparameters may be trained, or all hyperparameters may be trained. When some hyperparameters are trained, conventional methods may be used to train the remaining hyperparameters.

[0037] During the training process, sample data corresponding to the model to be trained is required for training, so that the model to be trained can have relevant functions after training.

[0038] S304, for any parameter vector, use the cosine pairing collaborative search method to perform a neighborhood search on the parameter vector to obtain a parameter vector after the neighborhood search; The cosine pairing collaborative search method is used to perform a neighborhood search on the parameter vector, and the parameter vector after a neighborhood search is obtained as follows:

[0039] in, Indicates t During the training n The parameter vector corresponds to d Dimension parameters, n =1,2,…,N, where N represents the total number of parameter vectors, d =1,2,…,D, D represents the total dimension of the parameter vector, T represents the maximum number of training times, Indicates n The parameter vector after a neighborhood search d Dimension parameters, represents pi, cos represents the cosine function, represents the first random number between (0,1), represents the second random number between (0,1), Indicates t During the training n The parameter vector corresponding to the collaboration parameter vector d dimensional parameters, each parameter vector is different from its corresponding collaborative parameter vector.

[0040] The cosine pairing collaborative search method provided in the embodiment of the present invention can effectively realize the search of the area between two parameter vectors, and adopts the cosine function to change the search step size and the search direction, which can effectively improve the search efficiency of the solution space.

[0041] S305, for any parameter vector after the neighborhood search, a gravitational search method is used to perform a secondary neighborhood search on the parameter vector to obtain a parameter vector after the secondary neighborhood search; The gravitational search method is used to perform a secondary neighborhood search on the parameter vector, and the parameter vector after the secondary neighborhood search is obtained as follows:

[0042]

[0043]

[0044]

[0045] in, Indicates t During the training i The parameter vector after the neighborhood search d Dimension parameters, i =1,2,…,N, where N represents the total number of parameter vectors, Indicates t During the training k The parameter vector after the neighborhood search d Dimension parameters, k =1,2,…,N, Indicates i The parameter vector after the second neighborhood search d Dimension parameters, represents the optimal parameter vector d dimensional parameters, the optimal parameter vector is the parameter vector with the smallest loss function value, represents the gravitational search coefficient, represents the upper limit of the gravitational search coefficient, represents the lower limit of the gravitational search coefficient, T represents the maximum number of training times, Indicates d The upper limit of the dimension parameter, Indicates d The lower limit of the dimension parameter, Indicates k The parameter vector after the neighborhood search d Dimension parameter i The parameter vector after the neighborhood search d The influencing parameter factor of the dimension parameter, Indicates k The parameter vector after the neighborhood search d Dimension parameter i The parameter vector after the neighborhood search d The gravitational strength of the dimensional parameter, Indicates i The parameter vector after the neighborhood search is k The Euclidean distance between the parameter vectors after the neighborhood search, Indicates the gravitational strength parameter, and is set to 0.5; represents a natural constant, represents the gravitational scale parameter and is set to 1.5.

[0046] The embodiment of the present invention adopts a gravitational search method, which can effectively enable the parameter vector to learn the information of other parameter vectors and search around the optimal parameter vector. While ensuring the efficiency of the algorithm, it can improve the search in unfamiliar areas and greatly avoid repeated searches. The early algorithm should conduct a large-scale search for the entire search domain, and the later local exploration based on the optimal individual can also avoid falling into the local optimal value to a certain extent.

[0047] S306, for any parameter vector after the secondary neighborhood search, use an adaptive random search method to perform a global search on the parameter vector to obtain a parameter vector after the global search; The parameter vector is searched globally using the adaptive random search method, and the parameter vector after the global search is:

[0048]

[0049] in, Indicates t During the training j The parameter vector after the quadratic neighborhood search, Indicates j The parameter vector after a global search, represents the global search coefficient, represents the third random number between (0,1), represents the fourth random number between (0,1), represents the fifth random number between (0,1), Expressed as j The parameter vector after the second neighborhood search randomly matches the first random vector d Dimension parameters, Expressed as j The parameter vector after the second neighborhood search randomly matches the second random vector d Dimension parameters; represents the sixth random number between (0,1), Represents the sine function.

[0050] The adaptive random search method provided by the embodiment of the present invention can enable the algorithm to have a larger search range in the early stage, greatly improving the global search capability, and gradually reduce the differential evolution capability in the later stage of the algorithm to improve the convergence accuracy of the algorithm. It is worth noting that in order to improve the search speed of the algorithm, a greedy strategy or an annealing simulation algorithm can also be used to control the global search process.

[0051] S307. Determine whether the current number of training times is greater than or equal to the maximum number of training times. If so, determine the final hyperparameters of the model to be trained based on the parameter vector after the global search, and obtain the model to be trained after training. Otherwise, return to the neighborhood search step once.

[0052] In the prior art, a gradient descent intelligent optimization algorithm is often used to train hyperparameters, which is prone to falling into local optimality, resulting in the inability of the trained model to effectively perform recognition tasks. Therefore, an embodiment of the present invention provides a new intelligent optimization algorithm to improve the model's recognition effect on data, and ultimately improve the accuracy of pickleball player motion analysis.

[0053] In the embodiment of the present invention, for any action type of a pickleball player, the trained portrait positioning detection model is used to detect the standard action frame set corresponding to the action type, and the portrait area corresponding to the standard action frame corresponding to each action type is obtained, including: For any action type of a pickleball player, a standard action frame in any standard action frame set corresponding to the action type is input into the trained portrait positioning detection model, and the area where the portrait detection frame is located is cropped to obtain the portrait area corresponding to the standard action frame corresponding to each action type.

[0054] In an embodiment of the present invention, the human body posture key point data corresponding to the standard action frame include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0055] It is worth noting that the above human body posture key point data is only a preferred example of the embodiment of the present invention, and other key points can also be used for recognition.

[0056] In an embodiment of the present invention, coordinate reconstruction is performed on the human body posture key point data corresponding to the standard action frame corresponding to each action type, and the coordinate value corresponding to each human body posture key point data is determined, including: Determine the left shoulder and the right shoulder according to the human body posture key point data corresponding to the standard action frame corresponding to each action type; The center point of the left shoulder and the right shoulder is used as the origin to construct a coordinate system, and based on the coordinate system, the coordinate value corresponding to each key point data of the human body posture is determined.

[0057] In an embodiment of the present invention, according to the coordinate values ​​corresponding to each human posture key point data, determining the action start frame, action end frame and common frame between the action start frame and the action end frame corresponding to each action type includes: For any action type, determine the action type start judgment condition and the action type end condition; For example, when the action category is serving, the action type start determination condition may include the right wrist-right elbow-right shoulder joint vertical angle of 170°±10° and the left wrist-left elbow-left shoulder joint vertical angle of 170°±10°; the action type end condition may include the right elbow-right shoulder-right hip joint angle of 150°~170° and the right shoulder-right elbow-right wrist joint angle of 170°±10°; When the action type is forehand hitting, the action type start determination condition may include the vector from the left shoulder to the right shoulder being 20°~50° and the right wrist-right shoulder-right hip joint angle being 40°~150°; the action type end condition may include the right wrist-right elbow-right shoulder joint angle being 100°~170°; When the action category is backhand stroke, the action type start judgment conditions may include the vector from left shoulder to right shoulder being 40°~100° and the right shoulder-right hip-right knee / left knee angle being 100°~160°; the action type end conditions may include the right wrist-right elbow-right shoulder joint angle being 170°±10°, the left wrist-left elbow-left shoulder joint angle being 170°±10° and the right elbow-right shoulder-right hip key point angle being 30°~80°.

[0058] When the action category is high pressure ball, the action type start determination conditions may include the right wrist-right elbow-right shoulder joint angle of 30°~120°, the right wrist is higher than the right shoulder, the left wrist-left elbow-left shoulder joint angle of 170°±10°, and the left elbow-left shoulder-left hip joint angle of 170°±10°; the action type end condition may include the right elbow-right shoulder-right hip joint angle of 170°±10°; When the action category is slicing, the action type start determination conditions may include the right wrist-right elbow-right shoulder joint angle being 120°~170°, the left wrist-left elbow-left shoulder joint angle being 110°~160°, the right shoulder being higher than the right elbow and higher than the right wrist, and the left shoulder being higher than the left elbow and higher than the left wrist; the action type end conditions may include the right wrist-right elbow-right shoulder joint angle being 170°±10°, the left wrist-left elbow-left shoulder joint angle being 170°±10°, and the right wrist-right elbow-right hip joint angle being 40°~70°.

[0059] When the action category is net volley, the action type start determination conditions may include the right wrist-right elbow-right shoulder joint angle being 100°~140°, the left wrist-left elbow-left shoulder joint angle being 100°~140°, the right shoulder being higher than the right elbow and higher than the right wrist, and the left shoulder being higher than the left elbow and higher than the left wrist; the action type end conditions may include the right elbow-right shoulder-right hip joint angle being 30°~70°.

[0060] However, it is worth noting that the above-mentioned action type start determination condition and action type end condition are merely used as the distance in the embodiment of the present invention, and other determination conditions may also be used for determination.

[0061] For any standard action frame, determine the key angle of the action according to the coordinate value corresponding to each key point data of the human body posture; According to the action key angle, the action type start determination condition and the action type end condition, the action start frame, the action end frame and the common frame between the action start frame and the action end frame corresponding to each action type are determined.

[0062] For example, based on the sequence of standard action frames, when a frame meets the action type start judgment condition, it can be determined as the action start frame, and then judged in sequence. When another frame meets the action type end condition, the action end frame can be determined, and the standard action frames between the action start frame and the action end frame are determined as ordinary frames.

[0063] In an embodiment of the present invention, for any action type, an action trajectory data matrix is ​​constructed according to the action start frame, the action end frame and the common frame, and the action analysis model is trained using the action trajectory data matrix to obtain the trained action analysis model, including: For any action type, the key point data of the human body posture of the action start frame, action end frame and ordinary frame are constructed into a matrix to obtain the trajectory data matrix; A motion analysis model is constructed, and the trajectory data matrix is ​​used as the input of the motion analysis model. The motion type corresponding to the trajectory data matrix is ​​used as the expected output of the motion analysis model. The motion analysis model is trained to obtain the trained motion analysis model.

[0064] The motion analysis model can be set to ST-GCN (space-time graph convolutional network), AGCN (adaptive graph convolutional neural network) or CNN (convolutional neural network). The motion analysis model is trained through the trajectory data matrix corresponding to each action type, so that the motion analysis model has recognition ability and can perform motion analysis.

[0065] It is worth noting that, although multiple standard action videos can be collected for the action type, during the training process, a standard action video should be used to obtain a trajectory data matrix for training. The action analysis model can be trained using the new intelligent optimization algorithm provided by the embodiment of the present invention, or the conventional intelligent optimization algorithm can be used to train the action analysis model. However, in order to improve the recognition effect, the embodiment of the present invention preferably uses the new intelligent optimization algorithm provided by the embodiment of the present invention to train the action analysis model.

[0066] In an embodiment of the present invention, video data to be analyzed of a pickleball player to be analyzed is obtained, and a trained portrait positioning detection model, a trained human posture estimation model, and a trained motion analysis model are scheduled to perform motion analysis of the pickleball player, and determine the motion analysis result, including: Obtain the video data to be analyzed of the pickleball player to be analyzed, and extract the video frames to be analyzed one by one according to time; The trained portrait positioning detection model is scheduled to analyze the video frames to be analyzed, and the portrait area corresponding to each video frame to be analyzed is determined; The motion analysis model after scheduling training analyzes the portrait area corresponding to the video frame to be analyzed, and determines the key point data of the human body posture corresponding to the video frame to be analyzed; According to the action type start judgment condition and the key data of human posture corresponding to the video frame to be analyzed, determine the start frame to be analyzed, the end frame to be analyzed, and the common frame to be analyzed between the start frame to be analyzed and the end frame to be analyzed; According to the start frame to be analyzed, the end frame to be analyzed, and the common frame structure to be analyzed between the start frame to be analyzed and the end frame to be analyzed, a trajectory data matrix to be analyzed is obtained; The motion analysis model after the scheduling training performs pickleball player motion analysis on the trajectory data matrix to be analyzed and determines the motion analysis results.

[0067] The motion analysis results may be some motion classification probability values ​​output by the motion analysis model, and the motion type corresponding to the maximum motion classification probability value may be used as the motion type corresponding to the video frame to be analyzed, thereby realizing the motion analysis of pickleball players.

[0068] In the embodiment of the present invention, the method further includes: determining the standard degree of the pickleball player's movements according to the movement analysis result.

[0069] According to the principles of deep learning, the closer the pickleball player's movements are to standard movements, the closer the movement classification probability value corresponding to the movement type corresponding to the video frame to be analyzed should be to 1. Therefore, the movement classification probability value corresponding to the movement type corresponding to the video frame to be analyzed can be used as the standard degree of the pickleball player's movements, thereby achieving further analysis.

[0070] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A computer vision method for analyzing the motion of a pickleball player, characterized in that: include: For any action type of a pickleball player, a standard action video corresponding to the action type is collected, and frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to each action type; Constructing a portrait positioning detection model and a human body posture estimation model, and training the portrait positioning detection model and the human body posture estimation model to obtain the trained portrait positioning detection model and the human body posture estimation model; For any action type of a pickleball player, the trained portrait positioning detection model is used to detect the standard action frame set corresponding to the action type, and the portrait area corresponding to the standard action frame corresponding to each action type is obtained; For the portrait area corresponding to the standard action frame corresponding to any action type, the trained human posture estimation model is used to detect the portrait area to obtain the human posture key point data corresponding to the standard action frame corresponding to each action type; Reconstruct the coordinates of the key point data of the human body posture corresponding to the standard action frame corresponding to each action type, and determine the coordinate value corresponding to each key point data of the human body posture; According to the coordinate values ​​corresponding to each human posture key point data, determine the action start frame, action end frame and common frame between the action start frame and the action end frame corresponding to each action type; For any action type, a motion trajectory data matrix is ​​constructed according to the action start frame, action end frame and common frame, and the action trajectory data matrix is ​​used to train the action analysis model to obtain the trained action analysis model; The video data of the pickleball player to be analyzed is obtained, and the trained portrait positioning detection model, the trained human posture estimation model and the trained motion analysis model are scheduled to perform motion analysis of the pickleball player and determine the motion analysis result.

2. The computer vision pickleball player motion analysis method according to claim 1, characterized in that: For any action type of a pickleball player, a standard action video corresponding to the action type is collected, and frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to each action type, including: The camera is placed at the boundary of the pickleball net with the central axis of the camera lens in a horizontal state, and the camera is rotated counterclockwise until the angle between the central axis of the camera lens and the pickleball net is 45 degrees, and the standard action video corresponding to each action type of the pickleball player is collected; wherein the number of standard action videos corresponding to each action type is at least one; For any standard action video, based on a preset data sampling frequency, frame extraction processing is performed on the standard action video to obtain a standard action frame set corresponding to the standard action video corresponding to each action type.

3. The computer vision pickleball player motion analysis method according to claim 1, characterized in that: A portrait positioning detection model and a human body posture estimation model are constructed, and after training the portrait positioning detection model and the human body posture estimation model, the trained portrait positioning detection model and the human body posture estimation model are obtained, including: The YOLOv3 neural network is used to build a portrait positioning detection model, and the ResNet50 is used to build a human posture estimation model; Use a portrait positioning detection model or a human body posture estimation model as a model to be trained; For the hyperparameters of the model to be trained, the random initialization method is used to randomly initialize the hyperparameters, and the initialized hyperparameters are encoded into vectors to obtain parameter vectors, and multiple different parameter vectors are obtained; For any parameter vector, a neighborhood search is performed on the parameter vector using the cosine pairing collaborative search method to obtain the parameter vector after the neighborhood search; For any parameter vector after neighborhood search, a gravitational search method is used to perform a secondary neighborhood search on the parameter vector to obtain the parameter vector after the secondary neighborhood search; For any parameter vector after the secondary neighborhood search, an adaptive random search method is used to perform a global search on the parameter vector to obtain the parameter vector after the global search; Determine whether the current number of training times is greater than or equal to the maximum number of training times. If so, determine the final hyperparameters of the model to be trained based on the parameter vector after the global search, and obtain the model to be trained after training. Otherwise, return to the neighborhood search step once.

4. The computer vision pickleball player motion analysis method according to claim 1, characterized in that: For any action type of a pickleball player, the trained portrait positioning detection model is used to detect the standard action frame set corresponding to the action type, and the portrait area corresponding to the standard action frame corresponding to each action type is obtained, including: For any action type of a pickleball player, a standard action frame in any standard action frame set corresponding to the action type is input into the trained portrait positioning detection model, and the area where the portrait detection frame is located is cropped to obtain the portrait area corresponding to the standard action frame corresponding to each action type.

5. The computer vision pickleball player motion analysis method according to claim 1, characterized in that: The human body posture key point data corresponding to the standard action frame include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

6. The computer vision pickleball player motion analysis method according to claim 1, characterized in that: Reconstruct the coordinates of the human body posture key point data corresponding to the standard action frame corresponding to each action type, and determine the coordinate value corresponding to each human body posture key point data, including: Determine the left shoulder and the right shoulder according to the human body posture key point data corresponding to the standard action frame corresponding to each action type; The center point of the left shoulder and the right shoulder is used as the origin to construct a coordinate system, and based on the coordinate system, the coordinate value corresponding to each key point data of the human body posture is determined.

7. The computer vision pickleball player motion analysis method according to claim 6, characterized in that: According to the coordinate values ​​corresponding to each human posture key point data, the action start frame, action end frame and common frame between the action start frame and action end frame corresponding to each action type are determined, including: For any action type, determine the action type start judgment condition and the action type end condition; For any standard action frame, determine the key angle of the action according to the coordinate value corresponding to each key point data of the human body posture; According to the action key angle, the action type start determination condition and the action type end condition, the action start frame, the action end frame and the common frame between the action start frame and the action end frame corresponding to each action type are determined.

8. The computer vision pickleball player motion analysis method according to claim 7, characterized in that: For any action type, the action trajectory data matrix is ​​constructed according to the action start frame, action end frame and common frame, and the action analysis model is trained using the action trajectory data matrix to obtain the trained action analysis model, including: For any action type, the key point data of the human body posture of the action start frame, action end frame and ordinary frame are constructed into a matrix to obtain the trajectory data matrix; A motion analysis model is constructed, and the trajectory data matrix is ​​used as the input of the motion analysis model. The motion type corresponding to the trajectory data matrix is ​​used as the expected output of the motion analysis model. The motion analysis model is trained to obtain the trained motion analysis model.

9. The computer vision pickleball player motion analysis method according to claim 8, characterized in that: Obtain the video data of the pickleball player to be analyzed, and schedule the trained portrait positioning detection model, the trained human posture estimation model, and the trained motion analysis model to perform motion analysis of the pickleball player and determine the motion analysis results, including: Obtain the video data to be analyzed of the pickleball player to be analyzed, and extract the video frames to be analyzed one by one according to time; The trained portrait positioning detection model is scheduled to analyze the video frames to be analyzed, and the portrait area corresponding to each video frame to be analyzed is determined; The motion analysis model after scheduling training analyzes the portrait area corresponding to the video frame to be analyzed, and determines the key point data of the human body posture corresponding to the video frame to be analyzed; According to the action type start judgment condition and the key data of human posture corresponding to the video frame to be analyzed, determine the start frame to be analyzed, the end frame to be analyzed, and the common frame to be analyzed between the start frame to be analyzed and the end frame to be analyzed; According to the start frame to be analyzed, the end frame to be analyzed, and the common frame structure to be analyzed between the start frame to be analyzed and the end frame to be analyzed, a trajectory data matrix to be analyzed is obtained; The motion analysis model after the scheduling training performs pickleball player motion analysis on the trajectory data matrix to be analyzed and determines the motion analysis results.

10. The computer vision pickleball player motion analysis method according to claim 9, characterized in that: Also includes: The standard degree of the pickleball player's movements is determined according to the movement analysis results.

Images