A goal segment classification method based on neural network
Through neural networks, the player's detection frame and the video frame when shooting in a basketball game are identified, combined with face and color recognition, the problem of the inability to classify goal clips in the existing technology is solved, and the effect of generating player's goal highlights is achieved.
Patent Information
- Application Number
- CN202111321234.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-09
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-11-09
AI Technical Summary
The existing technology cannot classify goals in basketball games by players, and cannot generate goals highlights for specific players.
A neural network-based method is adopted to identify the player's detection frame and the video frame when shooting through the detection model, calculate the sole coordinates and distance values, combine face recognition and color recognition to vote and classify the player's goal highlights.
The player classification of goal clips is realized, and more accurate player goal highlights are generated, which improves the accuracy and efficiency of classification.
Smart Images

Figure CN113989725B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of segment classification, and more specifically, the present invention relates to a goal segment classification method based on a neural network. Background Art
[0002] At present, there are a large number of basketball enthusiasts in China. A large number of basketball enthusiasts have a strong demand when playing in offline basketball courts, that is, after the game, they can generate a highlight video of all their shots or goals of the day to keep as a commemoration or post to Moments.
[0003] However, the technology currently on the market can only capture all goal clips, but cannot classify all goal clips by player to generate a goal highlight of a certain player. Summary of the invention
[0004] The purpose of the present invention is to provide a goal segment classification method based on a neural network to solve the technical problem in the above-mentioned prior art that all goal segments cannot be classified according to players.
[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:
[0006] A method for classifying goal fragments based on a neural network comprises the following steps:
[0007] Obtain a first shot video, input each to-be-identified video frame of the first shot video into a detection model to identify and obtain first detection frames of the players in all to-be-identified video frames and video frames representing when a shot is taken, and calculate first sole coordinates of the players in the video frames when the shot is taken according to the first detection frames;
[0008] Calculate the second sole coordinates in at least one second captured video according to the first sole coordinates, obtain the second detection frames of all players in the video frame representing the shooting in the second captured video, calculate the distance values between the second sole coordinates and all the second detection frames, and select the player in the second detection frame with the smallest distance value;
[0009] Perform face recognition and / or color recognition on the players in the first shot video and the at least one second shot video, calculate similarity based on the recognition results, vote on the players based on the similarity, and classify the goal clips of the first shot video and the at least one second shot video according to the players based on the voting results.
[0010] Preferably, the method further comprises the steps of:
[0011] Basketball training data is obtained, and the basketball training data is pre-processed and then input into a neural network model to train the neural network model to obtain a detection model.
[0012] Preferably, the step of acquiring the first captured video and inputting each to-be-recognized video frame of the first captured video into the detection model specifically comprises the following steps:
[0013] It includes a plurality of image collectors, and the image collectors are used to capture the entire basketball half court, the basketball restricted area and the three-point line area;
[0014] The image collector is used to collect the first shot video and at least one of the second shot videos;
[0015] All goal clips of the first shot video are obtained, wherein the goal clips include video frames of the basketball entering the basket, as well as several seconds forward and several seconds backward from the video frame of the basketball entering the basket, and each video frame to be identified in the goal clips is input into the detection model.
[0016] Preferably, calculating the second sole coordinates of the foot in at least one second captured video according to the first sole coordinates of the foot specifically includes the following steps:
[0017] arbitrarily obtaining a first video frame from the first captured video, and arbitrarily obtaining a second video frame from the second captured video;
[0018] Marking four corner points of the basketball restricted area in the first video frame and the second video frame respectively, and calculating an affine transformation matrix between the first video frame and the second video frame according to the four marked corner points;
[0019] The first sole coordinates are calculated using the affine transformation matrix to obtain the second sole coordinates in the second captured video.
[0020] Preferably, obtaining the second detection frames of all players in the video frame representing the shooting in the second shot video, calculating the distance values between the second sole coordinates and all the second detection frames, and selecting the player in the second detection frame with the smallest distance value specifically comprises the following steps:
[0021] Acquire all goal clips of the second shot video, wherein the goal clips include a video frame of a basketball entering a basket, and a number of seconds forward and a number of seconds backward from the video frame of the basketball entering the basket;
[0022] Inputting the video frame representing the shooting shot in the goal clip into the detection model to identify and obtain second detection frames of all players in the video frame representing the shooting shot;
[0023] The distance values between the second sole coordinate and all second detection frames are calculated, and the second detection frame with the smallest distance value is selected, where the second detection frame includes the player of the goal-scoring segment.
[0024] Preferably, the method further comprises the steps of:
[0025] Get a single photo of each participating player and calculate the HSV color histogram features of the player.
[0026] Preferably, obtaining a single photo of each participating player and calculating the HSV color histogram features of the player specifically includes the following steps:
[0027] Convert the BGR color space of the single-person photo into the HSV color space, and quantize each component of the HSV color space according to a quantization table;
[0028] The quantized HSV color space is merged into a single-channel image according to a ratio, and a color histogram feature is calculated according to the single-channel image to obtain and save a standard color histogram feature of the area where the player is located.
[0029] Preferably, the method further comprises the steps of:
[0030] A single photo of each participating player is obtained, and the face in the single photo is detected and the face feature vector is calculated using a face detection algorithm and a face recognition algorithm.
[0031] Preferably, obtaining a single photo of each participating player, detecting the face of the single photo and calculating the face feature vector by using a face detection algorithm and a face recognition algorithm, specifically includes the following steps:
[0032] Convert the BGR color space of a single person photo to the RGB color space, and normalize the pixel values in the RGB color space;
[0033] Detecting the single person photo using a face detection algorithm, and if a face in the single person photo is detected, extracting the detected face;
[0034] The face is corrected by a face correction algorithm, the corrected face is recognized by a face recognition algorithm, and a standard face feature vector is extracted and saved.
[0035] Preferably, calculating the similarity according to the recognition result and voting for the players according to the similarity specifically includes the following steps:
[0036] If a face is detected, the similarity between the face feature vector and the standard face feature vector is calculated. The similarity formula is as follows:
[0037]
[0038] Among them, X refers to the facial feature vector, Y refers to the standard facial feature vector, X*Y is the vector product of the two vectors, ||X||*||Y|| is the product of the modules of the two vectors, and the larger the calculated Sim(X,Y), the higher the similarity.
[0039] Vote for the standard facial feature vector with the highest similarity, calculate the number of votes for the player represented by each standard facial feature vector, and determine that the goal clip belongs to the player represented by the standard facial feature vector with the highest number of votes.
[0040] Preferably, calculating the similarity according to the recognition result and voting for the players according to the similarity specifically includes the following steps:
[0041] If no face is detected, the similarity between the color histogram feature and the standard color histogram feature is calculated respectively. The similarity formula is as follows:
[0042]
[0043] Among them, X represents the color histogram feature, Y represents the standard color histogram feature, X i represents the i-th component of X, Y i It represents the i-th component of Y. The smaller the calculated Sim(X,Y), the higher the similarity.
[0044] Voting for the standard color histogram feature with the highest similarity, calculating the number of votes for the player represented by each standard color histogram feature, and determining that the goal clip belongs to the player represented by the standard color histogram feature with the highest number of votes.
[0045] A computer-readable storage medium, characterized in that computer instructions are stored on the computer-readable storage medium, and the steps of the above-mentioned method are implemented when the computer instructions are executed by a processor.
[0046] The beneficial effects provided by the present invention are:
[0047] 1. The present invention calculates the second sole coordinates in at least one second captured video through the first sole coordinates, obtains the second detection frames of all players in the video frame representing the time of shooting in the second captured video, calculates the distance values between the second sole coordinates and all the second detection frames, and selects the player in the second detection frame with the smallest distance value, so that any point on the basketball court in any video frame can be converted to other video frames, and a more convenient algorithm is used to more quickly find the same goal clip and the corresponding player in different video frames.
[0048] 2. The present invention performs face recognition and / or color recognition on the players in the first shot video and at least one second shot video, calculates the similarity based on the recognition results, votes for the players based on the similarity, and classifies the goal clips of the first shot video and at least one second shot video according to the voting results, thereby generating a goal highlight of the player. At the same time, the classification method based on color features and face recognition can achieve a higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0050] Figure 1 It is a flow chart of the method for goal classification based on neural network;
[0051] Figure 2 is a schematic diagram of the position of the image collector;
[0052] The main component symbols are described as follows:
[0053] 1. First image collector 1; 2. Second image collector 2; 3. Third image collector 3; 4. Fourth image collector 4. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0055] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] Embodiment 1:
[0057] like Figure 1 As shown, this embodiment includes a goal segment classification method based on a neural network, comprising the following steps:
[0058] Obtain a first shot video, input each to-be-identified video frame of the first shot video into a detection model to identify and obtain first detection frames of the players in all to-be-identified video frames and video frames representing when a shot is taken, and calculate first sole coordinates of the players in the video frames when the shot is taken according to the first detection frames;
[0059] Calculate the second sole coordinates in at least one second captured video according to the first sole coordinates, obtain the second detection frames of all players in the video frame representing the time of shooting in the second captured video, calculate the distance values between the second sole coordinates and all the second detection frames, and select the player in the second detection frame with the smallest distance value;
[0060] Perform face recognition and / or color recognition on the players in the first shot video and at least one second shot video, calculate similarity based on the recognition results, vote on the players based on the similarity, and classify the goal clips of the first shot video and at least one second shot video according to the players based on the voting results.
[0061] Specifically, the first detection frame includes multiple types of players, such as a first detection frame representing a defensive player or a first detection frame representing a shooting player. In the present embodiment, the first detection frame of the player identified in all the video frames to be identified is the first detection frame representing the shooting player.
[0062] The method also includes the steps of obtaining basketball training data, pre-processing the basketball training data and inputting the data into a neural network model to train the neural network model and obtain a detection model.
[0063] Specifically, pictures of basketball games are collected, and the pictures of basketball games are basketball training data. The method for preprocessing the basketball training data is to manually mark the players, balls and baskets in the pictures with detection frames, and input the pictures with marked detection frames into the neural network model to train the neural network model until the precision and recall rate within a preset range is reached. After the training of the neural network model, a detection model is obtained, and the detection model is used to detect players, balls and baskets.
[0064] Acquire a first shot video, and input each to-be-identified video frame of the first shot video into a detection model, specifically comprising the following steps: include a plurality of image collectors, the image collectors are used to capture the entire basketball half court, the basketball restricted area, and the three-point line area;
[0065] The image collector is used to collect a first shot video and at least one second shot video, obtain all goal clips of the first shot video, the goal clips include video frames of the basketball entering the basket, and several seconds forward and backward from the video frame of the basketball entering the basket, and input each video frame to be identified in the goal clip into the detection model.
[0066] Specifically, in this implementation, if Figure 2 As shown, there are four image collectors, namely the first image collector 1, the second image collector 2, the third image collector 3 and the fourth image collector 4. The first image collector 1 and the second image collector 2 are respectively located on both sides of the center line of the basketball court. The first image collector 1 and the second image collector 2 are used to shoot the half court of the entire basketball court. The third image collector 3 and the fourth image collector 4 are set under the basketball stand. The third image collector 3 and the fourth image collector 4 are used to shoot the basketball restricted area and the three-point line area.
[0067] The first image collector 1 is used to collect the first shot video, the second image collector 2 is used to collect the second shot video, the third image collector 3 is used to collect the third shot video, and the fourth image collector 4 is used to collect the fourth shot video. During the game, the first image collector 1, the second image collector 2, the third image collector 3 and the fourth image collector 4 are operated simultaneously to shoot the basketball court at four different angles. In this embodiment, the at least one second shot video described includes the second shot video, the third shot video and the fourth shot video.
[0068] The second sole coordinates of the foot in at least one second captured video are calculated based on the first sole coordinates, and specifically include the following steps: arbitrarily obtain a first video frame from the first captured video, and arbitrarily obtain a second video frame from the second captured video; respectively calibrate the four corner points of the basketball restricted area in the first video frame and the second video frame, and calculate the affine transformation matrix between the first video frame and the second video frame based on the four calibrated corner points; the first sole coordinates are used to calculate the second sole coordinates in the second captured video through the affine transformation matrix.
[0069] Specifically, the positions of the four corner points are the upper left corner, lower left corner, upper right corner and lower right corner of the basketball restricted area.
[0070] The formula for calculating the second sole coordinates of the first foot in the second captured video through the affine transformation matrix is as follows: (x', y', w') = [u, v, 1] * A, where (u, v) is a point on the basketball court photographed by the first image acquisition device 1, and the matrix A is the first affine transformation matrix calculated based on the four corner points of the basketball restricted area calibrated in the first video frame and the second video frame. x' and y' are conversion coefficients, and 1 means no overall scaling.
[0071] The coordinates of the second foot sole are expressed as (x, y), where the calculation formula for x is: The calculation formula for y is: w' is automatically calculated by the system. The first sole coordinate is calculated based on the first affine transformation matrix by using the above formula to calculate the second sole coordinate in the second captured video.
[0072] Specifically, in the above embodiment, a third video frame is arbitrarily obtained from the third captured video, and a fourth video frame is arbitrarily obtained from the fourth captured video. The four corner points of the basketball restricted area are calibrated in the first video frame and the third video frame, respectively, and the four corner points of the basketball restricted area are calibrated in the first video frame and the fourth video frame, respectively. The second affine transformation matrix is calculated based on the four corner points of the basketball restricted area calibrated in the first video frame and the third video frame, and the third affine transformation matrix is calculated based on the four corner points of the basketball restricted area calibrated in the first video frame and the fourth video frame. The first sole coordinates are calculated based on the second affine transformation matrix by the above formula to calculate the third sole coordinates in the third captured video. The first sole coordinates are calculated based on the third affine transformation matrix by the above formula to calculate the fourth sole coordinates in the fourth captured video.
[0073] Acquire the second detection frames of all players in the video frame representing the time of shooting in the second shot video, calculate the distance values between the second sole coordinates and all the second detection frames, and select the player in the second detection frame with the smallest distance value, specifically including the following steps: acquire all goal clips of the second shot video, the goal clips including the video frame of the basketball entering the basket, and the video frame of the basketball entering the basket pushed forward a number of seconds and pushed backward a number of seconds;
[0074] The video frame representing the shooting in the goal clip is input into the detection model to identify the second detection frame of all players in the video frame representing the shooting; the distance value between the second sole coordinate and all the second detection frames is calculated, and the second detection frame with the smallest distance value is selected, and the second detection frame includes the player of the goal clip. Wherein, the second sole coordinate in at least one second shot video is calculated according to the first sole coordinate, the distance value between the second sole coordinate and all the second detection frames is calculated, and the second detection frame with the smallest distance value is selected, and the second detection frame includes the player of the goal clip. Since the first detection frame is the first detection frame representing the shooting player, and the second detection frame with the smallest distance value is selected, in this embodiment, the player in the second detection frame including the goal clip is the shooting player.
[0075] Specifically, all goal clips of the second shot video are obtained, and the goal clips include the video frame of the basketball entering the basket, and ...
[0076] The video frame representing the time when the shot is taken in the goal segment of the second shot video is input into the detection model to identify and obtain the second detection frame of all the players in the video frame representing the time when the shot is taken. The video frame representing the time when the shot is taken in the goal segment of the third shot video is input into the detection model to identify and obtain the third detection frame of all the players in the video frame representing the time when the shot is taken. The video frame representing the time when the shot is taken in the goal segment of the fourth shot video is input into the detection model to identify and obtain the fourth detection frame of all the players in the video frame representing the time when the shot is taken.
[0077] Calculate the distance between the second sole coordinate and all second detection frames, select the second detection frame with the smallest distance value, and the second detection frame includes the player of the goal clip. Calculate the distance between the third sole coordinate and all third detection frames, select the third detection frame with the smallest distance value, and the third detection frame includes the player of the goal clip in the third captured video. Calculate the distance between the fourth sole coordinate and all fourth detection frames, select the fourth detection frame with the smallest distance value, and the fourth detection frame includes the player of the goal clip in the fourth captured video.
[0078] The method also includes the steps of obtaining a single photo of each participating player and calculating the HSV color histogram feature of the player.
[0079] Obtain a single photo of each participating player and calculate the HSV color histogram features of the player, specifically including the following steps: convert the BGR color space of the single photo into the HSV color space, and quantize each component of the HSV color space according to the quantization table;
[0080] The quantized HSV color space is merged into a single-channel image according to the ratio, and the color histogram features are calculated based on the single-channel image to obtain the standard color histogram features of the player's area and save them.
[0081] Specifically, the quantization table is as follows:
[0082]
[0083] Among them, H, S, and V represent the three color components of the HSV color space respectively. Each row is the quantized value and the value range of the corresponding color component. Taking H∈[21,40] as an example, when the value of any position in the color component is between 21 and 40, it is quantized to 1.
[0084] The quantized HSV color space is merged into a single-channel image according to the ratio. The ratio formula is G=9H+3S+V. The standard color histogram features are calculated based on the single-channel image to obtain the standard color histogram features of the player's area and save them.
[0085] The method also includes the following steps: obtaining a single photo of each participating player, detecting the face in the single photo and calculating the face feature vector by using a face detection algorithm and a face recognition algorithm.
[0086] Obtain a single photo of each participating player, detect the face in the single photo and calculate the face feature vector by using a face detection algorithm and a face recognition algorithm, specifically including the following steps: convert the BGR color space of the single photo into an RGB color space, and normalize the pixel value of the RGB color space;
[0087] The single-person photo is detected by a face detection algorithm. If a face in the single-person photo is detected, the detected face is cut out; the face is corrected by a face correction algorithm, and the corrected face is recognized by a face recognition algorithm to extract a standard face feature vector.
[0088] Specifically, the calculation formula for normalizing the pixel value in the RGB color space is as follows:
[0089] p' i =((p i / 255)-mean i ) / std i
[0090] Among them, p i is the RGB color value of any position in the i-th channel, mean i is the mean of the RGB color values of the i-th channel, std i is the standard deviation of the RGB color values of the i-th channel.
[0091] The similarity is calculated based on the recognition results, and the players are voted based on the similarity, which specifically includes the following steps: If a face is detected, the similarity between the face feature vector and the standard face feature vector is calculated respectively. The similarity formula is as follows:
[0092]
[0093] Among them, X represents the facial feature vector, Y represents the standard facial feature vector, X*Y represents the vector product of the two vectors, ||X||*||Y|| represents the product of the modules of the two vectors, and the larger the calculated Sim(X,Y), the higher the similarity. Vote for the standard facial feature vector with the highest similarity, calculate the number of votes for the player represented by each standard facial feature vector, and determine that the goal clip belongs to the player represented by the standard facial feature vector with the highest number of votes.
[0094] Specifically, the face detection algorithm is used to perform face recognition detection on the first detection frame, the second detection frame with the smallest distance value, the third detection frame with the smallest distance value, and the fourth detection frame with the smallest distance value. In this embodiment, there are four players, and the four standard face feature vectors are recorded as: F1, F2, F3 and F4.
[0095] If the above four detection frames all detect faces, the corresponding face feature vectors are f1, f2, f3 and f4 respectively. The similarities between the four face feature vectors and the standard face feature vector are calculated respectively. The similarity formula is as follows:
[0096]
[0097] Among them, X represents any one of the vectors f1, f2, f3 and f4, Y represents any one of the vectors F1, F2, F3 and F4, X*Y represents the vector product of the two vectors, ||X||*||Y|| represents the product of the modules of the two vectors, and the larger the calculated Sim(X,Y), the higher the similarity.
[0098] Specifically, calculate the similarity between f1 and the four standard facial feature vectors F1, F2, F3, and F4. Assuming that f1 and F1 have the highest similarity, f1 will vote for the player with the standard facial feature vector of F1. Calculate the similarity between f2 and the four standard facial feature vectors F1, F2, F3, and F4. Assuming that f2 and F4 have the highest similarity, f2 will vote for the player with the standard facial feature vector of F4. Calculate the similarity between f3 and the four standard facial feature vectors F1, F2, F3, and F4. Assuming that f3 and F1 have the highest similarity, f3 will vote for the player with the standard facial feature vector of F1. Calculate the similarity between f4 and the four standard facial feature vectors F1, F2, F3, and F4. Assuming that f4 and F1 have the highest similarity, f4 will vote for the player with the standard facial feature vector of F1.
[0099] The number of votes for the player represented by each standard facial feature vector is calculated. In this embodiment, the number of votes for the player represented by the F1 standard facial feature vector is 3 votes, and the number of votes for the player represented by the F4 standard facial feature vector is 1 vote. The player represented by the F1 standard facial feature vector has the largest number of votes, so it is determined that the goal clip belongs to the player represented by the F1 standard facial feature vector.
[0100] The similarity is calculated based on the recognition results, and the players are voted based on the similarity, which specifically includes the following steps: If no face is detected, the similarity of the color histogram feature and the standard color histogram feature is calculated respectively. The similarity formula is as follows:
[0101]
[0102] Among them, X represents the color histogram feature, Y represents the standard color histogram feature, X i represents the i-th component of X, Y i It represents the i-th component of Y. The smaller the calculated Sim(X,Y), the higher the similarity.
[0103] Vote for the standard color histogram feature with the highest similarity, calculate the number of votes for the player represented by each standard color histogram feature, and determine that the goal clip belongs to the player represented by the standard color histogram feature with the highest number of votes.
[0104] Specifically, the face detection algorithm is used to perform face recognition detection on the first detection frame, the second detection frame with the smallest distance value, the third detection frame with the smallest distance value, and the fourth detection frame with the smallest distance value. In the above embodiment, there are four players. If the face detection algorithm does not detect faces, the color histogram features corresponding to the four detection frames are recorded as: g1, g2, g3 and g4, and the four standard color histogram features are recorded as: G1, G2, G3 and G4. The similarities between the four color histogram features and the standard color histogram features are calculated respectively, and the similarity formula is as follows:
[0105]
[0106] Where X represents any vector among g1, g2, g3, and g4, Y represents any vector among G1, G2, G3, and G4, and X i represents the i-th component of X, Y i Represents the i-th component of Y. The smaller the calculated Sim(X,Y), the higher the similarity.
[0107] Calculate the similarity between g1 and the four standard color histogram features of G1, G2, G3, and G4. Assuming that g1 and G1 have the highest similarity, g1 will vote for the player with the standard color histogram feature of G1. Calculate the similarity between g2 and the four standard color histogram features of G1, G2, G3, and G4. Assuming that g2 and G4 have the highest similarity, g2 will vote for the player with the standard color histogram feature of G4. Calculate the similarity between g3 and the four standard color histogram features of G1, G2, G3, and G4. Assuming that g3 and G1 have the highest similarity, g3 will vote for the player with the standard color histogram feature of G1. Calculate the similarity between g4 and the four standard color histogram features of G1, G2, G3, and G4. Assuming that g4 and G1 have the highest similarity, g4 will vote for the player with the standard color histogram feature of G1.
[0108] The number of votes for the player represented by each standard color histogram feature is calculated. In this embodiment, the number of votes for the player represented by the G1 standard color histogram feature is 3 votes, and the number of votes for the player represented by the G4 standard color histogram feature is 1 vote. The player represented by the G1 standard color histogram feature has the largest number of votes, so it is determined that the goal clip belongs to the player represented by the G1 standard color histogram feature.
[0109] Repeat the above steps for each goal clip and classify them according to the players. When the game is over, all the goal clips of a certain player will be automatically generated into a highlight video and provided to the players for download.
[0110] Embodiment 2:
[0111] A computer-readable storage medium stores computer instructions, which implement the steps of the method in Example 1 when executed by a processor.
[0112] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0113] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0116] It should be noted that:
[0117] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.
[0118] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0119] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.
Claims
1. A goal segment classification method based on a neural network, characterized in that: The following steps are involved: Obtain a first shot video, input each to-be-identified video frame of the first shot video into a detection model to identify and obtain first detection frames of the players in all to-be-identified video frames and video frames representing when a shot is taken, and calculate first sole coordinates of the players in the video frames when the shot is taken according to the first detection frames; Calculate the second sole coordinates in at least one second captured video according to the first sole coordinates, obtain the second detection frames of all players in the video frame representing the time of shooting in the second captured video, calculate the distance value between the second sole coordinates and all the second detection frames, and select the player in the second detection frame with the smallest distance value, wherein the second sole coordinates in at least one second captured video according to the first sole coordinates specifically include the following steps: arbitrarily obtain a first video frame from the first captured video, and arbitrarily obtain a second video frame from the second captured video; respectively mark the four corner points of the basketball restricted area in the first video frame and the second video frame, and calculate the affine transformation matrix between the first video frame and the second video frame according to the four marked corner points; the first sole coordinates calculate the second sole coordinates in the second captured video through the affine transformation matrix; Performing face recognition and color recognition on all players in the first shot video and the at least one second shot video, and calculating similarity based on the recognition results, voting for the players based on the similarity, and classifying the goal clips of the first shot video and the at least one second shot video according to the players based on the voting results, wherein calculating similarity based on the recognition results and voting for the players based on the similarity specifically include the following steps: if a face is detected, calculating the similarity between the face feature vector and the standard face feature vector respectively, and the similarity formula is as follows: , where X refers to the face feature vector, and Y refers to the standard face feature vector. is the vector product of these two vectors, is the product of the moduli of these two vectors, calculated The larger the value, the higher the similarity; vote for the standard facial feature vector with the highest similarity, calculate the number of votes for the player represented by each standard facial feature vector, and determine that the goal clip belongs to the player represented by the standard facial feature vector with the highest number of votes; If no face is detected, the similarity between the color histogram feature and the standard color histogram feature is calculated respectively. The similarity formula is as follows: , where X represents the color histogram feature and Y represents the standard color histogram feature. represents the i-th component of X, represents the i-th component of Y, and the calculated The smaller it is, the higher the similarity is; vote for the standard color histogram feature with the highest similarity, calculate the number of votes for the player represented by each standard color histogram feature, and determine that the goal clip belongs to the player represented by the standard color histogram feature with the highest number of votes.
2. A method for classifying goal fragments based on a neural network as claimed in claim 1, characterized in that: Also includes the steps: Basketball training data is obtained, and the basketball training data is pre-processed and then input into a neural network model to train the neural network model to obtain a detection model.
3. A method for classifying goal fragments based on a neural network as claimed in claim 1, characterized in that: The step of obtaining a first shot video and inputting each to-be-recognized video frame of the first shot video into a detection model specifically includes the following steps: It includes a plurality of image collectors, and the image collectors are used to capture the entire basketball half court, the basketball restricted area and the three-point line area; The image collector is used to collect the first shot video and at least one of the second shot videos; All goal clips of the first shot video are obtained, wherein the goal clips include video frames of the basketball entering the basket, as well as several seconds forward and several seconds backward from the video frame of the basketball entering the basket, and each video frame to be identified in the goal clips is input into the detection model.
4. A method for classifying goal fragments based on a neural network as claimed in claim 1, characterized in that: Obtaining the second detection frames of all players in the video frame representing the shooting in the second shot video, calculating the distance values between the second sole coordinates and all the second detection frames, and selecting the player in the second detection frame with the smallest distance value, specifically includes the following steps: Acquire all goal clips of the second shot video, wherein the goal clips include a video frame of a basketball entering a basket, and a number of seconds forward and a number of seconds backward from the video frame of the basketball entering the basket; Inputting the video frame representing the shooting shot in the goal clip into the detection model to identify and obtain second detection frames of all players in the video frame representing the shooting shot; The distance values between the second sole coordinate and all second detection frames are calculated, and the second detection frame with the smallest distance value is selected, where the second detection frame includes the player of the goal-scoring segment.
5. A method for classifying goal fragments based on a neural network as claimed in claim 1, characterized in that: Also includes the steps: A single photo of each participating player is obtained, and the HSV color histogram feature of each participating player is calculated.
6. A method for classifying goal fragments based on a neural network as claimed in claim 5, characterized in that: Obtaining a single photo of each participating player and calculating the HSV color histogram features of each participating player specifically includes the following steps: Convert the BGR color space of the single-person photo into the HSV color space, and quantize each component of the HSV color space according to a quantization table; The quantized HSV color space is merged into a single-channel image according to a ratio, and a color histogram feature is calculated according to the single-channel image to obtain and save a standard color histogram feature of the area where each participating player is located.
7. A method for classifying goal fragments based on a neural network as claimed in claim 1, characterized in that: Also includes the steps: A single photo of each participating player is obtained, and the face in the single photo is detected and the face feature vector is calculated using a face detection algorithm and a face recognition algorithm.
8. A method for classifying goal fragments based on a neural network as claimed in claim 7, characterized in that: Obtain a single photo of each participating player, detect the face in the single photo and calculate the face feature vector using a face detection algorithm and a face recognition algorithm, specifically including the following steps: Convert the BGR color space of a single person photo to the RGB color space, and normalize the pixel values in the RGB color space; Detecting the single-person photo using a face detection algorithm, and if a face in the single-person photo is detected, extracting the detected face; The face is corrected by a face correction algorithm, the corrected face is recognized by a face recognition algorithm, and a standard face feature vector is extracted and saved.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the steps of the neural network-based goal fragment classification method as described in any one of claims 1 to 8.
