Sports video action recognition method and device, computer equipment and storage medium
By segmenting and scoring sports videos and identifying keyframe segments, the problem of low accuracy in existing sports videos' actions recognition is solved, and the accuracy and accuracy of recognition are improved.
Patent Information
- Application Number
- CN202411540191.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-01-14
AI Technical Summary
The existing sports video action recognition methods are low in accuracy due to too much invalid information during identification. The human movements in different sports events have large intra-category variance and small inter-category variance, making it difficult to accurately distinguish the action categories, resulting in misjudgment.
By segmenting the sports video, a preset classifier is obtained to score the video clips, an action score matrix is generated, a keyframe video clip is determined from it, and a set of action matching probability sets are determined based on these frame segments, thereby identifying the action of the target sports video.
It improves the accuracy of sports video action recognition, avoids interference from invalid information, ensures that the action differences are not ignored, and enhances the accuracy of recognition.
Smart Images
Figure CN119314228A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a sports video action recognition method, device, computer equipment and storage medium. Background Art
[0002] Video action recognition technology is widely used in different scenarios. For example, in video security scenarios, video action recognition technology can be used to identify dangerous actions in monitoring so as to provide timely warnings; in human-computer interaction scenarios, video action recognition technology can be used to identify human gestures or actions so that the machine can give corresponding feedback. In addition, video action recognition technology is also needed in sports scenarios to facilitate intelligent scoring, highlight clip editing, and event analysis.
[0003] Existing sports action recognition methods use the feature information of all frames in the source video clip as the prediction basis of the action recognition model. However, the feature information of most frames in the video clip may be invalid information irrelevant to the sports action. The noise brought by this invalid information increases the difficulty of sports video action recognition and affects the accuracy of video action recognition. At the same time, human actions in different sports have the characteristics of large intra-class variance and small inter-class variance. This characteristic makes it difficult for the action recognition model to accurately distinguish different action categories, resulting in subtle action differences being ignored, and then misjudgment occurs. Summary of the invention
[0004] Based on this, it is necessary to provide a sports video action recognition method, device, computer equipment and storage medium to address the above technical problems, so as to solve the current problem of low accuracy in action recognition in sports videos.
[0005] A sports video action recognition method, comprising: Segment the target sports video to obtain multiple sports video clips; Obtaining a preset classifier corresponding to each candidate sports action, and scoring all the sports video clips using all the preset classifiers to obtain an action score matrix; Determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; A target sports action in the target sports video is determined from all the candidate sports actions according to the action matching probability set.
[0006] A sports video action recognition device, comprising: A video segmentation module is used to segment the target sports video to obtain multiple sports video clips; A clip scoring module is used to obtain a preset classifier corresponding to each candidate sports action, and score all the sports video clips using all the preset classifiers to obtain an action score matrix; A key frame determination module, configured to determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; The target action determination module is used to determine the target sports action of the target sports video from all the candidate sports actions according to the action matching probability set.
[0007] A computer device comprises a memory, a processor and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned sports video action recognition method when executing the computer-readable instructions.
[0008] A computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the sports video action recognition method as described above.
[0009] The sports video action recognition method, device, computer equipment and storage medium are as follows: the sports video action recognition method obtains a plurality of sports video clips by segmenting the target sports video; obtains preset classifiers corresponding to each candidate sports action, and scores all sports video clips by all preset classifiers to obtain an action score matrix; determines key frame video clips corresponding to each candidate sports action from all sports video clips according to the action score matrix, and determines the action matching probability set of the target sports video and all candidate sports actions according to the key frame video clips; determines the target sports action of the target sports video from all candidate sports actions according to the action matching probability set. The present invention obtains the action score matrix by segmenting and then scoring the sports video, which is not easy to ignore action differences. At the same time, based on the action score matrix, the key frame video clips are mined from the plurality of sports video clips after segmentation, thereby avoiding the interference of invalid information of other sports video clips. Finally, the key frame video clips are used as the main basis for predicting the target sports action, which greatly improves the accuracy of sports video action recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0011] Figure 1 is a schematic diagram of an application environment of a sports video action recognition method in one embodiment of the present invention; Figure 2 is a flow chart of a method for identifying sports video actions in one embodiment of the present invention; Figure 3 is a structural schematic diagram of a sports video action recognition device in one embodiment of the present invention; Figure 4 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0013] The intelligent question-answering processing method based on artificial intelligence provided by the embodiment of the present invention can be applied in Figure 1In an application environment, the client communicates with the server through a network. The server can receive a target sports video through the client, segment the target sports video, and obtain multiple sports video clips; obtain a preset classifier corresponding to each candidate sports action, and score all sports video clips through all preset classifiers to obtain an action score matrix; determine the key frame video clips corresponding to each candidate sports action from all sports video clips according to the action score matrix, and determine the action matching probability set of the target sports video and all candidate sports actions according to the key frame video clips; determine the target sports action of the target sports video from all candidate sports actions according to the action matching probability set. In the present invention, the target sports video can be a sports video shot in real time, a pre-recorded sports video, or a sports video downloaded from a public database. With the development of the business of the financial industry, it can deeply participate in sports events, and further promote the deep integration of "finance + sports" by providing technical support and data services. For example, real-time analysis of the athlete's motion trajectory, speed, strength and other parameters provides objective and accurate basis for referees, which helps to reduce human misjudgment and improve the fairness and viewing of the game. This cooperation model not only demonstrates the application potential of financial technology in the sports field, but also provides more market opportunities and cooperation space for the financial industry.
[0014] The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.
[0015] In one embodiment, if Figure 2 As shown, a sports video action recognition method is provided, comprising the following steps S10-S40: S10: Segment the target sports video to obtain multiple sports video clips.
[0016] Understandably, the target sports video is a sports video in which specific types of human body movements in the video need to be identified. Sports video clips refer to clips obtained after segmenting the target sports video. A video is usually composed of multiple video frames, and different video frames may be the same or different. In order to reduce the difficulty of action recognition of the target sports video, it is necessary to segment the target sports video into multiple sports video clips.
[0017] In one embodiment, in step S10, the target sports video is segmented to obtain a plurality of sports video segments, including: S101, determining a target sliding window according to a preset number of segmentation frames; S102: Slide the target sliding window on the target sports video to segment the target sports video to obtain at least two sports video segments.
[0018] Understandably, since a video is usually composed of multiple video frames, the target sports video can be segmented according to the number of frames. For example, when the total number of frames of the target sports video is 100 frames, the target sports video can be segmented according to a segment of 10 frames to obtain 10 sports video segments. The preset segmentation frame number refers to a specific frame number requirement for segmenting a video that is preset. The target sliding window refers to a sliding window whose sliding length of the window is the preset segmentation frame number. The sliding window is a technology that operates on a string or data group of a specific size, and adapts to different problem requirements by adjusting the starting and ending positions of the window. Specifically, in the segmentation process of the video, the implementation of the target sliding window mainly relies on a double pointer or iterator to identify the sequence of video segments currently being processed, the target sliding window moves step by step, and performs corresponding segmentation operations at each position, and finally obtains multiple sports video segments. In the process of sliding the target sliding window on the target sports video, there may be partial overlap between two adjacent slides, or there may be no overlap. The present embodiment preferably has no overlap, that is, the end position of the last window and the start position of the next window are two adjacent video frames.
[0019] In one embodiment, the entire segmentation process is represented as .in, represents the target sports video before segmentation, Represents each sports video clip after segmentation. Each sports video clip corresponds to an instance. Indicates the instance number. Represents the total number of instances, that is, the number of instances obtained after segmenting the target sports video Specifically, when the total number of frames of the target sports video is 100, the preset number of segmentation frames is 10, and the target sports video is segmented into segments of 10 frames to obtain 10 sports video segments. That is, the 1st to 10th frames are The sports video clip represented by The sports video clip represented by ... Frames 91 to 100 are Sports video clips represented.
[0020] This embodiment realizes the orderly segmentation of the target sports video based on the sliding window method, and at the same time ensures the integrity of the data after segmentation, thereby avoiding the omission of sports video clips.
[0021] In one embodiment, in step S10, that is, after the target sports video is segmented to obtain at least two sports video segments, the method further includes: S103, determining a tail video segment from at least two of the sports video segments, and obtaining the tail segment frame number of the tail video segment; the tail video segment is a sports video segment obtained by segmenting when the target sliding window is slid on the target sports video for the last time; S104, determining whether the number of frames of the tail segment is consistent with the preset number of segmentation frames; S105: When the frame number of the tail segment is inconsistent with the preset segmentation frame number, remove the tail video segment from at least two of the sports video segments.
[0022] Understandably, after the target sports video is segmented to obtain at least two sports video segments, post-processing is still required. Determine the tail video segment from the at least two sports video segments, and obtain the tail segment frame number of the tail video segment. Among them, the tail video segment refers to the sports video segment segmented when the target sliding window is slid on the target sports video for the last time, and the tail segment frame number is the number of video frames contained in the tail video segment. Compare the tail segment frame number with the preset segmentation frame number to determine whether the two are consistent. When the tail segment frame number is consistent with the preset segmentation frame number, retain all sports video segments. When the tail segment frame number is inconsistent with the preset segmentation frame number, remove the sports video segment segmented for the last time from all sports video segments, and retain the sports video segment before the last segmentation, that is, remove the tail video segment from at least two sports video segments.
[0023] In one embodiment, when the total number of frames of the target sports video is 95, the preset number of split frames is 10, and 10 sports video clips are obtained after splitting. That is, the 1st to 10th frames are The sports video clip represented by The sports video clip represented by ... Frames 91 to 95 are At this point, The sports video clip represented by is the tail video clip. The corresponding tail clip frame number is inconsistent with the preset segmentation frame number (5≠10), so it is necessary to Represents sports video clip removal.
[0024] This embodiment obtains the tail segment frame number of the tail video segment, and removes the tail video segment when the tail segment frame number is inconsistent with the preset segmentation frame number, so that each sports video segment has the same frame number, ensuring data consistency and helping to improve the accuracy and effectiveness of subsequent model predictions.
[0025] S20, obtaining a preset classifier corresponding to each candidate sports action, and scoring all the sports video clips using all the preset classifiers to obtain an action score matrix.
[0026] Understandably, candidate sports actions are all possible actions used to describe the action types of the target sports video. For example, in badminton, candidate sports actions include serves, high balls, draw balls, smashes, drives and diagonal balls, etc. For another example, in basketball, candidate sports actions include three-point goals, two-point goals, slam dunk goals, free throws and jump balls, etc. The preset classifier is a pre-trained neural network model used to determine whether an input video clip belongs to a specific candidate sports action, such as a binary classifier or a multi-classifier. Each candidate sports action corresponds to a preset classifier, that is, for each candidate sports action, it is necessary to collect the corresponding action video as a sample, and use the collected samples to train the model to obtain a preset classifier for determining whether the subsequent video clip belongs to the candidate sports action. The action score matrix refers to a matrix used to record the scores of different sports video clips after being scored by different preset classifiers. For example, when there is Sports video clips and When the preset classifier corresponding to the candidate sports action is used, a OK The action score matrix of columns.
[0027] In one embodiment, in step S20, that is, scoring all the sports video clips by all the preset classifiers to obtain an action score matrix, includes: S201, extracting features from the sports video clips by using a preset feature extractor to obtain video clip features of each of the sports video clips; S202, scoring each of the video clip features by using a preset classifier corresponding to each of the candidate sports actions to obtain a clip action score of each of the sports video clips corresponding to each of the candidate sports actions; S203, obtaining the segment action scores of all the sports video segments corresponding to the same candidate sports action, and obtaining an action score column vector corresponding to the candidate sports action; S204: Merge all the action score column vectors to generate an action score matrix.
[0028] Understandably, each sports video clip is input into a preset feature extractor, and the preset feature extractor is used to extract features of the sports video clip, so that the video clip features of each sports video clip can be obtained. The preset feature extractor is a neural network model for extracting human motion features in the clip, such as a convolutional neural network. The video clip feature is a vector used to characterize the human motion features in the video clip. The input of the preset classifier is the video clip feature of the sports video clip, and the output is the clip action score of the sports video clip belonging to a specific candidate sports action corresponding to the preset classifier. The clip action score is a score used to characterize the similarity between the human motion in the sports video clip and the candidate sports action. The higher the clip action score, the greater the similarity between the human motion in the sports video clip and the candidate sports action. The lower the clip action score, the smaller the similarity between the human motion in the sports video clip and the candidate sports action. When the last layer of the preset classifier is a softmax layer, the results of multi-classification can be presented in the form of probability. For example, when the similarity between the human body action in the sports video clip and the candidate sports action is 100%, the segment action score of the sports video clip is 1; when the similarity between the human body action in the sports video clip and the candidate sports action is 50%, the segment action score of the sports video clip is 0.5. That is, the segment action score is a value between 0 and 1.
[0029] Each sports video clip will generate a clip action score corresponding to a candidate sports action, and all the clip action scores are combined to form an action score matrix. The clip action scores of all sports video clips corresponding to the same candidate sports action are combined to obtain an action score column vector (one-dimensional column vector). Each action score column vector corresponds to a candidate sports action. All action score column vectors are merged to finally generate an action score matrix.
[0030] In one embodiment, when there is Sports video clips and When the preset classifiers corresponding to the candidate sports actions are used, the segment action scores of the sports video clips corresponding to the candidate sports actions are .in, Indicates Sports video clips in The segment action scores in the preset classifier corresponding to the candidate sports actions, Indicates The preset classifiers corresponding to the candidate sports actions, represents the preset feature extractor based on convolutional neural network, Indicates the first sports video clips. The action score column vector corresponding to the candidate sports actions is expressed as ,in, Indicates Sports video clips in The scores of the segment actions in the preset classifier corresponding to the candidate sports actions. Further, the action score matrix is expressed as ,in, represents the action score matrix, Indicates the sequence number of the sports video clip, represents the total number of sports video clips, Indicates the sequence number of the candidate sports action, Represents the total number of candidate sports actions.
[0031] This embodiment generates an action score matrix based on the scoring of sports video clips, thereby achieving a reasonable and clear display of the clip action scores. The degree of matching between each video clip and each candidate sports action can be intuitively seen, providing strong data support for subsequent action recognition.
[0032] S30. Determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine a set of action matching probabilities of the target sports video matching all the candidate sports actions according to the key frame video segments.
[0033] Understandably, after obtaining the action score matrix, the action score matrix is analyzed, and key frame video segments are determined from all sports video segments corresponding to each candidate sports action. The key frame video segment refers to the segment in the target sports video that best represents a specific candidate sports action, and the segment that best represents the specific candidate sports action is found from all sports video segments by comparing the segment action scores. Each candidate sports action corresponds to a key frame video segment, and each key frame video segment corresponds to a segment action score. At the same time, each segment action score corresponds to a probability. Therefore, the probabilities corresponding to all key frame video segments can form an action matching probability set. That is, the action matching probability set of the target sports video matching all candidate sports actions is determined based on the key frame video segments. The action matching probability set refers to the probability value set of the similarity between the target sports video and each candidate sports action.
[0034] In one embodiment, in step S30, that is, determining the key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, includes: S301, acquiring an action score column vector corresponding to the candidate sports action according to the action score matrix; S302: Sort the segment action scores of all the sports video segments in the same action score column vector, and determine the sports video segment with the highest score as the key frame video segment corresponding to the candidate sports action.
[0035] Understandably, in one embodiment, The action score column vector corresponding to the candidate sports actions is expressed as ,in, Indicates Sports video clips in The segment action scores in the preset classifier corresponding to the candidate sports actions, Indicates Sports video clips in The segment action scores in the preset classifier corresponding to the candidate sports actions. ~ The segment action scores of the sports video segments are sorted from high to low, and the sports video segment with the highest score is determined as the key frame video segment corresponding to the candidate sports action.
[0036] This embodiment sorts and compares the fragment action scores in the action score matrix in units of column vectors, and determines the sports video fragment with the highest score as the key frame video fragment. This can effectively determine the key frame video fragment corresponding to each candidate sports action from a large number of sports video fragments, thereby reducing the difficulty of subsequent data analysis.
[0037] In one embodiment, in step S30, determining a set of action matching probabilities of the target sports video and each of the candidate sports actions according to the key frame video segments includes: S303, obtaining the segment action scores of the key frame video segments corresponding to the candidate sports actions, and converting the segment action scores into action matching probabilities of each candidate sports action matching the target sports video; S304: Generate an action matching probability set according to all the action matching probabilities.
[0038] Understandably, after obtaining the segment action scores of the key frame video segments corresponding to each candidate sports action, the segment action scores are converted into the action matching probabilities of each candidate sports action (the converted action matching probabilities are values between 0% and 100%), and the data set of all action matching probabilities is the action matching probability set. The action matching probability refers to the probability value of the similarity between the target sports video and each candidate sports action.
[0039] In one embodiment, the probability of a candidate sports action matching a target sports video is: ,in, Indicates The action score column vector corresponding to the candidate sports actions, Indicates The maximum value of the segment action score in the action score column vector corresponding to the candidate sports actions. Indicates the sequence number of the candidate sports action, When representing the total number of candidate sports actions, arrive of The action matching probabilities are jointly determined as the action matching probability set.
[0040] This embodiment can quickly determine the action matching probability set of the target sports video based on the segment action scores of the key frame video segments.
[0041] S40 , determining a target sports action of the target sports video from all the candidate sports actions according to the action matching probability set.
[0042] In one embodiment, in step S40, determining the target sports action of the target sports video from all the candidate sports actions according to the action matching probability set includes: S401, sorting the action matching probabilities of all the candidate sports actions in the action matching probability set to obtain the maximum matching probability; S402: Determine the candidate sports action corresponding to the maximum matching probability as the target sports action of the target sports video.
[0043] Understandably, the action matching probabilities of all candidate sports actions in the action matching probability set are sorted from large to small to obtain the maximum matching probability. The maximum matching probability refers to the maximum probability value in the action matching probability set. The target sports action refers to the action type of the target sports video identified by model prediction.
[0044] In one embodiment, the candidate sports action corresponding to the maximum matching probability is determined as the target sports action of the target sports video, which is expressed as .in, a sports action label representing the target sports action, express arrive of The maximum probability among the action matching probabilities.
[0045] After obtaining the maximum match probability, you can also choose to compare the maximum match probability with a preset probability threshold. The preset probability threshold is a pre-set minimum probability critical value used to further determine whether the target sports action can be directly determined, such as 60%. When the maximum match probability is greater than or equal to the preset probability threshold, the candidate sports action corresponding to the maximum candidate action probability is determined as the target sports action of the target sports video. When the maximum match probability is less than the preset probability threshold, it indicates that the action matching probability of all candidate sports actions is less than 60%, and the target sports action cannot be determined. At this time, it is necessary to add a new preset classifier for the candidate sports action to score and re-determine the target sports action; This embodiment obtains a plurality of sports video clips by segmenting the target sports video; obtains preset classifiers corresponding to each candidate sports action, and scores all sports video clips through all preset classifiers to obtain an action score matrix; determines key frame video clips corresponding to each candidate sports action from all sports video clips according to the action score matrix, and determines an action matching probability set of the target sports video and all candidate sports actions according to the key frame video clips; determines the target sports action of the target sports video from all candidate sports actions according to the action matching probability set. This embodiment obtains the action score matrix by segmenting and then scoring the sports video, which makes it difficult to ignore action differences. At the same time, based on the action score matrix, key frame video clips are mined from the plurality of segmented sports video clips, avoiding interference from invalid information of other sports video clips, and finally using the key frame video clips as the main basis for predicting the target sports action, which greatly improves the accuracy of sports video action recognition.
[0046] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0047] In one embodiment, a sports video action recognition device is provided, and the sports video action recognition device corresponds to the sports video action recognition method in the above embodiment. Figure 3 As shown, the sports video action recognition device includes a video segmentation module 10, a segment scoring module 20, a key frame determination module 30 and a target action determination module 40. The functional modules are described in detail as follows: The video segmentation module 10 is used to segment the target sports video to obtain multiple sports video clips; The segment scoring module 20 is used to obtain a preset classifier corresponding to each candidate sports action, and score all the sports video segments using all the preset classifiers to obtain an action score matrix; A key frame determination module 30 is used to determine the key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine the action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; The target action determination module 40 is used to determine the target sports action of the target sports video from all the candidate sports actions according to the action matching probability set.
[0048] In one embodiment, the video segmentation module 10 includes: A sliding window determination unit, used to determine a target sliding window according to a preset number of segmentation frames; The sports video segmentation unit is used to slide the target sliding window on the target sports video to segment the target sports video to obtain at least two sports video segments.
[0049] In one embodiment, the video segmentation module 10 further includes: A tail segment frame number acquisition unit, used to determine a tail video segment from at least two of the sports video segments, and acquire the tail segment frame number of the tail video segment; the tail video segment is a sports video segment obtained by segmenting when the target sliding window is slid on the target sports video for the last time; A tail segment frame number determination unit, used to determine whether the tail segment frame number is consistent with the preset segmentation frame number; The tail segment removing unit is used to remove the tail video segment from at least two of the sports video segments when the frame number of the tail segment is inconsistent with the preset segmentation frame number.
[0050] In one embodiment, the segment scoring module 20 includes: A feature extraction unit, configured to extract features from the sports video clips by using a preset feature extractor to obtain video clip features of each of the sports video clips; A classifier scoring unit, used to score the features of each video clip by using a preset classifier corresponding to each candidate sports action, to obtain a clip action score of each sports video clip corresponding to each candidate sports action; An action score column vector generating unit, used for obtaining the segment action scores of all the sports video segments corresponding to the same candidate sports action, and obtaining an action score column vector corresponding to the candidate sports action; The action score matrix generating unit is used to merge all the action score column vectors to generate an action score matrix.
[0051] In one embodiment, the key frame determination module 30 includes: An action score column vector acquisition unit, used for acquiring an action score column vector corresponding to the candidate sports action according to the action score matrix; The key frame video segment determining unit is used to sort the segment action scores of all the sports video segments in the same action score column vector, and determine the sports video segment with the highest score as the key frame video segment corresponding to the candidate sports action.
[0052] In one embodiment, the key frame determination module 30 further includes: An action matching probability acquisition unit, used to acquire a segment action score of a key frame video segment corresponding to each of the candidate sports actions, and convert the segment action score into an action matching probability of each of the candidate sports actions matching the target sports video; A probability set generating unit is used to generate an action matching probability set according to all the action matching probabilities.
[0053] In one embodiment, the target action determination module 40 includes: A maximum matching probability determining unit, used for sorting the action matching probabilities of all the candidate sports actions in the action matching probability set to obtain a maximum matching probability; The target sports action determination unit is used to determine the candidate sports action corresponding to the maximum matching probability as the target sports action of the target sports video.
[0054] The specific definition of the sports video action recognition device can be found in the above definition of the sports video action recognition method, which will not be repeated here. The various modules in the above sports video action recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0055] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the sports video action recognition method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a sports video action recognition method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
[0056] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor executes the computer-readable instructions to implement the following steps: Segment the target sports video to obtain multiple sports video clips; Obtaining a preset classifier corresponding to each candidate sports action, and scoring all the sports video clips using all the preset classifiers to obtain an action score matrix; Determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; A target sports action in the target sports video is determined from all the candidate sports actions according to the action matching probability set.
[0057] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media store computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the following steps are implemented: Segment the target sports video to obtain multiple sports video clips; Obtaining a preset classifier corresponding to each candidate sports action, and scoring all the sports video clips using all the preset classifiers to obtain an action score matrix; Determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; A target sports action in the target sports video is determined from all the candidate sports actions according to the action matching probability set.
[0058] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0059] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0060] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A sports video action recognition method, characterized in that: include: Segment the target sports video to obtain multiple sports video clips; Obtaining a preset classifier corresponding to each candidate sports action, and scoring all the sports video clips using all the preset classifiers to obtain an action score matrix; Determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; A target sports action in the target sports video is determined from all the candidate sports actions according to the action matching probability set.
2. The sports video action recognition method according to claim 1, characterized in that: The target sports video is segmented to obtain a plurality of sports video clips, including: Determine the target sliding window according to the preset number of segmentation frames; The target sliding window is slid on the target sports video to segment the target sports video to obtain at least two sports video segments.
3. The sports video action recognition method as claimed in claim 2, characterized in that: After the target sports video is segmented to obtain at least two sports video segments, the method further includes: Determine a tail video segment from at least two of the sports video segments, and obtain the tail segment frame number of the tail video segment; the tail video segment is a sports video segment obtained by segmenting when the target sliding window is slid on the target sports video for the last time; Determine whether the number of frames of the tail segment is consistent with the preset number of segmentation frames; When the frame number of the tail segment is inconsistent with the preset segmentation frame number, the tail video segment is removed from at least two of the sports video segments.
4. The sports video action recognition method according to claim 1, characterized in that: The step of scoring all the sports video clips by using all the preset classifiers to obtain an action score matrix includes: Extracting features from the sports video clips by using a preset feature extractor to obtain video clip features of each of the sports video clips; Scoring the features of each video clip by using a preset classifier corresponding to each candidate sports action, to obtain a clip action score of each sports video clip corresponding to each candidate sports action; Obtaining the segment action scores of all the sports video segments corresponding to the same candidate sports action, and obtaining an action score column vector corresponding to the candidate sports action; All the action score column vectors are merged to generate an action score matrix.
5. The sports video action recognition method as claimed in claim 4, characterized in that: The step of determining the key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix comprises: Acquire an action score column vector corresponding to the candidate sports action according to the action score matrix; The segment action scores of all the sports video segments in the same action score column vector are sorted, and the sports video segment with the highest score is determined as the key frame video segment corresponding to the candidate sports action.
6. The sports video action recognition method according to claim 1, characterized in that: The step of determining a set of action matching probabilities of the target sports video and each of the candidate sports actions according to the key frame video segments includes: Obtaining a segment action score of a key frame video segment corresponding to each of the candidate sports actions, and converting the segment action score into an action matching probability of each of the candidate sports actions matching the target sports video; An action matching probability set is generated according to all the action matching probabilities.
7. The sports video action recognition method according to claim 1, characterized in that: The step of determining the target sports action in the target sports video from all the candidate sports actions according to the action matching probability set includes: Sorting the action matching probabilities of all the candidate sports actions in the action matching probability set to obtain the maximum matching probability; The candidate sports action corresponding to the maximum matching probability is determined as the target sports action of the target sports video.
8. A sports video action recognition device, characterized in that: include: A video segmentation module is used to segment the target sports video to obtain multiple sports video clips; A clip scoring module is used to obtain a preset classifier corresponding to each candidate sports action, and score all the sports video clips using all the preset classifiers to obtain an action score matrix; A key frame determination module, configured to determine key frame video segments corresponding to each of the candidate sports actions from all the sports video segments according to the action score matrix, and determine an action matching probability set of the target sports video matching all the candidate sports actions according to the key frame video segments; The target action determination module is used to determine the target sports action of the target sports video from all the candidate sports actions according to the action matching probability set.
9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the sports video action recognition method as described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the sports video action recognition method as described in any one of claims 1 to 7.