A sports video action positioning method, system and device based on deep learning

Abstract

This invention provides a method, system, and device for locating sports video actions based on deep learning, relating to the field of video analysis. The method comprises: acquiring sports game video data to generate an image data set; inputting the image data set into a pretrained video understanding deep learning network to learn video action features at each moment, and integrating the video action features from multiple consecutive moments into a complete video action feature; inputting the complete video action feature into a multi-scale refinement network based on a feature pyramid network to obtain time-point-based multi-scale offset information; combining the multi-scale offset information with the action localization results using an offset focusing strategy to obtain a refined action localization result; and performing a positioning error analysis based on the refined action localization result. This invention can achieve high-quality localization of sports video actions.

Description

Technical Field

[0001] The present invention relates to the field of video analysis, and in particular to a method, system and device for locating sports video actions based on deep learning. Background Art

[0002] Sports video analysis is a field of study that focuses on extracting and analyzing information about athletes and games from recorded sports games. Through in-depth analysis of sports videos, valuable insights can be obtained regarding player movements, tactics, strategies, and technical indicators. This information can be applied to training guidance, game decision-making, and event broadcasting.

[0003] In the field of sports video analysis, efficiently locating and identifying athlete movements is a challenging task. Deep learning-based video action localization methods utilize deep neural network models to automatically learn and extract complex video action features from raw video data to obtain localization results, greatly improving analysis efficiency.

[0004] However, practical applications still present several challenges. First, due to interference from preparatory and finishing movements, the start and end times of sports movements can easily deviate. Second, rapid movements and changes in perspective increase the difficulty of action localization. Third, different sports movements have different motion characteristics and contextual information, requiring flexible extraction of key information to reduce errors in action localization. Summary of the Invention

[0005] The purpose of the present invention is to provide a sports video action positioning method, system and device based on deep learning to achieve high-quality positioning of sports video actions.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A sports video action localization method based on deep learning, comprising:

[0008] Acquire sports game video data and generate image data sets;

[0009] Inputting the image data set into a pre-trained video understanding deep learning network to learn the video action features at each moment, and integrating the video action features of multiple consecutive moments into a complete video action feature;

[0010] The complete video action features are input into the multi-scale refinement network based on the feature pyramid network to obtain multi-scale offset information based on time points;

[0011] By using an offset focusing strategy, the multi-scale offset information is combined with the action localization result to obtain a refined action localization result;

[0012] A positioning error analysis is performed based on the refined motion positioning result.

[0013] Optionally, obtaining sports game video data and generating an image data set may include:

[0014] Each frame of the video image of the sports game video data is normalized to obtain a continuous image data set.

[0015] Optionally, the image data set is input into a pre-trained video understanding deep learning network to learn the video action features at each moment, and the video action features at multiple consecutive moments are integrated into a complete video action feature, specifically including:

[0016] Inputting continuous images from the image data set into the pre-trained video understanding deep learning network, and extracting video segment features according to a set sliding window; the pre-trained video understanding deep learning network is a video understanding network; the video segment features are video action features at each moment;

[0017] Integrate video action features of multiple consecutive moments into a complete video action feature.

[0018] Optionally, the complete video action features are input into a multi-scale refinement network based on a feature pyramid network to obtain multi-scale offset information based on a time point, specifically including: the multi-scale refinement network based on a feature pyramid network includes a feature pyramid network based on a convolution module and a multi-layer refinement module;

[0019] The complete video action features are extracted using a feature pyramid network based on a convolutional module and encoded into pyramid features; the complete video action is a multi-scale video action feature;

[0020] The pyramid features are decoded layer by layer using a multi-layer refinement module to obtain multi-scale offset information based on time points; the multi-scale offset information includes: boundary refinement information, fractional refinement information and boundary probability information.

[0021] Optionally, the multi-scale offset information is combined with the action localization result through an offset focus strategy to obtain a refined action localization result, specifically including:

[0022] Based on the offset focus strategy, the top-level information whose receptive field is lower than the set receptive field threshold is used to perform a rough offset adjustment on the action localization result to obtain a roughly adjusted action localization result;

[0023] The roughly adjusted action localization result is shifted layer by layer to achieve a refinement process from coarse to fine, thereby obtaining a refined action localization result.

[0024] Optionally, the offset focus strategy is to map specific position information of the action positioning result to a specific time point, thereby obtaining multi-scale offset information at the specific time point.

[0025] A sports video action positioning system based on deep learning, comprising:

[0026] An image data set generation module is used to obtain sports game video data and generate an image data set;

[0027] A complete video action feature integration module is used to input the image data set into a pre-trained video understanding deep learning network to learn the video action features at each moment and integrate the video action features of multiple consecutive moments into a complete video action feature;

[0028] The multi-scale offset information generation module is used to input the complete video action features into the multi-scale refinement network based on the feature pyramid network to obtain multi-scale offset information based on the time point;

[0029] A refined action localization result generation module is used to combine the multi-scale offset information with the action localization result through an offset focus strategy to obtain a refined action localization result;

[0030] The positioning error analysis module is used to perform positioning error analysis based on the refined motion positioning result.

[0031] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned sports video action positioning method based on deep learning.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned sports video action positioning method based on deep learning.

[0033] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: using a pre-trained video understanding network to extract sports video action features, based on the complete video feature sequence, applying a multi-scale refinement network based on a feature pyramid network to predict offset information, and combining the offset information with the positioning results according to the offset focusing strategy to achieve high-quality positioning of sports video actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flow chart of the sports video action positioning method based on deep learning provided by the present invention;

[0036] Figure 2 It is a structural diagram of a multi-scale refinement network based on a feature pyramid network provided by the present invention;

[0037] Figure 3 This is a flow chart of the offset focus strategy algorithm provided by the present invention;

[0038] Figure 4 This is the action positioning refinement result diagram provided by the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] The purpose of the present invention is to provide a sports video action positioning method, system and device based on deep learning, which can achieve high-quality positioning of sports video actions.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Example 1

[0043] like Figure 1 As shown, the present invention provides a sports video action positioning method based on deep learning, comprising:

[0044] Step 101: Acquire sports game video data and generate an image data set.

[0045] In practical applications, video acquisition equipment is used to obtain sports game video data; each frame of the video image in the sports game video data is normalized to [0,1], and a continuous image data set I={ , ,…, ,…,}, there are n images in total, For the nth image, t∈n, n=1, 2, 3, ....

[0046] Step 102: Input the image data set into a pre-trained video understanding deep learning network to learn the video action features at each moment, and integrate the video action features of multiple consecutive moments into a complete video action feature.

[0047] In practical applications, a classic video understanding network (Inflated 3D ConvNet, I3D) is used. Its structure mainly includes 3D global average pooling, fully connected layers, and 2D and 3D convolution kernels. After pre-training on the large-scale video classification dataset Kinetics, the I3D model parameters are frozen and used to extract video action features.

[0048] The input of I3D is a continuous image from the image data set I, denoted as , the output is the extracted video clip features ( ).

[0049] Specifically, for the complete I, a sliding window setting with a window size of 16 frames and a step size of 4 frames is used, which is divided into a series of segments { , ,…, ,…, }, where m is the total number of fragments, m=n / 4, fragment ={ , ,…, }, For the images, For the images, For the images, fragments ={ , ,…, }, For the images, For the images, For the Images, and images exceeding I are filled with 0. Apply I3D to extract video clip features one by one, and obtain the complete video action feature F={ , ,…, ,…, }, extracted video clip features Depend on ( ) calculated.

[0050] Step 103: Input the complete video action features into a multi-scale refinement network based on a feature pyramid network to obtain multi-scale offset information based on time points.

[0051] In practical applications, the network structure of the multi-scale refinement network RefineTAD based on the feature pyramid network is as follows: Figure 2 As shown, RefineTAD includes a feature pyramid network based on convolutional modules (i.e. Figure 2 The feature pyramid module in

[15] and the multi-layer refinement module.

[0052] RefineTAD is specifically designed to learn multi-scale offset information to assist in adjusting positioning results and achieve refined positioning. It also uses a time-point-based model. Compared to traditional methods based on positioning result refinement, it can better learn action boundary information, providing flexibility and efficiency.

[0053] RefineTAD uses a feature pyramid module based on a convolutional module as an encoder. The feature pyramid network based on the convolutional module receives the complete video action feature F as input, efficiently encodes F into a pyramid feature P and outputs it, as shown in the following example: Figure 2 As shown in the feature pyramid module, the input video action feature F, the feature pyramid module first applies convolution, normalization and other operations to obtain the first layer of pyramid features ={ , ,…, For example, is the mth feature of the first layer of pyramid features. Apply strided convolution to achieve feature downsampling and obtain the second pyramid feature ={ , ,…, }, is the m / 2th feature of the second pyramid feature. Similarly, let the number of pyramid layers L=6, and get the pyramid feature P={ , ,…, The downsampling operation enables each layer of the feature pyramid to have a different receptive field. The lower-layer features have more underlying details, while the upper-layer features have richer overall information. Therefore, it can flexibly learn action instances of different sizes and cope with different degrees of positioning errors.

[0054] RefineTAD uses a multi-layer refinement module as a decoder, inputs the pyramid feature P, decodes it into a pyramid-shaped time-point-based offset information D and outputs it. Figure 2 As shown in the multi-layer refinement module, the multi-layer refinement module includes three parallel prediction heads, whose internal structure is a simple convolutional neural network, which is used to encode the pyramid feature P into offset information. For example, for the i-th layer pyramid feature The jth feature of , which will be encoded as boundary refinement information ={ , }、Score refinement information ={ , } and boundary probability information ={ , },in, Refine the starting boundary information of the video at the jth time point on the i-th layer, Refine the ending boundary information of the video at the jth time point on the i-th layer. Similarly, To start the score refinement information, To refine the information for the end score, is the starting boundary probability information, is the end boundary probability information, s and e represent the offset prediction of the start frame and the end frame respectively. Specifically, Belongs to [-1,1], indicating the relative distance from the correct action boundary; It is the action category probability vector, the internal elements represent the probability of the corresponding category, after applying the sigmoid operation, it belongs to [0,1], and the sum is 1; Belongs to [0,1], indicating the probability that there is an action boundary within the receptive field.

[0055] Step 104: Combining the multi-scale offset information with the action localization result through an offset focus strategy to obtain a refined action localization result.

[0056] In practical applications, the multi-scale offset information output by RefineTAD is combined with the action localization results obtained by the traditional temporal action localization model through the offset focusing strategy to obtain the refined action localization results.

[0057] Apply the traditional temporal action localization model to predict the action localization results. Specifically, the complete video action features obtained in step 102 are input to the traditional temporal action localization model, and a set of predicted action instances is output, which is called the action localization result. For each action instance, the start frame , end frame , action category id and its score .

[0058] The input of the offset focus strategy is the action localization result and the multi-scale offset information predicted in step 103, and the output is the refined action localization result. Figure 3 As shown, the positioning result and offset information are obtained. The offset information has L layers, and i=L. It is determined whether the action length is greater than twice the offset range of the i-th layer. If so, the index of the corresponding offset information is calculated according to the positioning result, and the corresponding offset information is obtained. The offset information is combined with the positioning result to obtain a refined positioning result (i.e., the refined action positioning result). Let i=i-1 and determine whether i is equal to 0. If so, the process ends. If not, re-determine whether the action length is greater than twice the offset range of the i-th layer. If the action length is not greater than twice the offset range of the i-th layer, let i=i-1 and continue to determine whether i is equal to 0.

[0059] For an input action instance, its size needs to be filtered first, because the upper layer's offset information has a larger receptive field, which is not conducive to the fine positioning of short-term actions. Then, based on the positioning frame (the start / end frame operation is the same), the index of its corresponding offset information can be calculated. Since the index is discrete, the interpolation algorithm is used here to obtain the final offset information { , , }, To refine the boundary information, To refine the information for the score, is the boundary probability information, and t represents the interpolated offset information obtained based on the positioning frame t; by combining the offset information with the action instance, the refined action positioning result can be obtained.

[0060] The combination is as follows: First, , i represents the current layer number, offset the positioning frame t; then let ,in, is the score offset weight, k is the number of categories, is the action category number, the probability vector Perform sigmoid operation, Indicates that the final score of the corresponding category is obtained according to the id, minus make Belongs to [-1 / k,1-1 / k]; finally, let ), adjust the action category score. The above process is repeated L times, using the pyramid property of the offset information (L layers), from the top layer downwards, to achieve a gradual refinement from coarse to fine, and the action positioning refinement result is as follows Figure 4 shown.

[0061] Step 105: Perform positioning error analysis based on the refined motion positioning result.

[0062] In practical applications, the positioning target is recorded as a collection of real action instances, each of which includes a start frame, an end frame, and action classification information. Average positioning error analysis is performed based on the predicted refined action localization results. The positioning target is a series of actions in manually annotated videos, which is the target learned by a multi-scale refinement network based on a feature pyramid network. The refined action localization result is the output of the multi-scale refinement network based on the feature pyramid network, which is a prediction of the action instance. The closer the action localization result matches the positioning target, the better the model performance.

[0063] The expected mean average precision (mAP) was selected as the evaluation metric. The intersection over union (tIoU) between the action localization results and the ground-truth action instances was calculated. Results with tIoU greater than a set threshold were considered positive samples. Here, tIoU = {0.3, 0.4, 0.5, 0.6, 0.7} was set. Precision and recall were calculated to obtain the precision-recall curve, and the area under the curve (AP) was calculated. The average AP across all action categories was calculated to obtain mAP.

[0064] By calculating the mAP of the positioning results before and after refinement, the effectiveness of the present invention can be evaluated and further improvements can be made.

[0065] Example 2

[0066] In order to execute the method corresponding to the above-mentioned embodiment 1 and achieve the corresponding functions and technical effects, a sports video action positioning system based on deep learning is provided below.

[0067] A sports video action positioning system based on deep learning, comprising:

[0068] The image data set generation module is used to obtain sports game video data and generate an image data set.

[0069] A complete video action feature integration module is used to input the image data set into a pre-trained video understanding deep learning network, learn the video action features at each moment, and integrate the video action features of multiple consecutive moments into a complete video action feature.

[0070] The multi-scale offset information generation module is used to input the complete video action features into the multi-scale refinement network based on the feature pyramid network to obtain multi-scale offset information based on time points.

[0071] The refined action localization result generation module is used to combine the multi-scale offset information with the action localization result through an offset focus strategy to obtain a refined action localization result.

[0072] The positioning error analysis module is used to perform positioning error analysis based on the refined motion positioning result.

[0073] Example 3

[0074] An embodiment of the present invention provides an electronic device including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the sports video action positioning method based on deep learning provided in Example 1.

[0075] In practical applications, the above-mentioned electronic device may be a server.

[0076] In practical applications, an electronic device includes at least one processor, a memory, a bus, and a communications interface.

[0077] Wherein: the processor, the communication interface, and the memory communicate with each other via a communication bus.

[0078] Communication interface, used to communicate with other devices.

[0079] The processor is used to execute the program, and specifically can execute the method described in the above embodiment.

[0080] Specifically, the program may include program codes including computer operation instructions.

[0081] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in an electronic device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0082] Memory, used to store programs. Memory may include high-speed RAM (RAM) or non-volatile memory, such as at least one disk drive.

[0083] Based on the description of the above embodiments, the present invention provides a storage medium on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method described in any embodiment.

[0084] The sports video action positioning system based on deep learning provided in the embodiments of the present application exists in various forms, including but not limited to:

[0085] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and their primary purpose is to provide voice and data communications. These terminals include smartphones (such as iPhones), multimedia phones, feature phones, and low-end phones.

[0086] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access capabilities. These terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.

[0087] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0088] (4) Other electronic devices with data interaction functions.

[0089] Thus far, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0090] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0091] For ease of description, the above devices are described in terms of their functions and are divided into various units. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware. Those skilled in the art will appreciate that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0095] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0096] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0097] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM),

[0098] Digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, disk storage, or other magnetic storage devices

[0099] Or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media (transitory media) such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0101] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0102] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0103] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A sports video action positioning method based on deep learning, characterized in that: include: Acquire sports game video data and generate image data sets; Inputting the image data set into a pre-trained video understanding deep learning network to learn the video action features at each moment, and integrating the video action features of multiple consecutive moments into a complete video action feature; Inputting the complete video action features into a multi-scale refinement network based on a feature pyramid network to obtain multi-scale offset information based on time points, specifically including: the multi-scale refinement network based on the feature pyramid network includes a feature pyramid network based on a convolution module and a multi-layer refinement module; The complete video action features are extracted using a feature pyramid network based on a convolutional module and encoded into pyramid features; the complete video action is a multi-scale video action feature; Decoding the pyramid features layer by layer using a multi-layer refinement module to obtain multi-scale offset information based on time points; the multi-scale offset information includes: boundary refinement information, score refinement information, and boundary probability information; By using an offset focusing strategy, the multi-scale offset information is combined with the action localization result to obtain a refined action localization result; A positioning error analysis is performed based on the refined motion positioning result.

2. The sports video action positioning method based on deep learning according to claim 1 is characterized in that Obtain sports game video data and generate image data sets, including: Each frame of the video image of the sports game video data is normalized to obtain a continuous image data set.

3. The sports video action positioning method based on deep learning according to claim 1, characterized in that The image data set is input into a pre-trained video understanding deep learning network to learn the video action features at each moment, and integrate the video action features of multiple consecutive moments into a complete video action feature, specifically including: Inputting continuous images from the image data set into the pre-trained video understanding deep learning network, and extracting video segment features according to a set sliding window; the pre-trained video understanding deep learning network is a video understanding network; the video segment features are video action features at each moment; Integrate video action features of multiple consecutive moments into a complete video action feature.

4. The sports video action positioning method based on deep learning according to claim 1, characterized in that The multi-scale offset information is combined with the action localization result through the offset focus strategy to obtain a refined action localization result, specifically including: Based on the offset focus strategy, the top-level information whose receptive field is lower than the set receptive field threshold is used to perform a rough offset adjustment on the action localization result to obtain a roughly adjusted action localization result; The roughly adjusted action localization result is shifted layer by layer to achieve a refinement process from coarse to fine, thereby obtaining a refined action localization result.

5. The method for positioning sports video action based on deep learning according to claim 4, characterized in that: The offset focusing strategy is to map the specific position information of the action positioning result to a specific time point, thereby obtaining multi-scale offset information at the specific time point.

6. A sports video action positioning system based on deep learning, characterized in that: The sports video action positioning system based on deep learning adopts the sports video action positioning method based on deep learning according to any one of claims 1 to 5, and the sports video action positioning system based on deep learning includes: An image data set generation module is used to obtain sports game video data and generate an image data set; A complete video action feature integration module is used to input the image data set into a pre-trained video understanding deep learning network to learn the video action features at each moment and integrate the video action features of multiple consecutive moments into a complete video action feature; The multi-scale offset information generation module is used to input the complete video action features into the multi-scale refinement network based on the feature pyramid network to obtain multi-scale offset information based on the time point; A refined action localization result generation module is used to combine the multi-scale offset information with the action localization result through an offset focus strategy to obtain a refined action localization result; The positioning error analysis module is used to perform positioning error analysis based on the refined motion positioning result.

7. An electronic device, characterized in that: It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the sports video action positioning method based on deep learning as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that It stores a computer program, which, when executed by a processor, implements the sports video action positioning method based on deep learning as described in any one of claims 1 to 5.

Images