Unsupervised power video action positioning method, system and equipment and storage medium

Abstract

The invention relates to the technical field of video understanding, and particularly discloses an unsupervised power video action positioning method, system and device and a storage medium, and the method comprises the steps: constructing a video action positioning model comprising a to-be-trained teacher model and a to-be-trained student model for power video action positioning; using a teacher model to be trained to obtain video overall characteristics of a sample power video and performing spectral clustering to obtain a clustering pseudo-label text of the sample power video, and combining the confidence coefficient and the multi-mode large model to obtain a target pseudo-label video set and a non-label video set; training a video action positioning model by using the target pseudo-tag video set and the non-tag video set until the maximum number of iterations is reached, and obtaining a trained video action positioning model; and inputting a to-be-detected power video into the trained video action positioning model to obtain a video action positioning result. According to the invention, efficient and steady unsupervised power video action positioning can be realized.

Description

Background Art

[0002] In recent years, with the increasing emphasis on operational safety in the power industry and the rapid development of video surveillance technology, video surveillance has become a key tool for power operation safety monitoring. Especially at power operation sites, video has become an important medium for monitoring operational safety due to its intuitiveness and real-time nature. Faced with massive amounts of video data, how to quickly and accurately locate and identify actions with specific safety risks has become a key issue in power operation safety monitoring. Therefore, the application of video motion positioning technology in power operation safety monitoring has emerged.

[0003] Video action localization technology is oriented to a set of predefined safety action categories, and can realize accurate automatic localization of all relevant actions in the video and precise identification of the corresponding action categories. For example, given a predefined set of action categories (including safety-critical actions in power operations such as "not wearing a helmet", "entering a dangerous area by mistake", and "illegal operation of electrical equipment"), this technology can accurately locate and identify the time interval (starting at 5.5 seconds and ending at 12.3 seconds) where the action "not wearing a helmet" occurs in the video. However, existing video action localization technologies usually rely on full supervision conditions or weak supervision conditions to train action localization systems, that is, it is necessary to annotate all action segment categories and start and end times in the training video or annotate the action categories contained in the video. Both methods require watching all training videos and carefully annotating them, which results in a huge annotation cost and is not suitable for the current growing video scale. In contrast, video intelligent action localization under unsupervised settings only relies on the annotation information of the total number of predefined action categories in the entire video set, which can train a powerful video action localization system and has a wider application prospect.

[0004] The study of video action localization under unsupervised settings not only deepens the research of traditional video action localization, but also has stronger scalability and can cope with the growing scale of video data. At present, this research mainly faces the following three challenges:

[0005] ① Accurate overall video feature representation. Previous algorithms were simply based on the weighted sum of all video frame-level features and video attention weights, which resulted in inaccurate overall video feature representation and affected the video’s clustering effect on action categories. Therefore, how to improve the overall video feature representation is crucial for video-level pseudo-label generation.

[0006] ② Strong and robust positioning network training. Existing unsupervised video action positioning models directly use all samples for iterative “clustering-positioning” training. However, considering the unreliability of video-level pseudo-labels, using all samples for training may cause the model to learn incorrect action representations, damaging the attention generation used for clustering and the current network reasoning performance, thereby causing the transmission and accumulation of erroneous information in the iterative training process. Therefore, it is also challenging to reduce the negative impact of low-confidence pseudo-labels during iterative training, better model the temporal information perception of videos in the absence of labels, and achieve strong and robust positioning network training.

[0007] ③ Watching the video is unavoidable. Existing unsupervised video action localization methods usually use clustering to generate pseudo labels for videos, but such pseudo labels can only be used for fully supervised model training. Because the labels generated by this process are category labels such as 1, 2, 3..., and the specific category information of "not wearing a safety helmet", "entering a dangerous area by mistake", and "illegal operation of electrical equipment" cannot be known. In this way, during the test process, these methods can only give the timestamps of abnormal behavior fragments in the video and category labels such as "1, 2, 3". If you want to get the specific behavior category, you still need to watch the videos in the "1, 2, 3" clusters and obtain their specific behavior categories in order to achieve the correspondence between the cluster index labels and the specific behavior categories.

[0008] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the invention

[0009] To solve the above technical problems, the present invention provides an unsupervised power video action positioning method, system, device and storage medium.

[0010] In a first aspect, the present invention provides an unsupervised power video action positioning method, and the technical solution of the method is as follows:

[0011] S1. Constructing a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization;

[0012] S2. Using the to-be-trained teacher model corresponding to the current iteration round, obtain the high-action selective attention distribution corresponding to each original sample power video, and obtain the overall video features corresponding to each original sample power video based on the high-action selective attention distribution and video clip features corresponding to each original sample power video;

[0013] S3, performing spectral clustering on the overall video features corresponding to each original sample power video to obtain a clustered pseudo-label text of each original sample power video, determining an original sample power video whose confidence is not less than a target threshold as a first sample power video, and determining an original sample power video whose confidence is less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding clustered pseudo-label text and an unlabeled video set including each second sample power video;

[0014] S4, using the multimodal large model, obtaining the initial pseudo-label text of the action category of each first sample power video, and determining the target pseudo-label text of the action category of each first sample power video according to the clustered pseudo-label text and the initial pseudo-label text of the action category of each first sample power video, so as to obtain a target pseudo-label video set containing all the first sample power videos;

[0015] S5, using the target pseudo-label video set and the unlabeled video set to train the video action localization model, to obtain a target teacher model and a target student model corresponding to the current iteration round;

[0016] S6, determining the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determining the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action localization model;

[0017] S7, repeatedly executing S2-S6 until the maximum number of iterations is reached, determining the updated video action localization model as the trained video action localization model;

[0018] S8. Input the power video to be detected into the trained video motion localization model to obtain the video motion localization result of the power video to be detected.

[0019] The beneficial effects of the unsupervised power video action positioning method of the present invention are as follows:

[0020] The method of the present invention can improve the overall positioning performance and realize efficient and robust unsupervised power video action positioning by constructing a consistent unsupervised video action positioning model guided by a multimodal large model.

[0021] On the basis of the above scheme, the unsupervised power video action localization method of the present invention can also be improved as follows.

[0022] In an optional manner, the step of obtaining a high-action selective attention distribution corresponding to any original sample power video using the to-be-trained teacher model corresponding to the current iteration round includes:

[0023] Using the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branches and the action category-irrelevant attention weights of the action category-irrelevant branches corresponding to any original sample power video containing multiple video clips are obtained and fused to obtain the action-perceived attention distribution corresponding to any original sample power video;

[0024] The action-perceived attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-action selective attention distribution corresponding to the current iteration round.

[0025] In an optional manner, the step of obtaining the overall video features corresponding to any original sample power video according to the high-action selective attention distribution and video clip features corresponding to any original sample power video includes:

[0026] According to the video segment features corresponding to each video segment in any of the original sample power videos and the high-action selective attention distribution, the overall video features corresponding to any of the original sample power videos are obtained.

[0027] In an optional manner, the step of performing spectral clustering on the overall video features corresponding to each original sample power video to obtain clustered pseudo-label text of each original sample power video includes:

[0028] Calculate and construct an affinity matrix corresponding to all original sample power videos based on the affinity between every two original sample power videos;

[0029] The affinity matrix is ​​clustered using a spectral clustering algorithm to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain cluster pseudo-label text of each original sample power video.

[0030] In an optional manner, the step of determining the confidence of any original sample power video includes:

[0031] According to the affinity matrix, the intra-cluster compactness and inter-cluster separation of any original sample power video are calculated, and according to the intra-cluster compactness and inter-cluster separation of any original sample power video, the confidence of any original sample power video is determined.

[0032] In an optional manner, the step of determining the action category target pseudo label text of each first sample power video according to the clustered pseudo label text and the action category initial pseudo label text of each first sample power video includes:

[0033] The number of occurrences of the initial pseudo-label text of each action category in any cluster class is determined respectively, and the initial pseudo-label text of the action category with the largest number of occurrences in any cluster class is determined as the action category target pseudo-label text of each first sample power video in any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

[0034] In an optional manner, the step of training the video action localization model using the target pseudo-label video set and the unlabeled video set to obtain the target teacher model and the target student model corresponding to the current iteration round includes:

[0035] The target pseudo-label video set is used to train the teacher model to be trained corresponding to the current iteration round, and the unlabeled video set and the target pseudo-label video set after data perturbation are used to train the student model to be trained corresponding to the current iteration round, so as to obtain the target teacher model and target student model corresponding to the current iteration round; wherein the model parameters of the teacher model to be trained corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained corresponding to the current iteration round.

[0036] In a second aspect, the present invention provides an unsupervised power video action positioning system, the technical solution of which is as follows:

[0037] It includes: a construction module, a first training module, a second training module, a third training module, a fourth training module, a fifth training module, an iterative training module and a detection module;

[0038] The construction module is used to: construct a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization;

[0039] The first training module is used to: use the to-be-trained teacher model corresponding to the current iteration round to obtain the high-action selective attention distribution corresponding to each original sample power video, and obtain the overall video feature corresponding to each original sample power video according to the high-action selective attention distribution and video clip features corresponding to each original sample power video;

[0040] The second training module is used to: perform spectral clustering on the overall video features corresponding to each original sample power video to obtain a cluster pseudo-label text of each original sample power video, determine the original sample power video with a confidence not less than a target threshold as a first sample power video, and determine the original sample power video with a confidence less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding cluster pseudo-label text and an unlabeled video set including each second sample power video;

[0041] The third training module is used to: use the multimodal large model to obtain the initial pseudo-label text of the action category of each first sample power video, and determine the target pseudo-label text of the action category of each first sample power video according to the clustered pseudo-label text and the initial pseudo-label text of the action category of each first sample power video, so as to obtain a target pseudo-label video set containing all the first sample power videos;

[0042] The fourth training module is used to: train the video action localization model using the target pseudo-label video set and the unlabeled video set to obtain the target teacher model and the target student model corresponding to the current iteration round;

[0043] The fifth training module is used to: determine the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action positioning model;

[0044] The iterative training module is used to: repeatedly call the first training module to the fifth training module until a maximum number of iterations is reached, and determine the updated video action localization model as a trained video action localization model;

[0045] The detection module is used to: input the power video to be detected into the trained video action positioning model to obtain the video action positioning result of the power video to be detected.

[0046] The beneficial effects of the unsupervised power video action positioning system of the present invention are as follows:

[0047] The system of the present invention can improve the overall positioning performance and realize efficient and robust unsupervised power video action positioning by constructing a consistent unsupervised video action positioning model guided by a multimodal large model.

[0048] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0049] The invention comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the unsupervised power video action positioning method of the present invention are implemented.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0051] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the unsupervised power video action localization method of the present invention.

[0052] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0054] Figure 1 A schematic diagram of a flow chart of an embodiment of an unsupervised power video action localization method of the present invention;

[0055] Figure 2 This is a schematic diagram of the structure of the video action localization model;

[0056] Figure 3 It is a schematic diagram of the overall principle of this embodiment;

[0057] Figure 4 It is a structural schematic diagram of an embodiment of an unsupervised power video action positioning system of the present invention;

[0058] Figure 5 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0059] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0060] Figure 1A flow chart of an embodiment of an unsupervised power video motion positioning method provided by the present invention is shown, and the unsupervised power video motion positioning method can be executed by an electronic device such as a terminal device or a server. Among them, the terminal device can be any fixed or mobile terminal such as a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the unsupervised power video motion positioning method by means of a processor calling computer-readable instructions stored in a memory. Figure 1 As shown, the following steps are included:

[0061] S1. Construct a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization.

[0062] Among them, Figure 2 As shown in FIG. 1 , the structure of the teacher model to be trained is consistent with that of the student model to be trained. The video action localization model defaults to a consistent video action localization model.

[0063] S2. Using the teacher model to be trained corresponding to the current iteration round, obtain the high-action selective attention distribution corresponding to each original sample power video, and obtain the overall video features corresponding to each original sample power video based on the high-action selective attention distribution and video clip features corresponding to each original sample power video.

[0064] The current iteration round is any iteration round in the model iteration training process (1≤k≤m, k is the number of iterations corresponding to the current iteration round, and m is the maximum number of iterations). Each original sample power video contains multiple video clips. In this embodiment, the original sample power video is segmented with 16 frames as the minimum unit to obtain T video clips (T is a positive integer).

[0065] S3. Perform spectral clustering on the overall video features corresponding to each original sample power video to obtain a clustered pseudo-label text for each original sample power video, determine the original sample power video whose confidence is not less than a target threshold as a first sample power video, and determine the original sample power video whose confidence is less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding clustered pseudo-label text and an unlabeled video set including each second sample power video.

[0066] The cluster pseudo-label text is the number after clustering, such as "1, 2, 3", etc. The target threshold can be set according to the actual situation and is not limited here.

[0067] S4. Utilize the multimodal large model to obtain the initial pseudo-label text of the action category of each first sample power video, and determine the target pseudo-label text of the action category of each first sample power video based on the clustered pseudo-label text and the initial pseudo-label text of the action category, so as to obtain a target pseudo-label video set containing all first sample power videos.

[0068] Among them, the initial pseudo-label text of the action category is the specific action category that has not been clustered, such as: "Not wearing a safety helmet", "Mistakenly entering a dangerous area", "Illegal operation of electrical equipment", etc. The target pseudo-label text of the action category is the specific action category after clustering.

[0069] Among them, the zero-sample video understanding ability of the multimodal large model is used to generate the initial pseudo-label text of the action category. Specifically, for V l All first samples of power video v i In this embodiment, v i Input into the multimodal large language model (denoted by MLLM). Since all sample power videos have a predefined action category set A list As is known, this embodiment designs a corresponding prompt word template: "You are a video content analyst. The following video contains list >. You should watch the entire video and give the most likely action category based on the video content. While watching, you should try to exclude background information and focus on list >The action categories included and give accurate answers". Based on the prompt word template, the multimodal large language model MLLM will give the most likely action category corresponding to each first sample power video, namely: Represents the initial pseudo-label text of the action category of the i-th first sample power video given by the multimodal large model.

[0070] S5. Use the target pseudo-label video set and the unlabeled video set to train the video action localization model to obtain the target teacher model and the target student model corresponding to the current iteration round.

[0071] S6. Determine the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action localization model.​​

[0072] S7. Repeat S2-S6 until a maximum number of iterations is reached, and determine the updated video action localization model as the trained video action localization model.

[0073] S8. Input the power video to be detected into the trained video motion localization model to obtain the video motion localization result of the power video to be detected.

[0074] The power video to be tested is the power video to be video-localized in this embodiment. The video action localization result refers to: the target segment located by the model from the power video to be tested. Each target segment corresponds to an action category and the start and end time of the segment. For example, a target segment with an action category of "not wearing a helmet" appears in the 5-10s of the power video to be tested.

[0075] In an optional manner, the step of obtaining a high-action selective attention distribution corresponding to any original sample power video using the to-be-trained teacher model corresponding to the current iteration round includes:

[0076] By utilizing the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branches and the action category-irrelevant attention weights of the action category-irrelevant branches corresponding to any original sample power video containing multiple video clips are obtained and fused to obtain the action-perceived attention distribution corresponding to any original sample power video.

[0077] Among them, for any original sample power video v n T video clips, use the pre-trained I3D network to obtain the RGB features of each video clip With optical flow features Thus, the RGB features of the original sample power video are obtained With optical flow features The RGB features of the original sample power video are spliced ​​with the optical flow features to obtain the video clip features of the original sample power video

[0078] The teacher model to be trained includes two branches: action category related branch and action category irrelevant branch. n RGB features Optical flow features and video clip feature X n Input to the teacher model to be trained corresponding to the current iteration round to obtain the original sample power video v n The two-branch attention weights of the action category-related branch: the action category-related attention weights (CAS n∈R T*C ) Action-related attention weights of action-related branches (including: RGB attention weights and optical flow attention weights ), n represents the nth original sample power video, T represents the number of video clips, C represents the total number of preset action categories, and R represents the regularization parameter related to the attention weight, which is used to control the complexity or generalization ability of the model; CAS n represents the action category-related attention weight of the action category-related branch of the nth original sample power video, represents the RGB attention weight of the action-independent branch of the nth original sample power video, Represents the optical flow attention weight of the action category-independent branch of the nth original sample power video.

[0079] The content involved in the process of fusing the action category-related attention weights of the action category-related branches and the action category-irrelevant attention weights of the action category-irrelevant branches corresponding to any original sample power video to obtain the action-perceived attention distribution corresponding to the original sample power video includes:

[0080] ①For action category related attention weight CAS n , the action category-related attention distribution A corresponding to any original sample power video is obtained by the following method n : A n =Sigmoid(f sum (CAS n ));f sum (CAS n ) represents the summation of the attention weights of each preset action category in the time dimension, and Sigmoid represents the Sigmoid activation function.

[0081] ②For action-related branches, the RGB attention weights and optical flow attention weights Fusion is performed to obtain the action category-independent attention distribution corresponding to any original sample power video

[0082] ③ Attention distribution irrelevant to action category Attention distribution related to action category A n Fusion is performed to obtain the motion-aware attention distribution corresponding to any original sample power video The calculation method is: represents the action-aware attention distribution of the t-th video clip of the n-th original sample power video.

[0083] The action-perceived attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-action selective attention distribution corresponding to the current iteration round. Specifically:

[0084] ① For any original sample power video, the corresponding action category attention distribution A n Attention distribution A is independent of action category c n a , binarize them respectively to get the corresponding binary distribution (binarized 0-1 sequence). n For example, for each video clip: median(A n ) represents the calculation sequence Specifically, the binarization process converts the sequence A n Greater than median(A n ) becomes 1, which is less than or equal to median(A n ) becomes 0, and then the attention distribution A related to the action category is obtained n The corresponding binary distribution Similarly, after the same binarization process, the action category-independent attention distribution can be calculated The corresponding binary distribution

[0085] ② For binary distribution and Perform consistency comparison and filter the action-aware attention distribution based on the comparison results Screening for high action-perceived attention distribution The process can be described as: This process represents the attention distribution for action perception Keep All The values ​​of the corresponding video clips are set to 0 at the same time, and the values ​​of other video clips are set to 0 to obtain the high action perception attention distribution.

[0086] ③For any original sample power video, the corresponding high-action perception attention distribution Only the p video clips with the largest high action perceptual attention distribution in the original sample power video are selected as key video clips, and the high action selective attention distribution corresponding to the original sample power video can be obtained. Among them, the high action selection attention distribution of each video clip in the original sample power video is: Indicates that The values ​​in are arranged from large to small. It means selecting the largest p video clips after permutation.

[0087] In an optional manner, the step of obtaining the overall video features corresponding to any original sample power video according to the high-action selective attention distribution and video clip features corresponding to any original sample power video includes:

[0088] According to the video segment features corresponding to each video segment in any of the original sample power videos and the high-action selective attention distribution, the overall video features corresponding to any of the original sample power videos are obtained.

[0089] Among them, according to the video segment features corresponding to each video segment in any of the original sample power videos and the high-action selective attention distribution, the formula used to obtain the overall video features corresponding to any of the original sample power videos is: F n is the overall video feature corresponding to the nth original sample power video.

[0090] In an optional manner, the step of performing spectral clustering on the overall video features corresponding to each original sample power video to obtain clustered pseudo-label text of each original sample power video includes:

[0091] The affinity matrix corresponding to all the original sample power videos is constructed based on the affinity between every two original sample power videos.

[0092] Among them, the overall video features of all sample power videos are constructed into a fully connected graph using the Euclidean distance. The formula used to calculate the affinity between each two original sample power videos is: F i represents the ith original sample power video, F j represents the jth original sample power video, w ij Represents the affinity between the i-th original sample power video and the j-th original sample power video, 1≤i≤n, 1≤j≤n, i and j are both positive integers. The affinity between every two original sample power videos is calculated, and the affinity matrix W corresponding to all original sample power videos is constructed.

[0093] The affinity matrix is ​​clustered using a spectral clustering algorithm to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain cluster pseudo-label text of each original sample power video.

[0094] Among them, the cluster pseudo-label text corresponding to the original sample power video contained in each cluster is the same, for example, the cluster pseudo-label text corresponding to the original sample power video contained in the first cluster is 1, and the cluster pseudo-label text corresponding to the original sample power video contained in the second cluster is 2.

[0095] In an optional manner, the step of determining the confidence of any original sample power video includes:

[0096] According to the affinity matrix, the intra-cluster compactness and inter-cluster separation of any original sample power video are calculated, and according to the intra-cluster compactness and inter-cluster separation of any original sample power video, the confidence of any original sample power video is determined.

[0097] The formula used to calculate the intra-cluster density of any original sample power video is: a i represents the intra-cluster density of the ith original sample power video, C I represents the cluster corresponding to the i-th original sample power video, |C I | means C I The formula used to calculate the inter-cluster separation degree of any original sample power video is: b i represents the inter-cluster separation degree of the ith original sample power video, C J is a cluster other than the cluster corresponding to the i-th original sample power video, |C J | means C J The number of raw samples in the power video.

[0098] According to the intra-cluster compactness and inter-cluster separation of any original sample power video, the formula used to determine the confidence of any original sample power video is: s i Represents the confidence of the i-th original sample power video.

[0099] It should be noted that after determining the confidence of all original sample power videos, the first pseudo-label video set V l Expressed as: V l = {v i |v i ∈V set ,s i ≥δ}; unlabeled video set V u Expressed as: V u = {v i |v i ∈Vset ,s i <δ}. Among them, V set represents the sample power video set (including all original sample power videos), δ represents the target threshold; the first pseudo-label video set V l v i Defined as the first sample power video, the unlabeled video set V u v i Defined as the second sample power video.

[0100] In an optional manner, the step of determining the action category target pseudo label text of each first sample power video according to the clustered pseudo label text and the action category initial pseudo label text of each first sample power video includes:

[0101] The number of occurrences of the initial pseudo-label text of each action category in any cluster class is determined respectively, and the initial pseudo-label text of the action category with the largest number of occurrences in any cluster class is determined as the action category target pseudo-label text of each first sample power video in any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

[0102] Based on the initial pseudo-label texts of the action categories of all videos, this embodiment performs a one-to-one mapping between the cluster pseudo-label texts and the target pseudo-label texts of the action categories. Specifically, for each cluster C I , this embodiment calculates the number of occurrences of each action category (action category initial pseudo-label text) in the cluster, finds the action category initial pseudo-label text with the largest number of occurrences, and maps the cluster to this action category initial pseudo-label text. Specifically, let Y CI Represents cluster class C I The initial pseudo-label text set of all the action categories of the first sample power video, then for each cluster class C I , the initial pseudo-label text a for each action category can be calculated list ∈A list Number of occurrences in this cluster: a list is the initial pseudo label text for the action category, represents the indicator function; when 1 when , otherwise 0. Next, the present invention searches for I The initial pseudo-label text of the action category that appears most frequently (the target pseudo-label text of the action category) Finally, cluster C I Mapping to action categories Represents the action category target pseudo label text of the i-th first sample power video.

[0103] In an optional manner, S5 includes:

[0104] The target pseudo-label video set is used to train the teacher model to be trained corresponding to the current iteration round, and the unlabeled video set and the target pseudo-label video set after data perturbation are used to train the student model to be trained corresponding to the current iteration round to obtain the target teacher model and target student model corresponding to the current iteration round.

[0105] Among them, the model parameters of the teacher model to be trained corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained corresponding to the current iteration round.

[0106] For S5, specifically:

[0107] ① The input of the teacher model to be trained is the first sample power video v in the target pseudo-label video set n The input of the student model to be trained is the first sample power video after data perturbation in the target pseudo-label video set. During the training process, the parameters of the teacher model are updated by the student model using an exponential moving average strategy (EMA).

[0108] ②For video v n , the teacher model to be trained first obtains the RGB features and optical flow features of the video clip and The concatenation of the two is the feature representation of the video clip. Then, the convolutional layer and ReLU function of the teacher model to be trained are respectively embedded into Among them, represents the RGB features of the video clip, Represents the optical flow features of the video clip, x n,t The video clip features are obtained by concatenating the RGB features and optical flow features of the video clip.

[0109] ③ Embedding features As input, calculate the video v n The corresponding action category related attention weight CAS, action category irrelevant attention weight and It is expressed as:

[0110] CAS n =Conv(E n )

[0111]

[0112]

[0113] Among them, Conv, Conv Rgb 、Conv Flow is the learnable convolutional layer parameter in the neural network; and The attention weights of the action-related branches are obtained by fusion: For the action category related branch, the attention weight of each action category is also summed up in the time dimension to obtain the action category independent attention weight of the branch, which is expressed as: A n =Sigmoid(f sum (CAS n )).

[0114] ④ Randomly select λ feature channels, and randomly select The channel delay time dimension moves backward, The channel delay time dimension moves forward, and then randomly sets the value of some channels to 0 to complete the video v n The data perturbation is obtained Then the same operation as the teacher model is performed, and the student model is used to obtain

[0115] ⑤ Based on the action category-independent attention weight, the student model uses V l The sample videos in are compared and learned by mining simple video clips and difficult video clips. and and and CAS n , and The consistency loss based on mean square error is constructed to make the results as close as possible. l Using classification loss and contrast loss L con Train the network, for V u The network is jointly trained using contrastive loss and consistency loss, and the model parameters of the teacher model are updated using the exponential moving average of the model parameters of the student model.

[0116] For S8, the power video to be detected is input into the trained video action localization model, and the teacher model in the trained video action localization model is used to generate the category-related attention weights CAS n And video-level action prediction Select all that meet The power action category of the power video to be detected is first predicted; where τ represents the activation threshold of the predicted video action category. Then, in the corresponding attention weight The action proposal is generated and scored by the internal and external contrast loss, and the threshold τ is set a To obtain a set of video action proposals, non-maximum suppression is applied to repeated proposals, and finally video action localization is achieved to obtain the video action localization result of the power video to be detected.

[0117] Figure 3 Schematic diagram of the overall principle of this embodiment. Table 1 shows the positioning accuracy and efficiency comparison results obtained by comparing the positioning performance of the model of this embodiment with the internationally leading similar models.

[0118] Table 1:

[0119]

[0120] It can be seen from Table 1 that compared with similar comparative models, the positioning accuracy of the model (COPL) of this embodiment is significantly superior.

[0121] The technical solution of this embodiment can improve the overall positioning performance and achieve efficient and robust unsupervised power video action positioning by constructing a consistent unsupervised video action localization model guided by a multimodal large model.

[0122] Figure 4 FIG. 2 is a schematic diagram showing a structure of an embodiment of an unsupervised power video action positioning system 200 provided by the present invention. Figure 4 As shown, the system 200 includes: a construction module 210, a first training module 220, a second training module 230, a third training module 240, a fourth training module 250, a fifth training module 260, an iterative training module 270 and a detection module 280;

[0123] The construction module 210 is used to: construct a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization;

[0124] The first training module 220 is used to: use the to-be-trained teacher model corresponding to the current iteration round to obtain the high action selective attention distribution corresponding to each original sample power video, and obtain the overall video feature corresponding to each original sample power video according to the high action selective attention distribution and video clip features corresponding to each original sample power video;

[0125] The second training module 230 is used to: perform spectral clustering on the overall video features corresponding to each original sample power video to obtain a cluster pseudo-label text of each original sample power video, determine the original sample power video with a confidence not less than a target threshold as a first sample power video, and determine the original sample power video with a confidence less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding cluster pseudo-label text and an unlabeled video set including each second sample power video;

[0126] The third training module 240 is used to: use the multimodal large model to obtain the initial pseudo-label text of the action category of each first sample power video, and determine the target pseudo-label text of the action category of each first sample power video according to the clustered pseudo-label text and the initial pseudo-label text of the action category of each first sample power video, so as to obtain a target pseudo-label video set containing all the first sample power videos;

[0127] The fourth training module 250 is used to: train the video action localization model using the target pseudo-label video set and the unlabeled video set to obtain a target teacher model and a target student model corresponding to the current iteration round;

[0128] The fifth training module 260 is used to: determine the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action positioning model;

[0129] The iterative training module 270 is used to: repeatedly call the first training module 220 to the fifth training module 260 until a maximum number of iterations is reached, and determine the updated video action localization model as a trained video action localization model;

[0130] The detection module 280 is used to: input the power video to be detected into the trained video action positioning model to obtain the video action positioning result of the power video to be detected.

[0131] In an optional manner, the first training module 220 is specifically used for:

[0132] Using the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branches and the action category-irrelevant attention weights of the action category-irrelevant branches corresponding to any original sample power video containing multiple video clips are obtained and fused to obtain the action-perceived attention distribution corresponding to any original sample power video;

[0133] The action-perceived attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-action selective attention distribution corresponding to the current iteration round.

[0134] In an optional manner, the first training module 220 is specifically used for:

[0135] According to the video segment features corresponding to each video segment in any of the original sample power videos and the high-action selective attention distribution, the overall video features corresponding to any of the original sample power videos are obtained.

[0136] In an optional manner, the second training module 230 is specifically used to:

[0137] Calculate and construct an affinity matrix corresponding to all original sample power videos based on the affinity between every two original sample power videos;

[0138] The affinity matrix is ​​clustered using a spectral clustering algorithm to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain cluster pseudo-label text of each original sample power video.

[0139] In an optional manner, the second training module 230 is specifically used to:

[0140] According to the affinity matrix, the intra-cluster compactness and inter-cluster separation of any original sample power video are calculated, and according to the intra-cluster compactness and inter-cluster separation of any original sample power video, the confidence of any original sample power video is determined.

[0141] In an optional manner, the third training module 240 is specifically used for:

[0142] The number of occurrences of the initial pseudo-label text of each action category in any cluster class is determined respectively, and the initial pseudo-label text of the action category with the largest number of occurrences in any cluster class is determined as the action category target pseudo-label text of each first sample power video in any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

[0143] In an optional manner, the fourth training module 250 is specifically used for:

[0144] The target pseudo-label video set is used to train the teacher model to be trained corresponding to the current iteration round, and the unlabeled video set and the target pseudo-label video set after data perturbation are used to train the student model to be trained corresponding to the current iteration round, so as to obtain the target teacher model and target student model corresponding to the current iteration round; wherein the model parameters of the teacher model to be trained corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained corresponding to the current iteration round.

[0145] It should be noted that the beneficial effects of the unsupervised power video action positioning system provided by the above-mentioned embodiment are the same as the beneficial effects of the above-mentioned unsupervised power video action positioning method, which will not be repeated here. In addition, when the system provided by the above-mentioned embodiment realizes its functions, it only takes the division of the above-mentioned functional modules as an example. In practical applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above-mentioned embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0146] Among them, the unsupervised power video motion positioning system of the present invention can be a computer program (including program code) running in a computer device. For example, the unsupervised power video motion positioning system of the present invention is an application software that can be used to execute the corresponding steps in the unsupervised power video motion positioning method of the present invention.

[0147] In some embodiments, the unsupervised power video motion positioning system of the present invention can be implemented by a combination of software and hardware. As an example, the unsupervised power video motion positioning system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the unsupervised power video motion positioning method of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0148] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The name of a module does not limit the module itself in some cases.

[0149] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned unsupervised power video action localization methods is implemented. That is to say, an electronic device according to an embodiment of the present invention may include but is not limited to: a processor and a memory; a memory for storing a computer program; a processor for executing the unsupervised power video action localization method shown in any embodiment of the present invention by calling the computer program.

[0150] In an alternative embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0151] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0152] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 In the figure, only one thick line is used to represent the bus 4002, but this does not mean that there is only one bus or one type of bus.

[0153] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0154] The memory 4003 is used to store the application code (computer program) for executing the solution of the present invention, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0155] Among them, the electronic device can also be a terminal device, and the terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, and a smart car device.

[0156] It should be noted that Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0157] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned unsupervised power video action positioning methods is implemented.

[0158] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0159] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned unsupervised power video action positioning method.

[0160] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0161] It should be understood that the flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0162] The computer-readable storage medium provided by the embodiment of the present invention may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or component.

[0163] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0164] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

[0165] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0166] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0167] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An unsupervised power video action localization method, characterized in that: include: S1. Constructing a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization; S2. Using the to-be-trained teacher model corresponding to the current iteration round, obtain the high-action selective attention distribution corresponding to each original sample power video, and obtain the overall video features corresponding to each original sample power video based on the high-action selective attention distribution and video clip features corresponding to each original sample power video; S3, performing spectral clustering on the overall video features corresponding to each original sample power video to obtain a clustered pseudo-label text of each original sample power video, determining an original sample power video whose confidence is not less than a target threshold as a first sample power video, and determining an original sample power video whose confidence is less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding clustered pseudo-label text and an unlabeled video set including each second sample power video; S4, using the multimodal large model, obtaining the initial pseudo-label text of the action category of each first sample power video, and determining the target pseudo-label text of the action category of each first sample power video according to the clustered pseudo-label text and the initial pseudo-label text of the action category of each first sample power video, so as to obtain a target pseudo-label video set containing all the first sample power videos; S5, using the target pseudo-label video set and the unlabeled video set to train the video action localization model, to obtain a target teacher model and a target student model corresponding to the current iteration round; S6, determining the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determining the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action localization model; S7, repeatedly executing S2-S6 until the maximum number of iterations is reached, determining the updated video action localization model as the trained video action localization model; S8. Input the power video to be detected into the trained video motion localization model to obtain the video motion localization result of the power video to be detected.

2. The unsupervised power video action localization method according to claim 1, characterized in that: The steps of obtaining the high-action selective attention distribution corresponding to any original sample power video using the to-be-trained teacher model corresponding to the current iteration round include: Using the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branches and the action category-irrelevant attention weights of the action category-irrelevant branches corresponding to any original sample power video containing multiple video clips are obtained and fused to obtain the action-perceived attention distribution corresponding to any original sample power video; The action-perceived attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-action selective attention distribution corresponding to the current iteration round.

3. The unsupervised power video action positioning method according to claim 2, characterized in that: The step of obtaining the overall video feature corresponding to any original sample power video according to the high action selective attention distribution and video clip features corresponding to any original sample power video comprises: According to the video segment features corresponding to each video segment in any of the original sample power videos and the high-action selective attention distribution, the overall video features corresponding to any of the original sample power videos are obtained.

4. The unsupervised power video action localization method according to claim 1, characterized in that: The step of performing spectral clustering on the overall video features corresponding to each original sample power video to obtain cluster pseudo-label text of each original sample power video includes: Calculate and construct an affinity matrix corresponding to all original sample power videos based on the affinity between every two original sample power videos; The affinity matrix is ​​clustered using a spectral clustering algorithm to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain cluster pseudo-label text of each original sample power video.

5. The unsupervised power video action localization method according to claim 4, characterized in that: The steps of determining the confidence of any original sample power video include: According to the affinity matrix, the intra-cluster compactness and inter-cluster separation of any original sample power video are calculated, and according to the intra-cluster compactness and inter-cluster separation of any original sample power video, the confidence of any original sample power video is determined.

6. The unsupervised power video action localization method according to claim 4 or 5, characterized in that: The step of determining the target pseudo label text of the action category of each first sample power video according to the clustered pseudo label text and the initial pseudo label text of the action category of each first sample power video comprises: The number of occurrences of the initial pseudo-label text of each action category in any cluster class is determined respectively, and the initial pseudo-label text of the action category with the largest number of occurrences in any cluster class is determined as the action category target pseudo-label text of each first sample power video in any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

7. The unsupervised power video action localization method according to claim 1, characterized in that: The step of training the video action localization model by using the target pseudo-label video set and the unlabeled video set to obtain the target teacher model and the target student model corresponding to the current iteration round includes: The target pseudo-label video set is used to train the teacher model to be trained corresponding to the current iteration round, and the unlabeled video set and the target pseudo-label video set after data perturbation are used to train the student model to be trained corresponding to the current iteration round, so as to obtain the target teacher model and target student model corresponding to the current iteration round; wherein the model parameters of the teacher model to be trained corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained corresponding to the current iteration round.

8. An unsupervised power video action positioning system, characterized in that: include: A construction module, a first training module, a second training module, a third training module, a fourth training module, a fifth training module, an iterative training module and a detection module; The construction module is used to: construct a video action localization model including a teacher model to be trained and a student model to be trained for power video action localization; The first training module is used to: use the to-be-trained teacher model corresponding to the current iteration round to obtain the high-action selective attention distribution corresponding to each original sample power video, and obtain the overall video feature corresponding to each original sample power video according to the high-action selective attention distribution and video clip features corresponding to each original sample power video; The second training module is used to: perform spectral clustering on the overall video features corresponding to each original sample power video to obtain a cluster pseudo-label text of each original sample power video, determine the original sample power video with a confidence not less than a target threshold as a first sample power video, and determine the original sample power video with a confidence less than the target threshold as a second sample power video, to obtain a first pseudo-label video set including each first sample power video and the corresponding cluster pseudo-label text and an unlabeled video set including each second sample power video; The third training module is used to: use the multimodal large model to obtain the initial pseudo-label text of the action category of each first sample power video, and determine the target pseudo-label text of the action category of each first sample power video according to the clustered pseudo-label text and the initial pseudo-label text of the action category of each first sample power video, so as to obtain a target pseudo-label video set containing all the first sample power videos; The fourth training module is used to: train the video action localization model using the target pseudo-label video set and the unlabeled video set to obtain the target teacher model and the target student model corresponding to the current iteration round; The fifth training module is used to: determine the target teacher model corresponding to the current iteration round as the teacher model to be trained corresponding to the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained corresponding to the next round of the current iteration round, so as to obtain an updated video action positioning model; The iterative training module is used to: repeatedly call the first training module to the fifth training module until a maximum number of iterations is reached, and determine the updated video action localization model as a trained video action localization model; The detection module is used to: input the power video to be detected into the trained video action positioning model to obtain the video action positioning result of the power video to be detected.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the unsupervised power video action positioning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements the unsupervised power video action localization method as described in any one of claims 1 to 7.

Images