Unsupervised power video action localization method, system, device and storage medium

Abstract

The application relates to the technical field of video understanding, and particularly discloses an unsupervised power video action positioning method, system, device and storage medium, which comprises the following steps: constructing a video action positioning model comprising a teacher model to be trained and a student model to be trained for power video action positioning; obtaining the video overall features of sample power videos by using the teacher model to be trained, performing spectral clustering, obtaining the clustering pseudo-label texts of the sample power videos, combining the confidence and a multi-modal large model, obtaining a target pseudo-label video set and a no-label video set; training the video action positioning model by using the target pseudo-label video set and the no-label video set until the maximum iteration number is reached, obtaining the trained video action positioning model; and inputting the power video to be detected into the trained video action positioning model to obtain the video action positioning result. The application can realize efficient and robust unsupervised power video action positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video understanding, and in particular to an unsupervised power video action localization method, system, device and storage medium. BACKGROUND

[0002] In recent years, with the increasing emphasis on job safety in the power industry and the rapid development of video monitoring technology, video monitoring has become a key tool for monitoring power job safety. In particular, in the power job site, video has become an important medium for monitoring job safety due to its intuitiveness and real-time nature. In the face of massive video data, how to quickly and accurately locate and identify action behaviors with specific safety risks has become a key problem in power job safety monitoring. Therefore, the application of video action localization technology in power job safety monitoring has emerged as the times require.

[0003] Video action localization technology aims at a predefined set of safety action categories, and realizes accurate and automatic localization of all related actions in the video and accurate identification of the corresponding action categories. For example, given a predefined set of action categories (including "not wearing a safety helmet", "entering a dangerous area", "violating electrical equipment operation", and other safety-critical actions in power jobs), the technology can accurately locate and identify the time interval of the action "not wearing a safety helmet" (starting at 5.5 seconds and ending at 12.3 seconds) in the video. However, existing video action localization technology usually relies on complete supervision or weak supervision to train the action localization system, i.e. it needs to label the category and start and end time of all action segments in the training video or label the action categories contained in the video. Both of these methods require watching all training videos and carefully labeling, which results in a huge labeling cost and is not suitable for the current growing video size. In contrast, intelligent video action localization under an unsupervised setting only relies on the total number of predefined action categories in the entire video set, i.e. a powerful video action localization system can be trained with only this labeling information, which has a wider application prospect.

[0004] Through the study of video action localization under an unsupervised setting, not only does it deepen the research of traditional video action localization, but also has stronger scalability and can cope with the growing size of video data. Currently, this research mainly faces the following three challenges:

[0005] ① Accurate overall video feature representation. Previous algorithms simply based on the weighted sum of all video frame-level features and video attention weights, resulting in inaccurate overall video feature representation, which affects the clustering effect of the video on the action category. Therefore, how to improve the overall video feature representation is crucial for video-level pseudo-label generation.

[0006] ②Strong robustness of positioning network training. Existing unsupervised video action positioning models directly use all samples for "clustering-positioning" iterative training. However, considering the unreliability of video-level pseudo labels, training using all samples may cause the model to learn incorrect action representation, which will damage the attention generation for clustering and the current network inference performance, thus causing the transmission and accumulation of false information in the iterative training process. Therefore, how to reduce the negative impact of low-confidence pseudo labels in the iterative training process and better model the temporal information of videos in the unlabeled case to achieve strong robustness of positioning network training is also challenging.

[0007] ③Viewing videos cannot be avoided. Existing unsupervised video action positioning methods usually use clustering to generate pseudo labels for videos, but such pseudo labels can only be used for fully supervised model training. Because, through this process, the generated labels are class labels such as 1, 2, 3... and cannot know the specific "not wearing a safety helmet", "entering a dangerous area", "operating electrical equipment in violation of regulations" category information. In this way, in the testing process, these methods can only give the timestamp of the abnormal behavior segment in the video and the class label such as "1, 2, 3". If you want to get the specific behavior category, you still need to watch the videos in these clusters "1, 2, 3" and obtain their specific behavior categories to realize the correspondence between the clustering cluster index label and the specific behavior category.

[0008] Therefore, there is an urgent need to provide a technical solution to solve the above problems. SUMMARY

[0009] To solve the above technical problems, the present application provides an unsupervised power video action positioning method, system, device and storage medium.

[0010] In a first aspect, the present application provides an unsupervised power video action positioning method, and the technical scheme of the method is as follows:

[0011] S1, constructing a video action positioning model comprising a teacher model to be trained and a student model to be trained for power video action positioning;

[0012] S2, using the teacher model to be trained corresponding to the current iteration round, obtaining the high action selection attention distribution corresponding to each original sample power video, and obtaining the video overall feature corresponding to each original sample power video according to the high action selection attention distribution corresponding to each original sample power video and the video segment feature;

[0013] S3, spectrum clustering is performed on the video overall features corresponding to each original sample power video to obtain a clustering pseudo-label text of each original sample power video, original sample power videos with a confidence higher than a target threshold are determined as first sample power videos, and original sample power videos with a confidence lower than the target threshold are determined as second sample power videos, to obtain a first pseudo-label video set containing each first sample power video and a corresponding clustering pseudo-label text and a no-label video set containing each second sample power video;

[0014] S4, an action category initial pseudo-label text of each first sample power video is obtained by using a multi-modal large model, and an action category target pseudo-label text of each first sample power video is determined according to the clustering pseudo-label text and the action category initial pseudo-label text of each first sample power video, to obtain a target pseudo-label video set containing all first sample power videos;

[0015] S5, the video action positioning model is trained by using the target pseudo-label video set and the no-label video set, to obtain a target teacher model and a target student model corresponding to the current iteration round;

[0016] S6, the target teacher model corresponding to the current iteration round is determined as a to-be-trained teacher model corresponding to a next round of the current iteration round, and the target student model corresponding to the current iteration round is determined as a to-be-trained student model corresponding to the next round of the current iteration round, to obtain an updated video action positioning model;

[0017] S7, S2-S6 are repeatedly executed until a maximum iteration number is reached, and the updated video action positioning model is determined as a trained video action positioning model;

[0018] S8, a to-be-detected power video is input into the trained video action positioning model, to obtain a video action positioning result of the to-be-detected power video.

[0019] The beneficial effects of the unsupervised power video action positioning method of the present application are as follows:

[0020] The method of the present application can improve the overall positioning performance by constructing a consistent unsupervised video action positioning model guided by a multi-modal large model, and can realize efficient and robust unsupervised power video action positioning.

[0021] On the basis of the above-mentioned scheme, the unsupervised power video action positioning method of the present application can be further improved as follows.

[0022] In an optional manner, the step of obtaining a high action selection attention distribution corresponding to any original sample power video by using the to-be-trained teacher model corresponding to the current iteration round comprises:

[0023] obtaining, by using the teacher model corresponding to the current iteration round, the action category related attention weight of the action category related branch and the action category independent attention weight of the action category independent branch corresponding to the any original sample power video, and fusing the action category related attention weight and the action category independent attention weight to obtain an action perception attention distribution corresponding to the any original sample power video;

[0024] performing binarization processing and filtering screening on the action perception attention distribution corresponding to the any original sample power video to obtain a high action selection attention distribution corresponding to the current iteration round.

[0025] In an optional manner, the step of obtaining the video overall feature corresponding to the any original sample power video according to the high action selection attention distribution corresponding to the any original sample power video and the video segment feature includes:

[0026] obtaining the video overall feature corresponding to the any original sample power video according to the video segment feature and the high action selection attention distribution corresponding to each video segment in the any original sample power video.

[0027] In an optional manner, the step of performing spectral clustering on the video overall feature corresponding to each original sample power video to obtain a clustering pseudo-label text of each original sample power video includes:

[0028] calculating and constructing an affinity matrix corresponding to all original sample power videos according to the affinity between each two original sample power videos;

[0029] performing clustering on the affinity matrix by using a spectral clustering algorithm to obtain at least one cluster class, and determining the original sample power videos contained in each cluster class to obtain a clustering pseudo-label text of each original sample power video.

[0030] In an optional manner, the step of determining the confidence of the any original sample power video includes:

[0031] calculating the intra-cluster compactness and the inter-cluster separation degree of the any original sample power video according to the affinity matrix, and determining the confidence of the any original sample power video according to the intra-cluster compactness and the inter-cluster separation degree of the any original sample power video.

[0032] In an optional manner, the step of determining the action category target pseudo-label text of each first sample power video according to the clustering pseudo-label text and the action category initial pseudo-label text of each first sample power video includes:

[0033] Determine the occurrence number of each action category initial pseudo-label text in any cluster class respectively, and determine the action category initial pseudo-label text with the maximum occurrence number in the any cluster class as the action category target pseudo-label text of each first sample power video in the any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

[0034] In an optional manner, the video action positioning model is trained by using the target pseudo-label video set and the unlabeled video set, to obtain the target teacher model and the target student model corresponding to the current iteration round.

[0035] The target pseudo-label video set is used to train the to-be-trained teacher model corresponding to the current iteration round, and the unlabeled video set and the target pseudo-label video set after data disturbance are used to train the to-be-trained student model corresponding to the current iteration round, to obtain the target teacher model and the target student model corresponding to the current iteration round; wherein the model parameters of the to-be-trained teacher model corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the to-be-trained student model corresponding to the current iteration round.

[0036] In a second aspect, the present application provides an unsupervised power video action positioning system, and the technical scheme of the system is as follows:

[0037] The system comprises a construction module, a first training module, a second training module, a third training module, a fourth training module, a fifth training module, an iteration training module and a detection module.

[0038] The construction module is configured to construct a video action positioning model comprising a to-be-trained teacher model and a to-be-trained student model for power video action positioning.

[0039] The first training module is configured to obtain the high action selection attention distribution corresponding to each original sample power video by using the to-be-trained teacher model corresponding to the current iteration round, and obtain the video overall feature corresponding to each original sample power video according to the high action selection attention distribution corresponding to each original sample power video and the video segment feature.

[0040] The second training module is configured to perform spectral clustering on the video overall feature corresponding to each original sample power video to obtain the clustering pseudo-label text of each original sample power video, determine the original sample power video with a confidence not less than a target threshold as a first sample power video, and determine the original sample power video with a confidence less than the target threshold as a second sample power video, to obtain a first pseudo-label video set comprising each first sample power video and the corresponding clustering pseudo-label text, and an unlabeled video set comprising each second sample power video.

[0041] The third training module is configured to: acquire an action category initial pseudo-label text of each first sample power video by using a multi-modal large model, and determine an action category target pseudo-label text of each first sample power video according to the clustering pseudo-label text and the action category initial pseudo-label text of each first sample power video, to obtain a target pseudo-label video set containing all first sample power videos;

[0042] The fourth training module is configured to: train the video action positioning model by using the target pseudo-label video set and the unlabeled video set, to obtain a target teacher model and a target student model corresponding to the current iteration round;

[0043] The fifth training module is configured to: determine the target teacher model corresponding to the current iteration round as a to-be-trained teacher model corresponding to a next round of the current iteration round, and determine the target student model corresponding to the current iteration round as a to-be-trained student model corresponding to the next round of the current iteration round, to obtain an updated video action positioning model;

[0044] The iteration training module is configured to: repeatedly call the first training module to the fifth training module until a maximum iteration number is reached, and determine the updated video action positioning model as a trained video action positioning model;

[0045] The detection module is configured to: input a to-be-detected power video into the trained video action positioning model, to obtain a video action positioning result of the to-be-detected power video.

[0046] The unsupervised power video action positioning system of the present application has the following advantages:

[0047] The system of the present application can improve the overall positioning performance by constructing a consistent unsupervised video action positioning model guided by a multi-modal large model, and can realize efficient and robust unsupervised power video action positioning.

[0048] In a third aspect, the technical scheme of an electronic device of the present application is as follows:

[0049] The electronic device includes a memory, a processor, and a program stored on the memory and running on the processor, and the processor executes the program to realize the steps of the unsupervised power video action positioning method of the present application.

[0050] In a fourth aspect, the technical scheme of a computer readable storage medium provided by the present application is as follows:

[0051] The computer readable storage medium stores instructions, and when the computer readable storage medium reads the instructions, the computer readable storage medium executes steps of the unsupervised power video action localization method of the present application.

[0052] The above description is only a summary of the technical solutions of the present application. In order to make the technical means of the present application more clearly understood, the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated herein and constitute a part of the detailed description. It should be noted that in the accompanying drawings, the same reference numerals and signs are used to designate the same elements throughout the several views.

[0054] Figure 1 A flowchart of an embodiment of an unsupervised power video action localization method of the present application;

[0055] Figure 2 A structural diagram of a video action localization model;

[0056] Figure 3 A schematic diagram of the overall principle of the present embodiment;

[0057] Figure 4 A structural diagram of an embodiment of an unsupervised power video action localization system of the present application;

[0058] Figure 5 A structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0059] Exemplary embodiments of the present application will be described more fully hereinafter with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0060] Figure 1A flowchart of an embodiment of an unsupervised power video action positioning method provided by the present application is shown, which can be executed by electronic devices such as terminal devices or servers. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the unsupervised power video action positioning method by calling computer readable instructions stored in the memory through the processor. As shown in Figure 1 includes the following steps:

[0061] S1, constructing a video action positioning model comprising a teacher model to be trained and a student model to be trained for power video action positioning.

[0062] As shown in Figure 2 , the structure of the teacher model to be trained and the student model to be trained is consistent. The video action positioning model is a consistent video action positioning model by default.

[0063] S2, using the teacher model to be trained corresponding to the current iteration round, obtaining the high action selection attention distribution corresponding to each original sample power video, and obtaining the video overall feature corresponding to each original sample power video according to the high action selection attention distribution corresponding to each original sample power video and the video segment feature.

[0064] Wherein, the current iteration round is any iteration round (1≤k≤m, k is the iteration number corresponding to the current iteration round, and m is the maximum iteration number) in the model iteration training process. Each original sample power video comprises a plurality of video segments. In this embodiment, the original sample power video is segmented into T video segments (T is a positive integer) with 16 frames as the minimum unit.

[0065] S3, performing spectral clustering on the video overall feature corresponding to each original sample power video to obtain the clustering pseudo-label text of each original sample power video, determining the original sample power video with a confidence not less than a target threshold as a first sample power video, and determining the original sample power video with a confidence less than the target threshold as a second sample power video, to obtain a first pseudo-label video set comprising each first sample power video and the corresponding clustering pseudo-label text, and a no-label video set comprising each second sample power video.

[0066] Wherein, the clustering pseudo-label text is the number after clustering, such as: "1, 2, 3" and the like. The target threshold can be set according to the actual situation, which is not limited here.

[0067] S4, using a multi-modal large model, obtaining an action category initial pseudo-label text of each first sample power video, and determining an action category target pseudo-label text of each first sample power video according to the clustering pseudo-label text and the action category initial pseudo-label text of each first sample power video, to obtain a target pseudo-label video set containing all first sample power videos.

[0068] Wherein, the action category initial pseudo-label text is a specific action category without clustering, such as: "not wearing a safety helmet", "entering a dangerous area", "operating electrical equipment in violation of regulations" and the like. The action category target pseudo-label text is a specific action category after clustering.

[0069] Wherein, using the zero-shot video understanding ability of the multi-modal large model, the action category initial pseudo-label text is generated. Specifically, for all first sample power videos v l in the set V i , this embodiment inputs v i into a multi-modal large language model (denoted as MLLM). Since the pre-defined action category set A list of all sample power videos is known, this embodiment designs a corresponding prompt word template: "You are a video content analyst, the following video contains a corresponding segment of <A list > in some action category. You should watch the entire video and give the action category that the video is most likely to contain based on the video content. In the process of watching, you should try to exclude background information interference as much as possible, and focus on the action category contained in <A list > and give the correct answer". Based on this prompt word template, the multi-modal large language model MLLM will give the most likely action category corresponding to each first sample power video, that is: represents the action category initial pseudo-label text of the i-th first sample power video given by the multi-modal large model.

[0070] S5, using the target pseudo-label video set and the unlabeled video set, training the video action positioning model to obtain a target teacher model and a target student model corresponding to the current iteration round.

[0071] S6, determining the target teacher model corresponding to the current iteration round as a to-be-trained teacher model corresponding to the next round of the current iteration round, and determining the target student model corresponding to the current iteration round as a to-be-trained student model corresponding to the next round of the current iteration round, to obtain an updated video action positioning model.

[0072] S7. Repeat S2-S6 until the maximum number of iterations is reached, then determine the updated video motion localization model as the trained video motion localization model.

[0073] S8. Input the power video to be detected into the trained video motion localization model to obtain the video motion localization result of the power video to be detected.

[0074] In this embodiment, the power video to be tested is the power video for which video localization is to be performed. The video motion localization result refers to the target segment located by the model from the power video to be tested. Each target segment corresponds to an action category and a segment start and end time. For example, a target segment with the action category "not wearing a safety helmet" appears between 5-10 seconds in the power video to be tested.

[0075] In one alternative approach, the step of obtaining the high-action selection attention distribution corresponding to any original sample power video using the teacher model to be trained for the current iteration round includes:

[0076] Using the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branch and the action category-independent attention weights of the action category-independent branch corresponding to any original sample power video containing multiple video segments are obtained and fused to obtain the action-aware attention distribution corresponding to any original sample power video.

[0077] For any original sample power video v n Given T video clips, use a pre-trained I3D network to obtain the RGB features of each video clip. With optical flow characteristics Thus, the RGB features of the original sample power video are obtained. With optical flow characteristics The RGB features and optical flow features of the original sample power video are concatenated to obtain the video segment features of the original sample power video.

[0078] The teacher model to be trained includes two branches: a branch related to the action category and a branch unrelated to the action category. Any original sample power video v n RGB features Optical flow characteristics and video segment features X n The input is given to the teacher model to be trained in the current iteration round to obtain the original sample power video v. n Two-branch attention weights: Action category related attention weights (CAS) for the action category related branch. n∈R T*C ) and action category independent attention weights of action category independent branch (including: RGB attention weights and optical flow attention weights );n represents the nth original sample power video, T represents the number of video clips, C represents the total number of preset action categories, R represents a regularization parameter related to attention weights, used to control the complexity or generalization ability of the model;CAS n represents the action category dependent attention weights of the action category dependent branch of the nth original sample power video, represents the RGB attention weights of the action category independent branch of the nth original sample power video, represents the optical flow attention weights of the action category independent branch of the nth original sample power video.

[0079] Wherein, the process of fusing the action category dependent attention weights of the action category dependent branch and the action category independent attention weights of the action category independent branch corresponding to any original sample power video to obtain the action-aware attention distribution of the original sample power video involves:

[0080] ① For the action category dependent attention weights CAS n , the action category dependent attention distribution A n of any original sample power video is obtained by the following way: A n =Sigmoid(f sum (CAS n ));f sum (CAS n ) represents the time dimension summation of the attention weights of each preset action category, and Sigmoid represents the Sigmoid activation function.

[0081] ② For the action category independent branch, the RGB attention weights and the optical flow attention weights are fused to obtain the action category independent attention distribution

[0082] ③ The action category independent attention distribution and the action category dependent attention distribution A n are fused to obtain the action-aware attention distribution of any original sample power video. action-aware attention distribution of the t-th video segment of the n-th original sample power video.

[0083] The action-aware attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-action selection attention distribution corresponding to the current iteration round. Specifically:

[0084] ① For the action class related attention distribution A n and the action class independent attention distribution A c n a , respectively, binarization processing is performed to obtain the corresponding binarization distribution (binarization 0-1 sequence). Taking A n as an example, for each video segment: median(A n ) represents the median of the sequence . Specifically, the binarization process changes the part of the sequence A n greater than the median median(A n ) to 1, and the part less than or equal to median(A n ) to 0, thereby obtaining the binarization distribution corresponding to the action class related attention distribution A n Similarly, the binarization distribution corresponding to the action class independent attention distribution A can be calculated through the same binarization process.

[0085] ② The binarization distributions A and A are compared for consistency, and then the high-action-aware attention distribution A is filtered from the action-aware attention distribution A according to the comparison result. This process can be described as: This process means that for the action-aware attention distribution A , the values of all corresponding video segments are retained, and the values of other video segments are set to 0, thereby obtaining the high-action-aware attention distribution A

[0086] ③ For the high-action-aware attention distribution A corresponding to any original sample power video, only the p video segments with the highest high-action-aware attention distribution in the original sample power video are selected as key video segments, and the high-action selection attention distribution A corresponding to the original sample power video can be obtained.wherein the high-motion selection attention distribution of each video segment in the original sample power video is: denotes arranging values in from large to small, denotes selecting the largest p video segments after the arrangement.

[0087] In an optional manner, the step of obtaining the video overall feature corresponding to any original sample power video according to the high-motion selection attention distribution and the video segment feature of the original sample power video comprises:

[0088] obtaining the video overall feature corresponding to any original sample power video according to the video segment feature and the high-motion selection attention distribution of each video segment in the original sample power video.

[0089] wherein the formula used for obtaining the video overall feature corresponding to any original sample power video according to the video segment feature and the high-motion selection attention distribution of each video segment in the original sample power video is: F n is the video overall feature corresponding to the nth original sample power video.

[0090] In an optional manner, the step of obtaining the clustering pseudo-label text of each original sample power video by performing spectral clustering on the video overall feature corresponding to each original sample power video comprises:

[0091] calculating and constructing the affinity matrix corresponding to all original sample power videos according to the affinity between each two original sample power videos.

[0092] wherein the video overall features of all sample power videos are constructed into a fully connected graph using the Euclidean distance, and the formula used for calculating the affinity between each two original sample power videos is: F i denotes the ith original sample power video, F j denotes the jth original sample power video, w ij denotes the affinity between the ith original sample power video and the jth original sample power video, 1≤i≤n, 1≤j≤n, i and j are positive integers. The affinity between each two original sample power videos is calculated to construct the affinity matrix W corresponding to all original sample power videos.

[0093] performing clustering on the affinity matrix by using a spectral clustering algorithm to obtain at least one cluster class, determining the original sample power videos contained in each cluster class, and obtaining the clustering pseudo-label text of each original sample power video.​

[0094] wherein the original sample power videos included in each cluster class correspond to the same cluster pseudo-label text, such as the original sample power videos included in the first cluster class correspond to the cluster pseudo-label text of 1, and the original sample power videos included in the second cluster class correspond to the cluster pseudo-label text of 2.

[0095] In an optional manner, the step of determining the confidence of any original sample power video comprises:

[0096] According to the affinity matrix, the within-cluster compactness and the between-cluster separation of any original sample power video are calculated, and according to the within-cluster compactness and the between-cluster separation of any original sample power video, the confidence of the original sample power video is determined.

[0097] wherein the formula for calculating the within-cluster compactness of any original sample power video is: a i denotes the within-cluster compactness of the i-th original sample power video, C I denotes the cluster class corresponding to the i-th original sample power video, |C I denotes the number of original sample power videos in C I The formula for calculating the between-cluster separation of any original sample power video is: b i denotes the between-cluster separation of the i-th original sample power video, C J denotes the cluster class other than the cluster class corresponding to the i-th original sample power video, |C J denotes the number of original sample power videos in C J .

[0098] wherein the formula for determining the confidence of any original sample power video according to the within-cluster compactness and the between-cluster separation of the original sample power video is: s i denotes the confidence of the i-th original sample power video.

[0099] It should be noted that after determining the confidence of all original sample power videos, the first pseudo-label video set V l is denoted as: V l = {v i |v i ∈ V set , s i ≥ δ}; and the unlabeled video set V u is denoted as: V u = {v i |v i ∈ Vset i set wherein V l represents the set of sample power videos (containing all original sample power videos), and d represents the target threshold; the first pseudo-label video set V i is defined as the first sample power video, and the unlabeled video set V u is defined as the second sample power video. i

[0100] In an alternative way, the step of determining the action category target pseudo-label text of each first sample power video according to the cluster pseudo-label text and the action category initial pseudo-label text of each first sample power video comprises:

[0101] The occurrence number of each action category initial pseudo-label text in any cluster class is determined respectively, and the action category initial pseudo-label text with the largest occurrence number in the any cluster class is determined as the action category target pseudo-label text of each first sample power video in the any cluster class, until the action category target pseudo-label text of each first sample power video is determined.

[0102] Wherein, based on the action category initial pseudo-label text of all videos, the embodiment performs one-to-one mapping between the cluster pseudo-label text and the action category target pseudo-label text. Specifically, for each cluster class C I , the embodiment finds the action category initial pseudo-label text with the largest occurrence number by calculating the occurrence number of each action category (action category initial pseudo-label text) in the cluster class, and maps the cluster class to the action category initial pseudo-label text. Specifically, let Y CI represent the set of action category initial pseudo-label texts of all first sample power videos of the cluster class C I , then for each cluster class C I , the occurrence number of each action category initial pseudo-label text a list ∈A list in the cluster class can be calculated: a list is the action category initial pseudo-label text, indicates the indicator function; it is 1 when , otherwise it is 0. Next, the invention finds the action category initial pseudo-label text (action category target pseudo-label text) with the largest occurrence number in the cluster class C I Finally, the cluster class C I is mapped to the action category represents the action category target pseudo-label text of the i-th first sample power video.

[0103] ​​​​In an alternative mode, S5 comprises:

[0104] The target pseudo-labeled video set is used to train the to-be-trained teacher model corresponding to the current iteration round, and the unlabeled video set and the target pseudo-labeled video set after data perturbation are used to train the to-be-trained student model corresponding to the current iteration round, to obtain the target teacher model and the target student model corresponding to the current iteration round.

[0105] The model parameters of the to-be-trained teacher model corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the to-be-trained student model corresponding to the current iteration round.

[0106] For S5, specifically:

[0107] ①The input of the to-be-trained teacher model is the first sample power video v n in the target pseudo-labeled video set, and the input of the to-be-trained student model is the first sample power video in the target pseudo-labeled video set after data perturbation In the training process, the parameters of the teacher model are updated by the student model using the exponential moving average strategy (EMA).

[0108] ②For the video v n , the to-be-trained teacher model first obtains the video segment RGB feature and the optical flow feature and The two are spliced to obtain the video segment feature representation Then embedded into through the convolution layer and the ReLU function of the to-be-trained teacher model respectively; wherein, represents the RGB feature of the video segment, represents the optical flow feature of the video segment, and x n,t represents the video segment feature obtained by splicing the RGB feature and the optical flow feature of the video segment.

[0109] ③Taking the embedded feature as the input, the action class related attention weight CAS of the video v n corresponding, and the action class independent attention weight and are calculated as follows:

[0110] CAS n =Conv(E n )

[0111]

[0112]

[0113] wherein, Conv, Conv Rgb , Conv Flow are learnable convolutional layer parameters in the neural network; and and are fused to obtain the action-class-irrelevant attention weight of the branch: For the action-class-relevant branch, the action-class-irrelevant attention weight of the branch is also obtained by performing time-dimension summation on the attention weight of each action class, denoted as: A n = Sigmoid(f sum (CAS n )).

[0114] (4) Randomly select λ feature channels, and for each video, randomly select the channels of the video v to move backward in the time dimension, to move forward in the time dimension, and then randomly set the values of a part of the channels to 0, to complete the data perturbation of the video v n to obtain v Then, the operation of the student model is the same as that of the teacher model, and v

[0115] (5) Based on the action-class-irrelevant attention weight, the student model uses the sample videos in V l to perform contrastive learning by mining simple video clips and difficult video clips. At the same time, the consistency loss based on mean square error is constructed between V and V and V and CAS n , and to make the results as close as possible. For V l , the network is trained using the classification loss and the contrastive loss L con , and for V u , the network is jointly trained using the contrastive loss and the consistency loss, and the model parameters of the teacher model are updated using the exponential moving average of the model parameters of the student model.

[0116] For S8, the power video to be detected is input into the trained video action positioning model, and the teacher model in the trained video action positioning model is used to generate the class-related attention weight CAS n and the video-level action prediction All power action classes that satisfy are selected, so that the action class prediction of the power video to be detected is first realized; wherein τ represents the activation threshold of the predicted video action class. Then, the corresponding attention weight The action proposals are generated and scored by means of an inner-outer contrast loss, and a threshold τ is set a To obtain the set of video action proposals, non-maximum suppression is applied to the proposals that are repeated, and finally the video action localization is realized to obtain the video action localization result of the power video to be detected.

[0117] Figure 3 As a whole schematic diagram of the principle of the embodiment, Table 1 shows the positioning accuracy and efficiency comparison results obtained by comparing the positioning performance of the model of the embodiment with that of an internationally leading similar model.

[0118] Table 1:

[0119]

[0120] As can be seen from Table 1, compared with similar comparison models, the positioning accuracy of the model (COPL) of the embodiment has significant superiority.

[0121] The technical scheme of the embodiment can improve the overall positioning performance by constructing a consistency unsupervised video action localization model based on a multi-modal large model guide, and realize efficient and robust unsupervised power video action localization.

[0122] Figure 4 An embodiment of an unsupervised power video action localization system 200 provided by the application is shown in a structural schematic diagram. As shown in the figure, Figure 4 The system 200 includes a construction module 210, a first training module 220, a second training module 230, a third training module 240, a fourth training module 250, a fifth training module 260, an iterative training module 270 and a detection module 280.

[0123] The construction module 210 is configured to construct a video action localization model comprising a teacher model to be trained and a student model to be trained for power video action localization.

[0124] The first training module 220 is configured to obtain a high action selection attention distribution corresponding to each original sample power video by using the teacher model to be trained corresponding to the current iteration round, and obtain a video overall feature corresponding to each original sample power video according to the high action selection attention distribution corresponding to each original sample power video and a video segment feature.

[0125] The second training module 230 is configured to perform spectral clustering on the video overall features corresponding to each original sample power video to obtain a clustering pseudo-label text of each original sample power video, determine an original sample power video with a confidence higher than a target threshold as a first sample power video, and determine an original sample power video with a confidence lower than the target threshold as a second sample power video, and obtain a first pseudo-label video set containing each first sample power video and a corresponding clustering pseudo-label text, and a no-label video set containing each second sample power video.

[0126] The third training module 240 is configured to obtain an action category initial pseudo-label text of each first sample power video by using a multi-modal large model, and determine an action category target pseudo-label text of each first sample power video according to the clustering pseudo-label text and the action category initial pseudo-label text of each first sample power video, to obtain a target pseudo-label video set containing all first sample power videos.

[0127] The fourth training module 250 is configured to train the video action positioning model by using the target pseudo-label video set and the no-label video set, to obtain a target teacher model and a target student model corresponding to the current iteration round.

[0128] The fifth training module 260 is configured to determine the target teacher model corresponding to the current iteration round as a to-be-trained teacher model corresponding to a next round of the current iteration round, and determine the target student model corresponding to the current iteration round as a to-be-trained student model corresponding to the next round of the current iteration round, to obtain an updated video action positioning model.

[0129] The iteration training module 270 is configured to repeatedly call the first training module 220 to the fifth training module 260 until a maximum iteration number is reached, and determine the updated video action positioning model as a trained video action positioning model.

[0130] The detection module 280 is configured to input a to-be-detected power video into the trained video action positioning model to obtain a video action positioning result of the to-be-detected power video.

[0131] In an optional manner, the first training module 220 is specifically configured to:

[0132] The current iteration round corresponding to the to-be-trained teacher model is used to obtain action category related attention weights of an action category related branch and action category independent attention weights of an action category independent branch corresponding to the any original sample power video containing a plurality of video segments, and the action category related attention weights and the action category independent attention weights are fused to obtain an action perception attention distribution corresponding to the any original sample power video.

[0133] binarizing and filtering the action-aware attention distribution corresponding to any original sample power video to obtain a high-action selection attention distribution corresponding to the current iteration round.

[0134] In an optional manner, the first training module 220 is specifically configured to:

[0135] According to the video segment feature and the high-action selection attention distribution corresponding to each video segment in the any original sample power video, a video overall feature corresponding to the any original sample power video is obtained.

[0136] In an optional manner, the second training module 230 is specifically configured to:

[0137] An affinity matrix corresponding to all original sample power videos is constructed by calculating and according to the affinity between each two original sample power videos;

[0138] At least one cluster is obtained by clustering the affinity matrix using a spectral clustering algorithm, and original sample power videos contained in each cluster are determined to obtain a cluster pseudo-label text of each original sample power video.

[0139] In an optional manner, the second training module 230 is specifically configured to:

[0140] According to the affinity matrix, the intra-cluster compactness and the inter-cluster separation degree of the any original sample power video are calculated, and the confidence of the any original sample power video is determined according to the intra-cluster compactness and the inter-cluster separation degree of the any original sample power video.

[0141] In an optional manner, the third training module 240 is specifically configured to:

[0142] The occurrence number of each action category initial pseudo-label text in any cluster is determined respectively, and the action category initial pseudo-label text with the highest occurrence number in the any cluster is determined as the action category target pseudo-label text of each first sample power video in the any cluster, until the action category target pseudo-label text of each first sample power video is determined.

[0143] In an optional manner, the fourth training module 250 is specifically configured to:

[0144] The target pseudo-labeled video set is used to train the teacher model to be trained corresponding to the current iteration round, and the unlabeled video set and the target pseudo-labeled video set after data disturbance are used to train the student model to be trained corresponding to the current iteration round, to obtain a target teacher model and a target student model corresponding to the current iteration round; wherein the model parameters of the teacher model to be trained corresponding to the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained corresponding to the current iteration round.

[0145] It should be noted that the beneficial effects of the unsupervised power video action positioning system provided by the above embodiments are the same as those of the unsupervised power video action positioning method described above, and will not be repeated here. In addition, when the system provided by the above embodiments implements its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the above described functions. In addition, the system and method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0146] Among them, the unsupervised power video action positioning system of the present application can be a computer program (including program code) running in a computer device, for example, the unsupervised power video action positioning system of the present application is an application software, which can be used to execute the corresponding steps in the unsupervised power video action positioning method of the present application.

[0147] In some embodiments, the unsupervised power video action positioning system of the present application can be realized in a combination of software and hardware, for example, the unsupervised power video action positioning system of the present application can be a hardware decoding processor form of processor programmed to execute the unsupervised power video action positioning method of the present application, for example, the hardware decoding processor form of processor can use one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0148] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not limit the modules themselves.

[0149] An electronic device according to an embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above unsupervised power video action localization methods is implemented. That is, the electronic device according to an embodiment of the present application can include, but is not limited to, a processor and a memory; the memory is configured to store a computer program; and the processor is configured to execute the unsupervised power video action localization method according to any of the embodiments of the present application by invoking the computer program.

[0150] In an optional embodiment, an electronic device is provided, as shown in Figure 5 Figure 5 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, through a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual application, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not limit the embodiments of the present application.

[0151] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0152] ​The bus 4002 can include a path that transmits information between the above-described components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, or the like. For convenience of representation, Figure 5 The bus 4002 is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0153] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0154] The memory 4003 is used to store application code (computer program) for executing the scheme of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the application code stored in the memory 4003 to realize the content shown in the foregoing method embodiments.

[0155] The electronic device can also be a terminal device, and the terminal device can be any terminal device that can install an application and access a webpage through the application, including at least one of a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, and a smart vehicle device.

[0156] It should be noted that, Figure 5 The electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0157] The computer readable storage medium of the embodiment of the present application, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement any of the above unsupervised power video action positioning methods.

[0158] Optionally, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0159] In the exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the electronic device execute the above unsupervised power video action positioning method.

[0160] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0161] It should be understood that the flow diagrams and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each of the blocks of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0162] The computer readable storage medium of embodiments of the present application can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0163] The computer readable storage medium described above can bear one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0164] The above description merely illustrates the preferred embodiments of the present application and the principles of the technology applied. It should be understood that the disclosed scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present application (but not limited to) having similar functions.

[0165] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and do not represent a specific order or sequential order. The order of use of similar objects can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0166] Those skilled in the art know that the present application can be implemented as a system, a method or a computer program product, so the present application can be specifically implemented as follows: it can be a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present application can also be implemented as a computer program product in one or more computer readable media, which contains computer readable program code.

[0167] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. An unsupervised power video motion localization method, characterized in that, include: S1. Construct a video motion localization model that includes a teacher model to be trained and a student model to be trained for power video motion localization; S2. Using the teacher model to be trained corresponding to the current iteration round, obtain the high-action selection attention distribution corresponding to each original sample power video, and obtain the overall video features corresponding to each original sample power video based on the high-action selection attention distribution and video segment features. S3. Perform spectral clustering on the overall video features corresponding to each original sample power video to obtain the clustering pseudo-label text of each original sample power video. Determine the original sample power videos with a confidence level not less than the target threshold as the first sample power videos, and determine the original sample power videos with a confidence level less than the target threshold as the second sample power videos. This will result in a first pseudo-label video set containing each first sample power video and its corresponding clustering pseudo-label text, and an unlabeled video set containing each second sample power video. S4. Using a multimodal large model, obtain the initial pseudo-label text of the action category for each first sample power video, and determine the target pseudo-label text of the action category for each first sample power video based on the clustering pseudo-label text and the initial pseudo-label text of the action category for each first sample power video, so as to obtain a target pseudo-label video set containing all first sample power videos. S5. Using the target pseudo-labeled video set and the unlabeled video set, train the video action localization model to obtain the target teacher model and the target student model corresponding to the current iteration round; S6. Determine the target teacher model corresponding to the current iteration round as the teacher model to be trained in the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained in the next round of the current iteration round, so as to obtain the updated video action localization model; S7. Repeat S2-S6 until the maximum number of iterations is reached, and then determine the updated video action localization model as the trained video action localization model. S8. Input the power video to be detected into the trained video motion localization model to obtain the video motion localization result of the power video to be detected. The steps for performing spectral clustering on the overall video features corresponding to each original sample power video to obtain the clustering pseudo-label text for each original sample power video include: Calculate and construct an affinity matrix for all original sample power videos based on the affinity between every two original sample power videos; Using a spectral clustering algorithm, the affinity matrix is ​​clustered to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain the cluster pseudo-label text for each original sample power video. The steps for determining the confidence level of any original sample of power video include: Based on the affinity matrix, calculate the intra-cluster tightness and inter-cluster separation of any original sample power video, and determine the confidence level of any original sample power video based on the intra-cluster tightness and inter-cluster separation of any original sample power video. The steps of training the video action localization model using the target pseudo-labeled video set and the unlabeled video set to obtain the target teacher model and target student model corresponding to the current iteration round include: Using the target pseudo-labeled video set, the teacher model to be trained for the current iteration round is trained, and using the unlabeled video set and the target pseudo-labeled video set after data perturbation, the student model to be trained for the current iteration round is trained, thus obtaining the target teacher model and the target student model for the current iteration round; wherein, the model parameters of the teacher model to be trained for the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained for the current iteration round.

2. The unsupervised power video motion localization method according to claim 1, characterized in that, The steps for obtaining the high-action selection attention distribution corresponding to any original sample power video using the teacher model to be trained in the current iteration round include: Using the teacher model to be trained corresponding to the current iteration round, the action category-related attention weights of the action category-related branch and the action category-independent attention weights of the action category-independent branch corresponding to any original sample power video containing multiple video segments are obtained and fused to obtain the action-aware attention distribution corresponding to any original sample power video. The motion-aware attention distribution corresponding to any original sample power video is binarized and filtered to obtain the high-motion-activity selection attention distribution corresponding to the current iteration round.

3. The unsupervised power video motion localization method according to claim 2, characterized in that, The step of obtaining the overall video features corresponding to any original sample power video based on the high-activity selection attention distribution and video segment features corresponding to any original sample power video includes: Based on the video segment features corresponding to each video segment in any original sample power video and the high-activity selection attention distribution, the overall video features corresponding to any original sample power video are obtained.

4. The unsupervised power video motion localization method according to claim 1, characterized in that, The steps for determining the target pseudo-label text for the action category of each first sample power video, based on the clustering pseudo-label text and the initial pseudo-label text for the action category, include: The occurrence count of the initial pseudo-label text for each action category in any cluster is determined, and the initial pseudo-label text of the action category with the highest occurrence count in any cluster is determined as the target pseudo-label text of the action category for each first sample power video in any cluster, until the target pseudo-label text of the action category for each first sample power video is determined.

5. An unsupervised power video motion positioning system, characterized in that, include: The training modules include: a construction module, a first training module, a second training module, a third training module, a fourth training module, a fifth training module, an iterative training module, and a detection module. The construction module is used to: construct a video motion localization model that includes a teacher model to be trained and a student model to be trained for power video motion localization; The first training module is used to: use the teacher model to be trained corresponding to the current iteration round to obtain the high-action selection attention distribution corresponding to each original sample power video, and obtain the overall video features corresponding to each original sample power video based on the high-action selection attention distribution corresponding to each original sample power video and the video segment features; The second training module is used to: perform spectral clustering on the overall video features corresponding to each original sample power video to obtain the clustering pseudo-label text of each original sample power video; determine the original sample power videos with a confidence level not less than the target threshold as the first sample power videos; and determine the original sample power videos with a confidence level less than the target threshold as the second sample power videos; thereby obtaining a first pseudo-label video set containing each first sample power video and the corresponding clustering pseudo-label text, and an unlabeled video set containing each second sample power video. The third training module is used to: use a multimodal large model to obtain the initial pseudo-label text of the action category for each first sample power video, and determine the target pseudo-label text of the action category for each first sample power video based on the clustering pseudo-label text and the initial pseudo-label text of the action category for each first sample power video, so as to obtain a target pseudo-label video set containing all first sample power videos. The fourth training module is used to: train the video action localization model using the target pseudo-label video set and the unlabeled video set to obtain the target teacher model and the target student model corresponding to the current iteration round; The fifth training module is used to: determine the target teacher model corresponding to the current iteration round as the teacher model to be trained in the next round of the current iteration round, and determine the target student model corresponding to the current iteration round as the student model to be trained in the next round of the current iteration round, so as to obtain the updated video motion localization model; The iterative training module is used to: repeatedly call the first training module to the fifth training module until the maximum number of iterations is reached, and then determine the updated video motion localization model as the trained video motion localization model. The detection module is used to: input the power video to be detected into the trained video motion localization model, and obtain the video motion localization result of the power video to be detected; The second training module is specifically used for: Calculate and construct an affinity matrix for all original sample power videos based on the affinity between every two original sample power videos; Using a spectral clustering algorithm, the affinity matrix is ​​clustered to obtain at least one cluster, and the original sample power videos contained in each cluster are determined to obtain the cluster pseudo-label text for each original sample power video. The second training module is specifically used for: Based on the affinity matrix, calculate the intra-cluster tightness and inter-cluster separation of any original sample power video, and determine the confidence level of any original sample power video based on the intra-cluster tightness and inter-cluster separation of the original sample power video. The fourth training module is specifically used for: Using the target pseudo-labeled video set, the teacher model to be trained for the current iteration round is trained, and using the unlabeled video set and the target pseudo-labeled video set after data perturbation, the student model to be trained for the current iteration round is trained, thus obtaining the target teacher model and the target student model for the current iteration round; wherein, the model parameters of the teacher model to be trained for the current iteration round are updated using the exponential moving average of the model parameters of the student model to be trained for the current iteration round.

6. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the unsupervised power video motion localization method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer-readable storage medium to implement the unsupervised power video motion localization method as described in any one of claims 1 to 4.

Images