Power time sequence action detection method and system based on probability distribution optimization, and electronic equipment

Abstract

The invention belongs to the field of electric power system intelligent safety supervision, and discloses an electric power time sequence action detection method and system based on probability distribution optimization and electronic equipment, and the method comprises the following steps: obtaining historical electric power operation video data, historical electric power action training data and historical electric power action verification data; constructing a feature sequence division layer according to action categories and starting and ending boundaries in the historical power operation video data; constructing an action proposal prediction model according to historical electric power action training data in combination with the feature sequence division layer; obtaining reference data corresponding to each action proposal according to the historical power operation video data; inputting the historical electric power action verification data into an action proposal prediction model for prediction to obtain action proposal prediction data; matching the result of the motion proposal prediction data with the reference data to obtain offset probability distribution; and inputting the real-time power equipment action video data and the offset probability distribution into the action proposal prediction model for calculation to obtain a power equipment action detection result.

Description

Technical Field

[0001] This application belongs to the field of intelligent safety monitoring of power systems, and in particular relates to a power timing action detection method, system and electronic equipment based on probability distribution optimization. Background Technology

[0002] Temporal Action Detection (TAD) is an important task in the fields of computer vision and video understanding. It aims to automatically identify and locate the time range (i.e., the start and end boundaries of the action) of specific actions in unedited long videos, and simultaneously determine the action category. This technology has significant application value in scenarios such as power safety production monitoring.

[0003] In actual power construction video scenarios, the application of this technology faces several unique challenges: First, due to the fixed viewing angle of the monitoring camera and the complex working environment, key operation areas are often obstructed; second, the boundaries of temporal actions lack clear geometric feature definitions, and the annotation results of action intervals are often greatly affected by the subjective judgment of the annotators. These factors together lead to significant uncertainties in the localization modeling of temporal action detection. Summary of the Invention

[0004] This application addresses the problem in existing technologies where methods cannot model positioning uncertainty due to regression to fixed coordinates. It proposes a power timing action detection method, system, and electronic equipment based on probability distribution optimization. By performing positioning modeling through probability distribution, the problem of existing methods being unable to model positioning uncertainty due to regression to fixed coordinates is solved, achieving higher detection accuracy in power safety monitoring scenarios.

[0005] To achieve the above objectives, this application provides the following technical solution: a power time-series action detection method based on probability distribution optimization, comprising the following steps: acquiring historical data, including historical power operation video data, historical power action training data, and historical power action verification data; constructing a feature sequence segmentation layer based on the action categories and start / end boundaries in the historical power operation video data; constructing an action proposal prediction model based on the historical power action training data and the feature sequence segmentation layer; acquiring reference data corresponding to each action proposal based on the historical power operation video data; inputting the historical power action verification data into the action proposal prediction model for prediction to obtain action proposal prediction data; matching the result of the action proposal prediction data with the reference data to obtain an offset probability distribution; inputting real-time power equipment action video data and the offset probability distribution into the action proposal prediction model for calculation to obtain the power equipment action detection result.

[0006] Optionally, the following steps are performed in the feature sequence segmentation layer: obtain the detection target based on the action category and start and end boundaries; perform feature sequence segmentation of the power equipment action video data based on the detection target.

[0007] Optionally, constructing an action proposal prediction model based on the historical power operation training data and the feature sequence segmentation layer includes: extracting feature data from the feature sequence segmentation layer using a convolutional neural network to construct multi-level temporal features and feature grids; inputting the multi-level temporal features and feature grids into a sine and cosine position encoder to obtain refined temporal features; and constructing an action proposal prediction model based on the historical power equipment operation training data and the multi-level temporal features and feature grids.

[0008] Optionally, inputting the historical power action verification data into the action proposal prediction model for prediction, and obtaining action proposal prediction data includes: inputting refined time-series features into a multilayer perceptron for calculation to obtain the initial information of the predicted action proposal.

[0009] Optionally, inputting the historical power action verification data into the action proposal prediction model for prediction, and obtaining action proposal prediction data further includes: processing the initial information of the predicted action proposal based on the foreground score to obtain a query vector.

[0010] Optionally, matching the results of the action proposal prediction data with the reference data to obtain the offset probability distribution includes: inputting the reference data corresponding to each action proposal, the query vector, the initial information of the predicted action proposal, and the refined temporal features into the decoder, so that the decoder outputs the predicted action proposal information and the action category probability.

[0011] Optionally, the decoder includes a multi-head self-attention layer and a deformable self-attention layer; wherein, matching the result of the action proposal prediction data with the reference data to obtain the offset probability distribution further includes: each decoder layer optimizing and updating the query vector through the multi-head self-attention layer and the deformable self-attention layer; processing the optimized and updated query vector according to the offset probability distribution and candidate offset values ​​to obtain the center offset probability and width offset probability of the query vector; iteratively updating the center position and width of the predicted action proposal information according to the predicted offset probability distribution and candidate offset values; and outputting the action boundary and class probability according to the center offset probability and width offset probability of the query vector output by each layer and the center position and width of the predicted action proposal information.

[0012] Optionally, constructing an action proposal prediction model based on the historical power operation training data and the feature sequence layer division further includes: training with historical power equipment operation video data and an action proposal prediction model containing a loss function to obtain the loss function weights and correlation weights of the real-time power equipment action video data.

[0013] Another technical solution provided in this application is an electronic device, which includes a memory, a processor, and a battery module; the memory is used to store a program; the battery module is used to power the memory; and the processor is used to execute the program and implement the above-described time-series causality-based power action detection method when executing the program.

[0014] Another technical solution provided in this application is a computer-readable storage medium storing a computer program or instructions, which, when executed by a processing device, implements the above-described power action detection method based on time-series causality.

[0015] The beneficial effects of this application are as follows:

[0016] 1. By modeling uncertainty through probability distribution, capturing multi-scale actions through multi-level temporal features, and focusing effective areas with deformable attention, the localization accuracy of action boundaries is significantly improved.

[0017] 2. In response to the characteristics of fixed-viewpoint occlusion and large differences in action duration (such as "climbing" and "testing electricity") in power scenarios, the model is made more adaptable to different scenarios and action types through adaptive proposal selection, action proposal prediction model construction, and width-based center adjustment.

[0018] 3. By segmenting videos and filtering proposals, unnecessary calculations are reduced, which lowers the model's operational burden while ensuring accuracy, making it more suitable for the deployment needs of actual power safety monitoring systems;

[0019] 4. The use of probability distribution to replace fixed coordinate regression makes the model more tolerant of subjective differences from annotators; the deformable attention layer reduces the impact of occlusion on feature extraction, thus improving the overall robustness of the model in complex power scenarios. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a power timing action detection method based on probability distribution optimization in one embodiment of this application.

[0021] Figure 2 This is a schematic diagram of a power timing action detection method based on probability distribution optimization in one embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely one preferred embodiment of this application and are only used to explain this application. They do not limit the scope of protection of this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Example 1:

[0024] like Figure 1-2 As shown, a power timing action detection method based on probability distribution optimization includes the following steps:

[0025] Historical data is acquired, including historical power operation video data, historical power action training data, and historical power action verification data; among which, power equipment action video data refers to saving video surveillance footage. This video data is used for detection, facilitating subsequent auxiliary inspections.

[0026] A feature sequence segmentation layer is constructed based on action categories and start / end boundaries in historical power operation video data; where action category and start / end boundary {s i ,e i ,a i}, to guide model learning during training; s i e is the start time of the action. i a is the end time of the action. i Action category;

[0027] A motion proposal prediction model is constructed based on the historical power action training data and the feature sequence segmentation layer;

[0028] The historical power action verification data is input into the action proposal prediction model for prediction, and action proposal prediction data is obtained.

[0029] The results of the action proposal prediction data are matched with the reference data to obtain the offset probability distribution;

[0030] The real-time video data of the power equipment's actions and the offset probability distribution are input into the action proposal prediction model for calculation to obtain the power equipment action detection results.

[0031] In this embodiment, feature sequences are used to divide the model into layers, which guide model training and clarify the time boundaries and action categories of the learning target. This avoids interference from irrelevant features, thus more accurately capturing the essential patterns of power equipment actions. The deviation between the action proposal prediction data and the reference data is quantified through an "offset probability distribution relationship," essentially establishing a statistical correlation between the predicted time boundaries (start and end times) and the actual boundaries. During real-time detection, the model can correct prediction deviations based on this distribution, significantly improving the positioning accuracy of action start and end times. For power scenarios, the accuracy of action time boundaries is crucial, effectively reducing misjudgments caused by time positioning errors. Adapting to the specific characteristics of power equipment actions enhances the method's scenario relevance and practicality. By collaboratively outputting detection results using "real-time power equipment action video data + offset probability distribution + action proposal prediction model," the model retains its ability to learn stable patterns from historical data while dynamically correcting random deviations in real-time power equipment action video data through probability distribution, enabling the model to quickly adapt to the real-time dynamic changes in power equipment actions. This enhances the positioning accuracy of action time boundaries, meets the high requirements for timing accuracy in power scenarios, improves the reliability of action category recognition, and reduces the risk of false detection and missed detection in power scenarios. Simultaneously, through the combination of "model prediction + probability correction," the real-time requirements of power scenarios can be met while ensuring detection accuracy.

[0032] The following steps are performed in the feature sequence segmentation layer: Detection targets are obtained based on action categories and start / end boundaries; feature sequence segmentation of the power operation video data is then performed based on the detection targets. Specifically, the unedited video is divided into T consecutive non-overlapping video segments, each containing several video frames. These video segments are then fed into a pre-trained video feature extractor to construct the feature sequence segmentation layer, as shown in Formula 1 below:

[0033]

[0034] Where B is the batch size, L is the sequence length, and D is the number of batches. 0 For the feature dimension, pre-trained video feature extractors include I3D or VideoMAE V2.

[0035] Based on historical power equipment action training data and the aforementioned feature sequence segmentation, an action proposal prediction model is constructed, including:

[0036] The feature data of the feature sequence is extracted by using a convolutional neural network to construct multi-level temporal features and feature grids;

[0037] The multi-level temporal features and feature grids are input into the sine and cosine position encoder to obtain refined temporal features;

[0038] An action proposal prediction model is constructed based on historical power equipment action training data and the aforementioned multi-level temporal features and feature grids.

[0039] Specifically, the feature sequence is divided into layers X. 0 First, a total of N... embed The one-dimensional masked convolution and normalization operations of the layer achieve nonlinear transformation of the input features and preliminary capture of temporal information, as shown in Equation 2 below:

[0040] X 0 =LayerNorm(MaskedConv1D) (i) (X 0 ,M),i=0,1,2,…,N embed-1 (formula)(2);

[0041] Among them, X 0 For video features; MaskedConv1D(·) is a one-dimensional convolution operation with a mask; M 0 The initial mask is set to 1, where a value of 1 represents the actual video length and a value of 0 represents the padding portion. During this stage, since the video length remains unchanged, the initial mask is not modified.

[0042] Furthermore, MaskedConv1D(·) is a masked one-dimensional convolution operation: first, it undergoes a one-dimensional convolution operation with kernel_size set to 3, then it is multiplied by the mask M to mask out invalid regions, and then X... 0 A total of N applications branch One-dimensional masked convolution of layers, constructing N branch +1 layer of multi-level temporal features, where the stride of the one-dimensional convolution is set to 2 and the kernel_size is set to 3, making X i+1 The length is X i Half of it, and at the same time, the mask is updated synchronously as the feature length changes, making M 0 =M; as shown in Formula 3 below:

[0043] X i+1 M i+1 =LayerNorm(MaskedConv1D (i) (X i M i ;stride=2)),i=0,1,2,…,N branch-1 (formula)(3);

[0044] Among them, X i+1 The length of the downsampling operation becomes the lower-level temporal feature X. i Upper-layer temporal features of half the length; M i+1The length becomes the lower-layer mask M after the downsampling operation. i Half of the upper layer mask; X i M i These are the lower-level temporal features and the lower-level mask, respectively. Finally, the temporal features of each layer are... The splicing operation yields multi-level temporal features H. 0 As shown in Formula 4 below:

[0045]

[0046] Concat performs concatenation along the time dimension.

[0047] Specifically, a feature grid is constructed based on the video frame rate (fps), the video frame interval stride, and the video segment window size (window_size); and the temporal features X at each layer are used to construct the feature grid. i Length T i Create from 0 to T i -1 consecutive indices; calculate the center frame index corresponding to each feature based on the video frame interval stride and the video segment window size; as shown in Formula 5 below:

[0048]

[0049] Where indices is from 0 to T i -1 is the continuous index; stride is the video frame interval step size; window_size is the video segment window length; divide the center frame index by fps to obtain the feature grid, as shown in Formula 6 below:

[0050]

[0051] Where center_indices is the center frame index corresponding to each feature; fps is the video frame rate.

[0052] In obtaining refined temporal features based on sine and cosine position coding, multi-level temporal features, and feature grids, we first construct a sine and cosine position code Pos, and then combine it with a feature grid Grid and multi-level temporal features H. 0 These are input together into the encoder to obtain refined temporal feature memory. The encoder consists of a total of N... encoder The layer consists of a deformable attention layer (DA), as shown in Formula 7 below:

[0053] H (i) =DA (i) (H (i-1),Pos,Grid,H (i-1) ), l=1,2,…,N encoder (formula)(7);

[0054] Among them, H (i) This represents a new multi-level temporal feature optimized by a deformable attention layer; DA (i) This is a deformable attention layer. The implementation of the deformable attention layer references the Deformable-DETR algorithm in the object detection field, the only difference being that the original two-dimensional sampling method for image features is transformed into a one-dimensional sampling method adapted to temporal features; H (i-1) This represents the original multi-level temporal features output from the previous layer. After N layers, the output is...

[0055] Sine and cosine position coding

[0056] Where p∈[0,max_len-1] is the position index, and max_len is the multi-level temporal feature H. 0 The length of each layer of temporal features; pos∈[0,max_len-1] is the position index, and max_len is the length of the multi-layer temporal features H. 0 The length of the temporal features in each layer; i∈[0,D-1] is the dimension index; 10000 is a constant that controls the frequency difference, and exponential scaling makes different dimensions present different periods.

[0057] In constructing an action proposal prediction model based on multi-level temporal features, feature grids, and historical power equipment action training data, the action proposal prediction model is generated as shown in Formula 8 below:

[0058] Proposal-Init = {c ref ,w ref}(formula)(8);

[0059] Wherein, let the center reference position c ref =Grid, b is the base scaling scale, and the interval width is referenced from w. ref = b·fps×2 l-1 ,l∈{1,2,…,N branch}; fps is the video frame rate, which can be obtained in advance using ffmpeg.

[0060] The historical power action verification data is input into the action proposal prediction model for prediction, resulting in action proposal prediction data. This includes inputting the refined temporal features into a multilayer perceptron (MLP) to calculate the initial information for the predicted action proposal. Specifically, the refined temporal features (Memory) are input into different MLPs to predict a foreground score p for each temporal position. 0 Timing position center offset Δc 0 With width offset Δd 0 Finally, action proposal prediction information is obtained, including the center position c of the action interval. 0 With interval width d 0 As shown in Formula 9 below:

[0061] Proposal 0 =(c 0 ,d 0 )=(c ref +Δc 0 ·d ref ,d ref +Δd 0 ) (Formula)(9)

[0062] Among them, c ref d is the central reference position; ref c serves as a reference for the interval width. 0 The center position of the action interval; d 0 This represents the interval width.

[0063] The historical power action verification data is input into the action proposal prediction model for prediction. Obtaining the action proposal prediction data also includes processing the initial information of the predicted action proposal based on the foreground score to obtain a query vector. Specifically, based on the foreground score p... 0 An adaptive selection strategy is applied to the action proposal prediction information. This strategy divides the action proposal prediction information from each layer into S partitions, and selects the top K action proposal predictions with the highest foreground scores from each partition. A query vector Target is then generated based on the selected action proposal prediction information. 0 Construction process: First, the preset category index matrix labels∈R is constructed. B×L Initialize to the default value N cls N cls The total number of categories; then through the embedding layer Embedding∈R (N+1)×D Map the labels to the query vector, as shown in Formula 10 below:

[0064] Target 0 =Embedding(labels)∈RB×L×D (formula)(10)

[0065] Among them, Target 0 is the query vector; Embedding is the embedding layer; labels is the category index matrix.

[0066] Matching the action proposal prediction data with the reference data to obtain the offset probability distribution involves inputting the reference data corresponding to each action proposal, the query vector, the initial information of the predicted action proposal, and the refined temporal features into the decoder, so that the decoder outputs predicted action proposal information and action category probabilities. The decoder includes a multi-head self-attention layer and a deformable self-attention layer. Further, matching the action proposal prediction data with the reference data to obtain the offset probability distribution also includes: each decoder layer optimizing and updating the query vector through the multi-head self-attention layer and the deformable self-attention layer; processing the optimized and updated query vector according to the offset probability distribution and candidate offset values ​​to obtain the center offset probability and width offset probability of the query vector; iteratively updating the center position and width of the predicted action proposal information according to the predicted offset probability distribution and candidate offset values; and outputting the action boundary and category probability based on the center offset probability and width offset probability of the query vector output by each layer and the center position and width of the predicted action proposal information. The multi-head self-attention layer and the deformable self-attention layer can dynamically capture key action features, enhancing robustness to occluded areas (e.g., focusing on "hand operations" rather than the occluded "body").

[0067] Specifically, the query vector Target 0 Predicting action proposal initial information Proposal 0 The refined temporal features (Memory) output by the encoder are fed into the decoder. The decoder continuously optimizes the initial information of the predicted action proposal and the refined temporal features based on the probability distribution. The single-layer decoder layer (DecoderLayer) consists of a multi-head attention layer (MA) and a deformable attention layer (DA), for a total of N. decoder Layer. As shown in Formula 11 below:

[0068] Target i′ =MA(Target) i-1 ), i∈{1,2,...,N decoder}

[0069] Target i =DA(Target) i′ ,Pos,Proposal i-1(,Memory), i∈{1,2,...,N) decoder} (formula)(11)

[0070] Where Pos is the sine and cosine positional encoding; Memory is the refined temporal feature; Proposal 0 Provide prediction information for action proposals.

[0071] Specifically, based on Target i The predicted interval offset probability is shown in Formula 12 below:

[0072]

[0073] Among them, MLP (Target) i ) represents the query vector Target for each layer. i The unnormalized score output after inputting into a Multilayer Perceptron (MLP) is given, where the input and hidden dimensions are D, and the output dimension is 2(N). bin +1); The center position in the normalized interval offset probability The interval width in the normalized interval offset probability Softmax is a normalization operation, applied to MLP (Target) and MLP (Target) respectively. i The first N outputs bin +1 dimension features and the subsequent N bin The +1 dimension feature is normalized to obtain the normalized center position offset probability. With interval width offset probability

[0074] Furthermore, the prediction results include the probability distribution of the offset between the center position and width of the action interval. Each distribution predicts the probability of a set of candidate offset values. This probability distribution prediction model addresses positioning uncertainty, replacing fixed coordinate regression with a "probability-weighted sum of candidate offset values" to accommodate the subjective differences in the annotators' perception of action boundaries. The candidate offset values ​​are determined by the weighting function W(n), which contains N... bin There are several discrete intervals, where n is the specific number of the discrete interval. `bound` and `curv` control the upper bound and curvature of the weighting function, respectively. The upper bound of the weighting function represents the maximum value of the offset that may need to be adjusted between the center position and the interval width. The greater the curvature, the steeper the weighting function, meaning the offset changes more rapidly. See Formula 13 below:

[0075]

[0076] Where bound and curv are both constant values; N bindenoted as a discrete interval; n is the specific number of the discrete interval.

[0077] Furthermore, the action proposal prediction information is updated using candidate offset values ​​and offset probability distributions. i =(c i ,d i First, update the interval width d. i As shown in Formula 14 below:

[0078]

[0079] Where, d i d is the interval width; i-1 W(n) represents the interval width of the previous layer's output; W(n) is the weighting function. This represents the interval offset probability.

[0080] Secondly, update the center position c. i The adjustment amount at the center position is scaled using the interval width to ensure that the adjustment amount is proportional to the actual size of the frame, as shown in Formula 15 below:

[0081]

[0082] Among them, c i-1 The center position of the output from the previous layer; d i-1 The amount by which the center position of the range width of the output from the previous layer is adjusted; This represents the center position offset probability within the interval offset probability. Finally, based on the Target... i Predicting the probability of action category ClS i =MLP(Target) i This completely solves the core defect of existing methods, namely that "fixed coordinate regression cannot cope with annotation uncertainty and occlusion," and significantly improves the accuracy and robustness of action boundary localization.

[0083] The action proposal prediction model, constructed by dividing the historical power equipment operation training data into layers based on the feature sequence, also includes training the model using historical power equipment operation video data and a loss function containing action proposal loss and action category loss. This yields the loss function weights and correlation weights for the real-time power equipment operation video data. Specifically, the action proposal prediction information output from each layer is used to... i With action category probability ClS i Regression and classification loss functions are applied to guide model training. The total loss function is defined as shown in Equation 16 below:

[0084] L total =λ cls L cls+λ reg L reg (formula)(16)

[0085] Where L cls For classification loss, Focal Loss is used to address the class imbalance problem. reg For bounding box loss, possible loss implementations include DIoU loss, λ cls and λ reg These are the weights for the bounding box loss and the classification loss, respectively.

[0086] Example 2:

[0087] An electronic device includes a memory, a processor, and a battery module; the memory stores a program; the battery module powers the memory; the processor executes the program and, during program execution, implements the time-series causality-based power action detection method described above. The electronic device can be a personal computer, server, or network device, etc., and the memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] Example 3:

[0089] A computer-readable storage medium is provided for storing a computer program or instructions that, when executed by a processing device, implement the aforementioned time-series causality-based power action detection method. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives), etc. Similarly, in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0090] Example 4:

[0091] As shown in Table 1, to verify the beneficial effects of this application, scientific demonstration was conducted through economic benefit calculations and simulation experiments. This embodiment conducted experiments on both existing traditional methods and the method of this embodiment. This embodiment selected a time-series action detection dataset for power operation scenarios. This dataset focuses on typical operational behaviors during power outage operations on utility poles, covering five key actions: voltage testing, grounding wire installation, grounding wire removal, climbing, and pole removal. The dataset contains 3,104 unedited original power operation site videos with complete annotation information. The average action duration is 59.08 seconds. The sample size and average duration for each type are as follows: climbing 3260 segments (56.22 seconds), pole removal 2827 segments (49.92 seconds), voltage testing 1432 segments (21.12 seconds, shortest), grounding wire installation 789 segments (110.49 seconds, longest), and grounding wire removal 1452 segments (81.61 seconds). Overall, the samples for climbing and descending poles were the most abundant, while the samples for attaching ground wires were relatively scarce. In terms of duration, the durations for attaching and removing ground wires were significantly longer than the overall average, while the durations for voltage detection were significantly shorter than the average. Specifically, the training set contained 2,787 video segments, and the test set contained 317 video segments, which can provide data support for research on related temporal action detection technologies.

[0092] In terms of comparison models, advanced temporal action detection algorithms in recent years were selected as the comparison baseline. To ensure that the model parameters do not become a bottleneck restricting the model's capabilities, in addition to following the model's default settings, the width and height of the model were scaled to ensure that the number of parameters of these advanced temporal action detection algorithms is consistent with the number of parameters of the model proposed in this invention, and the algorithm performance is reported together.

[0093] The proposed model configuration is as follows:

[0094] 1. The pre-trained video feature extractor uses VideoMAE V2;

[0095] 2. The upper bound of the weighting function is set to 5, the curvature is set to 0.4, and the number of discrete intervals is N. bin Set to 64, encoder layer number N encoder Set to 3, decoder layer number N decoder Set to 5, N is the number of one-dimensional convolutional layers for local feature modeling. embed Set to 3, N is the number of one-dimensional convolutional layers constructed from multi-scale features. branch Set to 4;

[0096] 3. The initial learning rate used during training is 1×10⁻⁶. -4 The Adam optimizer optimizes the model with a weight decay coefficient of 0.05.

[0097] Therefore, this application uses the mAP (mean Average Precision) metric to evaluate model performance. mAP is a widely used evaluation standard in the field of temporal action detection, comprehensively reflecting the model's detection accuracy and recall across various action categories. Specifically, for each action category in the test set, the mean precision (AP) is first calculated, which is the area under the precision and recall curves of the model at different confidence thresholds. Then, the arithmetic mean of the mean precision (AP) for all categories is taken to obtain the mAP metric. This metric effectively evaluates the model's overall performance in multi-class detection tasks, balancing accuracy and detection capability, and has strong representativeness. This application's performance evaluation based on the mAP metric comprehensively and objectively reflects the effectiveness of the proposed method in temporal action detection scenarios.

[0098] In a time-series action detection dataset for power operation scenarios, the model proposed in this application achieves a significantly higher mAP compared to state-of-the-art methods, indicating that the proposed model can obtain more accurate time-series action detection results. Furthermore, adjusting the number of parameters in the comparison models did not yield better results, proving that the number of parameters in the original comparison models is sufficient to support their respective maximum modeling capabilities.

[0099] Table 1: mAP of the model

[0100]

[0101] The results show that, through validation using a time-series action detection dataset for power operation scenarios, the mAP results of the model proposed in this application are superior to other models, proving that the model can obtain more accurate time-series action behavior detection results.

[0102] The above-described specific embodiments are preferred embodiments of a power timing action detection method, system, and electronic device based on probability distribution optimization according to this application. They are not intended to limit the specific scope of this application. The scope of this application includes but is not limited to these specific embodiments. All equivalent changes made in accordance with the shape and structure of this application are within the protection scope of this application.

Claims

1. A power timing action detection method based on probability distribution optimization, characterized in that, Includes the following steps: Acquire historical data, including historical power operation video data, historical power action training data, and historical power action verification data; A feature sequence segmentation layer is constructed based on the action categories and start / end boundaries in the historical power operation video data. A motion proposal prediction model is constructed based on the historical power action training data and the feature sequence segmentation layer; Based on the historical power operation video data, reference data corresponding to each action proposal is obtained; The historical power action verification data is input into the action proposal prediction model for prediction, and action proposal prediction data is obtained. The results of the action proposal prediction data are matched with the reference data to obtain the offset probability distribution; The real-time video data of the power equipment's actions and the offset probability distribution are input into the action proposal prediction model for calculation to obtain the power equipment action detection results.

2. The power timing action detection method based on probability distribution optimization according to claim 1, characterized in that, The following steps are performed in the feature sequence partitioning layer: The detection target is obtained based on the action category and the start and end boundaries; Based on the detection target, perform feature sequence segmentation of the power action video data.

3. The power timing action detection method based on probability distribution optimization according to claim 1, characterized in that, The step of constructing an action proposal prediction model by dividing the historical power action training data into layers based on the feature sequence includes: The feature data of the feature sequence is extracted by using a convolutional neural network to construct multi-level temporal features and feature grids; The multi-level temporal features and the feature grid are input into a sine-cosine position encoder to obtain refined temporal features; An action proposal prediction model is constructed based on the historical power action training data, combined with the multi-level temporal features and the feature grid.

4. The power timing action detection method based on probability distribution optimization according to claim 3, characterized in that, The step of inputting the historical power action verification data into the action proposal prediction model for prediction to obtain action proposal prediction data includes: The refined temporal features are input into a multilayer perceptron for calculation to obtain the initial information for the predicted action proposal.

5. The power timing action detection method based on probability distribution optimization according to claim 4, characterized in that, The step of inputting the historical power action verification data into the action proposal prediction model for prediction to obtain action proposal prediction data further includes: The initial information of the predicted action proposal is processed based on the foreground score to obtain a query vector.

6. The power timing action detection method based on probability distribution optimization according to claim 5, characterized in that, The step of matching the result of the action proposal prediction data with the reference data to obtain the offset probability distribution includes: The reference data, the query vector, the initial information of the predicted action proposal, and the refined temporal features are input into the decoder so that the decoder outputs the predicted action proposal information and the action category probability.

7. The power timing action detection method based on probability distribution optimization according to claim 6, characterized in that, The decoder includes a multi-head self-attention layer and a deformable self-attention layer; wherein, the step of matching the result of the action proposal prediction data with the reference data to obtain the offset probability distribution further includes: Each layer of the decoder optimizes and updates the query vector through the multi-head self-attention layer and the deformable self-attention layer; The optimized update query vector is processed based on the offset probability distribution and candidate offset values ​​to obtain the center offset probability and width offset probability of the query vector. The center position and width of the predicted action proposal information are iteratively updated based on the predicted offset probability distribution and candidate offset values. Based on the center offset probability and width offset probability of the query vector output by each layer, and the center position and width of the predicted action proposal information, the action boundary and category probability are output.

8. The power timing action detection method based on probability distribution optimization according to claim 7, characterized in that, The step of constructing an action proposal prediction model by dividing the historical power action training data into layers based on the feature sequence further includes: By training a prediction model using historical power operation video data and an action proposal model that includes a loss function, the loss function weights and correlation weights of real-time power equipment action video data are obtained.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a battery module; The memory is used to store programs; The battery module is used to power the memory; The processor is configured to execute the program and, when executing the program, implement the power timing action detection method based on causal timing modeling as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by a processing device, implement the power timing action detection method based on causal timing modeling as described in any one of claims 1-8.

Images