Anomaly detection method and system of fusion size model cooperates cross time domain perception

Abstract

The application relates to the technical field of video detection, in particular to an abnormality detection method and system fusing a size model and cooperating with cross-time domain perception. The method comprises the following steps: configuring a small model, a multi-modal large model and cross-time domain parameters based on a power scene; acquiring a monitoring video stream; calculating a dynamic frame extraction frequency based on a scene change coefficient of continuous video frames of the monitoring video stream; extracting a frame sequence and recording a time stamp of each frame; inputting the frame sequence into the small model to obtain a candidate target, removing redundancy through non-maximum suppression, screening effective targets by combining single-frame confidence and inter-frame time domain continuous overlap rate, marking an alarm image and generating a preliminary analysis result; taking the alarm image time stamp as the center, collecting cross-time domain video frames, power scene sensor data and environmental data according to a dynamic time window, extracting visual time domain features and power scene sensor time sequence features, and fusing to obtain cross-time domain fusion features. The method solves the misjudgment and missed judgment problems caused by single-frame dependence in power scene abnormality detection.

Description

Technical Field

[0001] This invention relates to the field of video detection technology, and in particular to an anomaly detection method and system that integrates large and small models for collaborative cross-temporal perception. Background Technology

[0002] Video anomaly detection technology plays a crucial role in the safe production of smart power construction sites and smart power stations. Its core function is to monitor in real time abnormal situations such as construction personnel's illegal operation, improper stacking of materials, equipment icing, and station fires, so as to avoid safety accidents such as electric shock, equipment damage and fire, and ensure the stable operation of power equipment and the safety of personnel and property at the station.

[0003] With the development of deep learning technology, collaborative detection using small and large models has become a mainstream technical direction. Lightweight small models, such as the YOLO series and lightweight Transformer, have the advantages of high computational efficiency and low resource consumption, and can be adapted to edge devices to achieve high-frequency target initial detection, meeting real-time requirements. Multimodal large models, such as Qwen3-VL, with their powerful semantic understanding and reasoning capabilities, can parse power industry information and accurately judge fuzzy targets initially screened by small models, meeting the high-precision detection requirements of power scenarios. For example, CN120449105A proposes a device defect identification scheme based on collaborative detection of small and large models, which uses a small model to quickly locate the device area, and a large model to further identify the defect type.

[0004] However, existing methods for anomaly detection using both large and small models still suffer from key technical deficiencies. The core issue lies in their failure to fully utilize cross-temporal data and capture the temporal evolution characteristics of anomalies in power scenarios. Specifically, existing technologies often focus on single-frame image detection and anomaly assessment, neglecting the strong temporal correlation of anomalous events in power scenarios. For example, a power plant fire undergoes a continuous evolution process of "smoke → open flame → spread," equipment icing is a cumulative process under low-temperature conditions, and violations by construction workers also exhibit continuous action. Detection based solely on single-frame data is prone to misjudgment due to instantaneous interference, such as light and shadow reflections or dust obstruction, or to missed detection due to overlooking key features in the anomaly development process.

[0005] Furthermore, existing technologies for multimodal large-scale model fusion inference are limited to single-frame image and text information, without incorporating sensor time-series data. This results in an incomplete understanding of anomalies, and the data sampling and processing do not take into account the differences in the speed of anomaly evolution, making it difficult to balance real-time performance and detection accuracy.

[0006] Therefore, existing technologies, lacking a cross-time-domain data fusion mechanism, cannot effectively solve the problem of false positives and false negatives caused by single-frame dependence in power scenario anomaly detection, and are difficult to meet the anomaly detection needs of smart power construction sites. Summary of the Invention

[0007] To address the aforementioned problems, this invention provides an anomaly detection method and system that integrates large and small models for collaborative cross-temporal perception.

[0008] In a first aspect, the present invention provides an anomaly detection method that integrates large and small models for collaborative cross-temporal perception, comprising: Based on power scenario configuration, small models, multimodal large models, and cross-time domain parameters are configured; Acquire the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames in the monitoring video stream, extract the frame sequence and record the timestamp of each frame; The frame sequence is input into the small model to obtain candidate targets. Redundancy is removed by nonmaximum suppression. Valid targets are selected by combining single-frame confidence and inter-frame temporal continuous overlap rate. Alarm images are marked and preliminary analysis results are generated. Centered on the alarm image timestamp, cross-temporal video frames, power scene sensor data and environmental data are collected according to the dynamic time window. Visual temporal features and power scene sensor temporal features are extracted and fused to obtain cross-temporal fusion features. The alarm image, preliminary analysis results, and cross-temporal fusion features are input into the multimodal large model. Visual features and text features are extracted and weighted and fused with the cross-temporal fusion features. The authenticity of the anomaly is determined based on the fusion features, the severity of the detected anomaly is evaluated, and the detection results are output.

[0009] Furthermore, the small model configuration based on the power scenario includes: constructing anchor boxes for abnormal targets in the power scenario, ensuring that the anchor box size matches the actual characteristics of the power target, and configuring an optimized total loss function that includes bounding box regression loss, classification loss, and confidence loss, expressed by the formula: ,in, For bounding box regression loss, For classifying losses, The confidence loss is calculated using the binary cross-entropy loss formula. and These are preset weight parameters used to balance detection accuracy and inference efficiency.

[0010] Furthermore, the configuration of the multimodal large model includes: constructing a power scenario anomaly pattern library, which includes feature vectors corresponding to core anomaly types; these feature vectors include visual feature vectors, temporal feature vectors, and textual feature vectors; setting a confidence assessment mechanism; and calculating the confidence level through the joint probability of the multimodal large model's output sequences, expressed by the formula: When S is not lower than a preset high confidence threshold, it is determined to be a reliable inference result. For alarm images, For input text, The output sequence is the output sequence of a multimodal large model, where N is the length of the output sequence. Let S be the conditional probability of the i-th token in the output sequence, and S be the confidence level.

[0011] Furthermore, the setting of the cross-time domain parameters includes: A base time window is set based on the evolution speed of power anomaly types; The dynamic time window is obtained by dynamically adjusting the abnormal evolution velocity coefficient, as shown in the formula. ,in, Based on the time window, This represents the anomalous evolution rate coefficient. To preset the number of frames, For the first The area of ​​the abnormal region in the frame, for The average area of ​​frame anomalies; Set the sensor data sampling frequency so that the slowly changing sensor data sampling frequency is aligned with the frame skipping frequency, and the rapidly changing sensor data sampling frequency is a preset multiple of the frame skipping frequency, and the time synchronization error between all sensor data and video frame data does not exceed the preset synchronization threshold.

[0012] Furthermore, the calculation of the dynamic frame extraction frequency based on the scene change coefficients of continuous video frames from the monitored video stream, and the extraction of the frame sequence and recording of the timestamp of each frame, includes: Perform Gaussian blur denoising and bilinear interpolation on the video frames to scale them to a small model preset input size; The scene change coefficient is calculated by taking the average inter-frame pixel difference of a preset number of consecutive video frames, and the calculation formula is as follows: in, To preset the number of frames, For the first Original video frames, , These are the width and height of the original video frame, respectively. This is the scene change coefficient; The dynamic frame rate is calculated based on the scene change coefficient. The formula for calculating the dynamic frame rate is as follows: ,in The preset frame rate for the video. To set a preset adjustment coefficient, video frames are extracted at the dynamic frame extraction frequency to form a frame sequence, and the timestamp of each video frame is recorded.

[0013] Furthermore, the process of removing redundancy through nonmaximum suppression and selecting effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate includes: Non-maximum suppression is performed on the candidate targets output by the small model. The cross-union ratio threshold of the non-maximum suppression is set to a preset IoU threshold. When the cross-union ratio of two candidate targets exceeds the preset IoU threshold, the target with higher confidence is retained. The formula for calculating the temporal consecutive overlap rate between frames is as follows: ,in, The target bounding box for the current frame. , These are the bounding boxes of the same type in the previous and next frames, respectively. The inter-frame temporal continuous overlap rate; A dual screening condition is set to determine the effective target. The dual screening condition is that when the confidence of a candidate target in a single frame is not lower than a preset first confidence threshold and the continuous overlap rate in the temporal domain is not lower than a preset overlap rate threshold, it is marked as an effective target and an alarm image and preliminary analysis results are generated.

[0014] Furthermore, the step of collecting cross-time domain video frames, power scene sensor data, and environmental data according to a dynamic time window, centered on the alarm image timestamp, includes: An efficient encoding and decoding algorithm is used to encode and decode cross-temporal video frames. Kalman filtering is used to denoise the sensor data. Denoising data is obtained through state prediction, covariance prediction, Kalman gain calculation and state update. Based on the timestamp of the cross-temporal video frame, linear interpolation is used to perform time alignment between the video frame and the sensor data.

[0015] Furthermore, the extraction of visual temporal features and power scene sensor temporal features, and the fusion to obtain cross-temporal fusion features, includes: A temporal convolutional network is used to extract visual temporal features from cross-temporal video frames, and a long short-term memory network is used to extract sensor temporal features from sensor data. By assigning frame-level weights to visual temporal features through an attention mechanism and combining them with pre-defined sensor feature weights, cross-temporal fused features are obtained. ,in The frame-level attention weights are calculated using the cosine similarity between visual temporal features and sensor temporal features. To preset sensor feature weights, The number of frames for visual temporal features.

[0016] Furthermore, the process of determining the authenticity of anomalies based on fused features and assessing the severity of detected anomalies, outputting detection results, includes: A multimodal large model is used to extract visual features from alarm images, and the preliminary analysis results are converted into text features based on a word vector model. The cross-temporal fusion features are upscaled to a preset feature dimension through a fully connected layer. Weights are dynamically assigned based on the correlation scores between each feature and the abnormal event, and the final fusion features are obtained by weighted fusion. Calculate the cosine similarity between the final fused features and the features in the power anomaly pattern library, and combine the confidence level to determine the authenticity of the anomaly; The severity of power anomalies is assessed by setting intensity indicators based on the type of power anomaly. A linear regression model is used to fit the anomaly intensity data to obtain a trend curve. The development trend is determined based on the trend slope, and the anomaly intensity prediction value is output after a preset time interval.

[0017] Secondly, an intelligent auction system includes: The model configuration module is configured to configure small models, multimodal large models, and cross-time domain parameters based on power scenarios. The video acquisition module is configured to acquire the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames of the monitoring video stream, extract the frame sequence and record the timestamp of each frame. The small model initial screening module is configured to input frame sequences into the small model to obtain candidate targets, remove redundancy through non-maximum suppression, and filter effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate, mark alarm images and generate preliminary analysis results. The data processing module is configured to collect cross-temporal video frames, power scene sensor data and environmental data in a dynamic time window, centered on the alarm image timestamp, extract visual temporal features and power scene sensor temporal features, and fuse them to obtain cross-temporal fusion features. The large model re-judgment module is configured to input alarm images, preliminary analysis results, and cross-temporal fusion features into a multimodal large model, extract visual features and text features, and weightedly fuse them with the cross-temporal fusion features. Based on the fusion features, it determines the authenticity of the anomaly, evaluates the severity of the detected anomaly, and outputs the detection results.

[0018] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned anomaly detection method for fusion-size model collaborative cross-temporal perception.

[0019] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide the aforementioned anomaly detection method for fusion of size model collaborative cross-temporal perception.

[0020] In summary, the present invention has the following beneficial technical effects: 1. This invention utilizes a technique that calculates dynamic frame skipping frequency based on a scene change coefficient. First, it calculates the scene change coefficient using the average inter-frame pixel difference of a preset number of video frames. Then, it combines this with a preset video frame rate and an adjustment coefficient to achieve adaptive frame skipping frequency. When the scene is stable, the frame skipping frequency is reduced to decrease the computational and storage resource consumption of edge devices; when the scene is dramatic, the frame skipping frequency is increased to ensure the capture of abnormal dynamic details. This avoids the waste of resources associated with fixed frame skipping and also prevents the loss of details.

[0021] 2. In the initial screening of small models, this invention first performs non-maximum suppression on candidate targets to remove overlapping redundancy, and then eliminates false targets caused by instantaneous interference by using the inter-frame temporal continuous overlap rate, which significantly reduces the false judgment rate of small model initial screening, reduces the amount of invalid data transmitted to large models, and at the same time reduces the computational pressure of large models, thus solving the problem of high false judgment rate of single-frame dependent initial screening.

[0022] 3. This invention utilizes cross-time-domain data processing to set a base time window based on the evolution speed of power anomalies. It introduces an anomaly evolution speed coefficient and dynamically adjusts the window size to adapt to power anomalies with different evolution speeds. By mining the temporal correlation of anomalies, it supplements the missing contextual information in single-frame data, solving the problem of misjudgment and missed judgment caused by single-frame dependence in existing technologies. Simultaneously, it outputs multi-dimensional decision information, setting specific intensity indicators based on the type of power anomaly to assess severity, determine development trends, and predict intensity after a preset time interval, thereby reducing the risk of power safety accidents. Attached Figure Description

[0023] Figure 1 This is a flowchart of an anomaly detection method that integrates large and small models for collaborative cross-temporal perception, according to Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the initial screening process for a small-scale power scenario according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the cross-time domain data processing and large model input process of Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the multimodal large model fusion reasoning and result output process of Embodiment 1 of the present invention; Figure 5 This is a block diagram of a module in Embodiment 2 of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the accompanying drawings.

[0025] Example 1 Reference Figure 1 In this embodiment An anomaly detection method that integrates large and small models for collaborative cross-temporal awareness includes: S1. Configure small models, multimodal large models, and cross-time domain parameters based on power scenarios; To ensure that the detection model and parameters are well-suited to the anomaly detection needs of smart power construction sites and smart power plants, and to guarantee that the model's inference accuracy and real-time performance meet the requirements of power safety production, the following specific configuration steps are required to provide fundamental support for the entire subsequent anomaly detection process.

[0026] S11. The small model based on power scenario configuration includes: Anchor boxes are constructed for abnormal targets in the power scenario, ensuring that the anchor box size matches the actual characteristics of the power target. An optimized total loss function is configured, which includes bounding box regression loss, classification loss, and confidence loss, and is expressed by the formula: ,in, For bounding box regression loss, For classifying losses, The confidence loss is calculated using the binary cross-entropy loss formula. and These are preset weight parameters used to balance detection accuracy and inference efficiency.

[0027] For typical abnormal targets in power scenarios, specifically construction workers not wearing reflective vests, equipment icing, power plant fires, and illegally stacked materials, dedicated anchor frames are constructed. The size of the anchor frames is determined based on the actual physical size distribution of the aforementioned power abnormal targets. Specifically, the anchor frame size for construction workers not wearing reflective vests ranges from 35×70 pixels to 65×110 pixels; the anchor frame size for equipment icing ranges from 25×30 pixels to 65×70 pixels; the anchor frame size for power plant fires ranges from 15×20 pixels to 50×60 pixels; and the anchor frame size for illegally stacked materials ranges from 80×80 pixels to 150×150 pixels. To ensure accurate matching between the anchor frames and the actual characteristics of the power targets, a non-maximum suppression algorithm (NMS) is used to remove redundant detection boxes.

[0028] For classification loss, the binary cross-entropy loss formula is used for calculation, and the formula is as follows: In the formula, N is the batch size. This is the true category label for the power anomaly target; a value of 1 is assigned if the target exists, and a value of 0 is assigned if the target does not exist. The class probabilities predicted by the model; The confidence loss is also calculated using the binary cross-entropy loss formula, which is used to optimize the confidence score for the existence of power anomaly targets. and For the preset weight parameters, where The value range is from 1.0 to 1.5. The value range is from 0.5 to 1.0, preferably... =1.2、 =0.8, this weight configuration balances the detection accuracy and inference efficiency in the power scenario, and avoids the decline in model adaptability due to excessive pursuit of a single indicator.

[0029] S12. The configuration of the multimodal large model includes: A power scenario anomaly pattern library is constructed, comprising feature vectors corresponding to core anomaly types. These feature vectors include visual feature vectors, temporal feature vectors, and textual feature vectors. A confidence assessment mechanism is established, calculating the confidence level using the joint probability of the output sequences from a multimodal large model, expressed by the formula: When S is not lower than a preset high confidence threshold, it is determined to be a reliable inference result. For alarm images, For input text, The output sequence is the output sequence of a multimodal large model, where N is the length of the output sequence. Let S be the conditional probability of the i-th token in the output sequence, and S be the confidence level.

[0030] The power scenario anomaly pattern library contains core anomaly types for power scenarios, specifically feature vectors corresponding to construction worker violations, equipment icing, station fires, and improper material storage. Each feature vector has a uniform 1024-dimensional dimension and includes visual feature vectors, temporal feature vectors, and textual feature vectors. The visual feature vectors are generated based on image pixel features of the power anomaly target; the temporal feature vectors are constructed based on the temporal variation patterns of the anomaly events; and the textual feature vectors are obtained by encoding professional descriptive text for the power anomaly type. This anomaly pattern library is trained on historical power scenario anomaly data, containing over 5000 sets of labeled samples, providing a benchmark for subsequent anomaly determination.

[0031] A specific example of the input text is "Determine whether there is a fire at the site in this image, and explain the authenticity of the anomaly based on cross-time domain temperature change data"; Y is the output sequence of the multimodal large model, which specifically includes the anomaly authenticity determination result, relevant reasoning basis, and credibility explanation; N is the length of the output sequence, ranging from 50 to 200 tokens. The conditional probability of the i-th token in the output sequence is calculated by the softmax function of the output layer of the multimodal large model, and S is the confidence level. The preset high confidence threshold is 0.8. When S is not lower than 0.8, it is judged as a reliable inference result, ensuring that the anomaly detection conclusion output by the large model has high reliability.

[0032] S13. The setting of the cross-time domain parameters includes: S131. Set a basic time window based on the evolution speed of power anomaly types; A base time window is set based on the evolution speed of power anomaly types, including the base time window for station fires. Adapting to the rapid diffusion characteristics of flames and smoke, the basic time window for equipment icing. It adapts to the slow accumulation characteristics of icing and the basic time window for construction workers to violate regulations. Adapting to the rapid changes in personnel movements, and the basic time window for illegal material stacking. It adapts to the stable changes in the stacking state of materials, and the value of the basic time window is determined based on the actual evolution law of various abnormal types in the power scenario to ensure coverage of the core development stage of abnormal events.

[0033] S132. A dynamic time window is obtained by dynamically adjusting the abnormal evolution velocity coefficient, as shown in the formula. ,in, Based on the time window, This represents the anomalous evolution rate coefficient. To preset the number of frames, For the first The area of ​​the abnormal region in the frame, for The average value of the frame anomaly area, preferably For example, a frame represents the base time window for a fire at a fire station. The abnormal areas of five consecutive frames are 100 pixels, 150 pixels, 220 pixels, 300 pixels, and 380 pixels, respectively. The calculated values ​​are... , =0.243, dynamic time window =24.86s. This dynamic adjustment allows the time window to adapt to abnormal evolution speeds, avoiding information loss or redundancy caused by a fixed window.

[0034] S133. Set the sensor data sampling frequency so that the slowly changing sensor data sampling frequency is aligned with the frame skipping frequency, and the rapidly changing sensor data sampling frequency is a preset multiple of the frame skipping frequency, and the time synchronization error between all sensor data and video frame data does not exceed the preset synchronization threshold.

[0035] The sensor data sampling frequency is set. For slowly changing sensors, including temperature and humidity sensors, their sampling frequency is aligned with the video frame-dropping frequency to ensure time synchronization of data acquisition. For rapidly changing sensors, including device current and voltage sensors, their sampling frequency is set to a preset multiple of the frame-dropping frequency, preferably 2 times, to ensure the capture of rapid fluctuations in device operating parameters. The time synchronization error between all sensor data and video frame data does not exceed a preset synchronization threshold, which is 10ms. This setting ensures time consistency during cross-time domain data fusion, preventing time deviations from affecting the accuracy of abnormal feature extraction.

[0036] Reference Figure 2 S2. Obtain the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames of the monitoring video stream, extract the frame sequence and record the timestamp of each frame; The video streams from high-definition surveillance cameras deployed at smart power construction sites and smart power plants are accessed via Real-Time Streaming Protocol (RTSP). These cameras have resolutions adapted to the full coverage requirements of power scenarios, ensuring clear capture of abnormal details in construction and equipment areas. This step dynamically adjusts the frame sampling frequency to reduce data redundancy and conserve edge device resources during stable scenes, while increasing sampling density to capture abnormal dynamics during turbulent scenes. This provides high-quality, highly effective frame sequence data for subsequent small-scale model screening.

[0037] The calculation of the dynamic frame extraction frequency based on the scene change coefficient of continuous video frames in the monitoring video stream, and the extraction of the frame sequence and recording of the timestamp of each frame, includes: S21. Perform Gaussian blur denoising and bilinear interpolation scaling on the video frames to the small model preset input size; A Gaussian blur algorithm is used to denoise the original video frames. The Gaussian kernel size is set to 5×5, and the standard deviation σ is set to 1.5. This parameter configuration can effectively eliminate instantaneous interference such as dust and light reflection in power scenes, improve the purity of video frames, and avoid interference factors affecting the accuracy of subsequent target detection. After denoising, the video frames are scaled to the preset input size of the small model using a bilinear interpolation algorithm. This preset input size is uniformly set to 640×640 pixels, consistent with the small model input requirements configured in S11, ensuring that the video frames can be directly input into the small model for forward propagation calculations.

[0038] S22. Calculate the scene change coefficient, which is determined by the average inter-frame pixel difference of a consecutive preset number of video frames, and the calculation formula is as follows: in, To preset the number of frames, For the first Original video frames, , These are the width and height of the original video frame, respectively. This is the scene change coefficient; The optimal value for M is 5 frames. This number can reflect the changing trend of video content in a short period of time, while avoiding delays caused by excessive computation. This is the original video frame i, and the video frame after preprocessing in S21 still uses this identifier; , These are the width and height of the original video frame, respectively. The preferred resolution of the original video frame of the power scene monitoring camera is 1920×1080 pixels, i.e., W=1920, H=1080. This represents the sum of the absolute values ​​of the differences in grayscale values ​​between corresponding pixels in frame i and frame i+1, used to quantify the content differences between the two frames; The ratio represents the total number of pixels in a single frame of video, and it normalizes the differences between frames. k is the scene change coefficient, which ranges from [0,1]. The closer k is to 1, the more drastic the changes in the video content, such as the frequent movement of personnel and equipment in the construction area of ​​a power plant. The closer k is to 0, the more stable the video content, such as the static equipment area of ​​a power station. This coefficient directly reflects the dynamic characteristics of power scene videos.

[0039] S23. Calculate the dynamic frame skipping frequency based on the scene change coefficient. The formula for calculating the dynamic frame skipping frequency is as follows: ,in The preset frame rate for the video. To set a preset adjustment coefficient, video frames are extracted at the dynamic frame extraction frequency to form a frame sequence, and the timestamp of each video frame is recorded.

[0040] The preset frame rate for power scene monitoring video is preferably 25fps, which meets the industry standard for real-time monitoring. The preferred value for K is 5, which is used to balance the impact of scene changes on the frame sampling frequency. This indicates a floor function, and `max(1,・)` ensures the frame extraction frequency is no less than 1 frame / second, preventing data acquisition interruptions due to extremely stable scenes. After calculating the dynamic frame extraction frequency using the above formula, video frames are extracted from the preprocessed video stream at this frequency to form a continuous frame sequence. For example, when the power scene is a stationary area (k=0.1), That is, one frame is extracted every 16 frames, approximately 0.64 seconds per frame. This saves computing resources while ensuring device status monitoring. When the power scenario is a construction area (k=0.8), This means that one frame is extracted every five frames, approximately 0.2 seconds per frame, ensuring the capture of dynamic and abnormal details such as unauthorized actions by construction workers and equipment movement. During frame extraction, the timestamp of each video frame is recorded synchronously in the format "YYYY-MM-DDHH:MM:SS.ms", accurate to the millisecond level, providing a precise time reference for subsequent cross-time domain data acquisition and time alignment.

[0041] S3. Input the frame sequence into the small model to obtain candidate targets, remove redundancy through non-maximum suppression, and filter effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate, mark alarm images and generate preliminary analysis results; The frame sequence obtained in S2 is input into the small model configured in S11 in chronological order. The small model performs forward propagation and outputs candidate targets in each frame. The candidate targets include the detection box coordinates, confidence score, and anomaly category information. This step removes redundant targets and false targets caused by transient interference through multiple rounds of screening, and locks down valid anomaly targets with temporal continuity, providing clear alarm objects and basic information support for subsequent cross-temporal data processing.

[0042] The process of removing redundancy through nonmaximum suppression and selecting effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate includes: S31. Perform non-maximum suppression on the candidate targets output by the small model. The cross-union ratio (IoU) threshold for non-maximum suppression is set to a preset IoU threshold of 0.5. This threshold is adapted to the spatial distribution characteristics of abnormal targets in the power scenario, effectively removing redundant detection boxes with high overlap and avoiding the accidental deletion of different detection results for the same target. When the IoU ratio of two candidate targets exceeds the preset IoU threshold, the target with higher confidence is retained. When performing non-maximum suppression, all candidate targets within a single frame are first sorted by confidence level from highest to lowest. The candidate target with the highest confidence level is selected as the baseline target, and the intersection-over-union (IoU) ratio between the baseline target and the remaining candidate targets is calculated. The formula for calculating the IoU ratio is: Where A is the detection bounding box of the baseline target, B is the detection bounding box of the other candidate targets, Area(A∩B) is the intersection area of ​​the two detection bounding boxes, and Area(A∪B) is the union area of ​​the two detection bounding boxes. When the intersection-union ratio (IoU) of two candidate targets exceeds the preset IoU threshold (0.5), they are determined to be redundant targets, and the target with higher confidence is retained. All candidate targets are processed iteratively in sequence to finally obtain the deredundant candidate target set within a single frame.

[0043] S32. Calculate the inter-frame temporal continuous overlap rate, using the following formula: ,in, The target bounding box for the current frame. , These are the bounding boxes of the same type in the previous and next frames, respectively. The inter-frame temporal continuous overlap rate; Similar bounding boxes refer to bounding boxes that output anomalies of the same category as those in the smaller model, such as all bounding boxes for the station fire category. This represents the inter-frame temporal continuous overlap rate, with a value range of [0,1], used to quantify the spatial correlation of targets between consecutive frames. A higher value indicates a stronger temporal continuity of the target, making it more likely to be a real anomalous target rather than a false target caused by momentary interference.

[0044] S33. Set dual screening conditions to determine valid targets. The dual screening conditions are that when the confidence of a candidate target in a single frame is not lower than a preset first confidence threshold and the continuous overlap rate in the temporal domain is not lower than a preset overlap rate threshold, it is marked as a valid target and an alarm image and preliminary analysis results are generated.

[0045] The preset first confidence threshold is set to 0.6, which is determined based on the training effect of the small power scenario model and can effectively filter out suspected targets with low confidence. The preset overlap rate threshold is set to 0.3, which can distinguish the temporal continuity of real abnormal targets from the transient nature of false targets. When a candidate target meets the above dual screening conditions, it is determined to be a valid target. Video frames containing at least one valid target are marked as alarm images, and the identifier of the alarm image is consistent with the corresponding frame in the frame sequence. At the same time, a preliminary analysis result is generated, which contains complete information about the valid target, specifically the detection box coordinates, single-frame confidence value, anomaly category name, and the timestamp corresponding to the alarm image, which is consistent with the timestamp recorded in S23, providing structured input data for subsequent cross-temporal data acquisition and large model re-judgment. For example: A candidate target has a single-frame confidence score of 0.75, which is not lower than 0.6, and an inter-frame temporal continuous overlap rate of 0.4, which is not lower than 0.3. It meets the dual screening conditions and is determined to be a valid target. The frame containing the valid target is marked as an alarm image. An example of the generated preliminary analysis results is: detection box coordinates: 0.3, 0.4, 0.1, 0.2; confidence score: 0.75; anomaly category: station fire.

[0046] Reference Figure 3 S4. Centered on the alarm image timestamp, collect cross-temporal video frames, power scene sensor data and environmental data according to the dynamic time window, extract visual temporal features and power scene sensor temporal features, and fuse them to obtain cross-temporal fusion features; Using the alarm image timestamp marked by S3 as the core benchmark, and according to the dynamic time window determined by S13, multi-source cross-time domain data covering the key stages before and after the abnormal event are collected. Data quality and time consistency are ensured through preprocessing, and then time domain features are extracted and fused through a dedicated network to fully explore the time domain correlation of the abnormal event, supplement the missing context information of single frame data, and provide comprehensive and in-depth feature support for multimodal re-judgment of large models.

[0047] The process of collecting cross-time domain video frames, power scene sensor data, and environmental data according to a dynamic time window, centered on the alarm image timestamp, includes: S41. An efficient encoding and decoding algorithm is used to encode and decode the cross-time domain video frame. Kalman filtering is used to denoise the sensor data. Denoising data is obtained through state prediction, covariance prediction, Kalman gain calculation and state update. Based on the timestamp of the cross-time domain video frame, linear interpolation is used to perform time alignment between the video frame and the sensor data.

[0048] The preferred high-efficiency decoding algorithm is the H.265 (HEVC) algorithm, which compresses the video data volume by combining intra-frame prediction and inter-frame prediction. While ensuring that the frame features are not distorted, it optimizes the storage and transmission efficiency of cross-temporal video frames, and facilitates data interaction between edge devices and large models.

[0049] Kalman filtering is used to denoise the sensor data. The sensor data in the power scenario includes temperature, humidity, equipment current, and equipment voltage data, while environmental data includes wind speed and light intensity data. The core parameters of the Kalman filter are configured as follows: process noise covariance Q = 0.01, and observation noise covariance R = 0.1. These parameters are well-suited to the noise characteristics of power sensors and can effectively eliminate random errors. Specific processing steps include: State prediction: Predict the current state based on the denoised data from the previous time step, using the following formula: ; Covariance prediction: Update the covariance of the predicted state, the formula is as follows ; Kalman gain calculation: Adjust the weights of the observed data according to the covariance, using the following formula: ; State Update: The predicted state is corrected by combining the original sensor data to obtain the denoised data. The formula is as follows: ,in This is the raw sensor data.

[0050] Using the timestamps of cross-temporal video frames as a benchmark, linear interpolation is employed to perform time alignment between video frames and sensor and environmental data. When the timestamps of sensor data and video frames do not perfectly match, a linear interpolation formula is used... Calculate the aligned data, where For video frame timestamps, For the corresponding sensor or environmental data, , They are respectively with Adjacent original data timestamps ensure that the time synchronization error of multi-source data across time domains does not exceed the preset synchronization threshold of 10ms.

[0051] The extraction of visual temporal features and power scene sensor temporal features, and the fusion of these to obtain cross-temporal fusion features, include: By using a dedicated deep learning network to extract the visual dynamic features of cross-temporal video frames and the temporal trend features of sensor data, and then dynamically allocating weights through an attention mechanism, the two types of features are deeply fused to form a cross-temporal fusion feature that can comprehensively reflect the temporal evolution of abnormal events.

[0052] S42. A temporal convolutional network is used to extract visual temporal features from cross-temporal video frames, and a long short-term memory network is used to extract sensor temporal features from sensor data. S43. A temporal convolutional network (TCN) is used to extract visual temporal features from cross-temporal video frames. The TCN structure consists of three convolutional layers and one pooling layer. The kernel size is uniformly set to 3×3, and the dilation coefficients are 1, 2, and 4, respectively. The number of output channels is 64, 128, and 256, respectively. The activation function is ReLU, and the pooling layer is 2×2 max pooling. This structure expands the receptive field through dilated convolutions, effectively capturing the dynamic changes of abnormal targets in cross-temporal video frames, such as the direction of smoke diffusion, the trajectory of personnel movement, and the increase of icing area. The final output is visual temporal features. The feature vector is 256-dimensional, with dimensions consistent with subsequent sensor time-series features for easy fusion computation. A Long Short-Term Memory (LSTM) network is used to extract sensor time-series features from the sensor data. The LLSTM network structure consists of one LSTM layer with a hidden dimension of 256 and one fully connected layer with an output dimension of 256, with a dropout value of 0.2 to prevent overfitting. The LSTM layer effectively captures long-term dependencies and trend changes in sensor data, such as continuous temperature increases, humidity accumulation, and current fluctuations, through the synergistic effect of input gates, forget gates, and output gates. After processing by the fully connected layer, the sensor time-series features are output. It is a 256-dimensional feature vector.

[0053] S43. Frame-level weights are assigned to visual temporal features using an attention mechanism, and combined with preset sensor feature weights to obtain cross-temporal fused features. ,in The frame-level attention weights are calculated using the cosine similarity between visual temporal features and sensor temporal features. To preset sensor feature weights, The number of frames for visual temporal features.

[0054] The cosine similarity is calculated using the visual temporal features and the sensor temporal features. The formula for cosine similarity is: Frame-level attention weights The normalization formula is: This weighting mechanism assigns higher weights to video frame features that are more correlated with sensor trends, thereby improving the targeting of the fused features. The preset sensor feature weights have a value range of 0.2 to 0.4, preferably... This is used to balance the influence of visual temporal features and sensor temporal features. This represents the number of frames for visual temporal features, equal to the total number of frames in the cross-temporal video, ensuring that visual features from each frame participate in the fusion. For example, a cross-temporal video with 10 frames... The cosine similarity between the visual features and the sensor temporal features of frame 3 is 0.8, which is the maximum value among all frames. Therefore, its corresponding... Frames with higher weight than other frames will receive more attention during fusion due to abnormal dynamics they reflect, such as a sudden increase in flame area; combined with The sensor feature weights are then fused to obtain a 256-dimensional cross-temporal fusion feature. This feature integrates the visual dynamic evolution of abnormal events with sensor trend changes, providing core temporal support for large models to accurately determine anomalies.

[0055] Reference Figure 4 S5. Input the alarm image, preliminary analysis results and cross-temporal fusion features into the multimodal large model, extract visual features and text features and weightedly fuse them with the cross-temporal fusion features, determine the authenticity of the anomaly based on the fusion features, evaluate the severity of the detected anomaly, and output the detection results.

[0056] The alarm images and preliminary analysis results obtained in S3, along with the cross-temporal fusion features obtained in S4, are input into the multimodal large model configured in S12 according to a preset format. Qwen3-VL is preferred. Through the large model, deep fusion and intelligent reasoning of multimodal features are achieved, accurately determining whether the anomaly actually exists, quantitatively assessing the severity of the anomaly, and predicting the development trend of the anomaly. Finally, structured detection results are output, providing clear handling basis for power scenario operation and maintenance personnel.

[0057] The method for determining the authenticity of anomalies based on fusion features and assessing the severity of detected anomalies, outputting detection results including: S51. Employ a multimodal large model to extract visual features from alarm images, and convert the preliminary analysis results into text features based on a word vector model; The multimodal large model's built-in image encoder uses the ViT-L / 14 architecture. It extracts high-dimensional visual feature vectors by performing block embedding, multi-head self-attention calculation, and layer normalization operations on alarm images. The feature vector is set to 1024 dimensions, which can comprehensively characterize the visual attributes of abnormal targets in the alarm image, such as flame color, ice texture, and personnel posture. The preliminary analysis results are converted into text features based on a word vector model. The preferred word vector model is the Word2Vec model, and its training corpus is a professional corpus of power scenarios, such as descriptions of anomaly categories, equipment parameter descriptions, and safety specification texts. An example of the textual description of the preliminary analysis results is: "Anomaly category: Station fire; Confidence: 0.75; Detection box coordinates: 0.3, 0.4, 0.1, 0.2; Timestamp: 202x-xx-10 15:30:22.123". This text is encoded using the Word2Vec model. First, the text is split into independent words, generating low-dimensional word vectors for each word. Then, the average of all word vectors is taken to obtain a 1024-dimensional text feature vector. This enables the semantic quantification of preliminary analysis results.

[0058] S52. The cross-temporal fusion features are upgraded to a preset feature dimension through a fully connected layer. Weights are dynamically assigned based on the correlation scores between each feature and the abnormal event, and the final fusion features are obtained by weighted fusion. The preset feature dimension is consistent with the dimensions of visual and textual features, both being 1024 dimensions, ensuring dimension matching during multimodal feature fusion. The fully connected layer has an input dimension of 256 dimensions and an output dimension of 1024 dimensions, using ReLU as the activation function. The dimensionality increase formula is as follows: in, This is the weight matrix of the fully connected layer (dimension 256×1024). The bias vector (dimension 1024) is obtained by training and optimization using time-series data from power scenarios.

[0059] Weights are dynamically assigned based on the correlation scores between each feature and the abnormal event. The correlation score is calculated as follows: visual feature correlation. pass Cosine similarity with visual feature vectors of corresponding anomaly types in the power anomaly pattern library is used to determine the correlation of text features. pass Cosine similarity with the feature vectors of corresponding anomaly types in the pattern library is used to determine the correlation of temporal features. Calculations are based on the intensity of time-series trends (such as normalized scores corresponding to the rate of temperature rise and the rate of smoke diffusion).

[0060] The weighting formula is: in, ,satisfy .

[0061] The weighted fusion yields the final fusion feature, and the formula is: S53. Calculate the cosine similarity between the final fused features and the features in the power anomaly pattern library, and determine the authenticity of the anomaly based on the confidence level; The formula for cosine similarity is: in, The first in the power anomaly pattern library Feature vectors (1024 dimensions) of anomalies (such as station fires, equipment icing). , These are the L2 norms of the corresponding feature vectors. The maximum similarity is calculated by iterating through the feature vectors of all anomaly types in the pattern library. The confidence level calculated in conjunction with S12 Perform dual judgment: when A preset similarity threshold (value 0.8) is set and When the preset high confidence threshold (value 0.8) is set, it is judged as a real anomaly; otherwise, it is judged as a false alarm.

[0062] S54. Based on the type of power anomaly, set an intensity index to assess the severity, use a linear regression model to fit the anomaly intensity data to obtain a trend curve, determine the development trend based on the trend slope, and output the anomaly intensity prediction value after a preset time interval.

[0063] S54. Based on the type of power anomaly, an intensity index is set to assess the severity. The calculation formula for the intensity index is designed differently according to the type of anomaly: Station fire scenario: Intensity=0.5× +0.3× +0.2× ,in This represents the change in smoke area (in pixels) within a dynamic time window. The duration (in seconds) of the dynamic time window. This represents the temperature change (°C) within the window. This represents the proportion of the flame area to the total area of ​​the alarm image. Equipment icing scenario: Intensity=0.6× +0.4× ×0.1, where This represents the change in ice thickness within the window (mm). The change in humidity within the window is expressed as a percentage (%). The intensity index is graded as follows: Intensity > 0.7 is considered high severity, 0.3 ≤ Intensity ≤ 0.7 is considered medium severity, and Intensity < 0.3 is considered low severity.

[0064] A linear regression model is used to fit the abnormal intensity data to obtain a trend curve. The abnormal intensity data is the intensity index value corresponding to each frame within a dynamic time window, such as Intensity calculated every 1 second. The trend curve formula is y=kt+b, where k is the trend slope and b is the offset. The calculation formula is as follows: , ; in, This refers to a specific time point (in seconds) within a dynamic time window. , where n is the abnormal intensity index value at the corresponding time point, and n is the number of time points.

[0065] The development trend is determined based on the trend slope k: k > 0.1 indicates abnormal intensification, −0.1 ≤ k ≤ 0.1 indicates abnormal stability, and k < −0.1 indicates abnormal mitigation. A preset time interval Δt = 30 seconds is used to output the predicted anomaly intensity after this time interval, using the following formula: in, The number of seconds corresponding to the timestamp of the alarm image.

[0066] The final output detection result is a structured report, which includes anomaly authenticity (true / false), confidence score, anomaly type, severity (high / medium / low), development trend (intensifying / stabilizing / mitigating), intensity prediction value after a preset time interval, alarm image timestamp, and key feature description. Example report content is: "Anomaly authenticity: true; confidence score: 0.88; anomaly type: station fire; severity: high; development trend: intensifying; predicted intensity after 30 seconds: 0.92; timestamp: 20xx-xx-xx15:30:22.123; key features: smoke area growth rate 0.3 pixels / second, temperature rise 2.5°C, similarity to fire features in the pattern library 0.85."

[0067] Example 2 Reference Figure 5 This embodiment provides a system, including: The model configuration module is configured to configure small models, multimodal large models, and cross-time domain parameters based on power scenarios. The video acquisition module is configured to acquire the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames of the monitoring video stream, extract the frame sequence and record the timestamp of each frame. The small model initial screening module is configured to input frame sequences into the small model to obtain candidate targets, remove redundancy through non-maximum suppression, and filter effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate, mark alarm images and generate preliminary analysis results. The data processing module is configured to collect cross-temporal video frames, power scene sensor data and environmental data in a dynamic time window, centered on the alarm image timestamp, extract visual temporal features and power scene sensor temporal features, and fuse them to obtain cross-temporal fusion features. The large model re-judgment module is configured to input alarm images, preliminary analysis results, and cross-temporal fusion features into a multimodal large model, extract visual features and text features, and weightedly fuse them with the cross-temporal fusion features. Based on the fusion features, it determines the authenticity of the anomaly, evaluates the severity of the detected anomaly, and outputs the detection results.

[0068] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned anomaly detection method with fusion size model collaborative cross-temporal perception.

[0069] A terminal device includes a processor and a computer-readable storage medium, the processor being configured to implement various instructions; the computer-readable storage medium being configured to store multiple instructions adapted for loading and execution by the processor of the aforementioned anomaly detection method based on fusion of size models and collaborative cross-temporal perception.

[0070] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. An anomaly detection method that integrates large and small models for collaborative cross-temporal perception, characterized in that, include: Based on power scenario configuration, small models, multimodal large models, and cross-time domain parameters are configured; The settings for the cross-time domain parameters include: A base time window is set based on the evolution speed of power anomaly types; The dynamic time window is obtained by dynamically adjusting the abnormal evolution velocity coefficient, as shown in the formula. ,in, Based on the time window, This represents the anomalous evolution rate coefficient. To preset the number of frames, For the first The area of ​​the abnormal region in the frame, for The average area of ​​frame anomalies; Set the sensor data sampling frequency so that the slowly changing sensor data sampling frequency is aligned with the frame sampling frequency, and the rapidly changing sensor data sampling frequency is a preset multiple of the frame sampling frequency, and the time synchronization error between all sensor data and video frame data does not exceed the preset synchronization threshold. Acquire the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames in the monitoring video stream, extract the frame sequence and record the timestamp of each frame; The calculation of the dynamic frame extraction frequency based on the scene change coefficient of continuous video frames in the monitoring video stream, and the extraction of the frame sequence and recording of the timestamp of each frame, includes: Perform Gaussian blur denoising and bilinear interpolation on the video frames to scale them to a small model preset input size; The scene change coefficient is calculated by taking the average inter-frame pixel difference of a preset number of consecutive video frames, and the calculation formula is as follows: in, To preset the number of frames, For the first Original video frames, , These are the width and height of the original video frame, respectively. This is the scene change coefficient; The dynamic frame rate is calculated based on the scene change coefficient. The formula for calculating the dynamic frame rate is as follows: ,in The preset frame rate for the video. To set a preset adjustment coefficient, video frames are extracted at the dynamic frame extraction frequency to form a frame sequence, and the timestamp of each video frame is recorded. The frame sequence is input into the small model to obtain candidate targets. Redundancy is removed by nonmaximum suppression. Valid targets are selected by combining single-frame confidence and inter-frame temporal continuous overlap rate. Alarm images are marked and preliminary analysis results are generated. Centered on the alarm image timestamp, cross-temporal video frames, power scene sensor data and environmental data are collected according to the dynamic time window. Visual temporal features and power scene sensor temporal features are extracted and fused to obtain cross-temporal fusion features. The alarm image, preliminary analysis results, and cross-temporal fusion features are input into the multimodal large model. Visual features and text features are extracted and weighted and fused with the cross-temporal fusion features. The authenticity of the anomaly is determined based on the fusion features, the severity of the detected anomaly is evaluated, and the detection results are output.

2. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The small model based on the power scenario configuration includes: constructing anchor boxes for abnormal targets in the power scenario, ensuring that the anchor box size matches the actual characteristics of the power target, and configuring an optimized total loss function that includes bounding box regression loss, classification loss, and confidence loss, expressed by the formula: ,in, For bounding box regression loss, For classifying losses, The confidence loss is calculated using the binary cross-entropy loss formula. and These are preset weight parameters used to balance detection accuracy and inference efficiency.

3. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The configuration of the multimodal large model includes: constructing a power scenario anomaly pattern library, which includes feature vectors corresponding to core anomaly types. These feature vectors include visual feature vectors, temporal feature vectors, and textual feature vectors. A confidence assessment mechanism is also established, calculating the confidence level using the joint probability of the multimodal large model's output sequences, expressed by the formula: When S is not lower than a preset high confidence threshold, it is determined to be a reliable inference result. For alarm images, For input text, The output sequence is the output sequence of a multimodal large model, where N is the length of the output sequence. Let S be the conditional probability of the i-th token in the output sequence, and S be the confidence level.

4. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The process of removing redundancy through nonmaximum suppression and selecting effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate includes: Non-maximum suppression is performed on the candidate targets output by the small model. The cross-union ratio threshold of the non-maximum suppression is set to a preset IoU threshold. When the cross-union ratio of two candidate targets exceeds the preset IoU threshold, the target with higher confidence is retained. The formula for calculating the temporal consecutive overlap rate between frames is as follows: ,in, The target bounding box for the current frame. , These are the bounding boxes of the same type in the previous and next frames, respectively. The inter-frame temporal continuous overlap rate; A dual screening condition is set to determine the effective target. The dual screening condition is that when the confidence of a candidate target in a single frame is not lower than a preset first confidence threshold and the continuous overlap rate in the temporal domain is not lower than a preset overlap rate threshold, it is marked as an effective target and an alarm image and preliminary analysis results are generated.

5. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The process of collecting cross-time domain video frames, power scene sensor data, and environmental data according to a dynamic time window, centered on the alarm image timestamp, includes: An efficient encoding and decoding algorithm is used to encode and decode cross-temporal video frames. Kalman filtering is used to denoise the sensor data. Denoising data is obtained through state prediction, covariance prediction, Kalman gain calculation and state update. Based on the timestamp of the cross-temporal video frame, linear interpolation is used to perform time alignment between the video frame and the sensor data.

6. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The extraction of visual temporal features and power scene sensor temporal features, and the fusion of these to obtain cross-temporal fusion features, include: A temporal convolutional network is used to extract visual temporal features from cross-temporal video frames, and a long short-term memory network is used to extract sensor temporal features from sensor data. By assigning frame-level weights to visual temporal features through an attention mechanism and combining them with pre-defined sensor feature weights, cross-temporal fused features are obtained. ,in The frame-level attention weights are calculated using the cosine similarity between visual temporal features and sensor temporal features. To preset sensor feature weights, The number of frames for visual temporal features.

7. The anomaly detection method based on the fusion of large and small models for collaborative cross-temporal perception as described in claim 1, characterized in that, The method for determining the authenticity of anomalies based on fusion features and assessing the severity of detected anomalies, outputting detection results including: A multimodal large model is used to extract visual features from alarm images, and the preliminary analysis results are converted into text features based on a word vector model. The cross-temporal fusion features are upscaled to a preset feature dimension through a fully connected layer. Weights are dynamically assigned based on the correlation scores between each feature and the abnormal event, and the final fusion features are obtained by weighted fusion. Calculate the cosine similarity between the final fused features and the features in the power anomaly pattern library, and combine the confidence level to determine the authenticity of the anomaly; The severity of power anomalies is assessed by setting intensity indicators based on the type of power anomaly. A linear regression model is used to fit the anomaly intensity data to obtain a trend curve. The development trend is determined based on the trend slope, and the anomaly intensity prediction value is output after a preset time interval.

8. An anomaly detection system that integrates large and small models for collaborative cross-temporal perception, characterized in that, An anomaly detection method using the fusion size model collaborative cross-temporal perception method according to any one of claims 1-7: The model configuration module is configured to configure small models, multimodal large models, and cross-time domain parameters based on power scenarios. The video acquisition module is configured to acquire the monitoring video stream, calculate the dynamic frame extraction frequency based on the scene change coefficient of the continuous video frames of the monitoring video stream, extract the frame sequence and record the timestamp of each frame. The small model initial screening module is configured to input frame sequences into the small model to obtain candidate targets, remove redundancy through non-maximum suppression, and filter effective targets by combining single-frame confidence and inter-frame temporal continuous overlap rate, mark alarm images and generate preliminary analysis results. The data processing module is configured to collect cross-temporal video frames, power scene sensor data and environmental data in a dynamic time window, centered on the alarm image timestamp, extract visual temporal features and power scene sensor temporal features, and fuse them to obtain cross-temporal fusion features. The large model re-judgment module is configured to input alarm images, preliminary analysis results, and cross-temporal fusion features into a multimodal large model, extract visual features and text features, and weightedly fuse them with the cross-temporal fusion features. Based on the fusion features, it determines the authenticity of the anomaly, evaluates the severity of the detected anomaly, and outputs the detection results.

Images