Intelligent patrolling autonomous scheduling analysis method and device with multi-algorithm cooperation compatibility

Abstract

This invention discloses an intelligent patrol autonomous scheduling and analysis method and apparatus with multi-algorithm collaboration compatibility. The method includes the following steps: In the anomaly screening stage, multi-temporal patrol images are collected to construct a dynamic base map library, and a visual disturbance index is generated through a fusion function; In the analysis and scheduling stage, scheduling priority weights are generated based on the visual disturbance index, historical task response time, and resource usage data, and analysis strategies are assigned to each location; In the deep anomaly discrimination stage, for images of key patrol areas, a composite anomaly intensity is synthesized through dual-channel parallel analysis; In the target verification stage, if the composite anomaly intensity exceeds a threshold, an intrusion judgment index is calculated to determine the anomaly type; In the self-correction calibration stage, when an equipment anomaly is determined, a consistency ratio is calculated, and the cause of the anomaly is determined based on the ratio and its threshold to be either imaging system offset or actual equipment anomaly, and automatic correction is achieved. This invention improves the accuracy of anomaly analysis and detection during intelligent patrols.

Description

Technical Field

[0001] This invention relates to the field of electrical engineering, and in particular to an intelligent patrol autonomous scheduling and analysis method and device with multi-algorithm collaboration compatibility. Background Technology

[0002] Traditional intelligent substation inspection systems have long relied on static base map template matching or pixel / edge difference analysis based on fixed thresholds for anomaly detection. This approach lacks robustness in complex outdoor environments, easily generating numerous false alarms caused by environmental disturbances. Although some inspection systems have introduced lightweight deep learning models for initial screening and preset analysis priorities based on equipment type in recent years, their scheduling strategies are still mostly driven by static rules. This leads to computational overload and task backlog in high-concurrency scenarios due to full-scale deep analysis, or the omission of low-significance but high-risk structural anomalies due to uniform degradation. Furthermore, existing solutions have limited filtering capabilities against biological motion interference, often resulting in misjudgments due to a lack of semantic understanding and morphological-motion joint verification mechanisms. On the other hand, when the pan-tilt unit experiences slight shifts or the reference image becomes unusable due to equipment aging, the system often fails to automatically calibrate the observation perspective or update the cognitive benchmark, causing persistent missed detections. Therefore, this invention proposes an intelligent inspection autonomous scheduling and analysis method and device with multi-algorithm collaboration compatibility. Summary of the Invention

[0003] This invention provides an intelligent patrol autonomous scheduling and analysis method with multi-algorithm collaboration compatibility, characterized by comprising: S10. In the initial screening stage of anomalies, collect multi-temporal inspection images to construct a dynamic base map library, and use an exponential saturation fusion function to fuse texture complexity, edge stability and brightness deviation to generate a visual disturbance index. S20. In the analysis and scheduling phase, based on the visual disturbance index, historical task response time and resource usage data, the scheduling priority weight is generated through the load priority function, and analysis strategies are assigned to each point. S30. In the deep anomaly discrimination stage, for the key inspection area images, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference. S40. During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion judgment index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. S50. During the self-correction calibration stage, when an equipment malfunction is detected, the key features of the current frame are extracted and matched with the reference feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either an imaging system offset or a real equipment malfunction, and automatic correction is achieved.

[0004] The intelligent patrol autonomous scheduling and analysis method described above, which is compatible with multiple algorithm collaboration, involves the following sub-steps in the anomaly screening stage: First, multi-temporal patrol images are collected to construct a dynamic base map library. Then, texture complexity, edge stability, and brightness deviation are fused using an exponential saturation fusion function to generate a visual disturbance index. By extracting image frames from real-time video streams acquired by intelligent patrol devices, and after quality filtering and unsupervised consistency verification, the base map library is dynamically constructed and updated. Based on the local binary mode Shannon entropy, Canney edge structure similarity, and average gray level difference between the current patrol frame and the nearest neighbor base map, normalized perturbation components for texture, edge, and brightness are calculated respectively. The components are then input into an exponential saturation fusion function to generate a visual perturbation index that quantifies the degree to which the current image deviates from the normal state.

[0005] The intelligent patrol autonomous scheduling and analysis method described above, which is compatible with multiple algorithms, includes the following sub-steps in the analysis and scheduling phase: Based on the visual disturbance index, historical task response time, and resource usage data, a scheduling priority weight is generated through a load priority function to assign analysis strategies to each location. The system collects multi-dimensional resource load data of computing nodes in real time, and combines the visual disturbance index of the current inspection point with the historical average response time to calculate the scheduling priority weight, which is used to dynamically decide whether to initiate in-depth analysis at the point. Lightweight path quickly determines whether the current image is in a normal state with low overhead by performing optical flow consistency verification and key area template matching based on historical base map.

[0006] The intelligent inspection autonomous scheduling and analysis method with multi-algorithm collaboration compatibility described above, in the deep anomaly discrimination stage, for the key inspection area images, synthesizes a composite anomaly intensity to distinguish between real equipment defects and environmental interference through dual-channel parallel analysis, specifically divided into the following sub-steps: Frequency domain anomaly confidence is calculated by performing discrete cosine transform frequency domain channel analysis on preset key equipment areas; A Markov random field context channel is constructed in a preset key equipment area. The graph cut algorithm is used to solve the global energy minimization problem, output a binary confidence map, and calculate the spatial anomaly intensity. The frequency domain anomaly confidence and spatial anomaly intensity are fused through a dual-channel collaborative enhancement mechanism to generate a composite anomaly intensity. If the intensity exceeds a threshold, target verification is triggered.

[0007] The intelligent patrol autonomous scheduling and analysis method described above, which is compatible with multiple algorithms, includes the following sub-steps: In the target verification stage, if the intensity of the composite anomaly exceeds a threshold, the intrusion judgment index is calculated to determine the anomaly type based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library. By segmenting the moving foreground using online background modeling to form a binary image of the moving region, morphological and motion features are extracted from the image to provide criteria for target verification. By integrating morphological compactness, CLIP semantic similarity, and motion stability, an intrusion determination index is calculated. Only when the index exceeds the intrusion threshold is it determined to be biological interference and the alarm is suppressed to avoid falsely reporting biological intrusion as equipment failure. The intrusion determination results are recorded in the log and the interference statistics at the points are updated to continuously optimize the semantic prototype library and motion feature distribution.

[0008] The intelligent patrol autonomous scheduling and analysis method described above, which is compatible with multiple algorithms, involves segmenting the moving foreground using online background modeling to form a binary image of the moving region. Morphological and motion features are then extracted from this image to provide criteria for target verification. This process is specifically divided into the following sub-steps: For the current inspection point, a Gaussian mixture model is used to perform online background modeling on the real-time video frames continuously collected during the current inspection task. Static scenes that appear stably are classified as background, and pixels that deviate significantly from all background Gaussian distributions are identified as foreground. A binary image with the same size as the current frame is composed of the foreground and the static background. The binary image is subjected to morphological opening and closing operations to remove noise and holes, resulting in a connected binary image of the motion region. Based on this, morphological and motion features are extracted from the image for subsequent scale rationality judgment.

[0009] The intelligent patrol autonomous scheduling and analysis method described above, which is compatible with multiple algorithms, includes a self-correction calibration phase. When an equipment malfunction is detected, key features of the current frame are extracted and matched with a benchmark feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either imaging system offset or actual equipment malfunction, and automatic correction is achieved. This process is specifically divided into the following sub-steps: After excluding biological interference anomalies, multi-scale benchmark features were extracted from the equipment normal state reference image to construct a multi-scale benchmark feature library for subsequent cross-scale matching and geometric consistency verification. The feature matching consistency ratio is calculated based on the current frame features and the multi-scale benchmark feature library to determine whether the anomaly is caused by the imaging system offset, and to decide whether to perform self-calibration or enter feature-level anomaly verification. By performing gimbal coordinate self-calibration and cognitive adaptation on two types of anomalies—imaging system offset and real equipment anomaly—viewpoint drift compensation and optimized correction of equipment state evolution are achieved.

[0010] This invention also provides an intelligent patrol autonomous scheduling and analysis device with multi-algorithm cooperation compatibility, comprising: Anomaly screening module: In the anomaly screening stage, multi-temporal inspection images are collected to construct a dynamic base map library. Texture complexity, edge stability and brightness deviation are fused through an exponential saturation fusion function to generate a visual disturbance index. Analysis and scheduling module: During the analysis and scheduling phase, based on the visual disturbance index, historical task response time and resource usage data, the module generates scheduling priority weights through the load priority function and assigns analysis strategies to each point. Depth discrimination module: In the depth anomaly discrimination stage, for images of key inspection areas, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference; Target verification module: During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion judgment index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. Correction and calibration module: During the self-correction and calibration phase, when an equipment malfunction is detected, the key features of the current frame are extracted and matched with the benchmark feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either an imaging system offset or a real equipment malfunction, and automatic correction is achieved.

[0011] The beneficial effects achieved by this invention are as follows: This invention improves the accuracy of anomaly analysis and detection during intelligent patrols through multi-algorithm collaboration. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 This is a flowchart of an intelligent patrol autonomous scheduling and analysis method with multi-algorithm collaboration compatibility provided in Embodiment 1 of this application.

[0014] Figure 2 This is a schematic diagram of an intelligent patrol autonomous scheduling and analysis device with multi-algorithm collaboration compatibility provided in Embodiment 2 of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1

[0017] like Figure 1 As shown, Embodiment 1 of this application provides an intelligent patrol autonomous scheduling and analysis method with multi-algorithm collaboration compatibility, including: S10. In the initial screening stage of anomalies, collect multi-temporal inspection images to construct a dynamic base map library. Use an exponential saturation fusion function to fuse texture complexity, edge stability and brightness deviation to generate a visual disturbance index.

[0018] S11. By extracting image frames from the real-time video stream obtained from the intelligent inspection equipment, and after quality filtering and unsupervised consistency verification, the base map library is dynamically constructed and updated.

[0019] Real-time video streams are acquired from high-definition fixed PTZ cameras deployed in substations. High-quality still image frames are extracted from the real-time video at preset frame intervals as candidate basemap samples. After image frames are generated, blurred, overexposed, or motion-blurred frames are first removed. The remaining high-quality images are directly stored as seed basemaps in the basemap library and given initial context labels. The basemap library is managed by grouping according to context labels. Context labels include illumination level, season, and equipment load status. Whenever a new image frame is generated and preliminary quality filtering is completed, an unsupervised consistency check is performed between the frame and reference basemap images with the same context labels in the current dynamic basemap library. The consistency check consists of two parts: First, based on the context group to which the new frame belongs, ORB features are extracted and the similarity between the new frame and all basemaps in the group is calculated. The basemap with the smallest feature distance is selected as the nearest neighbor basemap. The proportion of interior points between the new frame and the nearest neighbor basemap, after being filtered by a random sampling consistency algorithm, is calculated. If the proportion is higher than a set proportion threshold, the new frame is considered to have a stable geometric structure. Secondly, by aligning the grayscale histogram and normalizing the local contrast, the overall frame illumination shift of the new frame compared to the nearest neighbor base map is evaluated to see if it is within tolerance. Only when both checks pass is the new frame marked as a valid normal observation and added to the sliding window base map pool of the corresponding label group using a first-in, first-out strategy, with a fixed capacity for each base map group. Simultaneously, groups with low-frequency access or those that have not been updated for a long time are periodically cleaned up to prevent the accumulation of historical noise. Through the above process, an independent dynamic base map library is established for each inspection point in the substation.

[0020] S12. Based on the local binary mode Shannon entropy, Canney edge structure similarity and average gray level difference of the current patrol frame and the nearest neighbor base map, calculate the normalized perturbation components of texture, edge and brightness respectively, and input the components into the exponential saturation fusion function to generate a visual perturbation index that quantifies the degree of deviation of the current image from the normal state.

[0021] Parallel feature extraction processes are performed on the new image frame of the current inspection point and the nearest neighbor base map in its corresponding context group: In the texture path, local binary pattern encoding is calculated for each of the two images, and the corresponding local binary pattern image is generated. Then, its gray-level histogram is calculated, and Shannon entropy is calculated as a texture complexity index. The higher the value, the more complex and irregular the local texture, suggesting abnormalities such as surface dirt, corrosion, or foreign object adhesion. In the edge path, the Cannibal edge detection operator is applied to the two images for double-threshold edge detection, and binary edge maps are generated for each. Then, the similarity between the two edge maps is calculated using a structural similarity algorithm to obtain the edge stability index, which takes a value range of [0,1]. The closer the value is to 1, the better the edge structure is maintained, and no obvious deformation or occlusion has occurred.

[0022] The absolute value of the difference between the Shannon entropy of the current frame and the nearest neighbor base image is calculated as the texture perturbation. This difference is then linearly normalized based on the maximum fluctuation value under historical normal conditions, yielding a texture perturbation component between [0,1]. The edge stability index is directly used as the edge stability component. The difference between the average grayscale values ​​of the grayscale histograms of the current frame and the nearest neighbor base image is calculated, divided by a preset maximum allowable offset, and then truncated to obtain a normalized brightness perturbation component between [0,1]. Subsequently, these three normalized components are input into a modified exponential saturation fusion function. This function suppresses multiple weak perturbations, and rapidly increases the output value when any one of them exhibits a significant anomaly. The fusion function outputs a visual perturbation index ranging from [0,1]. The closer the index is to 0, the more consistent the current image is with the normal state; the closer it is to 1, the more significant the texture disorder, edge instability, or sudden illumination changes caused by equipment defects, biological intrusion, or environmental interference. The above process is used to calculate the corresponding visual perturbation index for each inspection point in the substation.

[0023] S20. In the analysis and scheduling phase, based on the visual disturbance index, historical task response time and resource usage data, the scheduling priority weight is generated through the load priority function, and analysis strategies are assigned to each point.

[0024] S21. Real-time acquisition of multi-dimensional resource load of computing nodes, combined with the visual disturbance index of the current inspection point and the historical average response time, to calculate the scheduling priority weight, which is used to dynamically decide whether to start in-depth analysis of the point.

[0025] The system continuously collects key resource usage data from current computing nodes (edge ​​servers deployed in local substation computer rooms or near equipment areas; each server can store and maintain a dynamic base map library of multiple inspection points) through the operating system interface. This data includes CPU utilization, node memory usage, and network bandwidth pressure. These three indicators are synchronously sampled and combined at fixed time intervals to form a three-dimensional resource load vector. This vector dynamically reflects the real-time operating load of the system. Any dimension approaching its upper limit (CPU utilization exceeding 85%, memory usage exceeding 90%, or bandwidth utilization reaching 95%) indicates a decrease in the system's schedulable capacity. The system refers to the edge computing nodes and their operating environment that undertake real-time visual analysis tasks.

[0026] Scheduling priority weights are generated based on the visual disturbance index, multidimensional resource load, and the historical average response time of the current location, using a load priority function. The load priority function dynamically balances the severity of anomalies, system resource pressure, and historical processing efficiency to determine whether to initiate a costly fine-grained deep analysis process at the current location. The formula for calculating the load priority function is as follows: ,in, For disturbance load contention, The visual disturbance index. The visual disturbance sensitivity index. High visual disturbance index is significantly enhanced, while low visual disturbance index is suppressed. The initial value is 2, emphasizing that resources should only be allocated for in-depth analysis when the image is clearly abnormal. These represent the normalized CPU utilization, memory usage, and network bandwidth utilization, respectively, with values ​​ranging from [0,1]. 'a' represents the CPU load weight index, which controls... In the resource product term The contribution intensity of 'a' is such that the larger 'a' is, the stronger the inhibitory effect of high CPU load on scheduling weights. An initial value of 1 for 'a' indicates a linear effect. 'b' represents the memory usage weight index, which controls... The suppression strength is initially set to 1.2, and can be adjusted upwards if the system is memory-sensitive. c represents the network bandwidth weight index, which controls... The suppression strength is initially set to 1. If the network pressure at the edge nodes is stable, it is set to 0.5 to reduce its impact. This is a multidimensional resource coupling strength index, controlling the overall nonlinear amplification of the entire resource product term. The initial value is 1 to maintain the natural coupling characteristics of the resource term. For stronger robustness, then... The value range is [1.2, 1.5]. For time-dependent gain, The reference time constant is denoted as , and represents the maximum reasonable response time expected by the node. This represents the average response time for several past missions at this patrol point. It should be a very small positive number to prevent the denominator from being zero. This is the sensitivity coefficient for the time-dependent response, controlling the strength of the influence of changes in historical response time on the gain term. Its initial value is 1, and its range is [0.5, 2]. The larger the value, the greater the gain difference will be for the same historical response time, making the system more sensitive to the speed of point processing. For time-dependent nonlinear adjustment index, control The degree of nonlinear amplification, The larger the value, the more it amplifies the incentive gap between extremely fast and normal or extremely slow response points, thereby enhancing the preference for prioritizing high-efficiency points. The initial value is 1, and the range is [0.5, 2]. If the scheduling priority weight... If the value is greater than the preset weight threshold, the S30 depth anomaly detection process is triggered. If the weight is less than or equal to the weight threshold, a lightweight path is assigned.

[0027] S22. Lightweight path performs optical flow consistency verification and key area template matching based on historical base map, thereby quickly determining whether the current image is in a normal state with low overhead.

[0028] The lightweight path performs only low-overhead optical flow consistency checks to eliminate transient motion interference, and uses key area template matching based on historical base maps to verify whether the device status is consistent.

[0029] The optical flow consistency verification process is as follows: A lightweight dense optical flow algorithm is used to calculate the pixel-level motion vector field of the current frame and adjacent frames acquired at the same preset position in the previous time step. Regional statistics are performed on the motion vector field. If a certain proportion of pixels exhibit localized violent motion, or if a globally consistent small translation is detected, or if the proportion of pixel motion regions is lower than the motion threshold and the motion amplitude is less than the allowable value, the Boolean flag value of the optical flow verification output is set to False, directly terminating the analysis process for the current point, marking it as instantaneous dynamic interference, and neither template matching nor depth analysis is triggered. The criteria for determining localized violent motion are that more than a certain proportion of pixels in a certain region have a motion amplitude greater than 5 pixels / frame, the motion direction is discrete, or the motion pattern is abnormal, and it only occurs in a single frame or two consecutive frames.

[0030] Conversely, if the Boolean flag is set to True, the process of template matching for key regions based on historical base maps is initiated. This process preprocesses the current frame with the nearest neighbor base map selected from the dynamic base map library for that location using grayscale conversion and histogram equalization. Within the key equipment areas of the two preprocessed images, structural similarity is calculated. If the similarity score is higher than the preset tolerance, the equipment appearance is considered to be consistent with the historical normal state. At the same time, differential image thresholding is used to determine that there are no significantly abnormal areas, i.e., no large-area occlusion, no local missing or added objects, and the overall brightness shift is within the tolerance. In this case, the template matching result is set to True, and the system is considered to be in a normal state, and the task ends. Conversely, if the similarity score is lower than the preset tolerance and simultaneously meets the high-risk characteristics such as a low visual disturbance index, slight changes in the differential result, no violent movement, and no obvious structural damage, it is due to slow equipment degradation, base map labeling deviation, or minor environmental disturbances, which is not sufficient to constitute a valid fault. The location is simply marked for observation, logged, and re-inspection is delayed.

[0031] The path selection results (including point identifiers, image frames, and selected strategies) are encapsulated into structured task units and written into an asynchronous task queue. In the background, an asynchronous execution engine driven by multiple threads pulls tasks according to priority and calls the corresponding analysis modules to realize a resource-aware, non-blocking intelligent processing pipeline.

[0032] S30. In the deep anomaly discrimination stage, for the key inspection area images, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference.

[0033] S31. Calculate the frequency domain anomaly confidence level by performing frequency domain channel analysis of discrete cosine transform in the preset key equipment area.

[0034] Upon triggering the depth analysis path, a predefined Region of Interest (ROI) for that location is cropped from the current image frame. After grayscale conversion and size normalization, a two-dimensional discrete cosine transform (DCT) is applied to convert the region from the spatial domain to the frequency domain, resulting in a DCT coefficient matrix of the same size as the original region. Since the DCT coefficient matrix exhibits energy concentration in low frequencies and an increasing spatial frequency from the upper left to the lower right, the proportion of high-frequency energy is calculated by taking the sum of the squares of all coefficients in the high-frequency subband of the lower right quarter region as a percentage of the sum of the squares of all coefficients in the entire coefficient matrix. Next, using the three-standard-deviation principle, the current high-frequency energy percentage of the critical equipment area was compared with the average high-frequency energy percentage of the critical equipment area from multiple historical base maps of the same inspection point under normal conditions. and standard deviation Calculations were performed to obtain the frequency domain anomaly confidence level. The specific formula is as follows k controls the steepness of the exponent, with an initial value of 2. If the current high-frequency energy proportion is significantly higher than three times the upper limit, the frequency domain anomaly confidence value approaches 1, reflecting the presence of a large number of high-frequency details in the image (including crack edges, damage gaps, and foreign object attachments). If the frequency domain anomaly confidence value is 0.5, the energy is in a critical state, requiring collaborative judgment based on the spatial context channel. If the frequency domain anomaly confidence value approaches 0, it indicates that the surface texture of the device is smooth, without abrupt changes, and highly consistent with the historical normal state.

[0035] S32. Construct a Markov random field context channel in the preset key equipment area, use the graph cut algorithm to solve the global energy minimization problem, output a binary confidence map, and calculate the spatial anomaly intensity.

[0036] Taking a predefined critical device region in the current frame as the analysis scope, the local features of each pixel in the region are first calculated, including grayscale value, Sobel gradient magnitude and direction, and local binary pattern texture response. Each pixel is modeled as a node in a Markov random field, with one of two labels: normal or anomalous. A univariate potential is constructed based on the above three types of local features, which are then input into a predefined lightweight discrimination rule to evaluate the probability that each pixel belongs to an anomalous category. Simultaneously, a binary potential is defined for each pixel and its eight neighboring pixels. If neighboring pixels are highly similar in grayscale, gradient, or texture, a lower energy is assigned to encourage consistent labels; if the differences are significant, different labels are allowed. The complete energy function is obtained by adding the univariate and binary potentials in the standard form of a Markov random field. The minimized solution corresponds to the most probable global anomalous distribution. A graph cut algorithm is used to efficiently solve this energy function minimization problem, outputting a binary anomalous confidence map of the entire critical device region. In this process, isolated high-response points are assigned low anomaly probabilities due to the lack of surrounding context support, thus being effectively suppressed, while spatially coherent anomalous regions obtain high-confidence outputs due to neighborhood consistency, ultimately generating robust spatial context-enhanced anomaly discrimination results.

[0037] The proportion of anomalous pixels in the binary anomaly confidence map relative to the total number of pixels in the critical equipment area is calculated. This proportion is then weighted by the area of ​​the largest connected component in the anomalous region to obtain a scalar reflecting the spatial anomaly coverage and coherence. Finally, this scalar is mapped using a sigmoid function to obtain the spatial anomaly intensity in the range [0,1]. .

[0038] S33. By fusing frequency domain anomaly confidence and spatial anomaly intensity through a dual-channel collaborative enhancement mechanism, a composite anomaly intensity is generated. If the intensity exceeds the threshold, target verification is triggered.

[0039] Frequency domain anomaly confidence With spatial anomaly intensity The composite anomaly intensity is obtained by fusing the data using a dual-channel synergistic enhancement mechanism, and the specific formula is as follows: , The weighting parameters are positive real numbers, all taking a value of 2. The composite anomaly strength formula emphasizes that a high-confidence anomaly is only considered to exist when both channels simultaneously provide strong anomaly responses. If only the frequency domain channel or only the spatial channel results are prominent, the fusion value is significantly suppressed. Finally, if the CAI value exceeds a preset threshold... If a reliable structural anomaly is found at the current location, proceed to step S40 for target verification; otherwise, determine that the anomaly is caused by environmental interference or random noise, and terminate the depth analysis process.

[0040] S40. During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion judgment index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type.

[0041] S41. By segmenting the moving foreground through online background modeling to form a binary image of the moving region, morphological and motion features are extracted from the image to provide criteria for target verification.

[0042] For the current inspection points, a Gaussian mixture model (GMM) is used to perform online background modeling on the continuously acquired real-time video frames during the current inspection mission. The GMM maintains multiple Gaussian distributions for each pixel in the video frame, each defined by Gaussian components (weights, mean, and variance). The Gaussian distributions dynamically learn the color and brightness changes of pixels over time. Starting from the first received frame, the GMM is updated frame by frame, with each new frame used to adjust the Gaussian components of each pixel. As new frames arrive, the model adaptively captures subtle environmental disturbances such as slow changes in lighting and weather transitions, and classifies stable static scenes as the background. Based on this, pixels in the current frame that significantly deviate from all background Gaussian distributions are identified as foreground. A binary image of the same size as the current frame, composed of the foreground and static background, serves as the motion region mask. The binary image is then processed using morphological opening and closing operations to remove noise and holes, resulting in a connected motion region binary map. Based on this, the pixel area A and contour perimeter P of each connected region in the map are calculated, and the centroid position is tracked frame by frame, forming a centroid trajectory sequence. At the same time, the equivalent size of the moving target is estimated based on the area A, that is, the equivalent radius when the target is assumed to be circular, which is used for subsequent scale rationality judgment.

[0043] S42. By integrating morphological compactness, CLIP semantic similarity, and motion stability, an intrusion determination index is calculated. Only when the index exceeds the intrusion threshold is it determined to be biological interference and the alarm is suppressed to avoid misreporting biological intrusion as equipment failure.

[0044] Intrusion detection indicators are calculated using a target biological interference discrimination function to distinguish between moving targets that are genuine equipment malfunctions and biological interference, thereby avoiding false alarms. The specific function formula is as follows: , This is an invasion indicator; the closer its value is to 1, the more likely it is a biological invasion. The "shape compactness" term has a value of (0,1]. If the moving target is circular, this term has a value of 1. The more irregular the shape, the smaller the value. This term is used to exclude non-biological targets such as debris and ribbons. A is the pixel area of ​​the moving target, and P is the perimeter of the moving target's outline. Pi is the mathematical constant of a circle. The visual feature vector extracted by the CLIP image encoder after parameter freezing is cropped from the image of the current moving target. This is the prototype semantic vector of the target biological category. CLIP (Visual-Semantic Cosine Similarity) measures the similarity between the appearance of a target and the prototype of a category. This is the semantic similarity threshold; similarities below this value are considered irrelevant. Let Variants be the variances of the lateral and longitudinal displacements of the centroid trajectory across multiple consecutive frames, respectively, reflecting the degree of motion jitter. Indicates lateral displacement. Indicates longitudinal displacement. For the target equivalent radius, It is a very small constant to prevent division by zero. The weights for overall sensitivity, nonlinear enhancement intensity, and motion stability were controlled separately, with initial values ​​of 5, 2, and 3, respectively. If the value is greater than the intrusion threshold, it is determined to be biological intrusion interference and the fault alarm is suppressed to avoid false alarms as abnormalities of the equipment itself. If the value is less than or equal to the invasion threshold, it is considered a non-biological invasion.

[0045] S43. Record the intrusion determination results to the log and update the point interference statistics, and continuously optimize the semantic prototype library and motion feature distribution.

[0046] The output intrusion determination results, along with the timestamp, patrol point ID, original image fragment, motion region mask, and key features (including target compactness, CLIP similarity, and motion variance), are written into a structured event log. Simultaneously, if a biological intrusion is identified, a biological interference count is accumulated in the long-term statistical archive of that patrol point, and the corresponding visual feature sample is recorded. Periodically, based on this accumulated labeled data, the biological prototype semantic vector is incrementally updated by re-aggregating the mean CLIP features of frequently occurring biological images, and the centroid trajectory variance distribution of that point is adaptively calibrated. This closed-loop feedback mechanism continuously optimizes the ability to identify typical local biological interference, improving the accuracy and robustness of future judgments.

[0047] S50. During the self-correction calibration stage, when an equipment malfunction is detected, the key features of the current frame are extracted and matched with the reference feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either an imaging system offset or a real equipment malfunction, and automatic correction is achieved.

[0048] S51. After excluding biological interference anomalies, extract multi-scale benchmark features from the equipment normal state reference image to construct a multi-scale benchmark feature library for subsequent cross-scale matching and geometric consistency verification.

[0049] When the visual disturbance index and composite anomaly intensity exceed their set thresholds, but the analysis result in step S42 determines it to be a non-biological intrusion, anomaly analysis of the device body at the target location is performed. First, a high-quality reference image of the inspection point under normal conditions is preprocessed, including grayscale conversion and contrast enhancement. Based on this, multi-scale benchmark features are extracted: For efficiency, the ORB algorithm is used, which detects corner points in the device body region of the preprocessed image using the FAST corner detection operator and generates rotation-invariant binary descriptors. For robustness, the SIFT algorithm is used, which detects key points in the device body region of the preprocessed image using Gaussian difference and constructs scale-invariant floating-point descriptors. Both methods output the precise location coordinates, principal directions of local gradients, and scale information of the extracted feature points. All feature points, their descriptors, locations, directions, and scale information are organized into a structured multi-scale benchmark feature library, and the current frame features are extracted using the same method for subsequent cross-scale matching and geometric consistency verification.

[0050] S52. Calculate the feature matching consistency ratio based on the current frame features and the multi-scale benchmark feature library to determine whether the anomaly is caused by the imaging system offset, and decide whether to perform self-calibration or enter feature-level anomaly verification accordingly.

[0051] Graph structure matching is used to evaluate the geometric and topological consistency between the current frame and the reference graph, thereby determining whether the device's viewing angle has changed due to factors such as gimbal shift, lens shake, or loose mounting. The formula for calculating the feature matching consistency ratio is as follows: , The feature matching consistency ratio has a value range of (0,1). The closer this index is to 1, the more consistent the current frame is with the reference frame in terms of structure and scale. E is the set of matching mappings from the current frame feature points to feature points in the multi-scale benchmark feature library. E is the set of edges of the device body in the standard reference graph of the benchmark feature library, which connects point pairs with stable geometric relationships (including adjacent corner points and structurally symmetrical points) to form the topological skeleton of the device. V is the set of vertices of the device body in the standard reference graph of the benchmark feature library, with each vertex corresponding to a key feature point. Let be the relative position vector from feature point i to point j in the current frame. Let represent the Euclidean norm of the relative position vector from feature point i to point j in the current frame. For the corresponding feature point pairs in the benchmark library The relative position vector, For the corresponding feature point pairs in the benchmark library The Euclidean norm of the relative position vectors. Let i be the detection scale for feature point i in the current frame. For the corresponding point in the benchmark library The scale, The contributions of topological orientation error and scale error were adjusted separately, with initial values ​​of 1 and 0.5, respectively. This is a global sensitivity parameter that controls the exponential decay intensity. The larger the value, the more sensitive it is to topological consistency errors and local scale alignment errors. The initial value is 3, and the adjustment range is [2,5]. If the value is greater than the consistency threshold, it means that the device's geometry is consistent with the baseline from the current viewpoint, and no camera position correction is needed. If the feature points of the current frame are less than or equal to the consistency threshold and there is an overall correspondence between the feature points of the current frame and the reference image frame, and the descriptor similarity is high, the anomaly is determined to be an imaging system offset. If local feature mismatches still exist even after calibration (missing key points, excessive error in reprojection of matching points, low descriptor similarity), then feature-level anomaly verification is initiated.

[0052] S53. By performing gimbal coordinate self-calibration and cognitive adaptation on two types of anomalies—imaging system offset and real equipment anomaly—viewpoint drift compensation and optimized correction of equipment state evolution are achieved.

[0053] When step S52 determines that the anomaly is due to imaging system offset, a local fine-grained search is initiated within the neighborhood of the original critical equipment area. This search, centered on the current target coordinates, employs a spiral expansion method, sampling new sub-regions point by point in both the horizontal and vertical directions. Feature extraction and multi-scale geometric topological consistency evaluation are repeatedly performed on each candidate location. Once a suitable candidate location is found... For high consistency matching areas exceeding the threshold, the location is immediately adopted as the new observation center of the equipment, and the gimbal target coordinates of the inspection camera (including horizontal angle, pitch angle and zoom parameters) are updated. At the same time, the original coordinates, new coordinates, offset, consistency score and timestamp are written to the correction log. Subsequent inspection tasks will use this calibrated location as the benchmark to ensure imaging stability and anomaly detection reliability, thereby achieving adaptive self-calibration without human intervention.

[0054] The feature-level anomaly verification process is as follows: First, check whether the key structural points marked in the baseline feature library consistently fail to match in the current frame and during 2 to 3 consecutive inspections to rule out transient occlusion. Second, for successfully matched point pairs, calculate their reprojection error using the homography transformation model estimated by the random sampling consensus algorithm. If the average error exceeds a 3-pixel threshold, geometric distortion is determined. Simultaneously, analyze the local texture and brightness changes in the neighborhood of the matched points, and determine whether there are irreversible appearance mutations (rust, dirt, or component detachment) using SIFT descriptor cosine similarity or ORB Hamming distance. Only when any of the above anomalies is repeatable in time and structural in space is it confirmed as a real abnormal change in the equipment state. Subsequently, a cognitive adaptive process is executed: if the anomaly is an acceptable slow degradation and is manually reviewed and marked as a new normal state, the current frame is added to the dynamic base map library, replacing the original baseline image. If some key points become permanently unusable, the scope of the key equipment area will be narrowed or the matching weight of the point in the topology map will be reduced in subsequent analysis. At the same time, a structured alarm and calibration log containing the anomaly type, location coordinates, deviation, duration, and confidence level will be generated, and the operation and maintenance personnel will be notified to carry out physical repairs or fine-tune the sensor parameters as needed.

[0055] Example 2

[0056] like Figure 2 As shown, Embodiment 2 of this application provides an intelligent patrol autonomous scheduling and analysis device with multi-algorithm collaboration compatibility, including: Anomaly Screening Module: In the anomaly screening stage, multi-temporal inspection images are collected to construct a dynamic base map library. Texture complexity, edge stability, and brightness deviation are fused using an exponential saturation fusion function to generate a visual perturbation index. Specifically, it is divided into the following sub-modules: The base map library submodule dynamically builds and updates the base map library by extracting image frames from the real-time video stream obtained from intelligent patrol equipment, and after quality filtering and unsupervised consistency verification.

[0057] Visual perturbation submodule: Based on the local binary mode Shannon entropy, Canney edge structure similarity and average gray level difference between the current patrol frame and the nearest neighbor base map, normalized perturbation components of texture, edge and brightness are calculated respectively, and the components are input into the exponential saturation fusion function to generate a visual perturbation index that quantifies the degree to which the current image deviates from the normal state.

[0058] Analysis and Scheduling Module: During the analysis and scheduling phase, based on the visual disturbance index, historical task response time, and resource usage data, scheduling priority weights are generated using a load priority function to assign analysis strategies to each location. Specifically, it consists of the following sub-modules: The scheduling weight submodule collects multi-dimensional resource load data of computing nodes in real time and combines it with the visual disturbance index of the current inspection point and the historical average response time to calculate the scheduling priority weight, which is used to dynamically decide whether to initiate in-depth analysis at the point.

[0059] Lightweight Path Submodule: Lightweight Path quickly determines whether the current image is in a normal state with low overhead by performing optical flow consistency verification and key area template matching based on historical base map.

[0060] Depth Discrimination Module: In the depth anomaly discrimination stage, for images of key inspection areas, dual-channel parallel analysis is used to synthesize composite anomaly intensities to distinguish between real equipment defects and environmental interference. Specifically, it is divided into the following sub-modules: Frequency Domain Anomaly Submodule: Calculates the confidence level of frequency domain anomalies by performing discrete cosine transform frequency domain channel analysis in preset key equipment areas.

[0061] Spatial Anomaly Submodule: Constructs a Markov random field context channel in the preset key equipment area, uses the graph cut algorithm to solve the global energy minimization problem, outputs a binary confidence map, and calculates the spatial anomaly intensity.

[0062] Composite Anomaly Submodule: It fuses frequency domain anomaly confidence and spatial anomaly intensity through a dual-channel collaborative enhancement mechanism to generate composite anomaly intensity. If the intensity exceeds the threshold, it triggers target verification.

[0063] Target Verification Module: During the target verification phase, if the intensity of the composite anomaly exceeds a threshold, the intrusion determination index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. Specifically, it is divided into the following sub-modules: Feature extraction submodule: By segmenting the moving foreground through online background modeling to form a binary image of the moving region, morphological and motion features are extracted from the image to provide criteria for target verification.

[0064] Intrusion Detection Submodule: By integrating morphological compactness, CLIP semantic similarity, and motion stability, intrusion detection index is calculated. Only when the index exceeds the intrusion threshold is it determined to be biological interference and the alarm is suppressed to avoid falsely reporting biological intrusion as equipment failure.

[0065] The recording optimization submodule records the intrusion determination results to the log and updates the point interference statistics, which is used to continuously optimize the semantic prototype library and motion feature distribution.

[0066] Self-correction calibration module: During the self-correction calibration phase, when an equipment malfunction is detected, key features of the current frame are extracted and matched with a benchmark feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either imaging system offset or actual equipment malfunction, and automatic correction is achieved. Specifically, it is divided into the following sub-modules: Benchmark Feature Submodule: After excluding biological interference anomalies, multi-scale benchmark features are extracted from the reference image of normal equipment status to construct a multi-scale benchmark feature library for subsequent cross-scale matching and geometric consistency verification.

[0067] Feature consistency submodule: Calculates the feature matching consistency ratio based on the current frame features and the multi-scale benchmark feature library. This is used to determine whether the anomaly is caused by the imaging system offset and decide whether to perform self-calibration or enter feature-level anomaly verification.

[0068] Type Correction Submodule: By performing gimbal coordinate self-calibration and cognitive adaptation on two types of anomalies—imaging system offset and real equipment anomaly—it achieves optimized correction of viewpoint drift compensation and equipment state evolution.

[0069] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent patrol autonomous scheduling and analysis method with multi-algorithm collaboration compatibility, characterized in that, include: S10. In the initial screening stage of anomalies, collect multi-temporal inspection images to construct a dynamic base map library, and use an exponential saturation fusion function to fuse texture complexity, edge stability and brightness deviation to generate a visual disturbance index. S20. In the analysis and scheduling phase, based on the visual disturbance index, historical task response time and resource usage data, the scheduling priority weight is generated through the load priority function, and analysis strategies are assigned to each point. S30. In the deep anomaly discrimination stage, for the key inspection area images, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference. S40. During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion judgment index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. S50. During the self-correction calibration stage, when an equipment malfunction is detected, the key features of the current frame are extracted and matched with the reference feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either an imaging system offset or a real equipment malfunction, and automatic correction is achieved.

2. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 1, characterized in that, In the initial anomaly screening stage, multi-temporal inspection images are collected to construct a dynamic base map library. Texture complexity, edge stability, and brightness deviation are fused using an exponential saturation fusion function to generate a visual perturbation index. This process is divided into the following sub-steps: By extracting image frames from real-time video streams acquired by intelligent patrol devices, and after quality filtering and unsupervised consistency verification, the base map library is dynamically constructed and updated. Based on the local binary mode Shannon entropy, Canney edge structure similarity, and average gray level difference between the current patrol frame and the nearest neighbor base map, normalized perturbation components for texture, edge, and brightness are calculated respectively. The components are then input into an exponential saturation fusion function to generate a visual perturbation index that quantifies the degree to which the current image deviates from the normal state.

3. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 1, characterized in that, In the analysis and scheduling phase, based on the visual disturbance index, historical task response time, and resource usage data, scheduling priority weights are generated through the load priority function, and analysis strategies are assigned to each location. This process is divided into the following sub-steps: The system collects multi-dimensional resource load data of computing nodes in real time, and combines the visual disturbance index of the current inspection point with the historical average response time to calculate the scheduling priority weight, which is used to dynamically decide whether to initiate in-depth analysis at the point. Lightweight path quickly determines whether the current image is in a normal state with low overhead by performing optical flow consistency verification and key area template matching based on historical base map.

4. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 1, characterized in that, In the deep anomaly identification stage, for images of key inspection areas, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference. This is specifically divided into the following sub-steps: Frequency domain anomaly confidence is calculated by performing discrete cosine transform frequency domain channel analysis on preset key equipment areas; A Markov random field context channel is constructed in a preset key equipment area. The graph cut algorithm is used to solve the global energy minimization problem, output a binary confidence map, and calculate the spatial anomaly intensity. The frequency domain anomaly confidence and spatial anomaly intensity are fused through a dual-channel collaborative enhancement mechanism to generate a composite anomaly intensity. If the intensity exceeds a threshold, target verification is triggered.

5. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 1, characterized in that, During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion determination index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. This is specifically divided into the following sub-steps: By segmenting the moving foreground using online background modeling to form a binary image of the moving region, morphological and motion features are extracted from the image to provide criteria for target verification. By integrating morphological compactness, CLIP semantic similarity, and motion stability, an intrusion determination index is calculated. Only when the index exceeds the intrusion threshold is it determined to be biological interference and the alarm is suppressed to avoid falsely reporting biological intrusion as equipment failure. The intrusion determination results are recorded in the log and the interference statistics at the points are updated to continuously optimize the semantic prototype library and motion feature distribution.

6. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 5, characterized in that, The process involves segmenting the moving foreground using online background modeling to create a binary image of the moving region. Morphological and motion features are then extracted from this image to provide criteria for target verification. This process is divided into the following sub-steps: For the current inspection point, a Gaussian mixture model is used to perform online background modeling on the real-time video frames continuously collected during the current inspection task. Static scenes that appear stably are classified as background, and pixels that deviate significantly from all background Gaussian distributions are identified as foreground. A binary image with the same size as the current frame is composed of the foreground and the static background. The binary image is subjected to morphological opening and closing operations to remove noise and holes, resulting in a connected binary image of the motion region. Based on this, morphological and motion features are extracted from the image for subsequent scale rationality judgment.

7. The intelligent patrol autonomous scheduling and analysis method with multi-algorithm cooperation compatibility as described in claim 1, characterized in that, In the self-correction calibration phase, when an equipment malfunction is detected, key features of the current frame are extracted and matched with a benchmark feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either imaging system offset or actual equipment malfunction, and automatic correction is achieved. This process is divided into the following sub-steps: After excluding biological interference anomalies, multi-scale benchmark features were extracted from the equipment normal state reference image to construct a multi-scale benchmark feature library for subsequent cross-scale matching and geometric consistency verification. The feature matching consistency ratio is calculated based on the current frame features and the multi-scale benchmark feature library to determine whether the anomaly is caused by the imaging system offset, and to decide whether to perform self-calibration or enter feature-level anomaly verification. By performing gimbal coordinate self-calibration and cognitive adaptation on two types of anomalies—imaging system offset and real equipment anomaly—viewpoint drift compensation and optimized correction of equipment state evolution are achieved.

8. An intelligent patrol autonomous scheduling and analysis device with multi-algorithm collaboration compatibility, characterized in that, include: Anomaly screening module: In the anomaly screening stage, multi-temporal inspection images are collected to construct a dynamic base map library. Texture complexity, edge stability and brightness deviation are fused through an exponential saturation fusion function to generate a visual disturbance index. Analysis and scheduling module: During the analysis and scheduling phase, based on the visual disturbance index, historical task response time and resource usage data, the module generates scheduling priority weights through the load priority function and assigns analysis strategies to each point. Depth discrimination module: In the depth anomaly discrimination stage, for images of key inspection areas, a composite anomaly intensity is synthesized through dual-channel parallel analysis to distinguish between real equipment defects and environmental interference; Target verification module: During the target verification phase, if the intensity of the composite anomaly exceeds the threshold, the intrusion judgment index is calculated based on the morphological compactness of the interfering object and the semantic similarity score of the prototype library to determine the anomaly type. Correction and calibration module: During the self-correction and calibration phase, when an equipment malfunction is detected, the key features of the current frame are extracted and matched with the benchmark feature library to generate a consistency ratio. Based on the ratio and its threshold, the cause of the malfunction is determined to be either an imaging system offset or a real equipment malfunction, and automatic correction is achieved.

Images