Substation surrounding hidden danger identification method and system based on multi-source visual space fusion

Abstract

This invention relates to the field of substation hazard monitoring technology, and discloses a method and system for identifying substation perimeter hazards based on multi-source visual spatial fusion. The method includes at least: identifying hazards in a standardized multi-source monitoring dataset according to data source, obtaining candidate hazard identification results for each source of visual data; spatially mapping each candidate hazard identification result to obtain the spatial distribution characteristics of each candidate hazard target in the station area's planar coordinate system; grouping and merging a unified candidate set based on each spatial distribution characteristic to obtain multiple hazard event objects; calculating the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and performing gated fusion on each observation set based on each uncertainty score to obtain the target hazard identification result for each hazard event object. This invention can efficiently and accurately detect safety hazards around substations.

Description

Technical Field

[0001] This invention relates to the field of substation hazard monitoring technology, and in particular to a method and system for identifying hazards around substations based on multi-source visual spatial fusion. Background Technology

[0002] As substations expand and the external environment becomes more complex, common hazards around the substation area (such as external construction damage, piles of abandoned garbage, temporary sunshades, and haphazard placement of tarpaulins and plastic sheets by farmers) threaten the stable operation of power grid equipment and the safety of on-site personnel. Traditional manual inspections or single fixed monitoring methods generally suffer from incomplete coverage, low efficiency, poor real-time performance, and delayed response to sudden hazards, making it difficult to meet the needs of refined and intelligent safety management.

[0003] To enhance hazard identification and early warning capabilities, existing technologies widely employ deep learning-based visual recognition methods to perform target detection on images / videos from a single source, outputting results such as hazard category and confidence level. Simultaneously, multi-source collaborative off-site hazard monitoring technologies have emerged, such as integrating multi-source data (satellite remote sensing, UAV inspections, and ground monitoring) into a single platform, using image enhancement and target detection models to achieve hazard identification. However, these multi-source fusion technologies are prone to problems such as duplicate hazard reporting, cross-source miscorrelation, low-quality observation interference, and difficulty in quantifying hazard scale under asynchronous multi-source conditions. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention proposes a method and system for identifying potential hazards around substations based on multi-source visual spatial fusion. By using a unified spatial mapping framework, event-level cross-source correlation, and uncertainty-gated fusion decision-making, it achieves consistent output of hazard categories, spatial locations, and physical scales, thereby improving alarm accuracy and deduplication capabilities, and reducing the risks of false alarms and duplicate dispatches.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying potential hazards around substations based on multi-source visual spatial fusion, including: Obtain visual data of the substation's surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset; The standardized multi-source monitoring dataset is used to identify potential hazards according to the data source, and candidate hazard identification results are obtained for each source of visual data, wherein each candidate hazard identification result corresponds to a candidate hazard target; Based on the pre-constructed station area plane coordinate system, spatial mapping is performed on each of the candidate hazard identification results to obtain the spatial distribution characteristics of each candidate hazard target under the station area plane coordinate system; Each of the aforementioned candidate hidden danger targets within the same time window is aggregated into a unified candidate set, and the unified candidate set is grouped and merged in combination with the spatial distribution characteristics of each to obtain multiple hidden danger event objects. Each of the aforementioned hidden danger event objects corresponds to an observation set of a field hidden danger entity on different inspection equipment. Based on each observation set, an uncertainty score is calculated for each inspection device according to a predefined uncertainty scoring index. Then, gating fusion is performed on each observation set based on each uncertainty score to obtain the target hazard identification result for each hazard event object.

[0006] Preferably, after calculating the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and performing gated fusion on each observation set according to each uncertainty score to obtain the target hazard identification result of each hazard event object, the method further includes: Using a large model trained with enhanced knowledge of power scenarios and fine-tuned from historical hazard monitoring data, risk classification and alarm decisions are made for each of the target hazard identification results.

[0007] Preferably, the step of acquiring visual data of the substation perimeter collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset includes: Acquire visual data of the substation's surroundings collected by multiple inspection devices, including drones, fixed monitoring equipment, and inspection robots; The visual data around the substation from each source are formatted and synchronized in time to obtain a standardized multi-source monitoring dataset.

[0008] Preferably, the step of identifying potential hazards in the standardized multi-source monitoring dataset according to the data source to obtain candidate hazard identification results corresponding to visual data from each source includes: The standardized multi-source monitoring dataset is preprocessed to obtain preprocessed visual data of the substation perimeter for each source. A pre-trained instance segmentation model is used to identify potential hazards in the pre-processed visual data around the substation from each source, resulting in candidate hazard identification results for each source of visual data. The candidate hazard identification results include the category, segmentation mask, and confidence level of the candidate hazard target.

[0009] Preferably, the step of spatially mapping each candidate hazard identification result to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system based on the pre-constructed station area plane coordinate system includes: A flat plane within the station area that covers the inspection area is selected as the station area plane. The center point of the station area plane is taken as the origin, and a station area plane coordinate system is established by defining coordinate axes. Multiple anchor points are set up on the plane of the station area, and anchor point detection is performed on the visual data of the substation surroundings collected by each inspection device to obtain the observation value of each anchor point in the pixel coordinate system of the corresponding inspection device. Based on the correspondence between each observation value and the known coordinates in the station area plane coordinate system, a set of linear equations is constructed for each inspection device, and each set of linear equations is solved to obtain the homography matrix from the pixel domain to the station area plane coordinate domain for each inspection device. Extract the segmentation mask pixel set from each candidate hazard identification result, and project each segmentation mask pixel set onto the station area plane coordinate system through the corresponding homography matrix to obtain the station area plane point set corresponding to each candidate hazard target; Based on each set of points in the station area, the spatial distribution characteristics of each candidate potential hazard target in the station area's plane coordinate system are calculated.

[0010] Preferably, the step of calculating the spatial distribution characteristics of each candidate potential hazard target in the station area plane coordinate system based on each of the station area plane point sets includes: Calculate the geometric center of each set of points in the station area, and use the geometric center as the spatial position of the corresponding candidate hidden danger target in the station area plane coordinate system; Contour extraction is performed on each set of planar points in the station area to obtain the planar envelope boundary of the corresponding candidate hidden danger target, and the coverage area of ​​the corresponding candidate hidden danger target in the planar coordinate system of the station area is calculated based on each planar envelope boundary.

[0011] Preferably, the step of aggregating each candidate potential hazard target within the same time window into a unified candidate set, and grouping and merging the unified candidate set based on the spatial distribution characteristics of each target to obtain multiple potential hazard event objects, includes: Each of the aforementioned candidate potential hazards within the same time window is aggregated into a unified candidate set, and the unified candidate set is divided according to the hazard category to obtain multiple candidate hazard groups; Extract the spatial location from each of the spatial distribution features, and calculate the spatial distance between any two candidate hazard targets within each candidate hazard group based on the spatial location; A threshold comparison is performed on each of the spatial distances, and based on the comparison results, the candidate hazard targets within each candidate hazard group are connected and merged to obtain multiple hazard merging subgroups, wherein each hazard merging subgroup corresponds to a hazard event object.

[0012] Preferably, the step of calculating the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and performing gated fusion on each observation set according to each uncertainty score to obtain the target hazard identification result for each hazard event object, includes: The definition includes multiple uncertainty scoring metrics, including detection confidence, image sharpness, and temporal stability; Calculate the uncertainty component of the corresponding inspection equipment under each uncertainty scoring index based on each observation set; Each of the aforementioned uncertainty components is weighted and fused to obtain the uncertainty score corresponding to each of the aforementioned inspection devices; For each uncertainty score, a gating threshold comparison is performed, and the set of observations that pass the gating is determined based on the comparison results; Each of the observation sets that passes through the gating is weighted and fused to obtain the target hazard identification result for each of the hazard events.

[0013] Preferably, the definition includes multiple uncertainty scoring indicators, including detection confidence, image sharpness, and temporal stability, among others: Take the confidence level from the candidate hazard identification results, and define the difference between 1 and the confidence level as the uncertainty component corresponding to the detection confidence level; Calculate the sharpness value of the original image corresponding to the observation set, and obtain the image sharpness score based on the sharpness value and the preset sharpness upper and lower limit thresholds. The difference between 1 and the sharpness score is defined as the uncertainty component corresponding to the image sharpness. When the same inspection equipment forms multiple frames of observation of the same on-site hidden danger entity within the current time window, the degree of planar positioning fluctuation of the inspection equipment within the current time window is calculated, and the minimum value of the time series stability score obtained by normalizing the degree of planar positioning fluctuation is defined as the uncertainty component corresponding to the time series stability. When the same inspection equipment forms a single-frame observation of the same on-site hidden danger entity within the current time window, 0 is defined as the uncertainty component corresponding to the temporal stability.

[0014] Secondly, embodiments of the present invention provide a substation perimeter hazard identification system based on multi-source visual spatial fusion, comprising: The visual data acquisition module is used to acquire visual data of the substation surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset. The single-source hazard identification module is used to identify hazards in the standardized multi-source monitoring dataset according to the data source, and obtain the candidate hazard identification result corresponding to the visual data of each source, wherein each candidate hazard identification result corresponds to a candidate hazard target; The spatial mapping module is used to perform spatial mapping on each of the candidate hazard identification results based on a pre-constructed station area plane coordinate system to obtain the spatial distribution characteristics of each candidate hazard target under the station area plane coordinate system; The grouping and merging module is used to aggregate each of the candidate hidden danger targets within the same time window into a unified candidate set, and to group and merge the unified candidate set according to the spatial distribution characteristics of each to obtain multiple hidden danger event objects, wherein each hidden danger event object corresponds to an observation set of a field hidden danger entity on different inspection equipment. The evaluation fusion module is used to calculate the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and to perform gating fusion on each observation set according to each uncertainty score to obtain the target hazard identification result of each hazard event object.

[0015] Compared with existing technologies, the present invention provides a method and system for identifying potential hazards around substations based on multi-source visual spatial fusion. Its advantages include at least the following: (1) After single-source hazard identification and spatial mapping processing, each candidate hazard target is mapped from the pixel coordinate system to the pre-constructed station area plane coordinate system, realizing the spatial distribution characteristics of multi-source candidate hazard targets under a unified spatial reference, effectively solving the technical problems of inconsistent spatial feature descriptions of hazard location and coverage area and difficulty in accurately quantifying physical scale in traditional multi-source monitoring; (2) By clustering and merging candidate hidden danger targets within the same time window, multi-source candidate hidden danger targets pointing to the same on-site hidden danger entity are integrated into a single hidden danger event object, establishing a unique correspondence between on-site hidden danger entities and hidden danger event objects. This fundamentally avoids the problems of repeated reporting of hidden dangers and cross-source misassociation caused by asynchronous observation of multiple devices, and greatly improves the uniqueness and pertinence of hidden danger identification. (3) Based on the multi-dimensional uncertainty scoring index of detection confidence, image clarity and temporal stability, the observation quality of each inspection equipment is quantitatively evaluated. After screening out low-quality observation data by combining the gating mechanism, the data is then weighted and fused, which effectively suppresses the interference of low-quality observation on the identification results and ensures the accuracy and stability of the target identification results of hidden danger events in terms of category, spatial distribution characteristics and actual physical scale. Attached Figure Description

[0016] Figure 1This is a flowchart illustrating a method for identifying potential hazards around substations based on multi-source visual spatial fusion, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the spatial mapping process in an embodiment of the present invention; Figure 3 This is a schematic diagram of the uncertainty assessment and gating fusion process in an embodiment of the present invention; Figure 4 This is another flowchart illustrating a method for identifying potential hazards around substations based on multi-source visual spatial fusion according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of a substation perimeter hazard identification system based on multi-source visual spatial fusion according to an embodiment of the present invention; Figure label: 01. Visual data acquisition module; 02. Single-source hazard identification module; 03. Spatial mapping module; 04. Grouping and merging module; 05. Evaluation and fusion module. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by those skilled in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0019] like Figure 1 The diagram shown is a flowchart illustrating a method for identifying potential hazards around substations based on multi-source visual spatial fusion, according to an embodiment of the present invention. (Refer to...) Figure 1 This invention provides a method for identifying potential hazards around substations based on multi-source visual spatial fusion, comprising the following steps: S1. Obtain visual data of the substation surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset; Specifically, step S1 includes: 1) Acquire visual data of the substation's surroundings collected by multiple inspection devices, including drones, fixed monitoring equipment, and inspection robots; Synchronously acquire the supporting metadata of each inspection device when collecting visual data. The supporting metadata includes at least the visual data collection timestamp, the real-time location information of the device, and the device's pose parameters. Link and store the visual data of the substation surroundings collected by each inspection device with the corresponding metadata to form independent visual data units for each source.

[0020] 2) Standardize the format and synchronize the time of visual data around substations from each source to obtain a standardized multi-source monitoring dataset.

[0021] Specifically, for the visual data in each source visual data unit, the resolution, frame rate and encoding format of the image / video data output by each inspection device are unified to eliminate the problem of data format heterogeneity caused by differences in hardware parameters between different devices.

[0022] Furthermore, based on the acquisition timestamps associated in each source visual data unit, cross-source time axis alignment is performed on the multi-source visual data that has achieved unified format, calibrating all visual data to the same time reference, and finally forming a standardized multi-source monitoring dataset consisting of visual data with unified format and time alignment, as well as associated metadata.

[0023] S2. Identify potential hazards in the standardized multi-source monitoring dataset according to the data source, and obtain the candidate hazard identification results corresponding to the visual data from each source. Specifically, step S2 includes: 1) Preprocess the standardized multi-source monitoring dataset to obtain preprocessed visual data of the substation perimeter for each source; For visual data of substation perimeter from various sources in the standardized multi-source monitoring dataset, preprocessing operations such as image denoising, contrast enhancement, and lens distortion correction are performed according to the type of data acquisition equipment. This eliminates image quality problems caused by factors such as changes in on-site lighting, differences in equipment hardware, and environmental interference, thereby effectively improving the recognizability of visual data.

[0024] 2) A pre-trained instance segmentation model is used to identify potential hazards in the pre-processed visual data around the substation from each source, and the candidate hazard identification results corresponding to each source of visual data are obtained.

[0025] In this embodiment, the pre-trained instance segmentation model is a Mask R-CNN model that has been trained and fine-tuned based on historical inspection images of potential hazards around the substation.

[0026] The preprocessed visual data of the substation perimeter from various sources is input frame by frame into the model. Through feature extraction, instance segmentation, and target detection, the model accurately identifies and locates suspected hidden danger targets in each frame of visual data at the pixel level. The model generates category determination results, pixel-level segmentation masks, and confidence scores for each suspected hidden danger target. Each set of complete identification results, including category, segmentation mask, and confidence score, is a candidate hidden danger identification result for the corresponding source of visual data. Each candidate hidden danger identification result has a unique correspondence with an independent candidate hidden danger target.

[0027] S3. Based on the pre-constructed station area plane coordinate system, spatial mapping is performed on each candidate hazard identification result to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system; like Figure 2 As shown, this is a flowchart illustrating step S3. (Refer to...) Figure 2 Step S3 includes: S301. Select a flat plane within the station area that covers the inspection area as the station area plane. Establish the station area plane coordinate system by defining coordinate axes, with the center point of the station area plane as the origin. With the center point of the station area as the origin of the coordinate system, the positive direction of the X-axis is defined as the direction parallel to the east of the station area inspection road, and the positive direction of the Y-axis is defined as the direction parallel to the north of the station area inspection road. The coordinate unit is uniformly set to meters.

[0028] The coordinate axes are calibrated by the actual physical location of the control points at the station area boundary to ensure that the constructed station area plane coordinate system can fully cover the main inspection area of ​​the substation, and that the coordinate values ​​of the coordinate system correspond one-to-one with the actual physical location on site.

[0029] S302. Multiple anchor points are set up on the station area plane, and anchor point detection is performed on the visual data of the substation perimeter collected by each inspection equipment to obtain the observation value of each anchor point in the pixel coordinate system of the corresponding inspection equipment. The deployed anchor points must meet the requirements of known coordinates, non-collinearity, and reasonable geometric distribution. The actual physical coordinates of each anchor point in the station area's plane coordinate system must be measured and recorded in advance. For the visual data collected by each inspection device, a corner detection algorithm is used to detect the anchor point markers in the image, accurately extracting the two-dimensional pixel observation value of each anchor point in the corresponding inspection device's pixel coordinate system. The validity of the detection results is verified, and invalid observation values ​​caused by occlusion or blurring are removed to ensure that each anchor point can obtain a valid pixel-plane coordinate correspondence point pair.

[0030] S303. Based on the correspondence between each observation value and the known coordinates in the station area plane coordinate system, construct a set of linear equations corresponding to each inspection device, and solve each set of linear equations to obtain the homography matrix from the pixel domain to the station area plane coordinate domain corresponding to each inspection device. Based on the principle of homography transformation, the observed values ​​of the anchor point in the pixel coordinate system Known coordinates in the station area plane coordinate system Satisfying the homogeneous coordinate transformation relationship: in, The scale factor allows for "perspective scaling" of coordinates during projection, i.e., a non-zero constant in the projection transformation process. It is used to eliminate scale differences between the pixel coordinate system and the station area plane coordinate system during projection mapping. This represents a 3×3 homography matrix containing There are a total of 9 parameters to be solved.

[0031] Based on at least four pairs of valid anchor points, a system of linear equations is constructed using the Direct Linear Transformation (DLT) method. ( This represents the coefficient matrix consisting of the pairs of points corresponding to the anchor points. The homography matrix parameters are represented by column vectors expanded column by column. The linear equations are solved by singular value decomposition (SVD) to eliminate scale ambiguity and obtain the homography matrix corresponding to each inspection device, thus realizing the mapping relationship modeling from the pixel domain to the station area plane coordinate domain.

[0032] S304. Extract the segmentation mask pixel set from each candidate hazard identification result, and project each segmentation mask pixel set onto the station area plane coordinate system through the corresponding homography matrix to obtain the station area plane point set corresponding to each candidate hazard target. Iterate through the segmentation mask in each candidate hazard identification result, extract the pixel coordinates of all pixels within the mask, and form a segmentation mask pixel set. ( (This represents the total number of pixels within the mask).

[0033] The pixel coordinates of each pixel are converted into homogeneous coordinates, multiplied with the homography matrix of the corresponding inspection equipment to obtain homogeneous planar coordinates, and then normalized to finally obtain the set of planar points in the station area corresponding to the candidate hidden danger target. .

[0034] S305. Based on the set of points on the plane of each station area, calculate the spatial distribution characteristics of each candidate hidden danger target in the plane coordinate system of the station area.

[0035] Specifically, step S305 includes: 1) Calculate the geometric center of the plane point set of each station area, and use the geometric center as the spatial position of the corresponding candidate hidden danger target in the plane coordinate system of the station area; For each set of points on the plane of the station area, the coordinates of its geometric center are calculated by the arithmetic mean method. These coordinates accurately represent the core location of the candidate hidden danger target on the station area site.

[0036] 2) Extract the contours of each station area's planar point set to obtain the corresponding candidate hidden danger targets' planar envelope boundaries, and calculate the coverage area of ​​the corresponding candidate hidden danger targets in the station area's planar coordinate system based on each planar envelope boundary.

[0037] For each station area's planar point set, the Canny edge detection algorithm is used to extract edge points, and then the planar envelope boundary of the candidate hidden danger targets is obtained by fitting using the polygon approximation method. The boundary is composed of ordered contour points. ( The number of contour points is represented.

[0038] The coverage area is calculated using the polygon area calculation formula based on Green's formula. This area quantitatively represents the actual physical coverage range of the candidate hazard target in the station area.

[0039] S4. Aggregate each candidate hidden danger target within the same time window into a unified candidate set, and combine the unified candidate set with each spatial distribution characteristic to obtain multiple hidden danger event objects; Specifically, step S4 includes: 1) Aggregate each candidate potential hazard target within the same time window into a unified candidate set, and divide the unified candidate set according to the hazard category to obtain multiple candidate hazard groups; All candidate potential hazards that have been spatially mapped and are within the same time window are aggregated across the entire domain to form a unified candidate set containing core information such as the category of each candidate potential hazard and its spatial distribution characteristics under the station area's plane coordinate system.

[0040] Based on the hazard category corresponding to each candidate hazard target (such as external construction, waste, temporary tarpaulin, etc.), the unified candidate set is classified and divided. Candidate hazard targets belonging to the same hazard category are grouped into the same candidate hazard group, while candidate hazard targets of different hazard categories belong to different groups, thereby achieving centralized classification of hazards of the same category.

[0041] 2) Extract the spatial location from each spatial distribution feature, and calculate the spatial distance between any two candidate hazard targets within each candidate hazard group based on the spatial location; From the spatial distribution characteristics of each candidate hazard target, the geometric center coordinates in the station area's plane coordinate system are extracted as the core spatial location information. For each candidate hazard group, all candidate hazard targets within the group are traversed, and pairwise combinations of any two candidate hazard targets within the group are constructed. Using the planar Euclidean distance calculation formula, the spatial Euclidean distance between the geometric centers of the two candidate hazard targets in each pairwise combination is calculated sequentially.

[0042] 3) Perform threshold comparisons for each spatial distance, and based on the comparison results, connect and merge the candidate hidden danger targets within each candidate hidden danger group to obtain multiple hidden danger merged subgroups.

[0043] A pre-set spatial distance threshold is established, which is determined based on the spatial mapping accuracy and the positioning error of the inspection equipment. All spatial distance calculations within each candidate hazard group are compared one by one with the pre-set spatial distance threshold. If the spatial distance between two candidate hazard targets is less than or equal to the threshold, they are determined to point to the same on-site hazard entity, and a merged association relationship is established between the two candidate hazard targets.

[0044] Based on all established merging relationships, candidate hazard targets within each candidate hazard group are connected and merged. Candidate hazard targets with direct or indirect merging relationships are merged into the same hazard merging subgroup, while candidate hazard targets without any merging relationships are formed into a separate hazard merging subgroup.

[0045] Each hazard merging subgroup integrates all candidate hazard targets pointing to the same on-site hazard entity, and each hazard merging subgroup uniquely corresponds to a hazard event object, which corresponds to the observation set of the on-site hazard entity on different inspection equipment.

[0046] It should be noted that the observation set is composed of the full observation data of each inspection device corresponding to the on-site hidden danger entity. The observation data corresponding to each inspection device includes all candidate hidden danger identification results obtained by the device for the on-site hidden danger entity, as well as the relevant visual data of the substation surrounding the device and the corresponding data such as the collection timestamp and device pose. The observation data of each inspection device are independently collected and related to each other, and together they form the complete observation set corresponding to the hidden danger event object.

[0047] S5. Based on each observation set, calculate the uncertainty score corresponding to each inspection equipment according to the predefined uncertainty scoring index, and perform gating fusion on each observation set according to each uncertainty score to obtain the target hidden danger identification result of each hidden danger event object.

[0048] like Figure 3 As shown, this is a flowchart illustrating step S5. (Refer to...) Figure 3Step S5 includes: S501 defines multiple uncertainty scoring metrics, including detection confidence, image sharpness, and temporal stability. Specifically, step S501 includes: 1) Take the confidence level from the candidate hazard identification results, and define the difference between 1 and the confidence level as the uncertainty component corresponding to the detection confidence level; The confidence score in the candidate hazard identification results is the detection confidence score output by the instance segmentation model. Its value range is A higher confidence score indicates a stronger reliability of the corresponding candidate hazard identification result.

[0049] Directly calculate 1 and the detection confidence level. The difference is taken as the uncertainty component corresponding to the detection confidence level, and the value range of this component is synchronously defined. The smaller the component value, the lower the observation uncertainty in the detection confidence dimension.

[0050] 2) Calculate the sharpness value of the original image corresponding to the observation set, and obtain the image sharpness score based on the sharpness value and the pre-set upper and lower sharpness thresholds. The difference between 1 and the sharpness score is defined as the uncertainty component corresponding to the image sharpness. The original visual images of the substation surroundings corresponding to the observation set are converted into grayscale images. Laplacian filtering is then applied to the grayscale images, and the variance of the Laplacian response of the filtered image is calculated. This variance value is used as the image sharpness value. .

[0051] The lower limit threshold for sharpness is determined in advance based on the statistical results of historical images under normal operating conditions of the inspection equipment. and resolution upper limit threshold If the resolution value Then the sharpness score is 0; if the sharpness value is... Then the sharpness score is 1. The sharpness score is then calculated using linear normalization. Finally, the difference between 1 and this sharpness score is calculated and used as the uncertainty component corresponding to the image sharpness. The component's value range is... .

[0052] 3) When the same inspection equipment forms multiple frames of observation of the same on-site hidden danger entity within the current time window, calculate the degree of planar positioning fluctuation of the inspection equipment within the current time window, and define the minimum value of the time series stability score obtained by normalizing the degree of planar positioning fluctuation as the uncertainty component corresponding to the time series stability. Extract the geometric center coordinates of all candidate potential hazards in the station area's plane coordinate system from multiple frames of observations of the same potential hazard entity by the same inspection equipment, forming a coordinate sequence. ( (Indicates the number of observation frames).

[0053] The average position of the coordinate sequence is calculated, and the degree of planar positioning fluctuation is calculated using the root mean square offset formula. In other words, the root mean square offset reflects the degree of planar positioning fluctuation of the inspection equipment when performing multiple frame observations of the same potential hazard entity within the current time window. The root mean square offset is divided by a preset normalization scale (the maximum allowable positioning error scale) to obtain a temporal stability score. Finally, the minimum value between 1 and this temporal stability score is taken and defined as the uncertainty component corresponding to temporal stability, ensuring that the value of this component always remains within a certain range. Interval.

[0054] 4) When the same inspection equipment forms a single-frame observation of the same on-site hidden danger entity within the current time window, 0 is defined as the uncertainty component corresponding to the temporal stability.

[0055] Since single-frame observations lack multi-frame coordinate data for calculating the degree of planar positioning fluctuations and lack a basis for evaluating temporal stability, the uncertainty component corresponding to temporal stability is directly assigned a value of 0. This assignment rule indicates that single-frame observations have no observation uncertainty in the temporal stability dimension, which complements the component calculation logic of multi-frame observations and ensures the integrity and uniformity of temporal stability component calculations under different observation frame numbers.

[0056] S502. Calculate the uncertainty component of the corresponding inspection equipment under each uncertainty scoring index based on each observation set; For each inspection equipment's observation set corresponding to a potential hazard event, core data such as the confidence level of candidate hazard identification results, original visual images, and station area planar positioning coordinates of candidate hazard targets are extracted from the observation set. Based on the defined uncertainty component calculation rules corresponding to detection confidence, image clarity, and temporal stability, the uncertainty components of each inspection equipment under the three uncertainty scoring indicators are calculated one by one, and denoted as follows: (Detection confidence component) (Image sharpness component) (Time series stability components), each component takes the value of... .

[0057] S503. Weighted fusion of each uncertainty component to obtain the uncertainty score for each inspection device; Weighting factors matching the importance of observation quality are pre-defined for the three uncertainty components, with detection confidence being the dominant factor, followed by image sharpness, and temporal stability having a relatively smaller weight. Detection confidence directly reflects whether the target actually exists and is the core credibility indicator of the model output, therefore it should have a high weight. Image sharpness affects detection and localization accuracy; blurring can easily lead to boundary shifts, but it usually does not completely overturn the detection and localization results, so localization is of secondary importance. Temporal stability is only effective in multi-frame scenarios, therefore it is only used as an auxiliary factor.

[0058] This embodiment sets the weighting factor for the detection confidence component. Image sharpness component weighting factor Time-series stability component weighting factor ,and In practical applications, each weighting factor can be adjusted appropriately based on the equipment type and operating environment.

[0059] Through weighted fusion formula The three uncertainty components calculated for each inspection device are fused to obtain the uncertainty score corresponding to that inspection device. The range of values ​​for this rating is: The smaller the score, the higher the observation quality of the inspection equipment and the lower the overall observation uncertainty.

[0060] S504. For each uncertainty score, perform a gating threshold comparison and determine the set of observations that pass the gating based on the comparison results; Pre-set uncertainty threshold This threshold is determined by working backwards from the system's tolerance for false positives. A higher threshold favors "better to overreport than underreport," while a lower threshold favors "better to underreport than overreport." To reduce false positives and duplicate dispatches, and to avoid unacceptable observations from affecting the final identification result during fusion, the threshold should not be too high. The preferred threshold range is [range to be specified]. A more preferred threshold is 0.55.

[0061] The uncertainty score corresponding to each inspection device With gate threshold Compare them one by one, if If the observation set of the inspection equipment is determined to be valid observation data, it will be gated and participate in subsequent fusion calculations; if If the data is deemed low-quality, it will not be gated; instead, it will only be recorded and not fused, thus achieving precise screening of low-quality observations.

[0062] S505. Perform weighted fusion on each set of observations that have passed the gating to obtain the target hazard identification result for each hazard event object.

[0063] For each potential incident, an uncertainty score is assigned based on the corresponding gated observation sets. The fusion weights for each observation set are calculated using the following formula: in, Indicates the first The fusion weights correspond to the gated set of observations. The lower the uncertainty, the larger the fusion weight.

[0064] Based on the fusion weight, the probability of candidate hazard categories, the spatial coordinates of the station area, and the coverage area of ​​the hazard are weighted and fused respectively in each gated observation set to obtain the fused hazard category, fused spatial location, and fused coverage area of ​​the hazard event object. The above three fusion results together constitute the target hazard identification result of the hazard event object.

[0065] To achieve refined risk classification and intelligent alarm decision-making, and to further improve the response efficiency and accuracy of risk management around substations, it is necessary to use large models to carry out subsequent risk reasoning work.

[0066] like Figure 4 The diagram shown is another flowchart illustrating a method for identifying potential hazards around substations based on multi-source visual spatial fusion, according to an embodiment of the present invention. (Refer to...) Figure 4 This invention provides a method for identifying potential hazards around substations based on multi-source visual spatial fusion. Following step S5, the method further includes the following step: S6. Using a large model that has been enhanced with knowledge of power scenarios and fine-tuned with historical hazard monitoring data, risk classification and alarm decisions are made for the identification results of each target hazard.

[0067] The large model is a language model trained with power scenario knowledge enhancement and historical hidden danger monitoring data. Its input data is the target hidden danger identification results of each hidden danger event object, including the hidden danger category, the fused station area plane spatial location and the hidden danger coverage area.

[0068] First, the large model, combined with the planar coordinate data of the substation perimeter wall, calculates the minimum planar distance between the hazard fusion location and the perimeter wall boundary using a minimum distance algorithm from a point to the polygon boundary. Simultaneously, based on the hazard category, the corresponding coverage area parameter is extracted and compared with the preset category-specific scale safety threshold. Compare them.

[0069] Then, reasoning is performed based on the risk assessment rules: when Less than or equal to the preset safe distance threshold (Preferably 3 meters) or the area covered by the hazard exceeds (For example, the preferred depth for potential hazards such as accumulated debris is 1m) 2 When a risk alarm is triggered, the alarm level is increased if both conditions are met, and a general alarm or inspection and review task is generated if only one condition is met.

[0070] Finally, the large model outputs a risk level. Based on this level, the system automatically generates alarm information that includes the location, type, scale, and handling suggestions of the hidden danger. This information is then uploaded to the operation and maintenance management platform and a graded task is assigned, achieving intelligent closed-loop management of hidden danger alarms and handling.

[0071] After the tiered task assignment is completed, and on-site staff finish the hazard verification and handling, the operation and maintenance management platform will record the actual verification results, the true risk level of the hazard, and the final handling status in detail, and send this feedback data back to the system database. This feedback data will serve as the basis for optimization, used to update the historical hazard sample library, iteratively optimize the risk tiering strategy, and correct the gate threshold in the uncertainty assessment process, thereby continuously improving the accuracy of large-scale model risk inference and the overall hazard identification efficiency of the system.

[0072] This invention discloses a method for identifying potential hazards around substations based on multi-source visual spatial fusion. After single-source hazard identification and spatial mapping, each candidate hazard target is mapped from a pixel coordinate system to a pre-constructed station area planar coordinate system. This quantifies the spatial distribution characteristics of multi-source candidate hazard targets under a unified spatial reference, effectively solving the technical problems of inconsistent spatial feature descriptions such as hazard location and coverage area, and difficulty in accurately quantifying physical scale in traditional multi-source monitoring. Furthermore, by clustering and merging candidate hazard targets within the same time window, multi-source candidate hazard targets pointing to the same on-site hazard entity are integrated into a single hazard event object. A unique correspondence is established between on-site hazard entities and hazard event objects, fundamentally avoiding the problems of duplicate hazard reporting and cross-source misassociation caused by asynchronous observation of multiple devices, and significantly improving the uniqueness and pertinence of hazard identification. Based on the multi-dimensional uncertainty scoring index of detection confidence, image clarity, and temporal stability, the observation quality of each inspection device is quantitatively evaluated. Combined with the gating mechanism to filter out low-quality observation data before weighted fusion, the interference of low-quality observation on the identification results is effectively suppressed, ensuring the accuracy and stability of the target identification results of hazard event objects in terms of category, spatial distribution characteristics, and actual physical scale.

[0073] like Figure 5 The diagram shown is a structural schematic of a substation perimeter hazard identification system based on multi-source visual spatial fusion, according to an embodiment of the present invention. (Refer to...) Figure 5 This invention provides a substation perimeter hazard identification system based on multi-source visual spatial fusion, comprising: The visual data acquisition module 01 is used to acquire visual data around the substation collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset. The single-source hazard identification module 02 is used to identify hazards in the standardized multi-source monitoring dataset according to the data source, and obtain the candidate hazard identification results corresponding to the visual data of each source. Each candidate hazard identification result corresponds to a candidate hazard target. The spatial mapping module 03 is used to perform spatial mapping on each candidate hazard identification result based on the pre-constructed station area plane coordinate system to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system. The grouping and merging module 04 is used to aggregate each candidate hidden danger target within the same time window into a unified candidate set, and to group and merge the unified candidate set according to each spatial distribution feature to obtain multiple hidden danger event objects. Each hidden danger event object corresponds to an on-site hidden danger entity's observation set on different inspection equipment. The evaluation fusion module 05 is used to calculate the uncertainty score corresponding to each inspection equipment based on each observation set according to the predefined uncertainty scoring index, and to perform gating fusion on each observation set according to each uncertainty score to obtain the target hazard identification result of each hazard event object.

[0074] It should be noted that each module in the aforementioned substation perimeter hazard identification system based on multi-source visual spatial fusion can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, allowing the processor to call and execute the corresponding operations of each module. For specific limitations regarding the substation perimeter hazard identification system based on multi-source visual spatial fusion, please refer to the limitations of the substation perimeter hazard identification method based on multi-source visual spatial fusion described above; both have the same function and role, and will not be repeated here.

[0075] In summary, the present invention provides a method and system for identifying potential hazards around substations based on multi-source visual spatial fusion. Through single-source hazard identification and spatial mapping, each candidate hazard target is mapped from the pixel coordinate system to a pre-constructed station area plane coordinate system. This quantifies the spatial distribution characteristics of multiple candidate hazard targets under a unified spatial reference, effectively solving the technical problems of inconsistent spatial feature descriptions such as hazard location and coverage area, and difficulty in accurately quantifying physical scale in traditional multi-source monitoring. Furthermore, by clustering and merging candidate hazard targets within the same time window, multiple candidate hazard targets pointing to the same on-site hazard entity are integrated into a single hazard event. This approach establishes a unique correspondence between on-site hazard entities and hazard event objects, fundamentally avoiding the problems of duplicate hazard reporting and cross-source misassociations caused by asynchronous observations from multiple devices, and significantly improving the uniqueness and relevance of hazard identification. Based on multi-dimensional uncertainty scoring indicators such as detection confidence, image clarity, and temporal stability, the observation quality of each inspection device is quantitatively evaluated. Combined with a gating mechanism to filter out low-quality observation data before weighted fusion, the interference of low-quality observations on the identification results is effectively suppressed, ensuring the accuracy and stability of the hazard event object target identification results in terms of category, spatial distribution characteristics, and actual physical scale.

[0076] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0077] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A substation surrounding hidden danger identification method based on multi-source vision space fusion, characterized in that, include: Obtain visual data of the substation's surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset; The standardized multi-source monitoring dataset is used to identify potential hazards according to the data source, and candidate hazard identification results are obtained for each source of visual data, wherein each candidate hazard identification result corresponds to a candidate hazard target; Based on a pre-constructed station area plane coordinate system, spatial mapping is performed on each candidate hazard identification result to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system, including: A flat plane within the station area that covers the inspection area is selected as the station area plane. The center point of the station area plane is taken as the origin, and a station area plane coordinate system is established by defining coordinate axes. Extract the segmentation mask pixel set from each candidate hazard identification result, and project each segmentation mask pixel set onto the station area plane coordinate system through the corresponding homography matrix to obtain the station area plane point set corresponding to each candidate hazard target; Based on each set of points in the station area, the spatial distribution characteristics of each candidate potential hazard target in the station area plane coordinate system are calculated. Each of the aforementioned candidate hidden danger targets within the same time window is aggregated into a unified candidate set, and the unified candidate set is grouped and merged in combination with the spatial distribution characteristics of each to obtain multiple hidden danger event objects. Each of the aforementioned hidden danger event objects corresponds to an observation set of a field hidden danger entity on different inspection equipment. Based on each observation set, an uncertainty score is calculated for each inspection device according to a predefined uncertainty scoring index. Then, gating fusion is performed on each observation set based on each uncertainty score to obtain the target hazard identification result for each hazard event object, including: The definition includes multiple uncertainty scoring metrics, including detection confidence, image sharpness, and temporal stability; Calculate the uncertainty component of the corresponding inspection equipment under each uncertainty scoring index based on each observation set; Each of the aforementioned uncertainty components is weighted and fused to obtain the uncertainty score corresponding to each of the aforementioned inspection devices; For each uncertainty score, a gating threshold comparison is performed, and the set of observations that pass the gating is determined based on the comparison results; Each of the observation sets that passes through the gating is weighted and fused to obtain the target hazard identification result for each of the hazard events; Among them, the uncertainty scoring indicators corresponding to time series stability include: When the same inspection equipment forms multiple frames of observation of the same on-site hidden danger entity within the current time window, the degree of planar positioning fluctuation of the inspection equipment within the current time window is calculated, and the minimum value of the time series stability score obtained by normalizing the degree of planar positioning fluctuation is defined as the uncertainty component corresponding to the time series stability. When the same inspection equipment forms a single-frame observation of the same on-site hidden danger entity within the current time window, 0 is defined as the uncertainty component corresponding to the temporal stability.

2. The method for substation perimeter hazard identification based on multi-source vision space fusion according to claim 1, characterized in that, After calculating the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and performing gated fusion on each observation set based on each uncertainty score to obtain the target hazard identification result for each hazard event object, the method further includes: Using a large model trained with enhanced knowledge of power scenarios and fine-tuned from historical hazard monitoring data, risk classification and alarm decisions are made for each of the target hazard identification results.

3. The substation perimeter hazard identification method based on multi-source vision space fusion according to claim 1, characterized in that, The process involves acquiring visual data of the substation's surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset, including: Acquire visual data of the substation's surroundings collected by multiple inspection devices, including drones, fixed monitoring equipment, and inspection robots; The visual data around the substation from each source are formatted and synchronized in time to obtain a standardized multi-source monitoring dataset.

4. The substation perimeter hazard identification method based on multi-source vision space fusion according to claim 1, characterized in that, The step of identifying potential hazards in the standardized multi-source monitoring dataset according to the data source, and obtaining candidate hazard identification results for visual data from each source, includes: The standardized multi-source monitoring dataset is preprocessed to obtain preprocessed visual data of the substation perimeter for each source. A pre-trained instance segmentation model is used to identify potential hazards in the pre-processed visual data around the substation from each source, and candidate hazard identification results are obtained for each source of visual data. The candidate hazard identification results include the category, segmentation mask, and confidence level of the candidate hazard target.

5. The substation perimeter hazard identification method based on multi-source vision space fusion according to claim 1, characterized in that, The method of spatially mapping each candidate hazard identification result to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system based on the pre-constructed station area plane coordinate system further includes: Multiple anchor points are set up on the plane of the station area, and anchor point detection is performed on the visual data around the substation collected by each inspection device to obtain the observation value of each anchor point in the pixel coordinate system of the corresponding inspection device. Based on the correspondence between each observation value and the known coordinates in the station area plane coordinate system, a set of linear equations is constructed for each inspection device, and each set of linear equations is solved to obtain the homography matrix from the pixel domain to the station area plane coordinate domain for each inspection device.

6. The method for identifying potential hazards around substations based on multi-source visual spatial fusion according to claim 5, characterized in that, The calculation of the spatial distribution characteristics of each candidate potential hazard target in the station area's plane coordinate system based on each set of points in the station area includes: Calculate the geometric center of each set of points in the station area, and use the geometric center as the spatial position of the corresponding candidate hidden danger target in the station area plane coordinate system; Contour extraction is performed on each set of planar points in the station area to obtain the planar envelope boundary of the corresponding candidate hidden danger target, and the coverage area of ​​the corresponding candidate hidden danger target in the planar coordinate system of the station area is calculated based on each planar envelope boundary.

7. The method for identifying potential hazards around substations based on multi-source visual spatial fusion according to claim 1, characterized in that, The step of aggregating each candidate potential hazard target within the same time window into a unified candidate set, and grouping and merging the unified candidate set based on the spatial distribution characteristics of each target to obtain multiple potential hazard event objects, includes: Each of the aforementioned candidate potential hazards within the same time window is aggregated into a unified candidate set, and the unified candidate set is divided according to the hazard category to obtain multiple candidate hazard groups; Extract the spatial location from each of the spatial distribution features, and calculate the spatial distance between any two candidate hazard targets within each candidate hazard group based on the spatial location; A threshold comparison is performed on each of the spatial distances, and based on the comparison results, the candidate hazard targets within each candidate hazard group are connected and merged to obtain multiple hazard merging subgroups, wherein each hazard merging subgroup corresponds to a hazard event object.

8. The method for identifying potential hazards around substations based on multi-source visual spatial fusion according to claim 1, characterized in that, The uncertainty scoring index corresponding to the detection confidence level includes: taking the confidence level in the candidate hazard identification result, and defining the difference between 1 and the confidence level as the uncertainty component corresponding to the detection confidence level; The uncertainty scoring index corresponding to image sharpness includes: calculating the sharpness value of the original image corresponding to the observation set, and obtaining the image sharpness score based on the sharpness value and a pre-set upper and lower limit threshold of sharpness, and defining the difference between 1 and the sharpness score as the uncertainty component corresponding to image sharpness.

9. A substation perimeter hazard identification system based on multi-source visual spatial fusion, characterized in that, include: The visual data acquisition module is used to acquire visual data of the substation surroundings collected by multiple inspection devices to obtain a standardized multi-source monitoring dataset. The single-source hazard identification module is used to identify hazards in the standardized multi-source monitoring dataset according to the data source, and obtain the candidate hazard identification result corresponding to the visual data of each source, wherein each candidate hazard identification result corresponds to a candidate hazard target; The spatial mapping module is used to perform spatial mapping on each of the candidate hazard identification results based on a pre-constructed station area plane coordinate system to obtain the spatial distribution characteristics of each candidate hazard target in the station area plane coordinate system, including: A flat plane within the station area that covers the inspection area is selected as the station area plane. The center point of the station area plane is taken as the origin, and a station area plane coordinate system is established by defining coordinate axes. Extract the segmentation mask pixel set from each candidate hazard identification result, and project each segmentation mask pixel set onto the station area plane coordinate system through the corresponding homography matrix to obtain the station area plane point set corresponding to each candidate hazard target; Based on each set of points in the station area, the spatial distribution characteristics of each candidate potential hazard target in the station area plane coordinate system are calculated. The grouping and merging module is used to aggregate each of the candidate hidden danger targets within the same time window into a unified candidate set, and to group and merge the unified candidate set according to the spatial distribution characteristics of each to obtain multiple hidden danger event objects, wherein each hidden danger event object corresponds to an observation set of a field hidden danger entity on different inspection equipment. The evaluation fusion module is used to calculate the uncertainty score corresponding to each inspection device based on each observation set according to a predefined uncertainty scoring index, and to perform gating fusion on each observation set based on each uncertainty score to obtain the target hazard identification result for each hazard event object, including: The definition includes multiple uncertainty scoring metrics, including detection confidence, image sharpness, and temporal stability; Calculate the uncertainty component of the corresponding inspection equipment under each uncertainty scoring index based on each observation set; Each of the aforementioned uncertainty components is weighted and fused to obtain the uncertainty score corresponding to each of the aforementioned inspection devices; For each uncertainty score, a gating threshold comparison is performed, and the set of observations that pass the gating is determined based on the comparison results; Each of the observation sets that passes through the gating is weighted and fused to obtain the target hazard identification result for each of the hazard events; Among them, the uncertainty scoring indicators corresponding to time series stability include: When the same inspection equipment forms multiple frames of observation of the same on-site hidden danger entity within the current time window, the degree of planar positioning fluctuation of the inspection equipment within the current time window is calculated, and the minimum value of the time series stability score obtained by normalizing the degree of planar positioning fluctuation is defined as the uncertainty component corresponding to the time series stability. When the same inspection equipment forms a single-frame observation of the same on-site hidden danger entity within the current time window, 0 is defined as the uncertainty component corresponding to the temporal stability.

Images