An environmental obstacle dynamic identification and early warning method and system

Abstract

The application discloses an environment obstacle dynamic identification and early warning method and system, relates to the field of obstacle identification, and comprises the following steps: collecting a video stream in real time through an AI camera, and storing the video stream in a ring buffer queue after decoding; separating foreground from an image frame and generating a candidate box, and identifying an obstacle category through a neural network; calculating the motion speed and direction of a target through cross-frame association, and generating trajectory data; converting pixel coordinates into actual space coordinates based on three-dimensional coordinate mapping, and predicting the minimum approach distance in a future time window; if the distance is less than a safety threshold, triggering a hierarchical early warning according to a ratio, and sending alarm information to an operation and maintenance platform. The application has the advantages that AI visual detection, target tracking, three-dimensional space mapping and trajectory prediction technologies are combined, obstacles are accurately identified, the approach risk of the obstacles to live equipment is predicted, and the initiative of in-station safety protection and the intelligent level of operation and maintenance are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of obstacle recognition, and in particular to a method and system for dynamic recognition and early warning of environmental obstacles. Background Technology

[0002] As the core hub of the power transmission and distribution system, substations are densely packed with equipment and operate at high voltage. The safety and stability of their operating environment directly affects the power supply reliability of the entire power grid. Currently, substation operation and maintenance scenarios are becoming increasingly complex. In addition to static obstacles such as fixed equipment structures, substations frequently encounter various dynamic obstacles such as construction machinery, birds, floating debris, and intruding small animals. These obstacles can easily lead to safety accidents such as equipment collisions, line short circuits, and arc discharges, resulting in serious consequences such as equipment damage, power grid tripping, and even personal injury or death.

[0003] Most current obstacle recognition and early warning methods on the market lack independent circular buffers and read / write pointer control mechanisms for multi-channel monitoring video streams. Under high-concurrency acquisition, they are prone to frame loss, timing disorders, and processing blockages, making it difficult to ensure stable operation at the edge. The detection stage either relies solely on traditional differential algorithms, resulting in a high false negative and false negative rate, or it relies entirely on deep neural network inference, causing excessive computational overhead. It lacks a hierarchical detection mechanism that combines foreground and background coarse screening with model fine judgment. Target tracking often uses simple inter-frame matching with single features, which is prone to target loss and identity switching, and the accuracy of motion parameter calculation is insufficient. Spatial ranging often remains at the pixel level comparison or coarse two-dimensional mapping, lacking accurate camera calibration and three-dimensional spatial coordinate transformation, resulting in large errors in the actual distance calculation and failing to meet the safety protection requirements of equipment with different voltage levels. Risk warnings are mostly triggered after boundary crossings, lacking pre-trajectory prediction based on motion vectors and minimum approach distance prediction. They also generally adopt a single-level warning mode with a uniform threshold, and the linkage means are limited, making it difficult to achieve hierarchical and accurate protection. The overall proactiveness and scenario adaptability of safety protection are significantly insufficient. Summary of the Invention

[0004] To improve existing methods and systems, this paper provides a method and system for dynamic identification and early warning of environmental obstacles. This method integrates AI visual detection, target tracking, three-dimensional spatial mapping and trajectory prediction technologies to accurately identify obstacles and predict their proximity risk to energized equipment. Through a hierarchical early warning and linkage mechanism, it effectively enhances the initiative of on-site safety protection and the level of intelligent operation and maintenance.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for dynamic identification and early warning of environmental obstacles, comprising:

[0007] AI cameras deployed at multiple preset monitoring points within the substation collect real-time video streams of the substation environment. Each video stream is then decoded frame by frame, and the resulting continuous image frames are stored in a circular buffer queue in the order of their acquisition time.

[0008] The current image is extracted frame by frame from the circular buffer queue. Foreground and background separation processing is performed on the current image. Candidate obstacle boxes are generated based on the separated foreground regions. The candidate obstacle boxes are input into a pre-trained obstacle classification neural network, and the obstacle category label corresponding to each candidate obstacle box is output.

[0009] For the same obstacle target in multiple consecutive frames of images with identified obstacle category labels, cross-frame association matching is performed. Based on the pixel coordinate changes of the obstacle target in each frame image, and combined with the three-dimensional spatial coordinate mapping relationship, the motion velocity vector and motion direction angle in the actual space are calculated to generate the motion trajectory data of the obstacle target.

[0010] Based on the obtained motion trajectory data, combined with the pre-established three-dimensional spatial coordinate mapping relationship of the substation, the pixel coordinates of the obstacle target in each frame image are converted into the actual spatial coordinates of the substation, and the minimum approach distance between the obstacle target and each energized device in the future within a preset time window is predicted based on the motion velocity vector.

[0011] When the predicted minimum approach distance is less than the safe distance threshold for the corresponding energized equipment, a graded early warning mechanism is triggered. The early warning level is determined based on the ratio of the minimum approach distance to the safe distance threshold, and an alarm message containing the obstacle type, actual spatial coordinates, movement trajectory, and early warning level is sent to the substation operation and maintenance platform according to the early warning level.

[0012] Preferably, the construction and maintenance of the circular cache queue specifically includes:

[0013] Each AI camera is assigned an independent circular buffer queue. Each circular buffer queue contains a fixed number of buffer frame bits. Each buffer frame bit is used to store a frame of decoded image data and the timestamp of that frame.

[0014] The circular buffer queue is written and overwritten according to the first-in-first-out principle. The newly decoded image frame is written to the buffer frame position pointed to by the current write pointer, the write pointer moves forward one position, and when the write pointer reaches the end of the queue, it returns to the beginning of the queue.

[0015] A frame interval to be processed is maintained between the read pointer and the write pointer. Images are extracted from the frame interval to be processed frame by frame. The processed frame is marked as processed. The read pointer moves forward to release the processed frame.

[0016] When the write pointer catches up with the read pointer, it is determined that the circular buffer queue has overflowed. At this time, the read pointer is forcibly moved forward by a preset number of frame positions to free up buffer space, and the image frames corresponding to the forcibly overwritten frame positions are marked as discarded frames. The number of discarded frames and the timestamp range are recorded.

[0017] Preferably, the separation of the foreground and background specifically includes:

[0018] For each AI camera's fixed monitoring scenario, multiple frames of reference images of the camera in an unobstructed state are pre-acquired. The grayscale mean and grayscale variance of each frame of reference images are calculated pixel by pixel to generate the background statistical model corresponding to the camera.

[0019] The current image is compared pixel by pixel with the background statistical model. The deviation of each pixel in the current image from the mean gray value of the corresponding position in the background statistical model is calculated. When the deviation exceeds the threshold dynamically determined by the gray variance, the pixel is marked as a foreground pixel.

[0020] Connectivity analysis is performed on the binary image marked as foreground pixels. Adjacent foreground pixels are aggregated into connected regions. The bounding rectangle of each connected region is calculated. Connected regions with an area smaller than a preset area threshold or an aspect ratio exceeding a preset range are removed. The remaining connected regions are retained as candidate obstacle target boxes.

[0021] Preferably, the training and inference of the obstacle classification neural network specifically includes:

[0022] Image samples of various obstacles in the substation scenario are collected in advance. Boundary boxes and category labels are added to the obstacle targets in each sample image to construct a training dataset.

[0023] A feature extraction backbone network based on a convolutional neural network is used to extract features from sample images in the training dataset. A region proposal network is connected to the back end of the feature extraction network to generate candidate regions. Then, a classification sub-network is used to predict the category of the candidate regions, and a regression sub-network is used to fine-tune the bounding box position of the candidate regions. The classification loss and regression loss are trained together until the network converges.

[0024] The trained obstacle classification neural network is deployed to the edge computing device of the substation. After the candidate obstacle target boxes are input into the network, the probability distribution of each candidate box corresponding to each category is output. The category corresponding to the maximum probability is taken as the obstacle category label, and the maximum probability value is used as the confidence score.

[0025] Preferably, the cross-frame association matching and motion trajectory generation specifically includes:

[0026] Extract the appearance feature vector of each obstacle in the current frame image. The appearance feature vector includes color histogram features, texture features, and depth semantic features.

[0027] The similarity and Euclidean distance between the appearance feature vectors of each obstacle target in the current frame and the appearance feature vectors of each obstacle target in the previous frame are calculated, and the calculation results are weighted and fused to obtain the association cost matrix.

[0028] Based on the correlation cost matrix, the Hungarian algorithm is used to perform optimal matching between the target in the current frame and the target in the previous frame. For the same obstacle target that is successfully matched, the pixel coordinates of the target in the current frame are appended to the target's motion trajectory sequence. Based on the coordinate data of the latest few frames in the trajectory sequence, the current motion velocity vector and motion direction angle are calculated using a linear fitting method.

[0029] For obstacle targets that have not matched any target in the previous frame for a consecutive preset number of frames, mark them as newly appeared targets and initialize their motion trajectory sequence. For existing trajectories that have not matched any target in the current frame for a consecutive preset number of frames, remove the trajectory from the active trajectory list.

[0030] Preferably, the establishment of the three-dimensional spatial coordinate mapping relationship and the calculation of spatial distance of the substation specifically include:

[0031] The AI ​​cameras in the substation were calibrated to obtain the intrinsic parameter matrix and distortion coefficient. Six known three-dimensional coordinate calibration points were selected on site, and the pixel coordinates of the calibration points were extracted from the acquired images. Based on the intrinsic parameter matrix, the extrinsic parameter matrix of the camera containing the rotation matrix and translation vector was solved by the perspective n-point algorithm.

[0032] Based on the ground elevation of the camera installation points, a ground plane equation is constructed, the center pixel coordinates of the bottom of the obstacle target box are extracted, and the pixel back projection is completed by combining the inside and outside of the camera with the ground plane equation. The actual three-dimensional ground projection coordinates of the obstacle on the substation site are then calculated.

[0033] Retrieve pre-stored 3D coordinates and outline dimensions of live equipment in the substation, calculate the horizontal distance between the obstacle projection point and each live equipment, match the estimated height corresponding to the obstacle category to calculate the vertical distance, and fuse the two types of distances to obtain the real-time 3D spatial distance between the obstacle and the live equipment.

[0034] Preferably, the method for predicting the minimum approach distance within a future preset time window specifically includes:

[0035] Based on the motion velocity vector and motion direction angle, combined with the actual spatial coordinates of the obstacle target at the current moment, extrapolation is performed along the motion direction angle from the current moment according to the magnitude of the motion velocity vector to generate the predicted motion trajectory of the obstacle target within a future preset time window;

[0036] A number of predicted position points are sampled at preset time intervals on the predicted motion trajectory. The spatial distance calculation method is repeatedly executed for each predicted position point to calculate the predicted spatial distance between the predicted position point and each energized device.

[0037] Iterate through all predicted spatial distances of all predicted location points, and take the minimum value as the minimum approach distance between the obstacle target and each energized device within a future preset time window. Record the predicted time and the corresponding energized device number corresponding to the minimum approach distance.

[0038] Preferably, the tiered early warning mechanism specifically includes:

[0039] Set corresponding safety distance thresholds for different categories of live equipment, and configure the safety distance thresholds differently according to the voltage level and equipment type of the live equipment;

[0040] Calculate the ratio of the minimum approach distance to the corresponding safe distance threshold of the live equipment. When the ratio is greater than the first preset ratio, it is determined to be a no-warning state. When the ratio is between the second preset ratio and the first preset ratio, it is determined to be a level one warning. When the ratio is between the third preset ratio and the second preset ratio, it is determined to be a level two warning. When the ratio is less than or equal to the third preset ratio, it is determined to be a level three warning. The first preset ratio is greater than the second preset ratio and the third preset ratio.

[0041] When a Level 1 warning is issued, a text alarm message is sent to the operation and maintenance platform. When a Level 2 warning is issued, while sending a text alarm message, the AI ​​camera at the corresponding monitoring point is controlled to switch to optical zoom to magnify and track the obstacle target. When a Level 3 warning is issued, the sound and light alarm device at the substation site is further linked to issue a sound and light warning signal.

[0042] Furthermore, a dynamic environmental obstacle identification and early warning system is proposed, including:

[0043] Video acquisition and caching module: Acquires multiple monitoring video streams from the substation and decodes them frame by frame. It also configures an independent circular buffer queue for each camera to store image frames and timestamps in an orderly manner.

[0044] Obstacle detection module: Constructs a scene background statistical model to achieve foreground and background separation, generates candidate target boxes through connected component filtering, and completes obstacle classification and confidence scoring;

[0045] Target trajectory calculation module: It integrates multi-dimensional appearance features, uses the Hungarian algorithm to achieve cross-frame target association matching, fits and calculates the obstacle's movement speed and direction, and completes trajectory initialization, update and cleanup;

[0046] Spatial distance calculation module: completes camera calibration and parameter solving, establishes the three-dimensional spatial mapping relationship between pixels and substation, back-projects the actual coordinates of obstacles, and calculates the real-time three-dimensional spatial distance between them and the live equipment in the station;

[0047] Safety risk prediction module: extrapolates the future trajectory of the obstacle's movement based on the obstacle's motion parameters, calculates the predicted distance by sampling from multiple points, filters the minimum approach distance, and predicts the potential safety risks of the obstacle to the electrical equipment.

[0048] Tiered early warning and linkage module: Set differentiated safety thresholds based on equipment voltage level, divide into three levels of early warning according to distance ratio, push alarm information in a tiered manner, and link camera zoom tracking and on-site sound and light alarm devices.

[0049] Compared with the prior art, the advantages of the present invention are:

[0050] A multi-channel AI camera independent ring cache architecture is adopted, and read / write pointer management and overflow discarding strategies ensure stable processing of high-concurrency video streams, avoiding frame loss and processing blockage. Combining background statistical models for coarse screening of foreground and background with deep learning networks for fine recognition, it balances detection efficiency and recognition accuracy in fixed monitoring scenarios, effectively reducing false detection rates. A multi-feature fusion Hungarian algorithm achieves accurate cross-frame target association, establishing a 3D spatial mapping relationship based on camera calibration, converting pixel coordinates into real spatial coordinates, and accurately calculating the 3D distance between obstacles and energized equipment. Simultaneously, based on motion vector extrapolation, it predicts the minimum approach distance for future time windows, enabling proactive safety risk assessment. A graded early warning mechanism with differentiated safety thresholds adapts to the protection needs of equipment at different voltage levels, linking camera zoom tracking and graded responses from on-site audio-visual devices, effectively improving the intelligence level of substation operation and maintenance and proactive safety protection, reducing the risk of power accidents caused by foreign object intrusion. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of a dynamic identification and early warning method for environmental obstacles proposed in this invention;

[0052] Figure 2 This is a schematic diagram illustrating the construction and maintenance of the circular cache queue proposed in this invention;

[0053] Figure 3 This is a schematic diagram illustrating the training and inference of the obstacle classification neural network proposed in this invention. Detailed Implementation

[0054] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0055] An environmental obstacle dynamic identification and early warning system, comprising:

[0056] Video acquisition and caching module: Acquires multiple monitoring video streams from the substation and decodes them frame by frame. It also configures an independent circular buffer queue for each camera to store image frames and timestamps in an orderly manner.

[0057] Obstacle detection module: Constructs a scene background statistical model to achieve foreground and background separation, generates candidate target boxes through connected component filtering, and completes obstacle classification and confidence scoring;

[0058] Target trajectory calculation module: It integrates multi-dimensional appearance features, uses the Hungarian algorithm to achieve cross-frame target association matching, fits and calculates the obstacle's movement speed and direction, and completes trajectory initialization, update and cleanup;

[0059] Spatial distance calculation module: completes camera calibration and parameter solving, establishes the three-dimensional spatial mapping relationship between pixels and substation, back-projects the actual coordinates of obstacles, and calculates the real-time three-dimensional spatial distance between them and the live equipment in the station;

[0060] Safety risk prediction module: extrapolates the future trajectory of the obstacle's movement based on the obstacle's motion parameters, calculates the predicted distance by sampling from multiple points, filters the minimum approach distance, and predicts the potential safety risks of the obstacle to the electrical equipment.

[0061] Tiered early warning and linkage module: Set differentiated safety thresholds based on equipment voltage level, divide into three levels of early warning according to distance ratio, push alarm information in a tiered manner, and link camera zoom tracking and on-site sound and light alarm devices.

[0062] See Figure 1 As shown, a method for dynamic identification and early warning of environmental obstacles includes:

[0063] AI cameras deployed at multiple preset monitoring points within the substation collect real-time video streams of the substation environment. Each video stream is then decoded frame by frame, and the resulting continuous image frames are stored in a circular buffer queue in the order of their acquisition time.

[0064] The current image is extracted frame by frame from the circular buffer queue. Foreground and background separation processing is performed on the current image. Candidate obstacle boxes are generated based on the separated foreground regions. The candidate obstacle boxes are input into a pre-trained obstacle classification neural network, and the obstacle category label corresponding to each candidate obstacle box is output.

[0065] For the same obstacle target in multiple consecutive frames of images with identified obstacle category labels, cross-frame association matching is performed. Based on the pixel coordinate changes of the obstacle target in each frame image, and combined with the three-dimensional spatial coordinate mapping relationship, the motion velocity vector and motion direction angle in the actual space are calculated to generate the motion trajectory data of the obstacle target.

[0066] Based on the obtained motion trajectory data, combined with the pre-established three-dimensional spatial coordinate mapping relationship of the substation, the pixel coordinates of the obstacle target in each frame image are converted into the actual spatial coordinates of the substation, and the minimum approach distance between the obstacle target and each energized device in the future within a preset time window is predicted based on the motion velocity vector.

[0067] When the predicted minimum approach distance is less than the safe distance threshold for the corresponding energized equipment, a graded early warning mechanism is triggered. The early warning level is determined based on the ratio of the minimum approach distance to the safe distance threshold, and an alarm message containing the obstacle type, actual spatial coordinates, movement trajectory, and early warning level is sent to the substation operation and maintenance platform according to the early warning level.

[0068] See Figure 2 As shown, a contiguous memory region is allocated as a circular cache queue for each AI camera. This queue contains a fixed number of cached frame bits, Q in size. Each frame bit is equal to the size of the decoded image data for a single frame and its corresponding acquisition timestamp. The value of Q is determined based on the camera's acquisition frame rate f and the maximum tolerable latency T of the backend processing module, with an additional redundancy M. Where ⌈⋅⌉ represents rounding up, and M takes values ​​of 3 to 5 frames to absorb sudden processing jitter. For example, when f = 25 frames / second and T = 0.5 seconds, Q = ⌈12.5⌉ + 4 = 17 frames. Each frame has a status flag to indicate "idle", "pending", "processing", or "processed".

[0069] Write pointer during queue initialization and read pointer Both point to the start of the queue. After decoding a complete frame from a video stream, check... If the current frame bit status is "processed" or "idle", then write the frame pixel data and its timestamp. The frame bit pointed to is marked as "pending" after writing is complete. Move forward one position, that is If the status is "Pending," it indicates that the cache queue is full, triggering the overflow handling mechanism. The overflow determination condition is the number of writes before the data is processed. The frame bit being pointed to is marked as "pending processing".

[0070] When the backend processing module extracts images frame by frame from the circular buffer queue, The currently pointed-to frame is the starting point. If the frame's status is "Pending Processing," its image data and timestamp are read, and the status is changed to "Processing." After the frame is processed, the frame's status is changed to "Processed," and... Move forward one position, that is . and All the “to be processed” frame bits in between constitute the interval of frames to be processed, and the length L of this interval satisfies This represents the current backlog of frames awaiting processing. The processing module continuously extracts frames from this interval through polling or interruption to ensure that L remains within a reasonable range.

[0071] When an overflow occurs, the system forces... Move forward K frame positions to free up at least K frame positions for writing. For each frame position crossed by the forced move, its state changes from "pending" to "processed," and the corresponding image frames are marked as discarded frames. Simultaneously, the total number D of discarded frames and the timestamp range of these frames are recorded. The formula for D is D=min(K,L), and the actual number of dropped frames does not exceed the current backlog of frames. The recorded timestamp range is used to indicate the time period of missing data in subsequent alarm messages, facilitating network or decoding anomalies troubleshooting by operations and maintenance personnel. After forced overwrite, Updated to Write pointer Continue with the write operation.

[0072] For each camera, N frames of baseline grayscale images are continuously acquired during idle periods when there are no obstacles in the monitored scene. Calculate the grayscale mean pixel by pixel. and variance :

[0073]

[0074] in, (x,y) represents the grayscale value at pixel (x,y) of the reference image in frame t. The μ matrix and σ matrix formed by these values ​​constitute the background statistical model of the camera and are stored in the memory of the edge computing device.

[0075] After each frame is processed, for pixels identified as background, the background mean and variance are updated using a moving average method:

[0076]

[0077]

[0078] Here, α is the learning rate, ranging from 0.01 to 0.05. Foreground pixels are not updated initially to avoid contaminating the background model. This mechanism ensures that the background model can adapt to changes in illumination and slow environmental changes.

[0079] Each time a frame of the current image is retrieved from the circular buffer queue Similarly, convert it to grayscale, and for each pixel (x, y) in the current image, calculate its mean value relative to the background. absolute value of deviation The conditions for determining it as foreground are: Where k=3 is a multiplier factor and β=5 is a fixed offset used to suppress noise in low-variance regions. Pixels that meet the conditions are marked as foreground, otherwise as background, and a binary mask is generated.

[0080] A morphological opening operation is performed on the binary foreground mask to eliminate isolated noise points and smooth the edges of the foreground region. Then, an 8-neighborhood connected component labeling algorithm is used to aggregate adjacent foreground pixels into connected regions. For each connected region, the minimum x-coordinate among all its pixels is selected. Maximum value Minimum value of y-coordinate Maximum value ,by and Determine the minimum axis-aligned bounding rectangle for each diagonal vertex. Eliminate connected regions with an area smaller than a preset area threshold (e.g., 50 pixels) or an aspect ratio less than 0.2 or greater than 5, retaining the remaining regions as candidate bounding boxes. These retained rectangles are then used as candidate obstacle bounding boxes and fed into the classification network.

[0081] Images of the substation scene were pre-collected, covering five categories: personnel, animals, construction machinery, floating objects, and vehicles, with no fewer than 2000 images for each category. Bounding boxes and categories were manually labeled. A Faster R-CNN architecture was adopted, with a ResNet-50 backbone network followed by a region proposal network to generate candidate regions. These candidate regions were then pooled by RoI before being fed into classification and regression branches. Cross-entropy was used as the classification loss. The regression loss uses smoothing. loss Total loss Training with the SGD optimizer until the loss on the validation set no longer decreases.

[0082] After training, the model is quantized and deployed to edge computing devices. The input candidate boxes undergo forward computation, and the classification branch outputs the probabilities of each class. ,Pick As a category label, the corresponding maximum probability A confidence score is assigned. If the confidence score s ≥ 0.3, the target is sent to the subsequent trajectory association module; if s < 0.3, it is considered a low-confidence detection and is not discarded directly, but marked as a "target to be confirmed". It is given a lower appearance matching weight during trajectory association and allowed to enter tracking, but the warning trigger requires a comprehensive judgment based on the average confidence score of multiple consecutive frames. If the confidence score is below 0.3 for 5 consecutive frames, the target is abandoned.

[0083] See Figure 3 As shown, for each target bounding box in the current frame, three appearance features are extracted: HSV color histogram, where the H / S / V channels are quantized into 8, 4, and 4 bins respectively, concatenated into 128 dimensions and normalized; texture features: using the Local Binary Mode operator, a uniform mode LBP histogram is calculated within the target bounding box, with a dimension of 59; and depth semantic features, where the target bounding box is scaled to 224×224 and input into a ResNet-50 pooling layer, with a 2048-dimensional output. The three feature vectors are concatenated end-to-end and reduced to 512 dimensions using PCA to obtain the final appearance feature vector f of the target, which is stored in the target's trajectory data structure.

[0084] Suppose there are m targets in the current frame and n targets in the previous frame, construct an m×n cost matrix C. For the i-th target in the current frame and the j-th target in the previous frame, calculate the appearance cosine similarity. Its range is [−1, 1]. Simultaneously, the spatial Euclidean distance is calculated. Divide by the image diagonal to normalize to Convert appearance similarity into distance cost. The total association cost is:

[0085]

[0086] Where w=0.6 is the appearance weight, and (1−w)=0.4 is the spatial weight. (Complete) This constitutes the cost matrix.

[0087] Input C into the Hungarian algorithm to find the optimal match that minimizes the total cost, ensuring that each target in the current frame matches at most one target from the previous frame, and vice versa. After a successful match, the center pixel coordinates of the target bounding box in the current frame are set. Added to the target trajectory sequence Finally, take the pixel coordinates of the most recent K=5 frames, and first convert the pixel coordinates of each frame to world coordinates using the aforementioned 3D spatial coordinate mapping relationship. , (, 0), to obtain the world coordinate sequence. Linear least squares fitting is used. and Calculate the slope , As a velocity component in the world coordinate system, the velocity vector of motion Direction angle θ = arctan2( For newly created trajectories, the initial velocity is set to zero. It is recalculated after accumulating world coordinates for at least two frames, and subsequently converges rapidly to the actual velocity as the number of frames increases. A linear least squares velocity fitting method is used, suitable for short time windows, such as 5 frames, corresponding to a motion approximation within approximately 0.2 seconds. For complex motions such as pedestrians turning or vehicles changing lanes, the linear assumption has acceptable errors within a very short time. For long-term prediction, a higher-order motion model can be used; this invention is not limited to a linear model. This solution uses a linear model as an example to reduce computational overhead and adapt to the real-time processing needs of edge devices.

[0088] If a target in the current frame does not match any target in the previous frame, a new trajectory is created, with its initial coordinates set to the current values ​​and its velocity vector set to zero. If the target successfully matches for three consecutive frames, it is activated as a new target; otherwise, it is deleted. If a trajectory from the previous frame has no matching target in the current frame, it is marked as "awaiting elimination." If it still has no match after five consecutive frames, it is removed from the list of active trajectories and will no longer participate in subsequent predictions. This mechanism effectively filters out false detections and maintains a stable set of trajectories.

[0089] For each AI camera within the substation, the intrinsic parameter matrix A and distortion coefficients were pre-calculated using Zhang Zhengyou's checkerboard calibration method. Subsequently, six known 3D world coordinates were selected on-site. , , The calibration points, such as permanent ground markers or equipment corner points, are identified by capturing images through a camera and extracting the corresponding pixel coordinates (u, v). Using a perspective n-point algorithm, combined with the intrinsic parameter matrix A, the extrinsic parameter matrix of the camera is iteratively solved. Where R is a 3×3 rotation matrix and t is a 3×1 translation vector. The projection relationship from world coordinates to pixel coordinates is:

[0090]

[0091] Where s is the depth scale factor. After calibration, the intrinsic and extrinsic parameters are stored in the edge device for subsequent backprojection.

[0092] Based on the ground elevation data corresponding to the camera installation locations, a ground plane equation is established, using the substation site ground plane as... =0. For obstacle targets, extract the pixel coordinates (u,v) of the center point of the bottom of the detection box in the current frame. This point corresponds to the position where the obstacle's foot touches the ground, i.e., satisfying... =0. (u,v) and Substituting 0 into the projection equation above, and since A, R, and t are known, we can solve the two linear equations simultaneously. and Thus, the two-dimensional projection coordinates of the obstacle on the actual ground of the substation are obtained. , , 0). The actual height of the obstacle is obtained by looking up a table according to its category label, and its three-dimensional spatial coordinates are recorded as ( , , ).

[0093] The pre-stored substation equipment layout database is called to obtain the three-dimensional coordinates of each live equipment ( , , ) and their contour dimensions. For the obstacle coordinates ( , , ), the horizontal distance and the vertical distance are calculated, and then the three-dimensional spatial distance d is obtained. Traverse all live equipment, and take the minimum d value as the real-time safety distance between the obstacle and the surrounding equipment.

[0094] Assume that the current moment is , the current spatial coordinates of the obstacle are , and the velocity vector is , with the unit of m / s, which is obtained by fitting the world coordinate sequence. The trajectory within the future time window T is extrapolated in a uniform linear motion, and the position at time t is . Several predicted position points are generated within the window at a sampling interval Δt=0.5 seconds . For each , repeat the above distance calculation method to obtain the distance from each live equipment . Traverse all sampling points and all equipment, take the minimum value, and record the predicted moment and equipment number corresponding to the minimum value at the same time. This minimum value is the minimum approaching distance that the obstacle may reach with the live equipment within the future T seconds.

[0095] In the substation equipment management database, a safety distance threshold is preset for each live equipment. The safety distance threshold is determined comprehensively according to the voltage level U and type of the equipment: when U≤35, it is 2.0 meters; when 35<U≤110, it is 3.5 meters; when 110<U≤220, it is 5.0 meters; when U>220, it is 8.0 meters. Open-type equipment adds a margin of 1.0 meter on the above basis, and GIS enclosed equipment is implemented at 0.8 times the reference value. All thresholds are stored in the memory of the edge device in the form of a table.

[0096] Define the warning level coefficient , is the minimum distance value of each live equipment, is the safety distance threshold, and a smaller r value indicates a higher risk. Three preset ratios are set: the first preset ratio =0.8, the second preset ratio =0.5, the third preset ratio =0.25, which satisfies the condition. > > The decision rule is: when r > There is no warning at times, only log entries are recorded; when <r≤ It was determined to be a Level 1 warning at that time; <r≤ When r is determined to be a Level II warning; The time is determined to be a Level 3 warning. Each r value uniquely corresponds to a warning level.

[0097] When a Level 1 warning is detected, the system sends a structured text alarm message to the operation and maintenance platform via the substation's industrial Ethernet. The message includes the obstacle category label, current 3D spatial coordinates, a sequence of movement trajectory points within the last 5 seconds, the warning level, the corresponding energized equipment number, and the safe distance threshold. The alarm is pushed to the platform's message queue using the MQTT protocol. Upon receiving the alarm, the platform notifies the on-duty personnel via pop-up windows and voice announcements, and simultaneously stores the alarm record in the historical database.

[0098] When a Level 2 warning is issued, video tracking is added to all actions required for a Level 1 warning. The system sends a PTZ control command to the AI ​​camera at the monitoring point where the obstacle is located. The command includes the target center pixel coordinates and the current target bounding box size. Upon receiving the command, the camera automatically adjusts its optical zoom, increases its optical focal length, and continuously zooms in to track and capture images of the obstacle.

[0099] When a Level 3 warning is issued, the system further triggers an audible and visual alarm linkage within the substation, building upon all actions required for a Level 2 warning. The system, through the substation's intelligent linkage controller, sends an activation signal to the audible and visual alarms deployed in the area where the obstacle is located, emitting continuous red flashes and intermittent high-frequency alarm sounds to scare away animals or warn personnel. Simultaneously, the activation time and location of the audible and visual alarms are recorded.

[0100] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0101] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic identification and early warning of environmental obstacles, characterized in that, include: AI cameras deployed at multiple preset monitoring points within the substation collect real-time video streams of the substation environment. Each video stream is then decoded frame by frame, and the resulting continuous image frames are stored in a circular buffer queue in the order of their acquisition time. The current image is extracted frame by frame from the circular buffer queue. Foreground and background separation processing is performed on the current image. Candidate obstacle boxes are generated based on the separated foreground regions. The candidate obstacle boxes are input into a pre-trained obstacle classification neural network, and the obstacle category label corresponding to each candidate obstacle box is output. For the same obstacle target in multiple consecutive frames of images with identified obstacle category labels, cross-frame association matching is performed. Based on the pixel coordinate changes of the obstacle target in each frame image, and combined with the three-dimensional spatial coordinate mapping relationship, the motion velocity vector and motion direction angle in the actual space are calculated to generate the motion trajectory data of the obstacle target. Based on the obtained motion trajectory data, combined with the pre-established three-dimensional spatial coordinate mapping relationship of the substation, the pixel coordinates of the obstacle target in each frame image are converted into the actual spatial coordinates of the substation, and the minimum approach distance between the obstacle target and each energized device in the future within a preset time window is predicted based on the motion velocity vector. When the predicted minimum approach distance is less than the safe distance threshold for the corresponding energized equipment, a graded early warning mechanism is triggered. The early warning level is determined based on the ratio of the minimum approach distance to the safe distance threshold, and an alarm message containing the obstacle type, actual spatial coordinates, movement trajectory, and early warning level is sent to the substation operation and maintenance platform according to the early warning level.

2. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The construction and maintenance of the circular cache queue specifically includes: Each AI camera is assigned an independent circular buffer queue. Each circular buffer queue contains a fixed number of buffer frame bits. Each buffer frame bit is used to store a frame of decoded image data and the timestamp of that frame. The circular buffer queue is written and overwritten according to the first-in-first-out principle. The newly decoded image frame is written to the buffer frame position pointed to by the current write pointer, the write pointer moves forward one position, and when the write pointer reaches the end of the queue, it returns to the beginning of the queue. A frame interval to be processed is maintained between the read pointer and the write pointer. Images are extracted from the frame interval to be processed frame by frame. The processed frame is marked as processed. The read pointer moves forward to release the processed frame. When the write pointer catches up with the read pointer, it is determined that the circular buffer queue has overflowed. At this time, the read pointer is forcibly moved forward by a preset number of frame positions to free up buffer space, and the image frames corresponding to the forcibly overwritten frame positions are marked as discarded frames. The number of discarded frames and the timestamp range are recorded.

3. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The separation of foreground and background specifically includes: For each AI camera's fixed monitoring scenario, multiple frames of reference images of the camera in an unobstructed state are pre-acquired. The grayscale mean and grayscale variance of each frame of reference images are calculated pixel by pixel to generate the background statistical model corresponding to the camera. The current image is compared pixel by pixel with the background statistical model. The deviation of each pixel in the current image from the mean gray value of the corresponding position in the background statistical model is calculated. When the deviation exceeds the threshold dynamically determined by the gray variance, the pixel is marked as a foreground pixel. Connectivity analysis is performed on the binary image marked as foreground pixels. Adjacent foreground pixels are aggregated into connected regions. The bounding rectangle of each connected region is calculated. Connected regions with an area smaller than a preset area threshold or an aspect ratio exceeding a preset range are removed. The remaining connected regions are retained as candidate obstacle target boxes.

4. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The training and inference of the obstacle classification neural network specifically include: Image samples of various obstacles in the substation scenario are collected in advance. Boundary boxes and category labels are added to the obstacle targets in each sample image to construct a training dataset. A feature extraction backbone network based on a convolutional neural network is used to extract features from sample images in the training dataset. A region proposal network is connected to the back end of the feature extraction network to generate candidate regions. Then, a classification sub-network is used to predict the category of the candidate regions, and a regression sub-network is used to fine-tune the bounding box position of the candidate regions. The classification loss and regression loss are trained together until the network converges. The trained obstacle classification neural network is deployed to the edge computing device of the substation. After the candidate obstacle target boxes are input into the network, the probability distribution of each candidate box corresponding to each category is output. The category corresponding to the maximum probability is taken as the obstacle category label, and the maximum probability value is used as the confidence score.

5. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The cross-frame association matching and motion trajectory generation specifically include: Extract the appearance feature vector of each obstacle in the current frame image. The appearance feature vector includes color histogram features, texture features, and depth semantic features. The similarity and Euclidean distance between the appearance feature vectors of each obstacle target in the current frame and the appearance feature vectors of each obstacle target in the previous frame are calculated, and the calculation results are weighted and fused to obtain the association cost matrix. Based on the correlation cost matrix, the Hungarian algorithm is used to perform optimal matching between the target in the current frame and the target in the previous frame. For the same obstacle target that is successfully matched, the pixel coordinates of the target in the current frame are appended to the target's motion trajectory sequence. Based on the coordinate data of the latest few frames in the trajectory sequence, the current motion velocity vector and motion direction angle are calculated using a linear fitting method. For obstacle targets that have not matched any target in the previous frame for a consecutive preset number of frames, mark them as newly appeared targets and initialize their motion trajectory sequence. For existing trajectories that have not matched any target in the current frame for a consecutive preset number of frames, remove the trajectory from the active trajectory list.

6. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The establishment of the three-dimensional spatial coordinate mapping relationship and the calculation of spatial distance of the substation specifically include: The AI ​​cameras in the substation were calibrated to obtain the intrinsic parameter matrix and distortion coefficient. Six known three-dimensional coordinate calibration points were selected on site, and the pixel coordinates of the calibration points were extracted from the acquired images. Based on the intrinsic parameter matrix, the extrinsic parameter matrix of the camera containing the rotation matrix and translation vector was solved by the perspective n-point algorithm. Based on the ground elevation of the camera installation points, a ground plane equation is constructed, the center pixel coordinates of the bottom of the obstacle target box are extracted, and the pixel back projection is completed by combining the inside and outside of the camera with the ground plane equation. The actual three-dimensional ground projection coordinates of the obstacle on the substation site are then calculated. Retrieve pre-stored 3D coordinates and outline dimensions of live equipment in the substation, calculate the horizontal distance between the obstacle projection point and each live equipment, match the estimated height corresponding to the obstacle category to calculate the vertical distance, and fuse the two types of distances to obtain the real-time 3D spatial distance between the obstacle and the live equipment.

7. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The method for predicting the minimum approach distance within a future preset time window specifically includes: Based on the motion velocity vector and motion direction angle, combined with the actual spatial coordinates of the obstacle target at the current moment, extrapolation is performed along the motion direction angle from the current moment according to the magnitude of the motion velocity vector to generate the predicted motion trajectory of the obstacle target within a future preset time window; A number of predicted position points are sampled at preset time intervals on the predicted motion trajectory. The spatial distance calculation method is repeatedly executed for each predicted position point to calculate the predicted spatial distance between the predicted position point and each energized device. Iterate through all predicted spatial distances of all predicted location points, and take the minimum value as the minimum approach distance between the obstacle target and each energized device within a future preset time window. Record the predicted time and the corresponding energized device number corresponding to the minimum approach distance.

8. The method for dynamic identification and early warning of environmental obstacles according to claim 1, characterized in that, The tiered early warning mechanism specifically includes: Set corresponding safety distance thresholds for different categories of live equipment, and configure the safety distance thresholds differently according to the voltage level and equipment type of the live equipment; Calculate the ratio of the minimum approach distance to the corresponding safe distance threshold of the live equipment. When the ratio is greater than the first preset ratio, it is determined to be a no-warning state. When the ratio is between the second preset ratio and the first preset ratio, it is determined to be a level one warning. When the ratio is between the third preset ratio and the second preset ratio, it is determined to be a level two warning. When the ratio is less than or equal to the third preset ratio, it is determined to be a level three warning. The first preset ratio is greater than the second preset ratio and the third preset ratio. When a Level 1 warning is issued, a text alarm message is sent to the operation and maintenance platform. When a Level 2 warning is issued, while sending a text alarm message, the AI ​​camera at the corresponding monitoring point is controlled to switch to optical zoom to magnify and track the obstacle target. When a Level 3 warning is issued, the sound and light alarm device at the substation site is further linked to issue a sound and light warning signal.

9. A dynamic environmental obstacle identification and early warning system, used to implement the dynamic environmental obstacle identification and early warning method as described in any one of claims 1-8, characterized in that, include: Video acquisition and caching module: Acquires multiple monitoring video streams from the substation and decodes them frame by frame. It also configures an independent circular buffer queue for each camera to store image frames and timestamps in an orderly manner. Obstacle detection module: Constructs a scene background statistical model to achieve foreground and background separation, generates candidate target boxes through connected component filtering, and completes obstacle classification and confidence scoring; Target trajectory calculation module: It integrates multi-dimensional appearance features, uses the Hungarian algorithm to achieve cross-frame target association matching, fits and calculates the obstacle's movement speed and direction, and completes trajectory initialization, update and cleanup; Spatial distance calculation module: completes camera calibration and parameter solving, establishes the three-dimensional spatial mapping relationship between pixels and substation, back-projects the actual coordinates of obstacles, and calculates the real-time three-dimensional spatial distance between them and the live equipment in the station; Safety risk prediction module: extrapolates the future trajectory of the obstacle's movement based on the obstacle's motion parameters, calculates the predicted distance by sampling from multiple points, filters the minimum approach distance, and predicts the potential safety risks of the obstacle to the electrical equipment. Tiered early warning and linkage module: Set differentiated safety thresholds based on equipment voltage level, divide into three levels of early warning according to distance ratio, push alarm information in a tiered manner, and link camera zoom tracking and on-site sound and light alarm devices.

Images